Passing multiple HTTP API URL in PHP

Question

Am needing to call two or more HTTP API to GET META from Piwik Analytics. Unfortunately, their method of method=API.getBulkRequest will not work for a PHP call, and only for JSON and XML - both of which do not work in this request.

Is it possible to combine

https://demo.piwik.org/?module=API&method=Actions.get&idSite=7&period=month&date=last3&format=xml&token_auth=anonymous

and

https://demo.piwik.org/?module=API&method=VisitsSummary.get&idSite=7&period=month&date=last3&format=xml&token_auth=anonymous

Using this method of calling API

$url = "https://demo.piwik.org/";
$url .= "?module=API&method=Actions.get";
$url .= "&idSite=7&period=month&date=last3";
$url .= "&format=php&filter_sort_order=desc";
$url .= "&token_auth=anonymous";

so I can GET the data using

foreach ($content as $dates => $row) {
    $visits = @$row['nb_pageviews'] ?: 0;
    $unique = @$row['nb_uniq_pageviews'] ?: 0;
    $actions = @$row['nb_outlinks'] ?: 0;
    $bounce = @$row['bounce_count'] ?: 0;
    $bounce_rate = @$row['bounce_rate'] ?: 0;
    $time = @$row['avg_time_on_site'] ?: 0;

Thank you for your help and Merry Xmas tomorrow

UPDATE OF FULL CODE IN .PHP file

<?php
            $url = "https://demo.piwik.org/";
            $url .= "?module=API&method=Actions.get";
            $url .= "&idSite=7&period=month&date=last3";
            $url .= "&format=php&filter_sort_order=desc";
            $url .= "&token_auth=anonymous";

            $fetched = file_get_contents($url);
            $content = unserialize($fetched);
            krsort($content);

            // case error
            if (!$content) {
                print("<tr><td class='month subscriber subscriber-fixed-alone fixed-cell'></td><td class='stat number text-center'>Error, analytics not available at the moment.</td></tr>");
            }

            foreach ($content as $dates => $row) {
                $visits = @$row['nb_pageviews'] ?: 0;
                $unique = @$row['nb_uniq_pageviews'] ?: 0;
                $actions = @$row['nb_outlinks'] ?: 0;
                $bounce = @$row['bounce_count'] ?: 0;
                $bounce_rate = @$row['bounce_rate'] ?: 0;
                $time = @$row['avg_time_on_site'] ?: 0;

                $date = $dates;
                $action_percent = get_percentage($visits,$actions);
                $unique_percent = get_percentage($visits,$unique);

                print("<tr>");
                print("<td class='month subscriber subscriber-fixed-alone fixed-cell'>" .date('F Y',strtotime($date)). "</td>");
                print("<td class='stat number text-center'>$visits</td>");
                print("<td class='stat number text-center'>$unique ($unique_percent%)</td>");
                print("<td class='stat number text-center'>$actions ($action_percent%)</td>");
                print("<td class='stat number text-center'>$bounce ($bounce_rate)</td>");
                print("<td class='stat number text-center'>$time</td>");
                print("</tr>");
            }?>

@KIKOSoftware. I tried an additional $url = yet it just stopped all data from being retrieved. — Darren
– Darren, Commented Dec 24, 2017 at 8:45
"To send a bulk tracking request, an HTTP POST must be made with a JSON object to the Piwik tracking endpoint." ? that is from their api documentation. What is the problem sending that type of request data - json is nothing special — Professor Abronsius
– Professor Abronsius, Commented Dec 24, 2017 at 8:47

Professor Abronsius · Accepted Answer · 2017-12-24 09:50:46Z

In order to use the getBulkRequest ( details of which I could not find on their website ) or even the other API calls you might find that using curl is easier as you can quickly tailor the request by changing a few options passed to the curl function below and constructing a different $params payload.

function curl( $url=NULL, $options=NULL ){
    /*
        Download a copy of cacert.pem from
        https://curl.haxx.se/docs/caextract.html

        and then edit below as appropriate
    */
    $cacert='c:/wwwroot/cacert.pem';

    /* for advanced debugging info */
    $vbh = fopen('php://temp', 'w+');


    $res=array(
        'response'  =>  NULL,
        'info'      =>  array( 'http_code' => 100 ),
        'headers'   =>  NULL,
        'errors'    =>  NULL
    );
    if( is_null( $url ) ) return (object)$res;

    session_write_close();

    /* Initialise curl request object */
    $curl=curl_init();
    if( parse_url( $url,PHP_URL_SCHEME )=='https' ){
        curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, true );
        curl_setopt( $curl, CURLOPT_SSL_VERIFYHOST, 2 );
        curl_setopt( $curl, CURLOPT_CAINFO, $cacert );
    }

    /* Define standard options */
    curl_setopt( $curl, CURLOPT_URL,trim( $url ) );
    curl_setopt( $curl, CURLOPT_AUTOREFERER, true );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, true );
    curl_setopt( $curl, CURLOPT_FAILONERROR, true );
    curl_setopt( $curl, CURLOPT_HEADER, false );
    curl_setopt( $curl, CURLINFO_HEADER_OUT, false );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, true );
    curl_setopt( $curl, CURLOPT_BINARYTRANSFER, true );
    curl_setopt( $curl, CURLOPT_CONNECTTIMEOUT, 20 );
    curl_setopt( $curl, CURLOPT_TIMEOUT, 60 );
    curl_setopt( $curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36' );
    curl_setopt( $curl, CURLOPT_MAXREDIRS, 10 );
    curl_setopt( $curl, CURLOPT_ENCODING, '' );

    /* advanced debugging info */
    curl_setopt( $curl,CURLOPT_VERBOSE,true );
    curl_setopt( $curl,CURLOPT_NOPROGRESS,true );
    curl_setopt( $curl,CURLOPT_STDERR,$vbh );

    /* Assign runtime parameters as options */
    if( isset( $options ) && is_array( $options ) ){
        foreach( $options as $param => $value ) curl_setopt( $curl, $param, $value );
    }

    /* Execute the request and store responses */
    $res=(object)array(
        'response'  =>  curl_exec( $curl ),
        'info'      =>  (object)curl_getinfo( $curl ),
        'errors'    =>  curl_error( $curl )
    );

    rewind( $vbh );
    $res->verbose=stream_get_contents( $vbh );
    fclose( $vbh );

    curl_close( $curl );
    return $res;
}

/*
The original url
----------------
$url = "https://demo.piwik.org/";
$url .= "?module=API&method=Actions.get";
$url .= "&idSite=7&period=month&date=last3";
$url .= "&format=php&filter_sort_order=desc";
$url .= "&token_auth=anonymous";
*/

$url='https://demo.piwik.org/';

/* original url as array of parameters to POST */
$params=array(
    'module'            =>  'API',
    'method'            =>  'Actions.get',
    'idSite'            =>  7,
    'period'            =>  'month',
    'date'              =>  'last3',
    'format'            =>  'php',
    'filter_sort_order' =>  'desc',
    'token_auth'        =>  'anonymous'

);
/* Set the options for POSTing the data */
$options=array(
    CURLOPT_POSTFIELDS  =>  http_build_query( $params ),
    CURLOPT_POST        =>  true
);
/* Make the request */
$result = curl( $url, $options );

/* If the response status code is 200 (OK) do stuff with data */
if( $result->info->http_code==200 ){
    $data=unserialize( $result->response );
    echo '<pre>',print_r($data,true),'</pre>';
} else {
    /* panic */
    echo "Error: {$result->info->http_code}";
}

The above will output the following:

Array
(
    [2017-10] => Array
        (
            [nb_pageviews] => 37849
            [nb_uniq_pageviews] => 30775
            [nb_downloads] => 54
            [nb_uniq_downloads] => 50
            [nb_outlinks] => 1911
            [nb_uniq_outlinks] => 1808
            [nb_searches] => 1452
            [nb_keywords] => 925
            [avg_time_generation] => 0.672
        )

    [2017-11] => Array
        (
            [nb_pageviews] => 36409
            [nb_uniq_pageviews] => 29518
            [nb_downloads] => 50
            [nb_uniq_downloads] => 48
            [nb_outlinks] => 1841
            [nb_uniq_outlinks] => 1724
            [nb_searches] => 1238
            [nb_keywords] => 775
            [avg_time_generation] => 0.672
        )

    [2017-12] => Array
        (
            [nb_pageviews] => 28945
            [nb_uniq_pageviews] => 21623
            [nb_downloads] => 34
            [nb_uniq_downloads] => 28
            [nb_outlinks] => 1343
            [nb_uniq_outlinks] => 1280
            [nb_searches] => 819
            [nb_keywords] => 507
            [avg_time_generation] => 0.832
        )

)

To use the getBulkRequest feature of the api - something along these lines should get you started.

$params=array(
    'module'            =>  'API',
    'method'            =>  'API.getBulkRequest',
    'format'            =>  'json',
    'token_auth'        =>  'anonymous'
);
$urls=array(
    array( 'method' => 'VisitsSummary.get', 'idSite' => 7, 'date' => 'last3', 'period' => 'month' ),
    array( 'method' => 'VisitsSummary.get', 'idSite' => 7, 'date' => 'september', 'period' => 'day' )
);              
$tmp=array();
foreach( $urls as $index => $site ) $tmp[]='urls['.$index.']='.urlencode( http_build_query( $site ) );


/* Set the options for POSTing the data */
$options=array(
    CURLOPT_POSTFIELDS  =>  http_build_query( $params ) . '&' . implode('&',$tmp),
    CURLOPT_POST        =>  true
);


/* Make the request */
$result = curl( $url, $options );

/* If the response status code is 200 (OK) do stuff with data */
if( $result->info->http_code==200 ){
    $data=json_decode( $result->response );
    echo '<pre>',print_r($data,true),'</pre>';
} else {
    /* panic */
    echo "Error: {$result->info->http_code}";
}

The above ( for getBulkRequests ) will output:

Array
(
    [0] => stdClass Object
        (
            [2017-10] => stdClass Object
                (
                    [nb_uniq_visitors] => 16911
                    [nb_users] => 0
                    [nb_visits] => 21425
                    [nb_actions] => 41266
                    [nb_visits_converted] => 1
                    [bounce_count] => 14400
                    [sum_visit_length] => 2776649
                    [max_actions] => 59
                    [bounce_rate] => 67%
                    [nb_actions_per_visit] => 1.9
                    [avg_time_on_site] => 130
                )

            [2017-11] => stdClass Object
                (
                    [nb_uniq_visitors] => 16108
                    [nb_users] => 0
                    [nb_visits] => 20523
                    [nb_actions] => 39538
                    [nb_visits_converted] => 2
                    [bounce_count] => 13900
                    [sum_visit_length] => 2639042
                    [max_actions] => 171
                    [bounce_rate] => 68%
                    [nb_actions_per_visit] => 1.9
                    [avg_time_on_site] => 129
                )

            [2017-12] => stdClass Object
                (
                    [nb_uniq_visitors] => 12156
                    [nb_users] => 0
                    [nb_visits] => 15199
                    [nb_actions] => 31149
                    [nb_visits_converted] => 4
                    [bounce_count] => 10231
                    [sum_visit_length] => 2069782
                    [max_actions] => 830
                    [bounce_rate] => 67%
                    [nb_actions_per_visit] => 2
                    [avg_time_on_site] => 136
                )

        )

    [1] => stdClass Object
        (
            [nb_uniq_visitors] => 311
            [nb_users] => 0
            [nb_visits] => 347
            [nb_actions] => 592
            [nb_visits_converted] => 2
            [bounce_count] => 229
            [sum_visit_length] => 33879
            [max_actions] => 12
            [bounce_rate] => 66%
            [nb_actions_per_visit] => 1.7
            [avg_time_on_site] => 98
        )

)

Yes, I spent a rather long time reading the documentation and thought that their example of curl -i -X POST -d 'module=API&method=API.getBulkRequest&format=json&urls[0]=method%3dVisitsSummary.get%26idSite%3d3%26date%3d2012-03-06%26period%3dday&urls[1]=method%3dVisitorInterest.getNumberOfVisitsPerVisitDuration%26idSite%3d3%26date%3d2012-03-06%26period%3dday' https://demo.piwik.org/index.php would work. Yet, I cannot get to do so.
I'd not heard of piwik before now - I'm impressed so far and will, in the future, implement this for myself I think - Thanks Darren!
Wow. This is great and I accepted it as the correct answer. However, did you get the output as posted above? I currently get Error:0. With zero errors in the console or logs. Any idea?
Yes, Piwik is great. I find it a little slow but that may be the way I have implemented. We'll tidy that later. Using it firstly to host our own Analytics but also, Heatmaps and Screen Recordings are great. Along with the ability to create accounts programmatically with new user accounts.

Florian F. · Accepted Answer · 2017-12-24 09:24:28Z

0

Moin, can' t you you use the bulk request feature of the Piwik API: See https://developer.piwik.org/api-reference/reporting-api (Advanced Users: Send multiple API Requests at once) ?

Also, i would recommend to use a HTTP client library such as https://packagist.org/packages/guzzlehttp/guzzle which has a set of very nice features to facilitate server side HTTP calls (e.g. sync and async mode)

answered Dec 24, 2017 at 9:24

Florian F.

111 bronze badge

1 Comment

Darren Over a year ago

Thank you, Guzzle does look like a great library. Found it 20 minutes ago and look forward to delving further. Will give RamRaider answer a go first. Thank you again for your time.

Collectives™ on Stack Overflow

Passing multiple HTTP API URL in PHP

2 Answers 2

8 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

8 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related