Can we input array in curl function and follow the result for each element?

Question

My question is simple, but for me, it is creating so many confusions in my mind. I want to know that can we input an array to curl function ?
please note that 'I AM NOT POSTING DATA' (FOR POSTING DATA , I KNOW, ARRAY IS USED)

To make my question more clear, let me tell you the code ..

function mycurl($url){
$ch = curl_init(); // create a new cURL resource
// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$data = curl_exec($ch); // grab URL and pass it to the browser
//echo $data; //ncomment for debugging
curl_close($ch); 
return $data;

}

and array i am going to use is

myArray 
0 => string 'http://www.abc.com/a
1 => string 'http://www.abc.com/b
2 => string 'http://www.abc.com/c
3 => string 'http://www.abc.com/d

to use the array, I am using foreach loop the code is given below

foreach ($myArray as $temp){
    $heading= mycurl($temp);
    echo $heading;
    }

the purpose of the code is

go to each url of array using curl function
extract required data from the url
process the next element of array and extract data and so on

Can anybody guide me that How can i use array elements in curl function? How can i get my objective? If foreach loop is not the correct methodology here, then what should I do?

Mike Purcell · Accepted Answer · 2012-02-03 19:05:57Z

1

What your doing looks good, except, you shouldn't initialize the curl handler for every iteration, just initialize it once, then change the $url value for every iteration, would look something like:

function mycurl($ch, $url) {

    curl_setopt($ch, CURLOPT_URL,$url);

    return curl_exec($ch); 
}

$ch = curl_init(); // create a new cURL resource

curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);

foreach ($urls as $url) {
    $header = mycurl($ch, $url);
    var_dump($header);
}

curl_close($ch);

answered Feb 3, 2012 at 19:05

Mike Purcell

20k10 gold badges54 silver badges94 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Zaffar Saffee Over a year ago

thaks for the reply, i have done like it, but it is taking too much time, and randomly displays an error that "30second time out error", i know the reason that there are many elements of array , so taking time, is there any way to improve the performance and speed?

Mike Purcell Over a year ago

How far into the array stack does it reach before it throws the error? Try setting CURLOPT_CONNECTTIMEOUT to 0, and see if that helps.

Zaffar Saffee Over a year ago

i changed the time out in php config to 90 seconds..and it almost worked. if my network connection is okay, it works, if the connection is slow at the moment, it randomly gives error, and setting connect time out option fully resolve this problem.

Mike Purcell Over a year ago

Ahhh cool. Ya we had to set it to 0 for a project where we would remain logged in for a few hours pulling huge datasets.

Vyktor · Accepted Answer · 2012-02-03 19:03:01Z

1

You're doing it right. However you can use curl_multi_exec() (see examples) to launch all requests at once, classical curl can do only one request at the time.

Little more effective way to do your code would be:

$ch = curl_init(); // create a new cURL resource
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);

foreach( $myArray as $url){
    curl_setopt($ch, CURLOPT_URL,$url);
    $data = curl_exec( $ch);
    echo $data;
}

curl_close( $ch);

Or with correct object design:

class MyClass {
    protected $ch = null;

    public function __construct( ){
        $this->ch = curl_init();
        curl_setopt($this->ch, CURLOPT_HEADER, 0);
        curl_setopt($this->ch, CURLOPT_RETURNTRANSFER, TRUE);
    }

    public function __destruct(){
        curl_close( $this->ch);
    }

    public function getData( $url){
        curl_setopt($this->ch, CURLOPT_URL,$url);
        return curl_exec( $this->ch);
    }
}

$extractor = new MyClass();
foreach( $myArray as $url){
    $data = $extractor->getData( $url);
    echo $data;
}

answered Feb 3, 2012 at 19:03

Vyktor

21.1k6 gold badges69 silver badges98 bronze badges

1 Comment

Zaffar Saffee Over a year ago

thanks dear, thanks for the answer, i got read the example and what i understand is that, i should use curl multi exec if i have to run multiple instances at the same time, but i want to run one instance (one element of array), do some scrapping work, and then run the next instance, so the code remains same?

Collectives™ on Stack Overflow

Can we input array in curl function and follow the result for each element?

2 Answers 2

4 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related