PHP CURL sometimes return empty

Question

I want to ask something make me confuse. I'am using CURL to get html code from this link

echo set_user_agent_grab("https://www.bandros.co.id/produk/dress-atasan-baju-rajut-wanita-sad-500");

And This is my function

function set_user_agent_grab($link){
    $headers = ["text/html; charset=UTF-8"];
            $ch = curl_init();
            curl_setopt($ch, CURLOPT_URL, $link);
            curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.3 Safari/533.2');
            curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
            curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
            curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
            curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 30);
            curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
            curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
            curl_setopt($ch, CURLOPT_ENCODING, 'gzip');
            $result = curl_exec($ch);
            curl_close($ch);
            return $result;
    }

The problem, sometimes i got return empty, i dont know this is from my server or from the site protection with i dont know, please tell me, thank you.

I wonder how this would run anyway, because $ch is undefined — Jeff
– Jeff, Commented Dec 7, 2017 at 1:04
sorry i'am forgot, i've been edit my function by add $ch = curl_init(); — CSM Media
– CSM Media, Commented Dec 7, 2017 at 1:09
Using var_dump(curl_getinfo($ch)), var_dump(curl_errno($ch)) and var_dump(curl_error($ch)) after calling curl_exec() may give you some insight, at the moment you're not looking at any response state information, just the body. — Scuzzy
– Scuzzy, Commented Dec 7, 2017 at 1:11
sometimes i got return empty -- hmm, how often it happens? could you output the var_dumps suggested by @Scuzzy in the comment above to at least track on what occasion it returns empty. I believe this can be the target server did not want to answer your request. — Bagus Tesa
– Bagus Tesa, Commented Dec 7, 2017 at 1:15
var_dump(curl_getinfo($ch)) and get resource(1) of type (Unknown), what this mean ? — CSM Media
– CSM Media, Commented Dec 7, 2017 at 1:43

hanshenrik · Accepted Answer · 2017-12-07 12:00:12Z

1

CURLOPT_VERBOSE should reveal what happened. so just check if curl_exec fail, and if it does, throw a RuntimeException, then, next time it happens, check your php error logs. additionally you can check what curl_errno() and curl_error says.

function set_user_agent_grab($link) {
    $headers = [ 
            "text/html; charset=UTF-8" 
    ];
    $ch = curl_init ();
    $debugfileh = tmpfile ();
    $debugfile = stream_get_meta_data ( $debugfileh ) ['uri'];
    try {
        curl_setopt ( $ch, CURLOPT_VERBOSE, 1);
        curl_setopt ( $ch, CURLOPT_STDERR, $debugfileh);

        curl_setopt ( $ch, CURLOPT_URL, $link );
        curl_setopt ( $ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.3 Safari/533.2' );
        curl_setopt ( $ch, CURLOPT_FOLLOWLOCATION, true );
        curl_setopt ( $ch, CURLOPT_RETURNTRANSFER, true );
        curl_setopt ( $ch, CURLOPT_MAXREDIRS, 10 );
        curl_setopt ( $ch, CURLOPT_CONNECTTIMEOUT, 30 );
        curl_setopt ( $ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1 );
        curl_setopt ( $ch, CURLOPT_HTTPHEADER, $headers );
        curl_setopt ( $ch, CURLOPT_ENCODING, 'gzip' );
        $result = curl_exec ( $ch );
        if (! is_string ( $result )) {
            $errstr = "curl_exec failed: " . curl_errno ( $ch ) . ": " . curl_error ( $ch ) . ". debuginfo: " . file_get_contents ( $debugfile );
            throw new RuntimeException ( $errstr );
        }
        return $result;
    } finally{
        fclose ( $debugfileh );
        curl_close ( $ch );
    }
}

answered Dec 7, 2017 at 12:00

hanshenrik

22.3k4 gold badges62 silver badges113 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

CSM Media Over a year ago

Thanks before mr @hanshenrik, I do not see any error on the error_log file and does it possibly affect the cookies of the website? if from cookies how to use it?

hanshenrik Over a year ago

@CSMMedia libcurl doesn't use any cookies by default, every time this code runs, it starts fresh without any cookies. but is your error logging broken then? you should fix your error logging before debugging this further. as a simple testcase, run <?php throw new \Exception("error log test"); , and see if you can find error log test in your error logs... can you?

CSM Media Over a year ago

ok, thank you very much for helping but iam very curios even i using cronjob to run script every 5 minute and i not getting error in error_log, can we discuss privately so i can share my all code to you, I really appreciate if you can help me, please.

hanshenrik Over a year ago

@CSMMedia sure, try facebook. contact details on my profile.

Collectives™ on Stack Overflow

PHP CURL sometimes return empty

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related