2

I am using the following code and am not able to display amazon.com using php and curl. Im using curl_error and getting no errors, so I'm not sure what im doing wrong

<?php 

$curl = curl_init();

curl_setopt($curl, CURLOPT_URL, 'https://www.amazon.com');

curl_exec($curl);

curl_close ($curl);

I'm doing this on local host

2
  • What do you want to achieve? What exactly is not working with the given code? Commented Nov 3, 2018 at 8:50
  • im just trying to scrape amazon, this exact code worked just the day before, with the ssl_verifypeer. im new to this and do not understand the importance of debugging, but according to a later post, there is a bug that can be fixed by using curlopt_encoding Commented Nov 3, 2018 at 16:29

3 Answers 3

5

just display amazon then use this

echo file_get_contents("https://www.amazon.com");
Sign up to request clarification or add additional context in comments.

Comments

1

You should use the following:

$response = curl_exec($curl);

$result is an array. You can get for example the body of the request by using:

$header_size = curl_getinfo($curl,CURLINFO_HEADER_SIZE);
        $result['header'] = substr($response, 0, $header_size);
        $result['body'] = substr( $response, $header_size );
        $result['http_code'] = curl_getinfo($curl,CURLINFO_HTTP_CODE);
        $result['last_url'] = curl_getinfo($curl,CURLINFO_EFFECTIVE_URL);

echo $result['body'];

For more information: http://php.net/manual/de/function.curl-exec.php

Comments

1

when debugging curl code, use CURLOPT_VERBOSE, and post the CURLOPT_VERBOSE log when asking for help. also when debugging, do not ignore the return values of curl_setopt, because it returns bool(false) if there was an error, and if there was an error, that error would probably explain why the code isn't working. also do not ignore the return value of curl_exec, because it returns bool(false) if there was an error, which goes unnoticed if you ignore the return value (and your code does)

here is a version of your code that doesn't ignore any errors and enable CURLOPT_VERBOSE logging, it should reveal where your code fails:

<?php
$curl = curl_init();
if (! is_resource($curl)) {
    throw new \RuntimeException('curl_init() failed!');
}
ecurl_setopt($curl, CURLOPT_URL, 'https://www.amazon.com');
ecurl_setopt($curl, CURLOPT_VERBOSE, 1);
$curlstderr = etmpfile();
$curlstdout = etmpfile();
ecurl_setopt($curl, CURLOPT_STDERR, $curlstderr);
ecurl_setopt($curl, CURLOPT_FILE, $curlstdout);
if (true !== curl_exec($curl)) {
    throw new \RuntimeException("curl_exec failed! " . curl_errno($curl) . ": " . curl_error($curl));
}
rewind($curlstderr); // https://bugs.php.net/bug.php?id=76268
rewind($curlstdout); // https://bugs.php.net/bug.php?id=76268
$verbose = stream_get_contents($curlstderr);
$output = stream_get_contents($curlstdout);
curl_close($curl);
fclose($curlstderr);
fclose($curlstdout);
var_dump($verbose, $output);

function ecurl_setopt ( /*resource*/$ch, int $option , /*mixed*/ $value): bool
{
    $ret = curl_setopt($ch, $option, $value);
    if ($ret !== true) {
        // option should be obvious by stack trace
        throw new RuntimeException('curl_setopt() failed. curl_errno: ' . return_var_dump(curl_errno($ch)) . '. curl_error: ' . curl_error($ch));
    }
    return true;
}

function etmpfile()
{
    $ret = tmpfile();
    if (false === $ret) {
        throw new \RuntimeException('tmpfile() failed!');
    }
    return $ret;
}

also, it appears that https://www.amazon.com has a bug, see is it a bug to send response gzip-compressed to clients that doesn't specify Accept-Encoding: gzip?

  • in any case, to make curl automatically decompress the gzip-compressed response from amazon, add ecurl_setopt($curl,CURLOPT_ENCODING,''); , that tells libcurl to add the Accept-Encoding: gzip,deflate header, and automatically decompress the result.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.