How to avoid getting redirected when using cURL in PHP

Question

I want to call my Google App Script web service from PHP and I'm trying to use cURL for that, with GET method for starters. The service reponds with a simple text yes. I have found the PHP function (see below) to perform a GET request.

The first response from App Script service is always a redirect so I turned following redirects off in the configuration array in the fetching function, read the url and performed another request. However on the second request url I get redirected to the service url (with the text yes gets displayed as a result of the service) which is not desirable. I want to programatically process reponses, not having cURL influence the output or behavior in the browser.

Initial state and how it should work:

I have a functioning App Script web service URL.
The web service return the text yes.
I have a PHP script which calls the web service.
After the call the PHP script echoes the content returned by the service.
After displaying the content returned by the service the PHP script echoes the word hello.

What actually happens when opened the PHP script in browser:

I perform the first GET request and get response.
I fetch the redirect_url from the response and perform a second GET request to that.
The word hello get displayed on its for a second, wtill with the url of the PHP script.
Url in the browser changes to that of the service, hello disappears and the service response yes appears (probably just got redirected).

PHP script I Use:

$response = get_web_page($url);

$url2 = $response["redirect_url"];

$response = get_web_page($url2);

echo $response["content"];

echo "hello";

PHP function I use to fetch the URL:

function get_web_page( $url, $cookiesIn = '' ){
        $options = array(
            CURLOPT_RETURNTRANSFER => true,     // return web page
            CURLOPT_HEADER         => true,     //return headers in addition to content
            CURLOPT_FOLLOWLOCATION => false,     // follow redirects
            CURLOPT_ENCODING       => "",       // handle all encodings
            CURLOPT_AUTOREFERER    => false,     // set referer on redirect
            CURLOPT_CONNECTTIMEOUT => 120,      // timeout on connect
            CURLOPT_TIMEOUT        => 120,      // timeout on response
            CURLOPT_MAXREDIRS      => 10,       // stop after 10 redirects
            CURLINFO_HEADER_OUT    => true,
            CURLOPT_SSL_VERIFYPEER => true,     // Validate SSL Cert
            CURLOPT_HTTP_VERSION   => CURL_HTTP_VERSION_1_1,
            CURLOPT_COOKIE         => $cookiesIn
        );

        $ch      = curl_init( $url );
        curl_setopt_array( $ch, $options );
        $rough_content = curl_exec( $ch );
        $err     = curl_errno( $ch );
        $errmsg  = curl_error( $ch );
        $header  = curl_getinfo( $ch );
        curl_close( $ch );

        $header_content = substr($rough_content, 0, $header['header_size']);
        $body_content = trim(str_replace($header_content, '', $rough_content));
        $pattern = "#Set-Cookie:\\s+(?<cookie>[^=]+=[^;]+)#m"; 
        preg_match_all($pattern, $header_content, $matches); 
        $cookiesOut = implode("; ", $matches['cookie']);

        $header['errno']   = $err;
        $header['errmsg']  = $errmsg;
        $header['headers']  = $header_content;
        $header['content'] = $body_content;
        $header['cookies'] = $cookiesOut;
    return $header;
}

When I try to print the whole response of the second request:

The content is clearly empty yet.

Array ( [url] => https://www.google.com/a/MY_DOMAIN_NAME/ServiceLogin?service=wise&passive=1209600&continue=https://script.google.com/a/MY_DOMAIN_NAME/macros/s/AKfycbwg9-Y10kLyIJxGLYhpx9BnIu5f8AHs4qKEq1rSUoD-ugjsD3c/exec&followup=https://script.google.com/a/MY_DOMAIN_NAME/macros/s/AKfycbwg9-Y10kLyIJxGLYhpx9BnIu5f8AHs4qKEq1rSUoD-ugjsD3c/exec [content_type] => text/html; charset=UTF-8 [http_code] => 200 [header_size] => 511 [request_size] => 379 [filetime] => -1 [ssl_verify_result] => 0 [redirect_count] => 0 [total_time] => 0.146943 [namelookup_time] => 0.001001 [connect_time] => 0.026204 [pretransfer_time] => 0.102405 [size_upload] => 0 [size_download] => 1839 [speed_download] => 12515 [speed_upload] => 0 [download_content_length] => 1839 [upload_content_length] => 0 [starttransfer_time] => 0.146158 [redirect_time] => 0 [redirect_url] => [primary_ip] => 172.217.16.68 [certinfo] => Array ( ) [request_header] => GET /a/MY_DOMAIN_NAME/ServiceLogin?service=wise&passive=1209600&continue=https://script.google.com/a/MY_DOMAIN_NAME/macros/s/AKfycbwg9-Y10kLyIJxGLYhpx9BnIu5f8AHs4qKEq1rSUoD-ugjsD3c/exec&followup=https://script.google.com/a/MY_DOMAIN_NAME/macros/s/AKfycbwg9-Y10kLyIJxGLYhpx9BnIu5f8AHs4qKEq1rSUoD-ugjsD3c/exec HTTP/1.1 Host: www.google.com Accept: */* Accept-Encoding: deflate, gzip Cookie: [errno] => 0 [errmsg] => [headers] => HTTP/1.1 200 OK Content-Type: text/html; charset=UTF-8 X-Frame-Options: DENY Cache-control: no-cache, no-store Pragma: no-cache Expires: Mon, 01-Jan-1990 00:00:00 GMT Date: Sun, 08 Jul 2018 20:17:24 GMT X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block Content-Length: 1839 Server: GSE Set-Cookie: GAPS=1:qc-OvEMCdZAhgMZ6MhkFoUwlPrOIGg:_W0RGhF378FPmC1G;Path=/a;Expires=Tue, 07-Jul-2020 20:17:24 GMT;Secure;HttpOnly;Priority=HIGH Alt-Svc: quic=":443"; ma=2592000; v="43,42,41,39,35" [content] =>
[cookies] => GAPS=1:qc-OvEMCdZAhgMZ6MhkFoUwlPrOIGg:_W0RGhF378FPmC1G )

EDIT:

It turns out that that the service was set to access for me only. So the chain of events is clearer now: I accessed the service, Google demanded authentication and since I was logged in in the browser as the required user I immediately got redirected to the service. Once I had run the PHP script in a browser while logged out I got the Google log in page.

I have now deployed the service for anyone to access and I can acces the service without redirect and getting the content from the service. The differentiator of the redirecting behavior seems to be the /a/druna.cz/. Without it and when still deployed only for me to access I get the log in page html in content without redirecting. With it it always leads to redirecting. The actual url is https://script.google.com/a/druna.cz/macros/s/AKfycbzo6Y_XiXHLZFuNEb2rB7GLVXbXhtBpCGo9AlL8ul-gITmvv6k/exec

So even though with better settings the redirect can be avoided, the question stands - how does it force the redirect in the case with authentication required (/a/druna.cz/ present) and is there a way to avoid it (and simply fail in the PHP script)?

What's the question? Is it, "where's the 2nd response?" (If so, fix your title.) Is it, "how do I avoid being redirected?" (if so, fix your title.) Is it, "why does 'hello' print before 'yes'?" (If so, fix your title.) — tehhowch
– tehhowch, Commented Jul 19, 2018 at 22:57
can you provide the first URL so that we can try check the whole process? — SynapseIndia
– SynapseIndia, Commented Jul 20, 2018 at 9:44
@SynapseIndia thanks for your comment, I have deployed an equivalent service for you to try (now deployed as access only for me but for the demonstration of the behavior it should do) — David Apltauer
– David Apltauer, Commented Jul 20, 2018 at 13:21

SynapseIndia · Accepted Answer · 2018-07-21 03:48:21Z

I checked your code on command line using first URL https://script.google.com/a/druna.cz/macros/s/AKfycbzo6Y_XiXHLZFuNEb2rB7GLVXbXhtBpCGo9AlL8ul-gITmvv6k/exec

It printed lot of HTML and also printed "hello" In the HTML there is important statement "window.location.replace(redirectUrl);" Due to this HTML when accessed via browser the page redirects. When you are logged in browser window it might be redirecting to some other url due to similar JS code.

<!DOCTYPE html>
<html lang="en">
  <head>
  <meta charset="utf-8">
  <meta name="robots" content="noindex">
  <title>Sign in - Google Accounts</title>
  <meta http-equiv="refresh" content="1; url=https://www.google.com/accounts/AccountChooser?hd=druna.cz&amp;continue=https%3A%2F%2Fscript.google.com%2Fa%2Fdruna.cz%2Fmacros%2Fs%2FAKfycbzo6Y_XiXHLZFuNEb2rB7GLVXbXhtBpCGo9AlL8ul-gITmvv6k%2Fexec&amp;followup=https%3A%2F%2Fscript.google.com%2Fa%2Fdruna.cz%2Fmacros%2Fs%2FAKfycbzo6Y_XiXHLZFuNEb2rB7GLVXbXhtBpCGo9AlL8ul-gITmvv6k%2Fexec&amp;service=wise"></meta>
  </head>
  <body >
  <form id="hiddenget" action="https://www.google.com/accounts/AccountChooser?hd=druna.cz&amp;continue=https%3A%2F%2Fscript.google.com%2Fa%2Fdruna.cz%2Fmacros%2Fs%2FAKfycbzo6Y_XiXHLZFuNEb2rB7GLVXbXhtBpCGo9AlL8ul-gITmvv6k%2Fexec&amp;followup=https%3A%2F%2Fscript.google.com%2Fa%2Fdruna.cz%2Fmacros%2Fs%2FAKfycbzo6Y_XiXHLZFuNEb2rB7GLVXbXhtBpCGo9AlL8ul-gITmvv6k%2Fexec&amp;service=wise" method="get">
  <noscript>
  You should turn on Javascript support.
  <input type="submit" id="nojssubmit" value="Continue">
  </noscript>
</form>
  <script nonce="HobMA9FpZpT2/2X7NDFffIUgemA">
window.onload = function() {
  var redirectUrl = 'https:\x2F\x2Fwww.google.com\x2Faccounts\x2FAccountChooser?hd=druna.cz\x26continue=https%3A%2F%2Fscript.google.com%2Fa%2Fdruna.cz%2Fmacros%2Fs%2FAKfycbzo6Y_XiXHLZFuNEb2rB7GLVXbXhtBpCGo9AlL8ul-gITmvv6k%2Fexec\x26followup=https%3A%2F%2Fscript.google.com%2Fa%2Fdruna.cz%2Fmacros%2Fs%2FAKfycbzo6Y_XiXHLZFuNEb2rB7GLVXbXhtBpCGo9AlL8ul-gITmvv6k%2Fexec\x26service=wise';
  var domain = 'druna.cz';
  var hash = window.location.hash;
  if (hash) {
  var match = hash.match(/[#&]Email=([^&]+)/);
  if (match) {
  redirectUrl += "&Email=" + match[1] + "@" + domain;
  }
  }
  window.location.replace(redirectUrl);
};
</script>
  </body>
</html>hello

hanshenrik · Accepted Answer · 2018-07-21 13:30:06Z

the browser redirects after a second because of javascript or html tags printed in

echo $response["content"];

you have a couple of options here, you can HTML encode the response, so the html is rendered as text by the browser,

echo htmlentities ( $response["content"], ENT_QUOTES | ENT_HTML401 | ENT_SUBSTITUTE | ENT_DISALLOWED, 'UTF-8', true );

or, if you don't want to display the HTML directly, but just the text content of the html, you can extract the text:

$domd=@DOMDocument::loadHTML($response["content"]);
echo htmlentities ( $domd->textContent, ENT_QUOTES | ENT_HTML401 | ENT_SUBSTITUTE | ENT_DISALLOWED, 'UTF-8', true );

or ofc, if you just want to display a certain part of the response, you can parse out and only print that part.

Collectives™ on Stack Overflow

How to avoid getting redirected when using cURL in PHP

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related