Getting data from another url in Appscript

Question

I'm trying to scrape data from the site. The request url that get the product info shows in browser properly. But not getting the UrlFetchApp. My code is as following

function myFunction() {  
  var coles_url = "https://shop.coles.com.au/search/resources/store/20520/productview/bySeoUrlKeyword/mutti-tomato-passata-2349503p?catalogId=12064";
  Logger.log(jsonInitColes(coles_url));
  
}

function jsonInitColes(url){
   var options =
  {
   "method"  : "GET",   
   "followRedirects" : true,
   "muteHttpExceptions": true
    };
   var response = UrlFetchApp.fetch(url,options);
   var content = response.getContentText();
   return content; 
}

But I was getting the following HTML instead of data status_code is 429(Too many requests)

 <!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <link rel="shortcut icon" href="about:blank">
</head>
<body>
<script src="/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/j.js"></script>
<script src="/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/f.js"></script>
<script src="/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/fingerprint/script/kpf.js?url=/149e9513-01fa-4fb0-aad4-566afd725d1b/2d206a39-8ed7-437e-a3be-862e0f06eea3/fingerprint&token=c3436f36-e3c3-a537-25f3-bedc8592f189"></script>
</body>
</html>

Browser inspection of the request

Nimantha · Accepted Answer · 2021-11-05 02:56:44Z

1

Answer:

There's nothing you can do about this.

More Information:

Google Apps Script runs in the cloud, so your fetch will be coming from some unspecified Google IP.

Imagine how many users are using Google Apps Script. Of those users, how many are trying to fetch information from the same sites and servers. The Too many Requests is an unfortunately common error when using UrlFetchApp to external endpoints that don't require authentication

If you're getting a Too many requests error and you're not doing quick retries, then it's likely that there are just too many requests coming from all those other Apps Script scripts in the cloud making requests to the same server. There's nothing you can do about that.

edited Nov 5, 2021 at 2:56

Nimantha

6,5276 gold badges32 silver badges78 bronze badges

answered Jul 8, 2020 at 10:37

Rafa Guillermo

15.5k3 gold badges23 silver badges62 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Getting data from another url in Appscript

1 Answer 1

Answer:

More Information:

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Answer:

More Information:

Comments

Your Answer

Sign up or log in

Post as a guest

Related