I have some nested Node.js forEach loops which are hitting API rate limiters and I would like to slow them down so that they are not hitting the rate limit. I have tried adding a setTimeout on the end of the loop but that has no impact at all on the speed of the requests. Ideally, I would be able to send a request every 2 seconds. How can I achieve this? Everything online seems to talk about rate-limiting an API but nothing on the sending side of things.
-
Please show some code of where you're calling the API wihin a loopJamiec– Jamiec2022-01-26 16:27:04 +00:00Commented Jan 26, 2022 at 16:27
-
This question should answer your question: stackoverflow.com/questions/37977602/…Sorin– Sorin2022-01-26 16:34:56 +00:00Commented Jan 26, 2022 at 16:34
-
@Jamiec I don't think that would be of help too much past linking to what a forEach loop looks like. This is a general question about rate-limiting a forEach loop, not an exact rate limit my API. I have other places in my app that I would like to limit a forEach loop too so no point in a specific solution when I need general for multiple use casesjoshk132– joshk1322022-01-26 16:35:45 +00:00Commented Jan 26, 2022 at 16:35
2 Answers
To delay in a loop, first of all, your function must be async.
You can then await a promise that resolves after 2 seconds. Such a delay can be inserted as a one-liner like this:
await new Promise(resolve => setTimeout(resolve, 2000));
...or as a handy function like:
const delay = time => new Promise(resolve => setTimeout(resolve, time));
...which can then be used in a loop like:
async function fetchEverything() {
for (let thing of things) {
await fetchThing(thing);
await delay(2000);
}
}
1 Comment
forEach specifically, please see this questionYou can make a generic extension on Array which can insert a delay in between each iteration, you will want to make the function async so you can await it (and therefore know when it's finished).
Array.prototype.forEachLimited = async function(fn, timeBetweenCallsMs){
const delay = async () => new Promise(resolve => setTimeout(resolve, timeBetweenCallsMs));
for(var i=0;i<this.length;i++){
fn(this[i]);
await delay();
}
};
(async function(){
var myArr = [1,2,3,4,5];
await myArr.forEachLimited(i => console.log(i), 2000);
console.log("Finished");
})();
Some may frown at adding this to the Array.prototype, it will work just as well as a normal function taking the array as an argument, so the same thing without adding it to Array.prototype
async function forEachLimited(arr, fn, timeBetweenCallsMs) {
const delay = async() => new Promise(resolve => setTimeout(resolve, timeBetweenCallsMs));
for (var i = 0; i < arr.length; i++) {
fn(arr[i]);
await delay();
}
};
(async function() {
var myArr = [1, 2, 3, 4, 5];
await forEachLimited(myArr, i => console.log(i), 2000);
console.log("Finished");
})();