Async/Await not allowed in loops, TypeScript using Got package

Question

I am using the got package and trying to send a GET request to check items in an array. This request is looped for each item in the array and then these items are further sorted into more arrays, the rest of the code depends on this list. Afterwards one of these arrays will be sent to another server for DELETE requests.

The structure is as follows:

for (let i in arrTags){
    const {body} = await got(`https://example.com/path/to/${i}`)
   
      if (body.length > 0 &&
          // find current revision (Git SHA) to compare with i
          body.current_revision.substring(0, tag.length) !== i) {
        arrTagsToDelete.push(tag)
      } else {
        arrTagsIgnored.push(tag)
      }
}

later on in the code we delete items in arrTagsToDelete:

for (let i in arrTagsToDelete){
    await got.delete(`https://anotherwebsite.com/path/to/${i}`, {username: args.username, password: args.password}) //using oclif we get auth
}

I am getting an issue with ESLint telling me I cannot use Async-await inside a for-loop, I know I cannot use a forEach since async-await does not play nicely according to other answers on StackOverflow,it will run the rest of the code before the GET request and DELETE requests are done. I have seen as well that disabling ESLint warnings for this scenario isn't uncommon.

I would prefer to maybe have a cleaner solution to this problem than disabling the ESLint, unless that is the only way.

Thank you in advance.

This is an eslint suggestion that urges you to use Promise.all to do all requests concurrently instead of sequentially e.g. await Promise.all(arrTagsToDelete.map((value, i) => got.delete(...)) might work — apokryfos
– apokryfos, Commented Feb 3, 2021 at 21:25
Have you tried changing the for loop from a for in to a for of? — shn
– shn, Commented Feb 3, 2021 at 21:26
I have tried a for-in that is still against eslints rules, will attempt the Promise.all method. — Basil Zuberi
– Basil Zuberi, Commented Feb 3, 2021 at 23:14
Don't follow lint rules dogmatically without understanding them — Aluan Haddad
– Aluan Haddad, Commented Feb 4, 2021 at 1:42

Brad Zacher · Accepted Answer · 2021-02-08 01:20:54Z

1

Using await inside a loop is generally considered bad-practice.

This is because asynchronous functions are designed to run, well asynchronously. However, loops work serially - they process one item at a time.

If you use await inside the loop - you can only make one async call per iteration - this means that you have to wait for one to finish before you send off the next one.

This leads to a "waterfall", which will slow your code down.

The fix is to parallelise your work (as @apokryfos mentioned):

await Promise.all(arrTagsToDelete.map(
  (_, i) => got.delete(
    `https://anotherwebsite.com/path/to/${i}`,
    {username: args.username, password: args.password},
  ),
)

Or more verbosely:

const promises = [];
for (const i = 0; i < arrTagsToDelete.length; i += 1) {
  promises.push(
    got.delete(
      `https://anotherwebsite.com/path/to/${i}`,
      {username: args.username, password: args.password},
    )
  );
}
await Promise.all(promises);

By not awaiting inside the loop, we are firing off all of our calls at once and allowing them to run asynchronously in parallel. After they're all in flight, we use Promise.all to wait until they are all finished.

It's worth noting as an aside that there are some cases where you might want to process some asynchronous calls synchronously and serially.

These cases are rare, and you should be doing this consciously and cautiously. If you have one of these cases, then use an eslint-disable comment.

edited Feb 8, 2021 at 1:20

answered Feb 5, 2021 at 22:25

Brad Zacher

3,27520 silver badges37 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Aluan Haddad Over a year ago

There are rare cases where you would want to sequentially execute series of asynchronous operations. It's also not correct that you lose all the benefits of asynchrony. You're correct that it is generally a bad practice because most often such operations do not need to be executed in a specific order.

Brad Zacher Over a year ago

@AluanHaddad true that there are some rare cases that you want process things synchronously. "all benefits" was probably a bit broad when taken literally, but the sentiment is behind it was correct I'll update.

Bergi Over a year ago

No, it's not "generally a bad practice". These cases are less rare than you might think. I'd even argue that sequential execution is the better default, and you should parallelise conciously. Especially in the case of loops, it's often a bad idea to do an unlimited number of operations at the same time, firing off http requests without any rate limiting in place. "When you await them, you make them synchronous." is patently false - it's still asynchronous, meaning the server can process other tasks while the asynchronous loop is running; e.g. calling the asynchronous function multiple times.

Brad Zacher Over a year ago

@Bergi in my time using async await across a few languages, there have only been a small handful of times I've done an await in a loop. Across the codebases I've seen its the edge case. Which is why the lint rule exists and is widely used (in many languages). The browser limits on the number of concurrent requests so no need to limit it yourself unless you specifically care about resource contention.

Brad Zacher Over a year ago

If you disagree that my answer does not answer OP's question, then down vote. If it answers it but the wording isn't right, then don't down vote, comment or suggest an edit.

Collectives™ on Stack Overflow

Async/Await not allowed in loops, TypeScript using Got package

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related