Why use async.gather in Python?

Question

Let's say we have

await async_function_one_with_large_IO_request()
await async_function_two_with_large_IO_request()

versus

asyncio.gather(
  async_function_one_with_large_IO_request(), 
  async_function_two_with_large_IO_request())

In the first version, once we hit the 'large io request' part of function one, it's gonna move onto running function_two, that's the whole point of await, right?

Isn't that what version 2 with gather does too?

What's the performance difference between the two?

larsks · Accepted Answer · 2022-05-16 23:21:49Z

1

In the first version, once we hit the 'large io request' part of function one, it's gonna move onto running function_two, that's the whole point of await, right?

That's incorrect. In your first version, async_function_two_with_large_IO_request (which I will call function_two) won't run until async_function_one_with_large_IO_request (which I will call function_one) completes.

If function_one happens to await on another function, it will yield control to another running async task, but function_two hasn't been scheduled yet.

When you use asyncio.gather, the tasks are scheduled concurrently, so if function_one awaits on something, function_two has a chance to run (along with other async tasks).

Note that asyncio.gather creates an async task, which generally implies you have to await on it:

await asyncio.gather(...)

The Python documentation covers this in detail in Coroutines and Tasks.

edited May 16, 2022 at 23:21

answered May 16, 2022 at 21:09

larsks

318k49 gold badges474 silver badges482 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

user1008636 Over a year ago

so what does 'await' really do under the hood?

larsks Over a year ago

It causes the calling async task to yield control to the async scheduler, which may schedule another async task. When the await-ed function completes, the calling tasks is eligible to be scheduled again the next time another tasks calls await.

larsks Over a year ago

realpython.com/async-io-python is a good overview of asyncio programming in Python.

user1008636 Over a year ago

one more question. in that example, does asyncio.gather create its own mini-event loop that only includes the two functions? ie, async.gather will block until both functions passed in are finished?

larsks Over a year ago

asyncio.gather creates an async task. You need to await on the result. It is scheduled like any other async task.

Collectives™ on Stack Overflow

Why use async.gather in Python?

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related