Python asyncio non blocking for loop

Question

(I used Python 3.7)

import asyncio


result = 0


async def sub1():
    global result
    print('[*] sub1() start')
    for i in range(1, 10000000):
        result += i
    print('[*] sub1() end')

async def sub2():
    global result
    print('[*] sub2() start')
    for i in range(10000000, 20000000):
        result += i
    print('[*] sub2() end')

async def main():
    await asyncio.gather(
        asyncio.create_task(sub1()),
        asyncio.create_task(sub2()),
    )


asyncio.run(main())

When I execute above code, it's result is

[*] sub1() start
[*] sub1() end
[*] sub2() start
[*] sub2() end
[Finished in 1.7s]

You know that, sub2() is started after sub1() finished.

It seems doesn't work in parallel.

I want to give control over to sub2() until sub1() finished.

Maybe I have to insert await to somewhere, but I don't know where is the point.

Question

How can I execute both functions like parallel using for loop?

Thanks.

Asyncio (without using threads or additional processes) will not run code in parallel. It runs one task after the other. If you have blocking code like yours that does not release control it will run like being used synchronously. — Klaus D.
– Klaus D., Commented Jul 24, 2019 at 3:55
@KlausD. I know asyncio doesn't works in parallel. But coroutine give a effect that looks like asynchronous. — Hide
– Hide, Commented Jul 24, 2019 at 3:57
The first sentence does not match up with your question and the second is misleading, they are asynchronous if they are programmed like that. Just adding async is not enough. — Klaus D.
– Klaus D., Commented Jul 24, 2019 at 4:01
@KlausD. What is your point of comment? Modify subject of my post? You know that I summarize my question at the end of post. So I think maybe there is no confuing. — Hide
– Hide, Commented Jul 24, 2019 at 4:02
You have misunderstood the main concept of asyncio: write asynchronous code. Your code is synchronous and will not run asynchronously by just making it a coroutine. — Klaus D.
– Klaus D., Commented Jul 24, 2019 at 4:11

felipe · Accepted Answer · 2019-07-24 04:32:12Z

As @KlausD pointed out in his comment simply adding the async tag does not effectively make your code truly asynchronous. asyncio requires some sort of break-point within the for-loops to effectively know when to switch between functions. Here is a possible solution utilizing synchronous functions and running them inside the loop's executor (documentation here):

import asyncio

result = 0

def sub1():
    global result
    print('[*] sub1() start')
    for i in range(1, 10000000):
        result += i
    print('[*] sub1() end')

def sub2():
    global result
    print('[*] sub2() start')
    for i in range(10000000, 20000000):
        result += i
    print('[*] sub2() end')

async def main(loop):
    res = await asyncio.gather(
        loop.run_in_executor(None, sub1),
        loop.run_in_executor(None, sub2)
    )


loop = asyncio.get_event_loop()
loop.run_until_complete(main(loop))

It is worth noting that to write asynchronous code one must remember to engage in practices that ensure the code actually runs asynchronously. A native for-loop in Python is in fact a synchronous piece of code, and therefore when your functions are called, the for-loop is run until completion before making space in the thread for the next operation.

Another thing is the fact that since your code is simple additions, you aren't actually saving any time by running it asynchronously. Hell, you might be losing time due to asyncio operations having to switch between functions. Async functionalities are good for large processing that requires some sort of waiting between operations (i.e. doing various server calls, where one has to wait for servers to respond; reading in multiple files, where one has to be dependent on the hard drive reading speed, etc.) In these cases, async becomes useful since one can switch operations within the thread and ensure that all the calls (pull from server; get file from harddrive) are run before bothering moving on to processing the requests or the files.

Collectives™ on Stack Overflow

Python asyncio non blocking for loop

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related