Concurrent Async Function call in Python 3.6

Question

I have a script in which a Slow and a Fast function processes the same global object array. The Slow function is for filling up the array with new objects based on resource intensive calculations, the Fast is only for iterating the existing objects in the array and maintaining/displaying them. The Slow function only needs to be run only in every few seconds, but the Fast function is imperative to run as frequently as possible. I tried using asyncio and ensure_future calling the Slow process, but the result was that the Fast(main) function ran until I stopped it, and only at the end was the Slow function called. I need the Slow function to start running in the instance it is called in the background and complete whenever it can, but without blocking the call of the Fast function. Can you help me please? Thank you!

An example of what I tried:

import asyncio
variable = []

async def slow():
    temp = get_new_objects() #resource intensive
    global variable
    variable = temp

async def main():
    while True: #Looping
        if need_to_run_slow: #Only run sometimes
            asyncio.ensure_future(slow())

        do_fast_stuff_with(variable) #fast part

if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())
    loop.close()

Hi, I'm curious if my answer helped resolve your issue?

user4815162342
– user4815162342

2018-09-04 13:31:23 +00:00
Commented Sep 4, 2018 at 13:31 — user4815162342
– user4815162342, Commented Sep 4, 2018 at 13:31

user4815162342 · Accepted Answer · 2018-08-28 08:44:11Z

2

asyncio.ensure_future(slow()) only schedules slow() to run at the next pass of the event loop. Since your while loop doesn't await anything that can actually block, you are not giving the event loop a chance to run.

You can work around the issue by adding an await asyncio.sleep(0) after the call to the fast function:

async def main():
    while True:
        if need_to_run_slow:
            asyncio.ensure_future(slow())
        await asyncio.sleep(0)
        do_fast_stuff_with(variable)

The no-op sleep will ensure that at every iteration of the while loop (and between runs of the "fast" function") gives a chance for a previously scheduled slow() to make progress.

However, your slow() doesn't await either, so all of its code will run in a single iteration, which makes the above equivalent to the much simpler:

def main():
    while True:
        slow()  # slow() is an ordinary function
        do_fast_stuff_with(variable)

A code example closer to your actual use case would probably result in a more directly usable answer.

edited Aug 28, 2018 at 8:44

answered Aug 27, 2018 at 14:43

user4815162342

159k22 gold badges349 silver badges418 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Angezerus Over a year ago

I am a rookie in this area and I'm just trying to rewrite and combine existing codes, so probably I'm making no sense in that example code, but I want to accomplish what I explained, so I am open to any solution :)

user4815162342 Over a year ago

@Angezerus I've now amended the answer with a longer explanation of how to fix the issue, but also why the fix might not be a long-term solution, at least not for the code you've shown.

Collectives™ on Stack Overflow

Concurrent Async Function call in Python 3.6

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related