How to make an asynchronous principle of task execution from a task generator?

Question

I have a problem, do you have any ideas for a solution? You need to add asynchronous tasks to the event loop without waiting for them to be completed (using await is not an option, otherwise there is no point in such a program). Just expect new tasks and launch new ones. For example, the code would be clearer. He's not a worker.

import asyncio, random 

async def wrapper(word: str):
    print(word)
    await asyncio.sleep(1) # task simulation
 
def generator():
    abc = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
    while True:
        yield random.choice(abc)
        
async def manager():
    loop = asyncio.get_event_loop()
    for letter in generator():
        loop.create_task(wrapper(letter)) # here is something like performing tasks asynchronously
        
    
asyncio.run(manager())

I tried to create a longpull server from which I receive events from the user. I wanted to process certain events asynchronously. But so far it works only synchronously.

Put an await asyncio.sleep(0) in the for-loop of manager. This allows to suspend manager and to execute tasks. — Michael Butscher
– Michael Butscher, Commented Nov 30, 2023 at 16:13
but then the program will be suspended. I need task 1 to work while task 2 works in parallel. — aero.exe
– aero.exe, Commented Nov 30, 2023 at 16:25
I meant that manager is only halted temporarily to execute tasks and then the execution is resumed. If manager never halts=awaits something (as it is currently), no tasks can be executed in the same event loop. — Michael Butscher
– Michael Butscher, Commented Nov 30, 2023 at 16:30

jsbueno · Accepted Answer · 2023-11-30 18:30:28Z

2


async def manager():
    loop = asyncio.get_event_loop()
    for letter in generator():
        loop.create_task(wrapper(letter)) # here is something like performing tasks asynchronously

After you call create_task, just keep a reference around to the task created, and perform some awaits inside that "for" in your manager, so that the other tasks have an opportunity to run: the main difference between coding for async and coding for multi-threaded code, is that concurrent code is never exectuted if you don't yield to the main loop.

These changes in your manager are enough:

async def manager():
    with asyncio.TaskGroup() as tg():
        for letter in generator():
            tg.create_task(wrapper(letter)) # here is something like performing tasks asynchronously
            await asyncio.sleep(0)

(TaskGroup requires Python 3.11 >)

answered Nov 30, 2023 at 18:30

jsbueno

113k11 gold badges159 silver badges239 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to make an asynchronous principle of task execution from a task generator?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related