Python asyncio, possible to run a secondary event loop?

Question

Is there a way to create a secondary asyncio loop(or prioritize an await) that when awaited does not pass control back to the main event loop buts awaits for those 'sub' functions? IE

import asyncio
async def priority1():
    print("p1 before sleep")
    await asyncio.sleep(11)
    print("p1 after sleep")

async def priority2():
    print("p2 before sleep")
    await asyncio.sleep(11)
    print("p2 after sleep")

async def foo():
    while True:
        print("foo before sleep")
        #do not pass control back to main event loop here but run these 2 async 
        await asyncio.gather(priority1(),priority2())
        print("foo after sleep")

async def bar():
    while True:
        print("bar before sleep")
        await asyncio.sleep(5)
        print("bar after sleep")

async def main():
    await asyncio.gather(foo(),bar())


asyncio.run(main())

I would like foo to wait for priority1/2 to finish before passing control back to the main event loop.

Right now it would go:

foo before sleep
bar before sleep
p1 before sleep
p2 before sleep
bar after sleep

I would like to see:

foo before sleep
bar before sleep
p1 before sleep
p2 before sleep
p1 after sleep
p2 after sleep
bar after sleep

Is this possible? thanks

Not sure, if it is sensible to run two event loops in the same thread. It seems to me as though you might want to just run priority1 and priority2 in a separate thread. Or do you actually want a means of synchronization, so as to actually wait for them to finish before some other coroutine does its thing? Because I see no real point in blocking the entire event loop just because you want some functions to be executed sequentially. I think you need to elaborate a bit because this smells like an XY Problem. — Daniel Fainberg
– Daniel Fainberg, Commented Feb 5, 2023 at 9:24
You are correct, when foo and bar find the correct information, I need priority1/2 to run async with top priority IE foo and bar do not need to run and possibly use cpu(slowing the execution of p1/2) while priority1/2 are running however p1/2 need to run async. Does this clear it up? — Pearl
– Pearl, Commented Feb 5, 2023 at 19:04

jsbueno · Accepted Answer · 2023-02-06 19:33:16Z

It is not possible to run two event loops on the same thread. The code in asyncio and specs even look like they were thought in a way to permit that - but afterwards the API bent in a way it is no longer possible (for example, the explicit loop parameter for several of the relevant calls has been deprecated and removed)

In the same line, there is no way to prioritize a subset of tasks in the running loop. I answered a question on this line a couple weeks ago, and managed to get to a synchronization primitive to be used in place of asyncio.sleep which could take priority into account - but it requires all participating tasks to call it, so it would not be much different of a lock or something (I will link to it bellow - the idea is: your code call await custom.sleep() at certain points: it will only return when there are no other higher-prioritized tasks also calling that custom.sleep() ) -check it here: Execute asyncio task as soon as possible

When wrtting that code, I realized that it is possible to write an event loop which could take into account a "priority" attribute on tasks. But having a production-grade for this requires some non-hobby work: by using that loop, you could get what you are asking for, with no changes needed in the tasks code.

However, I think running a secondary loop in another thread, and then waiting synchronously on that thread to complete is a way to get your things accomplished.

import asyncio
import threading

def priority(func):
    async def wrapper(*args, **kwargs):
        result = None
        def runner(*args, **kw):
            nonlocal result
            result = asyncio.run(func(*args, **kw))
        t = threading.Thread(target=runner, args=args, kwargs=kwargs)
        await asyncio.sleep(0)
        t.start()
        # if one wants to perform limited yields to the main loop, it should be done here
        t.join()
        return result

    return wrapper


async def priority1():
    print("p1 before sleep")
    await asyncio.sleep(.11)
    print("p1 after sleep")

async def priority2():
    print("p2 before sleep")
    await asyncio.sleep(.11)
    print("p2 after sleep")

@priority
async def foo():

    print("foo before sleep")
    #do not pass control back to main event loop here but run these 2 async
    await asyncio.gather(priority1(),priority2())
    print("foo after sleep")

async def bar():
    print("bar before sleep")
    await asyncio.sleep(.05)
    print("bar after sleep")

async def main():
    await asyncio.gather(foo(),bar())


asyncio.run(main())

Collectives™ on Stack Overflow

Python asyncio, possible to run a secondary event loop?

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related