Calling async code from syncronous function in Python

Question

This problem is probably not complicated, but I'm new to async library and I just can't figure this out.

Let's say I have a synchronous function from which I want to call an async function without transforming the synchronous function into an asynchronous one. The synchronous function needs to wait for the async function to complete and return the value it provides. It's fine if it blocks execution in the meantime. All of this is necessary because the async function comes from a library.

How can I retrieve the result computed by the async function?

There is already a running event loop, and I have a reference to it.
They are running on the same thread, so I don't need another thread.
I can't use run_until_complete() because it somehow conflicts with the already running event loop.

import asyncio

class Sample:

    def __init__(self):
        self.state = self.compute_sync()  # I want to call async from this constructor

    async def io_function(self):
        print("started")
        x = await asyncio.sleep(1)  # using an async library here
        print("ended")
        return x.result()

    def compute_sync(self):
        io_result = self.io_function()  # I want the result of this without using await
        return 'local_data' + io_result

async def main():
    await asyncio.sleep(1)    # other IO stuff
    s = Sample()
    print(s)

asyncio.run(main())

(The code above is extremely simplified, maybe it does not reflect the real use case, but the comments will maybe help what I need to do here)

I'm looking for answers with the newest version of Python (which currently is 3.13).
I don't want to use an external async library if possible.

Generally speaking you can't. That's the gist of the coloured functions problem. — Aron
– Aron, Commented Oct 26, 2024 at 8:11
This question is similar to: How to call a async function from a synchronized code Python. If you believe it’s different, please edit the question, make it clear how it’s different and/or how the answers on that question are not helpful for your problem. — mkrieger1
– mkrieger1, Commented Oct 26, 2024 at 8:14
@Aron why is this the case? isn't that a real scenario where someone would like to use an async library but running it asynchronously would break some logic in the code? So synchronous execution would make much more sense (especially in a constructor)? — elaspog
– elaspog, Commented Oct 26, 2024 at 8:18
It boils down to how sync/async methods are scheduled. It would involve a deep dive into pretty complex C code. — Aron
– Aron, Commented Oct 26, 2024 at 8:20

VPfB · Accepted Answer · 2024-10-26 12:06:54Z

Your comment says "I want the result of this without using await". The easy part is to create a task that will come with the desired result (task's result is the coroutine's return value)

task = asyncio.create_task(coro())` # create_task is a regular function.

The real problem is how and when you get the result. It takes time till the result (of an async function) will be available. You cannot wait for it without await. Attempting to wait in a sync function would bring the program to a halt.

Option 1: polling in some intervals. Check with task.done(), retrieve the result with task.result().

Option 2: act immediately upon finished computation with a callback: Use task.add_done_callback().

However, I would first consider converting the regular/sync function in question to an async version. If such change is possible for the particular use-case and if it is also easy to make:

change the function definition from def to async def
call it always with await

Then there is no limitation which function types (sync vs async) can be invoked.

Booboo · Accepted Answer · 2024-10-26 16:01:22Z

This is your pattern:

You have a thread running an event loop and currently executing an asyncio function (let's call it async_function_1). You need that async function to somehow invoke a synchronous function (let's call it sync_function) and for that synchronous function to then somehow invoke an asynchronous function (let's call it async_function_2):

async_funtion_1 -> sync_function -> async_function_2

Once sync_function is invoked (1) the event loop running in the current thread is blocked and (2) there is no way for sync_function to execute an await expression since it is a synchronous function.

The only way for async_function_2 to be invoked at all from sync_function and get a returned result is to run it in a new event loop running in another thread. Probably the most efficient way to accomplish this is as follows:

import asyncio

_loop = None

def run_async(coro):
    global _loop

    if _loop is None:
        from threading import Thread
        
        _loop = asyncio.new_event_loop()
        Thread(target=_loop.run_forever, name="Async Runner", daemon=True).start()

    return asyncio.run_coroutine_threadsafe(coro, _loop).result()


class Sample:

    def __init__(self):
        self.state = self.compute_sync()  # I want to call async from this constructor

    async def io_function(self):
        print("started")
        x = await asyncio.sleep(1)  # using an async library here
        print("ended")
        # return x.result()  # x is None
        return "Some value"

    def compute_sync(self):
        # Note that this will block the current event loop
        io_result = run_async(self.io_function())
        return 'local_data' + io_result

async def main():
    await asyncio.sleep(1)    # other IO stuff
    s = Sample()
    print(s.state)

asyncio.run(main())

Prints:

started
ended
local_dataSome value

See: asyncio.run_coroutine_threadsafe

Collectives™ on Stack Overflow

Calling async code from syncronous function in Python

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related