Running an async coroutine in the background from a sync function using create_task and asyncio.run does not finish

Asyncio runs asynchronously which is not the same as in parallel, its not to be confused with threads which can be executed in parallel (on a high-level; take this with a grain of salt, see the comments). Asyncio is jumping from one async/await statement to the next executing code between two awaits in a sequential way.

In your code the task function is only scheduled via create_task. However the created task is not awaited it only executes until the first await through asyncio.create_task but then not further as there is no further await statement. For some hints see https://docs.python.org/3/library/asyncio-task.html#awaitables

starting the loop it reaches the first await.sleep and the sequence continues in your normal main loop. There the normal sleep does not communicate with asyncio and asyncio alone cannot execute further statements until the sequence reaches a new await, i.e. your background task is blocked.

Solution in Sequence

async def task():
    for i in range(5):
        print(f"Background task iteration {i}")
        await asyncio.sleep(0.1)
    print('finished')

async def background_task():
    print("a")
    scheduled_task = asyncio.create_task(task()) # here await was missing, 
    print("task scheduled")
    # do some stuff in between
    await scheduled_task # run your task function
    print("b")

def main():
    print("Main program started python", sys.version)

    asyncio.run(background_task())
    for i in range(3):
        sleep(0.5)
        print(f"Main program iteration {i}")

Output:

a
task sheduled
Background task iteration 0
Background task iteration 1
Background task iteration 2
Background task iteration 3
Background task iteration 4
finished
b
Main program iteration 0
Main program iteration 1
Main program iteration 2

Solution in Parallel: Combine with threading: to allow parallel execution of async code to your normal code.

import asyncio
from time import sleep
import sys
import threading

async def task():
    for i in range(5):
        print(f"Background task iteration {i}")
        await asyncio.sleep(1)
    print('finished')


async def background_task():
    print("a")
    await task() # Note if you do not need a task object this is sufficient.
    print("b")

def main():
    print("Main program started python", sys.version)

    t = threading.Thread(target=lambda: asyncio.run(background_task()))
    t.start()
    for i in range(3):
        sleep(3)
        print(f"Main program iteration {i}")

Output

a
Background task iteration 0
Background task iteration 1
Background task iteration 2
Main program iteration 0
Background task iteration 3
Background task iteration 4
finished
b
Main program iteration 1
Main program iteration 2

The biggest problem, along side with the one already explained by some comments that you should use await asyncio.gather() instead of asyncio.create_task(), is that asyncio.run() runs in the main thread and thus pauses the execution. You can't run synchronous code at the same time as asynchronous code, unless you use a thread. (Duplicate: Python run non-blocking async function from sync function).

Another workaround would be to make main also asynchronous, as so:

import asyncio
import sys


async def task():
    for i in range(5):
        print(f"Background task iteration {i}")
        await asyncio.sleep(1)
    print('finished')


async def background_task():
    print("a")
    await asyncio.gather(task())
    print("b")


async def main():
    print("Main program started python", sys.version)

    for i in range(3):
        await asyncio.sleep(3)
        print(f"Main program iteration {i}")


async def runner():
    await asyncio.gather(main(), background_task())


if __name__ == "__main__":
    asyncio.run(runner())

This produces the result:

Main program started python 3.11.4 (tags/v3.11.4:d2340ef, Jun  7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)]
a
Background task iteration 0
Background task iteration 1
Background task iteration 2
Main program iteration 0
Background task iteration 3
Background task iteration 4
finished
b
Main program iteration 1
Main program iteration 2

Process finished with exit code 0