Asyncio event loop per python process (aioprocessing, multiple event loops)

Question

I have two processes; a main process and a subprocess. The main process is running an asyncio event loop, and starts the subprocess. I want to start another asyncio event loop in the subprocess. I'm using the aioprocessing module to launch the subprocess.

The subprocess function is:

def subprocess_code():
     loop = asyncio.get_event_loop()
     @asyncio.coroutine
     def f():
        for i in range(10):
            print(i)
            yield from asyncio.sleep(1)
     loop.run_until_complete(f())

But I get an error:

    loop.run_until_complete(f())
  File "/usr/lib/python3.4/asyncio/base_events.py", line 271, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.4/asyncio/base_events.py", line 239, in run_forever
    raise RuntimeError('Event loop is running.')
RuntimeError: Event loop is running.

Is it possible to start a new, or restart the existing, asyncio event loop in the subprocess? If so, how?

process = aioprocessing.AioProcess(target=target) process.start() — solarw
– solarw, Commented Apr 17, 2015 at 17:08
I think aioprocessing was written by our very own @dano. You could add it to the title to get his attention. — tdelaney
– tdelaney, Commented Apr 17, 2015 at 17:20
@tdelany I found the question anyway :). I filed a bug about this back when I was working on aioprocessing initially: bugs.python.org/issue22087 — dano
– dano, Commented Apr 17, 2015 at 19:45

solarw · Accepted Answer · 2015-04-17 17:19:31Z

23

Sorry for disturb! I found a solution!

policy = asyncio.get_event_loop_policy()
policy.set_event_loop(policy.new_event_loop())
loop = asyncio.get_event_loop()

put this code to start new asycnio event loop inside of subprocess started from process with asyncio event loop

answered Apr 17, 2015 at 17:19

solarw

4613 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Andrea Corbellini Over a year ago

No need to be sorry! Answers to your own questions are more than welcome, as they may help future visitors!

tdelaney Over a year ago

That's interesting. aioprocessing goes through multiprocessing which means that the child has the parent memory space (at least in linux). So, there is a non-active event loop that needs restarting.

solarw Over a year ago

I found that asyncio checks loops at threads, so new loop in new thread should be started without questions. Suppose some kind of atfork handler should remove inactive event loop automatically, to help starting new loop without tricks with policy

dano Over a year ago

@solarw There's a patch in the bug I linked to above that works around the issue, but it looks like there's another patch in yet another similar bug that implements atfork type handlers to address it. See here: bugs.python.org/issue21998. There hasn't been activity on it in a few months, though. For now, what you're doing is the right way to work around the issue.

amohr Over a year ago

note this broke in 3.5.3, patch is in the works:github.com/python/asyncio/pull/452

Collectives™ on Stack Overflow

Asyncio event loop per python process (aioprocessing, multiple event loops)

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related