asyncio create_subprocess_shell python 3.7 on windows

Question

I'm trying to test some asyncio functionality on Windows. I'm using Python 3.7.

This complains bitterly about some NotImplementedError

import asyncio
import os
import time
import sys

#this works
async def say_after(delay, what):
    await asyncio.sleep(delay)
    print(what)

#this doesn't
async def test_async(num):
    print('Task #{0} start'.format(num))

    proc = await asyncio.create_subprocess_shell(
        'C:/Python37/python test_repl.py',
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE,
        stdin=asyncio.subprocess.PIPE )

    stdout, stderr = await proc.communicate("sleep 10")
    cmd = 'python'
    print(f'[{cmd!r} exited with {proc.returncode}]')
    if stdout:
        print(f'[stdout]\n{stdout.decode()}')
    if stderr:
        print(f'[stderr]\n{stderr.decode()}')

async def test_loop():
    task1 = asyncio.create_task(
        test_async(1))

    task2 = asyncio.create_task(
        test_async(2))

    print(f"started at {time.strftime('%X')}")

    await task1
    await task2

    print(f"finished at {time.strftime('%X')}")

loop = asyncio.ProactorEventLoop()
asyncio.set_event_loop(loop)
asyncio.run(test_loop())

I've tried several loop types (Protractor and Default). I've tried different functions through the shell and directly invoking the program (_shell and _exec in the subprocess lingo). Nothing seems to work. For a much hype I've read about asyncio, it can't possibly be a Linux only thing. I must be doing something wrong. Can you please point me in the right direction.

The specific error I get is the following:

--------------Async--------------------
started at 22:39:55
Task #1 start
Task #2 start
Traceback (most recent call last):
  File "multirun.py", line 45, in <module>
    asyncio.run(test_loop())
  File "C:\Python37\lib\asyncio\runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "C:\Python37\lib\asyncio\base_events.py", line 568, in run_until_complete
    return future.result()
  File "multirun.py", line 38, in test_loop
    await task1
  File "multirun.py", line 20, in test_async
    stdin=asyncio.subprocess.PIPE)
  File "C:\Python37\lib\asyncio\subprocess.py", line 202, in create_subprocess_shell
    stderr=stderr, **kwds)
  File "C:\Python37\lib\asyncio\base_events.py", line 1486, in subprocess_shell
    protocol, cmd, True, stdin, stdout, stderr, bufsize, **kwargs)
  File "C:\Python37\lib\asyncio\base_events.py", line 444, in _make_subprocess_transport
    raise NotImplementedError
NotImplementedError
Task exception was never retrieved
future: <Task finished coro=<test_async() done, defined at multirun.py:12> exception=NotImplementedError()>
Traceback (most recent call last):
  File "multirun.py", line 20, in test_async
    stdin=asyncio.subprocess.PIPE)
  File "C:\Python37\lib\asyncio\subprocess.py", line 202, in create_subprocess_shell
    stderr=stderr, **kwds)
  File "C:\Python37\lib\asyncio\base_events.py", line 1486, in subprocess_shell
    protocol, cmd, True, stdin, stdout, stderr, bufsize, **kwargs)
  File "C:\Python37\lib\asyncio\base_events.py", line 444, in _make_subprocess_transport
    raise NotImplementedError
NotImplementedError

Also, I got the same error when trying to run the notepad.

Can you show the resulting exception and the traceback? Does asyncio.create_subprocess_shell work when you execute some other command? — user4815162342
– user4815162342, Commented Nov 3, 2018 at 8:47
@user4815162342 I've edited the question a little bit with the information about the specific error I'm getting. — carlosdc
– carlosdc, Commented Nov 4, 2018 at 2:42
I think the problem is that you're not actually using the ProactorEventLoop, despite appearances. asyncio.run() creates a new event loop based on the current loop creation policy, which you've never changed. (The [documentation[(docs.python.org/3/library/…) example frankly looks incompatible with asyncio.run.) Try replacing asyncio.run(test_loop()) with loop.run_until_complete(test_loop()) and see if that helps. — user4815162342
– user4815162342, Commented Nov 4, 2018 at 4:18
@user4815162342: that worked! Do you mind putting that as an answer so I can mark it? — carlosdc
– carlosdc, Commented Nov 4, 2018 at 18:18
Note that asyncio.create_subprocess_shell() is not implemented for Windows cmd shell as of python version 3.7.2 — John Mark
– John Mark, Commented Feb 15, 2019 at 19:52

user4815162342 · Accepted Answer · 2021-03-04 12:16:06Z

4

As of Python 3.8, this issue should no longer exist, as the proactor event loop is now the default event loop on Windows. Original answer follows below.

The problem is that, despite appearances, you're not actually using the ProactorEventLoop. asyncio.run() creates a new event loop based on the current loop creation policy, which you've never changed. Creating a new loop for each run is normally a feature because it guarantees cleanup of the resources associated with the loop - but in this case it's incompatible with the example from the documentation. (Edit: the example has since been removed because ProactorEventLoop was made the default.)

A quick fix is to change asyncio.run(test_loop()) to loop.run_until_complete(test_loop()). A better fix is to set the event loop policy to one that creates the proactor loop.

edited Mar 4, 2021 at 12:16

answered Nov 4, 2018 at 23:24

user4815162342

159k22 gold badges350 silver badges418 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

bicarlsen Over a year ago

The link provided for set the event loop policy is broken, however I believe it was linking to this content: docs.python.org/3.7/library/asyncio-policy.html

user4815162342 Over a year ago

@bicarlsen Fixed the link, thanks. The idea was to directly link to the docs of the set_event_loop_policy method.

Collectives™ on Stack Overflow

asyncio create_subprocess_shell python 3.7 on windows

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related