How to launch a list of python scripts asynchronously?

Question

I tried making a script that reads an array of scripts directories in it, and launches them asynchronously. The problem in my script is that it launches only the last script in the array.

import json 
import asyncio
import firstscript
import secondscript

def main():
    list = json.load(open('list.json'))
    launch_list = []
    for i in range(len(exchanges_list['e'])):
        get_exchange = exchanges_list['e'][i][0]
        launch_exchange = str(get_exchange.lower() + '.' + get_exchange.lower() + '()')
        launch_list.append(launch_exchange)

    async def handle_script(uri):
        exec(uri)
    async def handler():
        await asyncio.wait([handle_socket(uri) for uri in launch_list])
    
    asyncio.run(handler())
if __name__ == "__main__":
    main()

Markus · Accepted Answer · 2022-10-31 13:52:53Z

1

I know this is not perfectly the solution you asked for, but using Python itself to run concurrent scripts poses some drawbacks (e.g. having real tasks distributed over multiple physical cores on the CPU for each script).

If you're using a UNIX-system with e.g. a bash-shell you can use the taskset to run scripts asynchronuously if you do not care to share information between them. taskset even allows you to choose the respective core you want your script to run on.

You could do it like this:

cd /your/folder/containingTheScripts

taskset --cpu-list 0 python3 yourPythonScript1.py &
taskset --cpu-list 1 python3 yourPythonScript2.py &
taskset --cpu-list 2 python3 yourPythonScript3.py &
taskset --cpu-list 3 python3 yourPythonScript4.py &

wait

The & pushes the respective tasks to the background
wait waits until each script has finished.
--cpu-list X lets you choose your physical core

Of course you can also make selecting the script generic etc. Note: You can also issue these commands directly from Python with the subprocess-module.

I hope this at least helps to tackle your issue to some extend :)

edited Oct 31, 2022 at 13:52

answered Oct 31, 2022 at 10:37

Markus

2,4975 gold badges34 silver badges60 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Jishan Shaikh Over a year ago

This answer doesn't debug their script, but I'll give +1 for a solution any day

Collectives™ on Stack Overflow

How to launch a list of python scripts asynchronously?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related