run multiple instances of python script simultaneously

Question

I am trying to create 86 instances of task.py to run simultaneously.

import sys
import subprocess

for file in range(86):
    subprocess.call([sys.executable,'task.py',str(file)+'in.csv',str(filen)+'out.csv'])

@Alex the problem was "subprocess.call waits for command to complete. Use subprocess.Popen instead:" — user14372
– user14372, Commented Oct 3, 2013 at 10:59

Pe Dro · Accepted Answer · 2020-05-29 06:38:46Z

22

subprocess.call waits for command to complete. Use subprocess.Popen instead:

import sys
import subprocess

procs = []
for i in range(86):
    proc = subprocess.Popen([sys.executable, 'task.py', '{}in.csv'.format(i), '{}out.csv'.format(i)])
    procs.append(proc)

for proc in procs:
    proc.wait()

edited May 29, 2020 at 6:38

Pe Dro

3,1273 gold badges29 silver badges48 bronze badges

answered Oct 3, 2013 at 10:13

falsetru

371k69 gold badges769 silver badges659 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Lisa Over a year ago

Is this solution a "multithread" solution? Or actually it launches multiple instance of python?

falsetru Over a year ago

@Lisa, As the name of the module suggest, it launches multiple instances (sub"process"). (OP's original code also launches multiple instances)

Pawankumar Dubey Over a year ago

I was just searching for something like this for so much time. Thank you

MasayoMusic Over a year ago

I can't get it to work when looping and appending: out =subprocess.Popen("hugo server -D", stdout = subprocess.PIPE) outputs.append(out) And then iterating through outputs and calling output.wait(). It opens up the first program, but then doesn't open up the next one unless I close the first program.

falsetru Over a year ago

@MasayoMusic, Replace "hugo server -D" -> ['hugo', 'server', '-D']. If that does not work, please post a separate question so that others can answer you.

|

Collectives™ on Stack Overflow

run multiple instances of python script simultaneously

1 Answer 1

8 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related