python script execution in concurrently without thread

Question

How can I execute single python script multiple time concurrently using different PID without using Threads and Multiprocessing from another python script?

And I need to get results from each execution

I tried Multiprocessing module simple program I got AttributeError: 'module' object has no attribute 'f' and it need to work on both Linux and windows. previous post's solution not working for me

After verifying with previous posts I posted this again.

Try running "python yourscript.py >> output.txt &" several times?? I am testing this. You can print pid in your script. — BAE
– BAE, Commented Jan 20, 2016 at 14:46
It is good to write to some file in the script instead of ">> output.txt", too. If this meet your requirements, I will provide detailed answer. — BAE
– BAE, Commented Jan 20, 2016 at 15:14

BlackJack · Accepted Answer · 2016-01-20 16:23:29Z

1

A different PID means you need a different process. Not using multiprocessing you may start another process with the subprocess module and get the result via stdout:

#!/usr/bin/env python
# coding: utf8
from __future__ import absolute_import, division, print_function
import pickle
import sys
from subprocess import PIPE, Popen


def main():
    processes = [
        Popen([sys.executable, 'test.py'], stdout=PIPE)
        for _ in xrange(5)
    ]
    results = [pickle.loads(p.stdout.read()) for p in processes]
    for process in processes:
        process.wait()
    print(results)


if __name__ == '__main__':
    main()

test.py needs to write the result serialized with pickle to its stdout.

answered Jan 20, 2016 at 16:23

BlackJack

4,7231 gold badge22 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

BAE · Accepted Answer · 2016-01-20 21:31:39Z

May this help?

test.py:(just for demo)

import time, os, datetime, fcntl

with open("output.txt", "a") as g:
    fcntl.flock(g, fcntl.LOCK_EX)
    g.write( "PID [" + str(os.getpid()) + "]," + str(datetime.datetime.now()) + "\n" )
    fcntl.flock(g, fcntl.LOCK_UN)


time.sleep(30)

with open("output.txt", "a") as g:
    fcntl.flock(g, fcntl.LOCK_EX)
    g.write( "PID [" + str(os.getpid()) + "]," + str(datetime.datetime.now()) + "\n" )
fcntl.flock(g, fcntl.LOCK_UN)

Run it several times:

c@chen:~/src$ python test2.py &
[1] 29265
c@chen:~/src$ python test2.py &
[2] 29266
c@chen:~/src$ python test2.py &
[3] 29268
c@chen:~/src$ python test2.py &
[4] 29269
c@chen:~/src$ vim test2.py 
[1]   Done                    python test2.py
[2]   Done                    python test2.py
[3]-  Done                    python test2.py
[4]+  Done                    python test2.py

Output:

c@chen:~/src$ tail -f -n 100 output.txt 
PID [29265],2016-01-20 16:28:20.373244
PID [29266],2016-01-20 16:28:21.068946
PID [29268],2016-01-20 16:28:21.911043
PID [29269],2016-01-20 16:28:22.547805
PID [29265],2016-01-20 16:28:50.403474
PID [29266],2016-01-20 16:28:51.075268
PID [29268],2016-01-20 16:28:51.914001
PID [29269],2016-01-20 16:28:52.564706

Collectives™ on Stack Overflow

python script execution in concurrently without thread

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related