Best way to stop a Python script even if there are Threads running in the script

Question

I have a python program run_tests.py that executes test scripts (also written in python) one by one. Each test script may use threading.

The problem is that when a test script unexpectedly crashes, it may not have a chance to tidy up all open threads (if any), hence the test script cannot actually complete due to the threads that are left hanging open. When this occurs, run_tests.py gets stuck because it is waiting for the test script to finish, but it never does.

Of course, we can do our best to catch all exceptions and ensure that all threads are tidied up within each test script so that this scenario never occurs, and we can also set all threads to daemon threads, etc, but what I am looking for is a "catch-all" mechanism at the run_tests.py level which ensures that we do not get stuck indefinitely due to unfinished threads within a test script. We can implement guidelines for how threading is to be used in each test script, but at the end of the day, we don't have full control over how each test script is written.

In short, what I need to do is to stop a test script in run_tests.py even when there are rogue threads open within the test script. One way is to execute the shell command killall -9 <test_script_name> or something similar, but this seems to be too forceful/abrupt.

Is there a better way?

Thanks for reading.

burnpanck · Accepted Answer · 2015-03-01 22:22:32Z

2

To me, this looks like a pristine application for the subprocess module. I.e. do not run the test-scripts from within the same python interpreter, rather spawn a new process for each test-script. Do you have any particular reason why you would not want to spawn a new process and run them in the same interpreter instead? Having a sub-process isolates the scripts from each other, like imports, and other global variables.

If you use the subprocess.Popen to start the sub-processes, then you have a .terminate() method to kill the process if need be.

answered Mar 1, 2015 at 22:22

burnpanck

2,2661 gold badge19 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

wookie919 Over a year ago

Reading this answer made me realise that I've actually asked the wrong question. Now that I have figured out the correct question to ask, I have managed to solve my own problem. So thank you, you still helped me. :) BTW, I was already using the subprocess module.

wookie919 · Accepted Answer · 2015-03-01 23:10:17Z

1

What I actually needed to do was tidy up all threads at the end of each test script rather than at the run_tests.py level. I don't have control over the main functions of each test script, but I do have control over the tidy up functions.

So this is my final solution:

for key, thread in threading._active.iteritems():
        if thread.name != 'MainThread':
                thread._Thread__stop()

I don't actually need to stop the threads. I simply need to mark them as stopped with _Thread__stop() so that the test script can exit. I hope others find this useful.

answered Mar 1, 2015 at 23:10

wookie919

3,13428 silver badges35 bronze badges

Collectives™ on Stack Overflow

Best way to stop a Python script even if there are Threads running in the script

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related