Python - Why doesn't multithreading increase the speed of my code?

Question

I tried improving my code by running this with and without using two threads:

from threading import Lock
from threading import Thread
import time

start_time = time.clock()

arr_lock = Lock()

arr = range(5000)

def do_print():
    # Disable arr access to other threads; they will have to wait if they need to read
    a = 0
    while True:
        arr_lock.acquire()
        if len(arr) > 0:
            item = arr.pop(0)
            print item
            arr_lock.release()
            b = 0
            for a in range(30000):
                b = b + 1
        else:
            arr_lock.release()
            break
thread1 = Thread(target=do_print)
thread1.start()
thread1.join()

print time.clock() - start_time, "seconds"

When running 2 threads my code's run time increased. Does anyone know why this happened, or perhaps know a different way to increase the performance of my code?

Python isn't very compatible with multithreading because of how it is implemented. There are very specific uses where multithreading gives a significant boost in performance. It has something to do with the GIL, you can find more info here — Juan C
– Juan C, Commented Sep 4, 2019 at 19:02
@JuanC, Your comment would make more sense if you said "multiprocessing" instead of "multithreading." Multiple threads in a CPython program can do pretty much everything that multiple threads in any other language can do except use more than one CPU at the same time. That's what the GIL prevents. — Solomon Slow
– Solomon Slow, Commented Sep 4, 2019 at 19:45
@עומר ששוני, It never makes sense to write t.join() on the very next line after t.start(). The entire point of creating a new thread is to let the caller to do something else while the thread is running (i.e., after the start() call and before the join() call.) — Solomon Slow
– Solomon Slow, Commented Sep 4, 2019 at 19:48
@JuanC, I see nothing wrong in that article, but I made a mistake when I said, "multiprocessing." I forgot, Python programmers use that word in a specific way, namely, to describe a single Python app that is composed of several cooperating processes. Python is "very compatible with" multiprocessing in that sense. But "multiprocessing" more generally means any technique for exploiting multiple CPUs on a single host. Python threads are not useful as a means to achieve multi-processing in the broader sense, but they are useful for pretty much all of the other reasons people use threads. — Solomon Slow
– Solomon Slow, Commented Sep 4, 2019 at 21:16

selbie · Accepted Answer · 2019-09-04 19:27:18Z

The primary reason you aren't seeing any performance improvements with multiple threads is because your program only enables one thread to do anything useful at a time. The other thread is always blocked.

Two things:

Remove the print statement that's invoked inside the lock. print statements drastically impact performance and timing. Also, the I/O channel to stdout is essentially single threaded, so you've built another implicit lock into your code. So let's just remove the print statement.

Use a proper sleep technique instead of "spin locking" and counting up from 0 to 30000. That's just going to burn a core needlessly.

Try this as your main loop

while True:
    arr_lock.acquire()
    if len(arr) > 0:
        item = arr.pop(0)
        arr_lock.release()
        time.sleep(0)
    else:
        arr_lock.release()
        break

This should run slightly better... I would even advocate getting the sleep statement out altogether so you can just let each thread have a full quantum.

However, because each thread is either doing "nothing" (sleeping or blocked on acquire) or just doing a single pop call on the array while in the lock, the majority of the time spent is going to be in the acquire/release calls instead of actually operating on the array. Hence, multiple threads aren't going to make your program run faster.

Collectives™ on Stack Overflow

Python - Why doesn't multithreading increase the speed of my code?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related