How to make this Python program faster using multithreading?

Question

I made a simple program that performs some mathematical operations, and I want to make it faster using multithreading. I understand that Python's GIL prevents true multithreading from taking place, but that there are still situations where the execution time may be improved.

In this case, the threaded and non-threaded execution time of the program seem similar, and so there doesn't appear to be any performance improvement. I'm aware of the multiprocessing module but I want to avoid that due to its limitations when performing shared memory operations. Is there any way I can achieve a significant speed increase in this program using multithreading?

Thanks

import string, random, threading

def getLetters():
  letterList = []
  for letter in string.ascii_lowercase:
    letterList.append(letter)
  return letterList

def getNum(letter):
  letterVal = ord(letter)
  rand = random.randint(1,5000000)
  result = 0

  for num in range(0,rand):
    if num%2 == 0:
      result+=num
    else:
      result-=num

  result*=letterVal
  print(f"Result: {result}")
  return result

def mainWithThreading():
  letters = getLetters()
  threadList = []
  for letter in letters:
    th = threading.Thread(target=getNum, args=[letter], daemon=True)
    threadList.append(th)
  for th in threadList:
    th.start()
  for th in threadList:
    th.join()

def mainWithoutThreading():
  letters = getLetters()
  for letter in letters:
    getNum(letter)

start = time.time()
mainWithoutThreading()
end = time.time()
print(f"Time taken: {end-start:.2f}s\n")

start = time.time()
mainWithThreading()
end = time.time()
print(f"Time taken: {end-start:.2f}s\n")

After taking a quick look, it seems your program is entirely CPU bound, so no, multithreading will not improve performance in CPython — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Aug 27, 2022 at 16:47
As @juanpa.arrivillaga has correctly stated, multithreading is very unlikely to help. However, multiprocessing may. Is that something you've considered? Also, your getLetters() function is pointless — jackal
– jackal, Commented Aug 27, 2022 at 17:14
@Vlad Yeah it was just random for an example. Thanks for letting me know about multithreading not working for CPU bound operations. — zohani
– zohani, Commented Aug 27, 2022 at 17:54

jackal · Accepted Answer · 2022-08-27 17:35:55Z

Some code changed - mainly to utilise multiprocessing but also removed some redundancy.

from concurrent.futures import ProcessPoolExecutor
from string import ascii_letters
from random import randint
from time import perf_counter

def getNum(c):
    letterVal = ord(c)
    result = 0
    for num in range(randint(1, 5_000_000)):
        # the calculations within this loop serve no purpose other
        # than to make Python do "work"
        if num % 2 == 0:
            result += num
        else:
            result -= num
    result *= letterVal
    return result

def main():
    start = perf_counter()
    with ProcessPoolExecutor() as executor:
        executor.map(getNum, ascii_letters)
    end = perf_counter()
    print(f'Duration after subprocess mapping = {end-start:.2f}s')
    start = perf_counter()
    for c in ascii_letters:
        getNum(c)
    end = perf_counter()
    print(f'Duration after linear processing = {end-start:.2f}s')

if __name__ == '__main__':
    main()

Output:

Duration after subprocess mapping = 1.35s
Duration after linear processing = 8.68s

Note:

Due to the randomisation in the getNum() function, you would need to run this several times to get a clearer picture of how multiprocessing gives benefits for this kind of thing. A better test would be to take out the random factor.

By changing the loop in getNum() to a constant range of two million the result are:

Duration after subprocess mapping = 1.08s
Duration after linear processing = 7.69s

...which is a much more realistic test

Collectives™ on Stack Overflow

How to make this Python program faster using multithreading?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related