How to make my code to not run sequentially in Python using multithreading

Question

I created simple code which sum the array using Python and multithreading, but I know that It works sequentially. My question is, how can I change my code to not work sequentially using multithreading ? The code for sum:

import threading
matrix = [6,5,4,3,2,1,0]
def sum(n, **totalSum):
    sumThreads['sumThreads'] += matrix[n]

sumThreads = {"sumThreads":0}
for i in range(len(matrix)):
    t = threading.Thread(target=sum, args=(i,), kwargs=sumThreads)
    t.start()
    t.join()
print("Suma: ",sumThreads['sumThreads'])

The code for finding minimum:

import threading
matrix = [6,5,4,3,2,1,0]
def min(n, **total):
    if matrix[n-1] <= matrix[n]:
        if (matrix[n] < minThreads['minThreads']):
            minThreads['minThreads'] = matrix[n]
    else:
        if (matrix[n] < minThreads['minThreads']):
            minThreads['minThreads'] = matrix[n]


minThreads = {"minThreads": matrix[1]}
for i in range(len(matrix)):
    t = threading.Thread(target=min, args=(i,), kwargs=minThreads)
    t.start()
    t.join()
print("Minimum: ",minThreads['minThreads'])

I tried to achive it using multiprocessing like so:

if __name__ == '__main__':
    import multiprocessing
    my_input = [6,5,4,3,2,1,0]
    # Pool over all CPUs
    print(sum(multiprocessing.Pool().map(int, my_input)))
    print(min(multiprocessing.Pool().map(int, my_input)))

but I need to use multithreading. Any help would be greatly appreciated!

Does this answer your question? What is the global interpreter lock (GIL) in CPython? — Askold Ilvento
– Askold Ilvento, Commented Nov 18, 2021 at 14:58

Corralien · Accepted Answer · 2021-11-18 19:29:03Z

1

You cannot share your list my_input between processes. A good strategy is to split your data into small chunks then compute partial data and finally process all partial results:

You code could be:

import multiprocessing as mp
import time
import random

CHUNKSIZE = 3

def partial_sum(l):
    # Include a latency for demo
    time.sleep(random.random())
    s = sum(l)
    print(f"Sum of {str(l):<15}: {s:>3}")
    return s

if __name__ == '__main__':
    matrix = [6,5,4,3,2,1,0]
    chunks = (matrix[i:i+CHUNKSIZE]
                  for i in range(0, len(matrix), CHUNKSIZE))

    with mp.Pool(mp.cpu_count()) as pool:
        global_sum = sum(pool.map(partial_sum, chunks))
    print('---------------------------')
    print(f"Global total: {global_sum}")

Output:

Sum of [3, 2, 1]      :   6
Sum of [6, 5, 4]      :  15
Sum of [0]            :   0
---------------------------
Global total: 21

Update

Find the minimum:

import multiprocessing as mp

CHUNKSIZE = 3

def local_min(l):
    m = min(l)
    print(f"Local minimum of {l} is {m}")
    return m

if __name__ == '__main__':
    matrix = [6,5,4,3,2,1,0]
    chunks = (matrix[i:i+CHUNKSIZE]
                  for i in range(0, len(matrix), CHUNKSIZE))

    with mp.Pool(mp.cpu_count()) as pool:
        l = pool.map(local_min, chunks)
        m = min(l)
    print('--------------------------------')
    print(f"Global minimum of {l} is {m}")

Output:

Local minimum of [6, 5, 4] is 4
Local minimum of [3, 2, 1] is 1
Local minimum of [0] is 0
--------------------------------
Global minimum of [4, 1, 0] is 0

edited Nov 18, 2021 at 19:29

answered Nov 18, 2021 at 13:56

Corralien

121k8 gold badges43 silver badges68 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Corralien Over a year ago

Sorry I think my answer is out of topic. I can remove it if you prefer

neekitit Over a year ago

It's a little bit out of topic, but I have a question. How would your example work If I wanted to sum the array using multiprocessing ? Like in my example when using array named matrix. If you could change your code to work with the array I would be very grateful. Maybe that is the way instead of using multithreading.

Corralien Over a year ago

matrix is not defined in your code.

neekitit Over a year ago

Yeah I forgot to add it. Now It is defined.

neekitit Over a year ago

Yeah It's fantastic. Thanks!

|

Collectives™ on Stack Overflow

How to make my code to not run sequentially in Python using multithreading

1 Answer 1

7 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related