0

I have used multi threading execute independent code ( that did not have any common code between them ), but this time, I have to use a common function that would be called in each of the thread. So, I am a little confused if it would work. For example,

thread1:
 #do something
 input_list = [5,6,7,8]
 output_list = common_function(input_list)
 print output_list

thread2:
 #do something
 input_list = [1,2,3,4]
 output_list = common_function(input_list)
 print output_list

Would the above code be a problem, perhaps because of the race condition? Or python automatically takes care of this? The common_function(input_data) returns a list based on the input provided and this list is then displayed. For a small dataset it work, but I my question is if the input_list becomes huge, would it lead to a problem in the code?

3 Answers 3

1

If common_function is a function without any side effects, this is save. In other words, if common_function only works on the input_list and uses nothing else (no shared data, no service), you can call the function in parallel. The size of the input data does not matter as long as it is not shared with any other thread.

Sign up to request clarification or add additional context in comments.

2 Comments

there is no shared data, but there are lots of local variables inside the common_function(). So, there would be different common_functions instances on the stack, depending on which thread called the common_function() ..is it?
Yes, that's right. Local variables are on the stack of the function call.
0

If you are using the threading package then you can use semaphore to lock a variable before writing to it. Like the Example here:

import threading

lock = threading.BoundedSemaphore()

def function():
    lock.acquire()  # Lock the Variable
    # Write to your List
    lock.release()  # Release the Variable

So all you write between acquire and release is save for threading

1 Comment

For working on function arguments using a lock is not necessary.
0

As @lutz had written, there is no danger of race conditions in case there are no shared data between two instances of the common_function. This is what is meant by referential transparency and generally all programming languages aim that such functions should be thread safe. Sometimes, you will need to write and use functions which change some global state. In such cases the modern adage is to use event driven programming - which means to not communicate directly between threads but communicate via some thread safe queueing system. In python I am a huge fan of the queue module for that matter. Another good queue module is the multiprocessing.queue for which a good example is here. I am also pasting the code here.

from multiprocessing import Process, Queue

def f(q):
    q.put([42, None, 'hello'])

if __name__ == '__main__':
    q = Queue()
    p = Process(target=f, args=(q,))
    p.start()
    print(q.get())    # prints "[42, None, 'hello']"
    p.join()

Finally in case you are not confident about some function (probably it is a big function and you dont understand the nuts and bolts), I would recommend that you use the fuzzying method. Here you define a simple function

FUZZ = True
def fuzz():
    """
    fuzzing is a technique to make the race condition errors more visible
    """
    if FUZZ:
        time.sleep(random.random())

and then drop this function at random places inside your code. That should amplify any race conditions that there are in the code. This is of-course not a guaranteed method and so if your function is in a production application that gets called millions of times the better strategy will be to break the function to smaller more digestible parts. Watch Raymond Hettinger deliver a lecture about concurrent code in his famous talk on python threading. You can get the code that he is talking about here.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.