How to write data into a text file using multithreading in Python?

Question

I want to append the data into the same text file using multithreading in Python.

And this is my code:

import threading
my_list = ["a", "b", "c", "d", "e", "f"]
def f1():
     for item in my_list:
       file = open("order_log.txt", "a")
       file.write(f"It is f1 {item}")
       file.close()
def f2():
     for item in my_list:
       file = open("order_log.txt", "a")
       file.write(f"It is f2 {item}")
       file.close()
def f3():
     for item in my_list:
       file = open("order_log.txt", "a")
       file.write(f"It is f3 {item}")
       file.close()
def f4():
     for item in my_list:
       file = open("order_log.txt", "a")
       file.write(f"It is f4 {item}")
       file.close()
t1 = threading.Thread(target=f1)
t2 = threading.Thread(target=f2)
t3 = threading.Thread(target=f3)
t4 = threading.Thread(target=f4)
t1.start()
t2.start()
t3.start()
t4.start()
t1.join()
t2.join()
t3.join()
t4.join()

But when I execute the program it's not writing anything.

Hi Christy, what benefit does multithreading a single resource (a file locked by another thread) provide? — Shawn
– Shawn, Commented May 12, 2021 at 4:38
Trying to write into a single file from multiple threads simultaneously is going to make an unusable mess of the file's contents. If you change your code so that each thread writes to a different file (e.g. "order_log_1.txt", "order_log_2.txt", etc), OTOH, that will work okay. — Jeremy Friesner
– Jeremy Friesner, Commented May 12, 2021 at 4:41
If you have to write into the same file then you'll need to share one file-handle amongst all of the threads (rather than having each thread open its own separate file-handle), and you'll need to guard all uses of that file-handle with a mutex to avoid race conditions. That will serialize access to the file so that only one thread can write to it at a time. — Jeremy Friesner
– Jeremy Friesner, Commented May 12, 2021 at 5:11

Danya02 · Accepted Answer · 2021-05-12 07:44:14Z

3

You need to make it so that only one thread can write to the file at one time. This is done by having the thread hold a Lock on the file. Once the thread is done with the file, it can release the Lock and it'll get taken by one of the threads waiting for it.

Here's an example of how to use locks, along with a Queue because most often the reason you want to use threads is because you want to perform the same operation on many data objects in parallel.

import threading
import queue
import random
import time

tasks = queue.Queue()

output = open('output.txt', 'w')
output_lock = threading.Lock()

def worker(thread_number):
    while not tasks.empty():
        task = tasks.get()
        # perform long calculation here...
        time_to_spend = random.random()
        time.sleep(time_to_spend)
        result = task * task

        # now we have result, want to write it
        with output_lock:  # this will block until the lock is available
            print(thread_number, ': square of', task, 'is', result, '; took', time_to_spend, file=output)
        tasks.task_done()

for i in range(100):
    tasks.put(i)

for thread in range(8):
    threading.Thread(target=worker, args=(thread,)).start()

print('waiting for tasks to complete')
tasks.join()
print('done')

The output I got was:

6 : square of 6 is 36 ; took 0.02233345201885739
7 : square of 7 is 49 ; took 0.0352967148552743
4 : square of 4 is 16 ; took 0.1043699083780637
7 : square of 9 is 81 ; took 0.2158108589024338
1 : square of 1 is 1 ; took 0.3330501408298937
4 : square of 10 is 100 ; took 0.3564233912485101
5 : square of 5 is 25 ; took 0.8496825534757959
0 : square of 0 is 0 ; took 0.8807306770021203
4 : square of 13 is 169 ; took 0.4420943872313102
7 : square of 11 is 121 ; took 0.6772180132068408
5 : square of 14 is 196 ; took 0.1101644871869385
6 : square of 8 is 64 ; took 0.944739067078435
3 : square of 3 is 9 ; took 0.9699315957506418
2 : square of 2 is 4 ; took 0.9903787965119304
3 : square of 20 is 400 ; took 0.029847547355710158
1 : square of 12 is 144 ; took 0.696648284935379
...

answered May 12, 2021 at 7:44

Danya02

1,4061 gold badge14 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Serge Ballesta Over a year ago

This is a nice demo and it even display the id of the thread that processed a number, but in real world programming, I would use a ThreadPoolExecutor. +1 anyway...

Danya02 Over a year ago

Yes, I forgot that this was a thing. Do you think I should edit this to add a demo of that?

Collectives™ on Stack Overflow

How to write data into a text file using multithreading in Python?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related