python - logging in Multi Threads

Question

I am trying to store logs from multiple threads into a single file. But in my log file, logs from different threads are getting merged. I don't want separate log files for different threads. But when one thread is writing logs to the file, the other thread should wait till the previous thread finishes writing all the logs. I have tried different ways to do so, but finally didn't find appropriate solution.

from threading import Thread, current_thread
import threading
import time
import logging


logging.basicConfig(filename='LogsThreadPrac.log', level=logging.INFO)
logger = logging.getLogger(__name__)


def worker(val):
    print('value from the thread: {}'.format(val))
    logger.info('+++++++++++++++++++++++++++++++++++++++')
    print(current_thread().name)
    logger.info('value from the thread: {}'.format(val))
    time.sleep(3)
    logger.info('+++++++++++++++++++++++++++++++++++++++')

t1 = Thread(target=worker, args=[1,], name='th'+str(1))
t1.start()
t2 = Thread(target=worker, args=[2,], name='th'+str(2))
t2.start()

What is your use case? The logging module is not intended to deal with synchronisation requirements of higher layers of abstraction. — code_onkel
– code_onkel, Commented Feb 12, 2018 at 11:40

jtagle · Accepted Answer · 2018-02-12 12:39:41Z

2

You could pass a Lock as an argument to your workers, so you manage to synchronize the threads the way you want.

from threading import Thread, current_thread, Lock
import threading
import time
import logging


logging.basicConfig(filename='LogsThreadPrac.log', level=logging.INFO)
logger = logging.getLogger(__name__)
logger_lock = Lock()

def worker(val, lock):
    print('value from the thread: {}'.format(val))
    with lock:
        logger.info('+++++++++++++++++++++++++++++++++++++++')
        print(current_thread().name)
        logger.info('value from the thread: {}'.format(val))
        time.sleep(3)
        logger.info('+++++++++++++++++++++++++++++++++++++++')
t1 = Thread(target=worker, args=[1, logger_lock], name='th'+str(1))
t1.start()
t2 = Thread(target=worker, args=[2, logger_lock], name='th'+str(2))
t2.start()

You can check more about the lock uses and examples here

answered Feb 12, 2018 at 12:39

jtagle

3121 silver badge8 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Billy T Over a year ago

This solution is going to take at least 6 seconds to finish both workers, which is pretty much the same time it takes without using threading.

Metehan Yıldırım Over a year ago

The reason is that the threads are accessing the same file and waiting for them. This is not a true concurrency design.

jtagle Over a year ago

Guess you guys don't really understand the problem of concurrency and how did the guy wanted to apply it...

Collectives™ on Stack Overflow

python - logging in Multi Threads

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related