Python logging handler to append to list

Question

Somewhat similar to this (unanswered) question here:

https://stackoverflow.com/questions/33136398/python-logging-handler-that-appends-messages-to-a-list

I am looking to use a handler to simply append anything (passing a filter) to a python list. I would envisage some kind of code like this:

import logging
import sys
mylog = logging.getLogger()
mylog.setLevel(logging.DEBUG)

log_list = []
lh = logging.SomeHandler(log_list)
lh.setLevel(logging.DEBUG)

mylog.addHandler(lh)
mylog.warning('argh')
print log_list[0]

The question is therefore - how can I implement this? What SomeHandler can I use?

the linked question has been removed from Stack Overflow for moderation reasons. Could you please remove the link from the question body, as it is no longer helpful? Thank you! — gt6989b
– gt6989b, Commented Jun 27, 2023 at 11:32

imriqwe · Accepted Answer · 2019-08-08 08:12:11Z

18

Here is a naive, non thread-safe implementation:

import logging

class ListHandler(logging.Handler): # Inherit from logging.Handler
        def __init__(self, log_list):
                # run the regular Handler __init__
                logging.Handler.__init__(self)
                # Our custom argument
                self.log_list = log_list
        def emit(self, record):
                # record.message is the log message
                self.log_list.append(record.msg)

edited Aug 8, 2019 at 8:12

answered Apr 4, 2016 at 17:05

imriqwe

1,45511 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Dave Over a year ago

This looks good on paper, but I get the error AttributeError: 'LogRecord' object has no attribute 'message' when trying to implement it (as per code in OP). Any ideas?

Dave Over a year ago

OK - I figured it out. This could be version specific (I'm running 2.7.6), but the emit function needs to append record.msg, not record.message, ie: self.logs_list.append(record.msg) --if you could correct this @imriqwe, I'll mark it as the accepted answer. Cheers!

imriqwe Over a year ago

Glad I could help :)

tavor999 Over a year ago

@imriqwe having a little difficulty putting this into action. My goal is simply to create a second handler that writes to the array when a message is logged. Can you expand on answer a bit?

Community · Accepted Answer · 2017-05-23 12:00:34Z

@imriqwe's answer is correct for a non thread-safe implementation, but if you need to be thread-safe, one solution is to use a queue.Queue() instead of a list. Here is some code I am using in an in-process project to generate a tkinter log window.

import logging
import queue

class QueuingHandler(logging.Handler):
    """A thread safe logging.Handler that writes messages into a queue object.

       Designed to work with LoggingWidget so log messages from multiple
       threads can be shown together in a single ttk.Frame.

       The standard logging.QueueHandler/logging.QueueListener can not be used
       for this because the QueueListener runs in a private thread, not the
       main thread.

       Warning:  If multiple threads are writing into this Handler, all threads
       must be joined before calling logging.shutdown() or any other log
       destinations will be corrupted.
    """

    def __init__(self, *args, message_queue, **kwargs):
        """Initialize by copying the queue and sending everything else to superclass."""
        logging.Handler.__init__(self, *args, **kwargs)
        self.message_queue = message_queue

    def emit(self, record):
        """Add the formatted log message (sans newlines) to the queue."""
        self.message_queue.put(self.format(record).rstrip('\n'))

To use, create a queue, create the handler using the queue, then add it to the logger (this example also creates a log file in the current directory):

LOG_FORMAT = '%(asctime)s: %(name)8s: %(levelname)8s: %(message)s'
#  Setup root logger to write to a log file.
logging.basicConfig(filename='gui-test.log',
                    filemode='w',
                    format=LOG_FORMAT,
                    level=logging.DEBUG
                   )

#  Get a child logger
logger = logging.getLogger(name='gui')

#  Build our QueuingHandler
message_queue = queue.Queue()
handler = QueuingHandler(message_queue=message_queue, level=logging.DEBUG)

#  Change the date/time format for the GUI to drop the date
formatter = logging.Formatter(LOG_FORMAT)
formatter.default_time_format = '%H:%M:%S'
handler.setFormatter(formatter)

#  Add our QueuingHandler into the logging heirarchy at the lower level
logger.addHandler(handler)

Now all you have to do is read your messages from the queue.

Collectives™ on Stack Overflow

Python logging handler to append to list

2 Answers 2

4 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related