Different levels of logging in python

Question

I want to do something which I thought is simple.

Actually with the python logging module, I am interested logging everything on the command line at the level given from the command line arguments, and log to file to a fixed DEBUG level.

Creating two different loggers with different levels doesn't work, but setting the levels of two different handlers both added to the root logger doesn't work either, so any idea about how I should actually do it? (reading on other links the second approach should work, so am I doing something else stupid?)

This is the code which sets up my logging system at the moment:

class LoggerSetup(object):
    """Setup the different logger objects
    """

    def __init__(self):
        self.root_logger = logging.getLogger()
        self.shell_hdlr = logging.StreamHandler()

    #TODO: add another logging handler which stores to a temporary file
    #which should be cleaned up later
    def setup_shell_logger(self, log_level):
        self.root_logger.setLevel(LOG_LEVELS[log_level])
        # in this way the root logger is not set but the handlers are set
        self.shell_hdlr = logging.StreamHandler()
        self.shell_hdlr.setLevel(LOG_LEVELS[log_level])
        self.shell_hdlr.setFormatter(StarFormatter())
        #FIXME: add the support for regular expression exclusion too
        self.root_logger.addHandler(self.shell_hdlr)

    def setup_log_include(self, log_include):
        """Set up the filter to include log messages
        """
        if log_include:
            incl = FilterInclude(log_include)
            self.shell_hdlr.addFilter(incl)

    def setup_log_exclude(self, log_exclude):
        """Set up the filters to exclude log messages
        """
        if log_exclude:
            excl = FilterExclude(log_exclude)
            self.shell_hdlr.addFilter(excl)

    def setup_file_logging(self):
        """Set up the file logger, which always logs in DEBUG mode
        even if the top level logger is set to another level
        """
        #XXX: not working, one possible way to make it work is to create
        #only one log, and different handler/filters to make to handle all
        #the different outputs
        file_handler = logging.FileHandler(LOG_FILENAME)
        # the file logging is always in debug mode
        file_handler.setLevel(logging.DEBUG)
        formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s - %(asctime)s')
        file_handler.setFormatter(formatter)
        self.root_logger.addHandler(file_handler)

"but setting the levels of two different handlers both added to the root logger doesn't work"? Can you clarify, please? — S.Lott
– S.Lott, Commented Jan 12, 2012 at 15:48

Zach Young · Accepted Answer · 2015-08-12 20:31:01Z

13

This is something I'm using with all of my Python command-line apps. It's a little verbose, but you should be able to get a logger that accepts an optional argument to create a console logger at any level, irrespective of what's being logged to the file:

#!/usr/bin/env python
import logging
from argparse import ArgumentParser

COMPANY_LOGGER = 'COMPANY.Python.Logger'
CONSL_LEVEL_RANGE = range(0, 51)
LOG_FILE = 'company.log'
FORMAT_STR = '%(asctime)s %(levelname)s %(message)s'

parser = ArgumentParser()
parser.add_argument('-c', '--console-log', metavar='ARG',
                    type=int, choices=range(0, 51),
                    action='store', dest='console_log',
                    default=None,
                    help='Adds a console logger for the level specified in the range 1..50')

args = parser.parse_args()

# Create logger
logger = logging.getLogger(COMPANY_LOGGER)
logger.setLevel(logging.DEBUG)
formatter = logging.Formatter(FORMAT_STR)

# Add FileHandler and only log WARNING and higher
fh = logging.FileHandler(LOG_FILE)
fh.name = 'File Logger'
fh.level = logging.WARNING
fh.formatter = formatter
logger.addHandler(fh)

# Add optional ConsoleHandler
if args.console_log:
    ch = logging.StreamHandler()
    ch.name = 'Console Logger'
    ch.level = args.console_log
    ch.formatter = formatter
    logger.addHandler(ch)

logger.debug('DEBUG')
logger.info('INFO')
logger.warning('WARNING')
logger.critical('CRITICAL')

Whenrun from the command line we can see the differences in logged levels.

-c1 equates to "DEBUG and higher" (the most verbose), but company.log is still only logging WARNING and higher:

~ zacharyyoung$ ./so.py -c1
2012-01-12 08:59:50,086 DEBUG DEBUG
2012-01-12 08:59:50,086 INFO INFO
2012-01-12 08:59:50,087 WARNING WARNING
2012-01-12 08:59:50,087 CRITICAL CRITICAL

~ zacharyyoung$ cat company.log 
2012-01-12 08:59:50,087 WARNING WARNING
2012-01-12 08:59:50,087 CRITICAL CRITICAL

-c20 equates to INFO:

~ zacharyyoung$ ./so.py -c20
2012-01-12 09:00:09,393 INFO INFO
2012-01-12 09:00:09,393 WARNING WARNING
2012-01-12 09:00:09,393 CRITICAL CRITICAL

~ zacharyyoung$ cat company.log 
2012-01-12 08:59:50,087 WARNING WARNING
2012-01-12 08:59:50,087 CRITICAL CRITICAL
2012-01-12 09:00:09,393 WARNING WARNING
2012-01-12 09:00:09,393 CRITICAL CRITICAL

edited Aug 12, 2015 at 20:31

answered Jan 12, 2012 at 17:04

Zach Young

11.4k4 gold badges38 silver badges57 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

logc Over a year ago

+1 Just a suggestion: I think you can avoid defining the getConsoleLevel function if you use the type=int and choices=xrange(51) keywords of the parser.add_argument method. See the end of this section on the doc. And you can set a default of 21 and get the same default level as the logging module.

Zach Young Over a year ago

@logc: thank you for the suggestion... I'm not sure why I didn't do this previously (maybe I was concerned about the way the help statement printed), but I can see it works just as well using only built-ins. Thank you :)

Collectives™ on Stack Overflow

Different levels of logging in python

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related