Python/Django logging with extra options

Question

I have a logging config file in Django that takes sys.exc_info() tuple & passes them as extra options while logging (see the variables type & value in formatter below)

'formatters': {
    'basic': {
        'format': '%(levelname)s %(lineno)d %(message)s %(type)s %(value)s'
    },
},

Here's how I log the error:

    except Exception, e:
        extra = {'type':sys.exc_info()[0], 'value':sys.exc_info()[1]}
        logger.warning('My message', extra=extra)

However if I simply write

    except Exception, e:
        logger.warning('My message')

I get an exception because the variables type & value are undefined in formatter now. How can I tell the formatter to treat these variables as optional i.e. if I pass them while logging, then use them, else skip them.

I updated my answer in response to your comment.

Vinay Sajip
– Vinay Sajip

2014-04-11 12:40:46 +00:00
Commented Apr 11, 2014 at 12:40 — Vinay Sajip
– Vinay Sajip, Commented Apr 11, 2014 at 12:40
Added further update with Filter example.

Vinay Sajip
– Vinay Sajip

2014-04-11 18:49:16 +00:00
Commented Apr 11, 2014 at 18:49 — Vinay Sajip
– Vinay Sajip, Commented Apr 11, 2014 at 18:49

Vinay Sajip · Accepted Answer · 2014-04-11 18:48:57Z

2

I would suggest that you don't pass the exc_info tuple parts explicitly in your logging calls. Instead, note that if an exception occurs and you pass exc_info=True or exc_info=sys.exc_info(), then the tuple is already stored in the LogRecord's exc_info attribute. You can access these in your formatter, so you can either use a Formatter subclass or a Filter subclass to convert parts of the tuple to other LogRecord attributes which can be referred to in the format string (or even otherwise, handled by a custom Formatter subclass).

Update: You can reference as %(exc_info)s in the format string, but that will just display the tuple. Note that a Formatter's formatException method can be overridden to display the exception. The default will format a standard traceback - do I understand that you don't want that to happen?

class MyFormatter(logging.Formatter):
    def formatException(self, exc_info):
        return 'exception: %s %s' % exc_info[:2])

That would print a one-liner rather than a full traceback, but on a new line. Or you could use an approach like:

class MyFormatter(logging.Formatter):
    def format(self, record):
        if isinstance(record.exc_info, tuple):
            record.exc_data = ' %s %s' % record.exc_info[:2]
        else:
            record.exc_data = ''
        return super(MyFormatter, self).format(record)

    def formatException(self, exc_info):
        return ''

and then use a format string with %(exc_data)s in it.

With both of these, you need to ensure that a true exc_info is passed into the logging call, to ensure that the exception data is saved in the LogRecord. The exception() method of loggers does this (with a level of ERROR).

Further update: To do this with a Filter subclass, you could do something like:

class MyFilter(logging.Filter):
    def filter(self, record):
        if isinstance(record.exc_info, tuple):
            record.exc_data = ' %s %s' % record.exc_info[:2]
        else:
            record.exc_data = ''
        # The next bit is to prevent the standard Formatter from writing
        # a traceback
        record.exc_info = None
        return True # otherwise, the record doesn't get output

This should have the same effect as the earlier example, so you can use the same format string.

edited Apr 11, 2014 at 18:48

answered Apr 11, 2014 at 7:53

Vinay Sajip

100k15 gold badges184 silver badges196 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user Over a year ago

This sounds like an approach worth exploring. (1) Should I access exc_info in formatter as %(exc_info)s? (2) Could you show an example of how to extract exc_info[0] inside formatter. -- P.S. python docs docs.python.org/2/library/logging.html#logrecord-attributes says You shouldn’t need to format this yourself for exc_info

user Over a year ago

Good learning. Thanks. Could you also point to an example of using the Filter subclass.

Collectives™ on Stack Overflow

Python/Django logging with extra options

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related