python logging print traceback only in debug

Question

I am currently loading the python logger like this:

import logging
logging.basicConfig(level=logging.INFO)
log = logging.getLogger("myprogram")

and using it e. g. like this:

[...]
except FileNotFoundError:
    log.exception("could not open configuration file")
    sys.exit(1)

However, this will always print the traceback along with the error message:

ERROR:myprogram:could not open configuration file
Traceback (most recent call last):
[...]
FileNotFoundError: [Errno 2] No such file or directory: 
'not/existing/file.yml'

I do not want the traceback in the normal error output. Instead it should only print my error message and the exception info ("No such file...").

What is the recommended way of showing the traceback only when the loglevel is set to logging.DEBUG?

Martijn Pieters · Accepted Answer · 2018-09-23 12:45:34Z

12

Log the exception at DEBUG level instead and set exc_info=True. logger.exception() is essentially a logger.error(..., exc_info=True) call, but you can log exception tracebacks at any level:

log.debug("could not open configuration file", exc_info=True)

It's the exc_info option that's important; from the documentation:

If exc_info does not evaluate as false, it causes exception information to be added to the logging message. If an exception tuple (in the format returned by sys.exc_info()) or an exception instance is provided, it is used; otherwise, sys.exc_info() is called to get the exception information.

You perhaps want to use printing (to stdout or stderr) to communicate with the end-user:

except FileNotFoundError as e:
    log.debug("could not open configuration file", exc_info=True)
    print("Could not open configuration file:", e.strerror, file=sys.stderr)
    sys.exit(1)

I included the system error message in the print output without the FileNotFoundError(...) representation.

If you use a command-line argument parser like argparse or click, then do use their user feedback API (which usually includes exiting too).

You can make the logging module produce user-level messages too, but if you want a single logger call to produce debug-friendly tracebacks in a file and user-friendly output on a console, you'd have to configure separate handlers for these use-cases with the console handler using a custom Formatter() class to override the formatException() method to alter how exceptions are shown. It's just much easier and clearer to separate logging and end-user communication.

edited Sep 23, 2018 at 12:45

answered Sep 23, 2018 at 12:16

Martijn Pieters

1.1m326 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Sebastian Stark Over a year ago

ok, I was under the wrong impression that the logging module would take exactly this burden off the programmer. I indeed use argparse, which I should use here for handling this particualr error, but I have more places in the program where I need this. Thanks a lot!

Martijn Pieters Over a year ago

@SebastianStark: logging can, sort-of, take care of this, but then you'd have to configure an extra handler that shows only end-user messages, and use a custom Formatter subclass to alter how exception information is included in the output. I usually just use what I outlined in my answer instead.

Sebastian Stark Over a year ago

In this case it is a daemon and I want everything always in the log. Although I think what you propose is the right way usually, I prefer something simpler for this.

DeepSpace · Accepted Answer · 2018-09-23 12:21:54Z

7

I'd use a combination of exc_info and .getEffectiveLevel:

try:
    ...
except FileNotFoundError as ex:
   logger.error(ex, exc_info=log.getEffectiveLevel() == logging.DEBUG)

This way, the exception itself (FileNotFoundError) is always logged, but the stacktrace will only be logged if log level is debug.

edited Sep 23, 2018 at 12:21

answered Sep 23, 2018 at 12:16

DeepSpace

82.1k12 gold badges119 silver badges166 bronze badges

1 Comment

Sebastian Stark Over a year ago

I ended up defining a helper function like this: def err(*args, **kwargs): log.error(*args, exc_info=log.getEffectiveLevel() == logging.DEBUG, **kwargs), does exactly what I want, thanks!

Hashimoto · Accepted Answer · 2022-01-02 20:25:41Z

1

You can also directly use logging.debug plus traceback:

try:
    do_something()
except Exception as e:
    logger.error("Unhandled exception: %s", e)
    logger.debug("Traceback: %s", traceback.format_exc())

Edit: logger.debug("", exc_info=True) can be used instead of logger.debug("Traceback: %s", traceback.format_exc()).

edited Jan 2, 2022 at 20:25

answered Nov 15, 2020 at 5:33

Hashimoto

3464 silver badges8 bronze badges

Collectives™ on Stack Overflow

python logging print traceback only in debug

3 Answers 3

3 Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related