294

In a python script I am writing, I am trying to log events using the logging module. I have the following code to configure my logger:

ERROR_FORMAT = "%(levelname)s at %(asctime)s in %(funcName)s in %(filename) at line %(lineno)d: %(message)s"
DEBUG_FORMAT = "%(lineno)d in %(filename)s at %(asctime)s: %(message)s"
LOG_CONFIG = {'version':1,
              'formatters':{'error':{'format':ERROR_FORMAT},
                            'debug':{'format':DEBUG_FORMAT}},
              'handlers':{'console':{'class':'logging.StreamHandler',
                                     'formatter':'debug',
                                     'level':logging.DEBUG},
                          'file':{'class':'logging.FileHandler',
                                  'filename':'/usr/local/logs/DatabaseUpdate.log',
                                  'formatter':'error',
                                  'level':logging.ERROR}},
              'root':{'handlers':('console', 'file')}}
logging.config.dictConfig(LOG_CONFIG)

When I try to run logging.debug("Some string"), I get no output to the console, even though this page in the docs says that logging.debug should have the root logger output the message. Why is my program not outputting anything, and how can I fix it?

Improve this question
0

11 Answers 11

Reset to default
398

Many years later there seems to still be a usability problem with the Python logger. Here's some explanations with examples:

import logging
# This sets the root logger to write to stdout (your console).
# Your script/app needs to call this somewhere at least once.
logging.basicConfig()

# By default the root logger is set to WARNING and all loggers you define
# inherit that value. Here we set the root logger to NOTSET. This logging
# level is automatically inherited by all existing and new sub-loggers
# that do not set a less verbose level.
logging.root.setLevel(logging.NOTSET)

# The following line sets the root logger level as well.
# It's equivalent to both previous statements combined:
logging.basicConfig(level=logging.NOTSET)


# You can either share the `logger` object between all your files or the
# name handle (here `my-app`) and call `logging.getLogger` with it.
# The result is the same.
handle = "my-app"
logger1 = logging.getLogger(handle)
logger2 = logging.getLogger(handle)
# logger1 and logger2 point to the same object:
# (logger1 is logger2) == True

logger = logging.getLogger("my-app")
# Convenient methods in order of verbosity from highest to lowest
logger.debug("this will get printed")
logger.info("this will get printed")
logger.warning("this will get printed")
logger.error("this will get printed")
logger.critical("this will get printed")


# In large applications where you would like more control over the logging,
# create sub-loggers from your main application logger.
component_logger = logger.getChild("component-a")
component_logger.info("this will get printed with the prefix `my-app.component-a`")

# If you wish to control the logging levels, you can set the level anywhere 
# in the hierarchy:
#
# - root
#   - my-app
#     - component-a
#

# Example for development:
logger.setLevel(logging.DEBUG)

# If that prints too much, enable debug printing only for your component:
component_logger.setLevel(logging.DEBUG)


# For production you rather want:
logger.setLevel(logging.WARNING)

A common source of confusion comes from a badly initialised root logger. Consider this:

import logging
log = logging.getLogger("myapp")
log.warning("woot")
logging.basicConfig()
log.warning("woot")

Output:

woot
WARNING:myapp:woot

Depending on your runtime environment and logging levels, the first log line (before basic config) might not show up anywhere.

Improve this answer
Sign up to request clarification or add additional context in comments.

12 Comments

My logging isn't working, in that it produces no output file. Do you see anything I'm doing that is clearly wrong?logging.basicConfig( filename='logging.txt', level=logging.DEBUG) logger = logging.getLogger() logger.info('Test B') logging.info('Test A')
I noticed when I drop a break point after logger = logging.getLogger(), the level is set to WARNING even though I specified the level as DEBUG. Do you know what I'm doing wrong?
Note that if you are using Matplotlib and run import matplotlib.pyplot as plt after the logging import but before the rest of the code (which would follow pep8), the Matplotlib import borks the logging configuration and none of the logging messages are shown.
every few weeks I'll forget this and land up back at this answer
Why on earth is 'logging.basicConfig()' not just called automatically when the logger is instantiated? That blows my mind a little - would be a big improvement from a usability perspective.
|
198

The default logging level is warning. Since you haven't changed the level, the root logger's level is still warning. That means that it will ignore any logging with a level that is lower than warning, including debug loggings.

This is explained in the tutorial:

import logging
logging.warning('Watch out!') # will print a message to the console
logging.info('I told you so') # will not print anything

The 'info' line doesn't print anything, because the level is higher than info.

To change the level, just set it in the root logger:

'root':{'handlers':('console', 'file'), 'level':'DEBUG'}

In other words, it's not enough to define a handler with level=DEBUG, the actual logging level must also be DEBUG in order to get it to output anything.

Improve this answer

11 Comments

The documentation says thats its default level is NOTSET which is a level of 0 which should output everything... Why is this not true?
@Ben according to the docs the loggers are traversed to find the first parent with level != NOTSET or the root (if none is found). The root has WARNING level by default. This is written in the section you've linked to (Logger.setLevel).
Keep in mind that after importing logging you need to call logging.basicConfig() at least once. Otherwise you might be badly surprised that child loggers will not print anything. Logging functions on the root logger call it lazily.
What do you mean by set it in root logger 'root':{'handlers':('console', 'file'), 'level':'DEBUG'} ?
This is not what the documentation (not tutorial!) says! The documentation says to just use e.g. logging.basicConfig(level=logging.DEBUG). Then logging.debug(...) is supposed to be printed (They also show what is printed). Well, not in my case either! As for 'root':{'handlers':('console', 'file'), 'level':'DEBUG'} ... How and where is this used??? You can't throw in a thing like this without an example of its application! I really wonder how such a bad and useless answer got so many upvotes! It should be downvoted instead! (I didn't, because I don't like that.)
|
149

For anyone here that wants a super-simple answer: just set the level you want displayed. At the top of all my scripts I just put:

import logging
logging.basicConfig(level = logging.INFO)

Then to display anything at or above that level:

logging.info("Hi you just set your fleeb to level plumbus")

It is a hierarchical set of five levels so that logs will display at the level you set, or higher. So if you want to display an error you could use logging.error("The plumbus is broken").

The levels, in increasing order of severity, are DEBUG, INFO, WARNING, ERROR, and CRITICAL. The default setting is WARNING.

This is a good article containing this information expressed better than my answer:
https://www.digitalocean.com/community/tutorials/how-to-use-logging-in-python-3

Improve this answer

3 Comments

Add execution time using logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s: %(message)s')
I defined in a separate script a simple function that does both actions above (setting the logging level to info and the formatting of the message). The function is then used in the main script, just after the declaration id __name__ == "__main__". But that does not work, basic configurations are not properly set. Where can be the problem?
@mattiatantardini configuring across modules can be tricky: stackoverflow.com/questions/15727420/…
53

This problem wasted me so much time, so I'll just invest some more to write an answer and save yours

Problem

Cannot set logging level for custom loggers. (e.g: to DEBUG level)

What DOESN'T work

Setting logging level to the handler.

import logging

# Get logger
logger = logging.getLogger("my logger")

# Create a handler and set logging level for the handler
c_handler = logging.StreamHandler()
c_handler.setLevel(logging.DEBUG) # <- Here things went wrong

# link handler to logger
logger.addHandler(c_handler)

# test
logger.debug('This is a debug message') # WILL NOT WORK

SOLUTION

Set the logging level via the logger object (instead of the handler) Customlogger.setLevel(logging.DEBUG)

import logging

# Get logger
logger = logging.getLogger("my logger")

# Create a handler
c_handler = logging.StreamHandler()

# link handler to logger
logger.addHandler(c_handler)

# Set logging level to the logger
logger.setLevel(logging.DEBUG) # <-- THIS!

# test
logger.debug('This is a debug message') # WILL WORK
Improve this answer

3 Comments

Hmm... what about actually having different log levels for handlers? I'd like console log to be INFO and file log to be DEBUG
Creating the handler is the thing that I was missing. Thank you!
You need to set the default log level of the logger object also along with the handler level. log level set at logger will be the base level and can give higher log level to handlers
26

Maybe try this? It seems the problem is solved after remove all the handlers in my case.

for handler in logging.root.handlers[:]:
    logging.root.removeHandler(handler)

logging.basicConfig(filename='output.log', level=logging.INFO)
Improve this answer

5 Comments

SyntaxError: invalid syntax
Why is this necessary? What handlers come stock with the python logger and why are they there to begin with? Or maybe the question is, why doesn't basicConfig override them / replace them?
@yue dong, this has worked for me as well but can you please explain more about this ? I mean why this is required?
@jrh Perhaps the behavior was different for earlier versions of python but for 3.8, an unconfigured root logger will have 0 handlers by default (check by printing logging.root.handlers -- it will be an empty list). Calling logging.basicConfig() adds a StreamHandler. So, the clearing of handlers is redundant.
@Rimov can confirm it's empty in Linux with Python 3.8 (inside or outside of Spyder), I do remember it not being empty when I tried it a few years ago. Unfortunately I don't remember what version of Python that was, or what platform, probably Windows, maybe Python 2.7, maybe Spyder.
9

That simply works fine for me ...

import logging

LOGGER = logging.getLogger("my-fetcher")
logging.basicConfig(level=logging.INFO)

LOGGER.info("Established Connection Successfully!")
# > INFO:my-fetcher:Established Connection Successfully!
Improve this answer

Comments

8

Nothing of the above worked for me when executing locally on my python environment, the following worked though,

import logging

logging.basicConfig()
logger = logging.getLogger()
logger.setLevel(logging.INFO)

in my case I was missing the logging.basicCofig()

Improve this answer

1 Comment

Wow, the whole page docs.python.org/3.10/library/logging.html does not contain the word baseConfig. :facepalm:
6

import logging
log = logging.getLogger()
log.setLevel(logging.DEBUG)

this code will set the default logging level to DEBUG.

Improve this answer

4 Comments

No, it will not.
I'm not sure why this answer has been marked down, as it works. In fact, it's the way that AWS document the usage of logging within a Lambda - docs.aws.amazon.com/lambda/latest/dg/…
This won't work as last resort (docs.python.org/3/library/logging.html#logging.lastResort) handler will be invoked, effectively using WARNING level.
this solution is the correct and simple one for my problem. 2022-07-26
3

The key point is that you must set the logging level on a logging handler object, as well as the logger object itself:

Here's an example setup. (You place this somewhere at the top of your executable Python script, outside of def main() / if __name__ == '__main__'.)

logger = logging.getLogger(__name__)

stdout_log_formatter = logging.Formatter(
    '%(name)s: %(asctime)s | %(levelname)s | %(filename)s:%(lineno)s | %(process)d | %(message)s'
)

stdout_log_handler = logging.StreamHandler(stream=sys.stdout)
stdout_log_handler.setLevel(logging.INFO)
stdout_log_handler.setFormatter(stdout_log_formatter)

logger.addHandler(stdout_log_handler)
logger.setLevel(logging.INFO)

I have included a couple of extras like a format specification for your convenience.

Improve this answer

Comments

0

Calling removeHandler() function leaves stdout/stderr output even though all handlers have been removed.

one way to cleanup a logger is to empty the list of handlers, i.e. logger.handlers = [] or logger.root.handlers = []

This worked for me.

Improve this answer

Comments

0

I was also using the root logger and not seeing any logs, i.e.

import logging

.. other code

logging.debug("Hello World")

I tried setting the basicConfig but I found the only way it would work, was to use force = True.

import logging

logging.basicConfig(
    level=logging.DEBUG,
    force=True
)

.. other code

logging.debug("Hello World")

https://docs.python.org/3/library/logging.html#logging.basicConfig

force. If this keyword argument is specified as true, any existing handlers attached to the root logger are removed and closed, before carrying out the configuration as specified by the other arguments.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.