Duplicate messages while logging in Python

Question

I think I've tried just about any combination and I keep hitting the wall with regards to logging. I've been trying to get my head around the Logging cookbook and various posts mentioning the same issue, but I just don4t get it. My code is fairly simple:

Main module (main.py):

#my modules
import manager
import metadata

import logging
import logging.config

if config_file:
    logging.config.fileConfig(config_file)
else:
    logging.basicConfig(level=logging.INFO)

Then in both modules (manager.py and metadata.py) that are imported in the main.py module:

import logging
logger = logging.getLogger(__name__)
logger.addHandler(logging.NullHandler())

Finally, the logger config file:

[loggers]
keys=root, manager, metadata

[logger_root]
level=NOTSET
handlers=console
qualname=root

[logger_manager]
level=DEBUG
handlers=console
qualname=manager

[logger_metadata]
level=DEBUG
handlers=console
qualname=metadata

If I now run the main.py, I will end up with only logging for root.

On the other hand, if I change my code as follows:

manager.py:

import logging
logger = logging.getLogger('manager')
logger.addHandler(logging.NullHandler())

metadata.py:

import logging
logger = logging.getLogger('metadata')
logger.addHandler(logging.NullHandler())

I'll get duplicate logging for manager and metadata.

Any hints would be greatly appreciated.

oxyum · Accepted Answer · 2014-02-04 15:24:13Z

1

For the first part of your question, check the real value of _name_ variable. As it depends on file location and import method, it can be not what you expect:

import logging
print __name__
logger = logging.getLogger(__name__)
logger.addHandler(logging.NullHandler())

For the second part:

From Logging HOWTO:

Child loggers propagate messages up to the handlers associated with their ancestor loggers. Because of this, it is unnecessary to define and configure handlers for all the loggers an application uses. It is sufficient to configure handlers for a top-level logger and create child loggers as needed. (You can, however, turn off propagation by setting the propagate attribute of a logger to False.)

So your config should looks like:

[loggers]
keys=root, manager, metadata

[logger_root]
level=NOTSET
handlers=console
qualname=root

[logger_manager]
level=DEBUG
handlers=console
qualname=manager
propagate=0

[logger_metadata]
level=DEBUG
handlers=console
qualname=metadata
propagate=0

or like:

[loggers]
keys=root, manager, metadata

[logger_root]
level=NOTSET
handlers=console
qualname=root

[logger_manager]
level=DEBUG
qualname=manager

[logger_metadata]
level=DEBUG
qualname=metadata

answered Feb 4, 2014 at 15:24

oxyum

6,9071 gold badge17 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

DocZerø Over a year ago

I added a different answer, but you're quite right; this answer allows you to keep the separation in loggers.

Community · Accepted Answer · 2017-05-23 10:31:42Z

0

Found the answer here: Python logging: different behavior between using fileconfig and programmatic config.

Basically, I had to add disable_existing_loggers=False to the logging.config.fileConfig call. Thus, the code that worked for me is:

if config_file:
    logging.config.fileConfig(config_file, disable_existing_loggers=False)
else:
    logging.basicConfig(level=logging.INFO)

This also allowed me to simplify the logging configuration to just the root logger:

[loggers]
keys=root

[logger_root]
level=DEBUG
handlers=console
qualname=root

edited May 23, 2017 at 10:31

CommunityBot

11 silver badge

answered Feb 4, 2014 at 15:24

DocZerø

8,61511 gold badges44 silver badges75 bronze badges

Collectives™ on Stack Overflow

Duplicate messages while logging in Python

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related