python logging - different level for specific function

Question

i'm trying to reduce the amount of logging that the napalm library sends to syslog, but also allow for info logs to be sent from other parts of the code. I set up logging.basicConfig to be INFO but then i'd like the napalm function to be WARNING and above.

So i have code like this:

from napalm import get_network_driver
import logging
import getpass

logging.basicConfig(
    filename="/var/log/myscripts/script.log", level=logging.INFO, format="%(asctime)s %(message)s")

def napalm(device):
    logging.getLogger().setLevel(logging.WARNING)
    username = getpass.getuser()
    driver = get_network_driver("junos")
    router = driver(str(device), username, "", password="", timeout=120)
    router.open()
    return router

router = napalm('myrouter')
config = "hostname foobar"
router.load_merge_candidate(config=config)
show = router.compare_config()
logging.info(show)

The issue is the logging.info output never makes it to the log file. If i do logging.warning(show) it does, but i'd like this to be info. The reason i want the function to be WARNING is that it generates so much other logging at the info level that is just noise. So trying to cut down on that.

Setting the log level is "sticky". Why not just set back to INFO at the end of the function? — 0x5453
– 0x5453, Commented Jan 29, 2021 at 16:45
Does this answer your question? Turn off logging in schedule library — MisterMiyagi
– MisterMiyagi, Commented Jan 29, 2021 at 16:50
@0x5453 oh, so i could just set logging.getLogger().setLevel(logging.INFO) at the end of the function? — PhillipCres
– PhillipCres, Commented Jan 29, 2021 at 17:04

lllrnr101 · Accepted Answer · 2021-01-29 17:14:20Z

3

A nice trick from the book Effective Python. See if it helps your situation.

def napalm(count):
    for x in range(count):
        logger.info('useless log line')

@contextmanager
def debug_logging(level):
    logger = logging.getLogger()
    old_level = logger.getEffectiveLevel()
    logger.setLevel(level)
    try:
        yield
    finally:
        logger.setLevel(old_level)

napalm(5)
with debug_logging(logging.WARNING):
    napalm(5)
napalm(5)

answered Jan 29, 2021 at 17:14

lllrnr101

2,3432 gold badges6 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Shmuel Kamensky Over a year ago

Since the logger level is shared by all threads, wouldn't this be a bad idea in a multithreaded environment? You may end up logging things you don't mean to, or worse, lose logs you were supposed to capture. EDIT: I see that the behavior is different if you use the root logger (thread unsafe) as opposed to getting a custom logger (seems to be thread safe).

Czaporka · Accepted Answer · 2021-01-30 10:13:30Z

1

By calling logging.getLogger() without a parameter you are currently retrieving the root logger, and overriding the level for it also affects all other loggers. You should instead retrieve the library's logger and override level only for that specific one.

The napalm library executes the following in its __init__.py:

logger = logging.getLogger("napalm")

i.e. the library logger's name is "napalm". You should thus be able to override the level of that specific logger by putting the following line in your script:

logging.getLogger("napalm").setLevel(logging.WARNING)

Generic example:

import logging

logging.basicConfig(level=logging.DEBUG, format="%(levelname)s: %(message)s")
A = logging.getLogger("A")
B = logging.getLogger("B")

A.debug("#1 from A gets printed")
B.debug("#1 from B gets printed")

logging.getLogger("A").setLevel(logging.CRITICAL)

A.debug("#2 from A doesn't get printed")  # because we've increased the level
B.debug("#2 from B gets printed")

Output:

DEBUG: #1 from A gets printed
DEBUG: #1 from B gets printed
DEBUG: #2 from B gets printed

Edit:

Since this didn't work well for you, it's probably because there's a bunch of various other loggers in this library:

$ grep -R 'getLogger' .
napalm/junos/junos.py:log = logging.getLogger(__file__)
napalm/base/helpers.py:logger = logging.getLogger(__name__)
napalm/base/clitools/cl_napalm.py:logger = logging.getLogger("napalm")
napalm/base/clitools/cl_napalm_validate.py:logger = logging.getLogger("cl_napalm_validate.py")
napalm/base/clitools/cl_napalm_test.py:logger = logging.getLogger("cl_napalm_test.py")
napalm/base/clitools/cl_napalm_configure.py:logger = logging.getLogger("cl-napalm-config.py")
napalm/__init__.py:logger = logging.getLogger("napalm")
napalm/pyIOSXR/iosxr.py:logger = logging.getLogger(__name__)
napalm/iosxr/iosxr.py:logger = logging.getLogger(__name__)

I would then resort to something like this (related: How to list all existing loggers using python.logging module):

# increase level for all loggers
for name, logger in logging.root.manager.loggerDict.items():
    logger.setLevel(logging.WARNING)

or perhaps it will suffice if you just print the various loggers, identify the most noisy ones, and silence them one by one.

edited Jan 30, 2021 at 10:13

answered Jan 29, 2021 at 18:16

Czaporka

2,4363 gold badges13 silver badges28 bronze badges

5 Comments

PhillipCres Over a year ago

hmm trying this without luck.. would i need to change what i'm importing from napalm?

Czaporka Over a year ago

@PhillipCres Can you please add the imports to the question? I'll then try to figure out.

PhillipCres Over a year ago

thanks! i added the imports to the question

Czaporka Over a year ago

@PhillipCres I've edited the answer, try out the for loop.

PhillipCres Over a year ago

thanks!! Turned out it was a totally different logger causing all the noise. I was able to set that specific one to WARN using the logging.root.manager.loggerDict.items() approach.

Christian Bates · Accepted Answer · 2024-09-13 15:25:49Z

0

In the napalm documentation it lists the drivers that are being used for each different type of network device it supports https://napalm.readthedocs.io/en/latest/support/index.html

I had the same issue only when attempting to connect to cisco devices running ios and fixed it by setting the logger level to warning for ncclient which is the driver used by napalm under the hood for ios devices, refer to the above link.

logging.getLogger("ncclient").setLevel(logging.WARNING)

answered Sep 13, 2024 at 15:25

Christian Bates

11 bronze badge

Collectives™ on Stack Overflow

python logging - different level for specific function

3 Answers 3

1 Comment

Generic example:

Edit:

5 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Generic example:

Edit:

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related