How to avoid duplicate outputs using logging module

Question

I am using the logging module to output what used to be print statements in my script to both the console and a log file. However, each time I run the script, the output seems to append to what was printed in the console and the log, rather than overwrite what was there. For example, the first time I run the script I get this output in my console and log file:

This print statement is going to both the console and the log
The year is: 2015

The second time I run the script, I get this in the console:

This print statement is going to both the console and the log
This print statement is going to both the console and the log
The year is: 2015
The year is: 2015

The third time:

This print statement is going to both the console and the log
This print statement is going to both the console and the log
This print statement is going to both the console and the log
The year is: 2015
The year is: 2015
The year is: 2015

etc.. and the log file keeps appending so that I end up with:

This print statement is going to both the console and the log
The year is: 2015
This print statement is going to both the console and the log
This print statement is going to both the console and the log
The year is: 2015
The year is: 2015
This print statement is going to both the console and the log
This print statement is going to both the console and the log
This print statement is going to both the console and the log
The year is: 2015
The year is: 2015
The year is: 2015

What I want is for both the log file and the console to only give me the following, no matter how many times I rerun the script:

This print statement is going to both the console and the log
The year is: 2015

Note: the only way I can currently get what I want is to shut down Python in between runs of the script, but that is not a practical solution.

I have experimented with using flush() and have tried doing something like this:

logger = logging.basicConfig(filemode='w', level=logging.DEBUG)

but I have not been able to get it to work with my code. Can someone please help? Here is my code:

import inspect
import logging
import datetime

def getDate():
    now = datetime.datetime.now()
    m = now.month
    d = now.day
    y = str(now.year)
    if m < 10:
        m = "0"+str(m)
    else:
        m = str(m)
    if d < 10:
        d = "0"+str(d)
    else:
        d = str(d)
    y = y[2:]
    formatted_date = m+d+y
    return formatted_date

def function_logger(file_level, console_level = None):
    function_name = inspect.stack()[1][3]
    logger = logging.getLogger(function_name)
    logger.setLevel(logging.DEBUG) #By default, logs all messages

    if console_level != None:
        ch = logging.StreamHandler() #StreamHandler logs to console
        ch.setLevel(console_level)
        ch_format = logging.Formatter('%(message)s')
        ch.setFormatter(ch_format)
        logger.addHandler(ch)

    log_name = 'Test_log' + getDate() + '.log'
    fh = logging.FileHandler(log_name.format(function_name))
    fh.setLevel(file_level)
    fh_format = logging.Formatter('%(message)s')
    fh.setFormatter(fh_format)
    logger.addHandler(fh)
    return logger

def f1():
    global logger
    year = '2015'
    logger.info("The year is: %s" % (year))

def main():

    global logger
    logger = function_logger(logging.INFO, logging.INFO)
    logger.info('This print statement is going to both the console and the log')
    f1()
    logging.shutdown()

if __name__ == '__main__':
    main()

You're seeing this because every handler will emit a record once, and if you run your script multiple times, you will register a new handler with every addHandler() call. Either keep track of your logging handlers and remove them again in a try..finally, or check if there already is one registered for the type you're about to add before adding them. — Lukas Graf
– Lukas Graf, Commented Feb 5, 2015 at 18:28
(Print logger.handlers during each run to see what's going on) — Lukas Graf
– Lukas Graf, Commented Feb 5, 2015 at 18:33
Thank you. Placing both of my addHandlers under if not logger.handlers: solved the problem of the duplicate entries. However, there is a secondary problem relating only to the log file in that the log is still appended to instead of overwritten. How can I solve this remaining log problem? — Chris Nielsen
– Chris Nielsen, Commented Feb 5, 2015 at 19:22
If you look at FileHandler's constructor, it's mode defaults to 'a' (append). So use FileHandler(log_name.format(function_name), mode='w') to overwrite the existing file instead of appending (semantics for mode are the same as with open()). — Lukas Graf
– Lukas Graf, Commented Feb 5, 2015 at 21:50

Chris Nielsen · Accepted Answer · 2015-02-09 19:54:34Z

Thanks to Lucas Graf, here is how I fixed the code to do exactly what I wanted:

def function_logger(file_level, console_level = None):
    function_name = inspect.stack()[1][3]
    logger = logging.getLogger(function_name)
    logger.setLevel(logging.DEBUG) #By default, logs all messages
    log_name = 'Test_log_' + getDate() + '.log'

    if not logger.handlers:
        if console_level != None:
            ch = logging.StreamHandler() #StreamHandler logs to console
            ch.setLevel(console_level)
            ch_format = logging.Formatter('%(message)s')
            ch.setFormatter(ch_format)
            logger.addHandler(ch)

        fh = logging.FileHandler(log_name.format(function_name), mode='w')
        fh.setLevel(file_level)
        fh_format = logging.Formatter('%(message)s')
        fh.setFormatter(fh_format)
        logger.addHandler(fh)

    return logger

There are three things to note about this fix:

1: I moved both addHandlers under the test condition:

if not logger.handlers:

2: I added mode='w' to the Filhandler:

fh = logging.FileHandler(log_name.format(function_name), mode='w')

3: I cleared the handlers at the bottom of main():

logger.handlers = []

Jean Castro · Accepted Answer · 2020-07-09 00:07:49Z

0

Just do that:

def LOG_insere(file, format, text, level):
    infoLog = logging.FileHandler(file)
    infoLog.setFormatter(format)
    logger = logging.getLogger(file)
    logger.setLevel(level)
    
    if not logger.handlers:
        logger.addHandler(infoLog)
        if (level == logging.INFO):
            logger.info(text)
        if (level == logging.ERROR):
            logger.error(text)
        if (level == logging.WARNING):
            logger.warning(text)
    
    infoLog.close()
    logger.removeHandler(infoLog)
    
    return

answered Jul 9, 2020 at 0:07

Jean Castro

3312 silver badges7 bronze badges

Collectives™ on Stack Overflow

How to avoid duplicate outputs using logging module

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related