Python module "logging" double output

Question

I want to use the logging module, but I'm having some trouble because it is outputting twice. I've read a lot of posts with people having the same issue and log.propagate = False or `log.handlers.pop()´ fixed it for them. This doesn't work for me.

I have a file called logger.py that looks like this:

import logging

def __init__():
    log = logging.getLogger("output")
    filelog = logging.FileHandler("output.log")
    formatlog = logging.Formatter("%(asctime)s %(levelname)s %(message)s")
    filelog.setFormatter(formatlog)
    log.addHandler(filelog)
    log.setLevel(logging.INFO)

    return log

So that I from multiple files can write:

import logger
log = logger.__init__()

But this is giving me issues. So I've seen several solutions, but I don't know how to incorporate it in multiple scripts without defining the logger in all of them.

"it is outputting twice": to where and where? To output.log and stderr? In any case yes it looks like you should be setting log.propagate = False in your function. Or (less likely) does each message appear twice in output.log? — BrianO
– BrianO, Commented Sep 2, 2018 at 21:56
It outputs the same message twice to output.log, but I've written an answer to it: stackoverflow.com/a/52141311/9875740 — HelloThereToad
– HelloThereToad, Commented Sep 2, 2018 at 21:58
PS log.propagate = False would guard against every message also being written by the root logger, if the root has a handler. If this is the only configuration you do, that'll never happen so you actually don't have to touch propagate. But if you call logging.log(...), logging.debug(), ... or logging.critical(), they will "helpfully" create a root handler if there aren't any. — BrianO
– BrianO, Commented Sep 2, 2018 at 22:07

HelloThereToad · Accepted Answer · 2018-09-02 21:56:18Z

1

I found a solution which was really simple. All I needed to do was to add an if statement checking if the handlers already existed. So my logger.py file now looks like this:

import logging

def __init__():
    log = logging.getLogger("output")

    if not log.handlers:
        filelog = logging.FileHandler("output.log")
        formatlog = logging.Formatter("%(asctime)s %(levelname)s %(message)s")
        filelog.setFormatter(formatlog)
        log.addHandler(filelog)
        log.setLevel(logging.INFO)

    return log

answered Sep 2, 2018 at 21:56

HelloThereToad

2791 gold badge4 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

user145440 · Accepted Answer · 2018-09-02 20:09:47Z

0

multiple scripts lead to multiple processes. so you unluckily create multiple objects returned from the logger.__init__() function.

usually, you have one script which creates the logger and the different processes, but as i understand, you like to have multiple scripts logging to the same destination.

if you like to have multiple processes to log to the same destination, i recommend using inter process communication as "named pipes" or otherwise using a UDP/TCP port for logging.

There are also queue modules available in python to send a (atomic) logging entry to be logged in one part (compared to append to a file by multiple processes - which may lead from 111111\n and 22222\n to 11212121221\n\n in the file)

otherwise think about named pipes...

Code snippet for logging server Note: i just assumed to log everything as error...

import socket
import logging

class mylogger():
    def __init__(self, port=5005):
    log = logging.getLogger("output")
    filelog = logging.FileHandler("output.log")
    formatlog = logging.Formatter("%(asctime)s %(levelname)s %(message)s")
    filelog.setFormatter(formatlog)
    log.addHandler(filelog)
    log.setLevel(logging.INFO)
    self.log = log
    UDP_IP = "127.0.0.1"  # localhost
    self.port = port
    self.UDPClientSocket = socket.socket(family=socket.AF_INET, type=socket.SOCK_DGRAM)
    self.UDPClientSocket.bind((UDP_IP, self.port))
def pollReceive(self):
    data, addr = self.UDPClientSocket.recvfrom(1024)  # buffer size is 1024 bytes
    print("received message:", data)
    self.log.error( data )
pass

log = mylogger()

while True:
    log.pollReceive()

edited Sep 2, 2018 at 20:09

answered Sep 2, 2018 at 20:00

user145440

1465 bronze badges

3 Comments

BrianO Over a year ago

I could be wrong, but I don't think OP means multiple scripts running concurrently, just that he'd like to use this as a library routine in various scripts.

HelloThereToad Over a year ago

In certain cases yes.

BrianO Over a year ago

Ah, then each time it's called it adds another handler to the logger :) In general you want to configure logging once, only. Shameless self-promo: github.com/Twangist/prelogging, a sensible API for setting up logging.

YBrush · Accepted Answer · 2020-02-17 15:29:13Z

You have to be very carefully by adding new handler to your logger with:

log.addHandler(...)

If you add more than one handler to your logger, you will get more than one outputs. Be aware that this is only truth for logger which are running in the same thread. If you are running a class which is derived from Thread, then it is the same thread. But if you are running a class which is derived from Process, then it is another thread. To ensure that you only have one handler for your logger according to this thread, you should use the following code (this is an example for SocketHandler):

logger_root = logging.getLogger()
if not logger_root.hasHandlers():
    socket_handler = logging.handlers.SocketHandler('localhost', logging.handlers.DEFAULT_TCP_LOGGING_PORT)
    logger_root.addHandler(socket_handler)

Collectives™ on Stack Overflow

Python module "logging" double output

3 Answers 3

Comments

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related