python 2.4 logging - dynamic log file name inside loop

Question

I'm trying to write to a separate log file which is created inside a loop using basicConfig. So far I managed to create only one log file which has all the subsequent logging calls. Am I missing something trivial here? I checked the docs it had hints about filemode but nothing else.

import logging


class MultipleInstancesStarter:
    def __init__(self):
        pass


    def startMany(self):
        objs = [Main(str(i)) for i in xrange(1, 10)]
        print objs
        for obj in objs:
            obj._start()


class Main:
    def __init__(self, i):
        self.index = i


    def _start(self):
        name = self.index
        logging.basicConfig(
                        filename="log_" + name + ".log",
                        filemode="w",
                        format="%(asctime)s -  %(levelname)s - %(filename)s:%(module)s:%(lineno)d - %(message)s",
                        level=logging.DEBUG)

        logging.debug("%s Message Debug" % name)
        logging.info("%s Message Info" % name)
        logging.error("%s Message Error" % name)


if __name__ == '__main__':
    MultipleInstancesStarter().startMany()

This creates only log_1.log with log entries from 2nd iteration as well. I tried creating an object outside the loop and it creates a separate file no problem though as shown below.

def startMany(self):
    obj2 = Main("sample")
    obj2._start()

    objs = [Main(str(i)) for i in xrange(1, 10)]
    print objs
    for obj in objs:
        obj._start()

Not sure what I'm doing wrong though, any help would be appreciated. Cheers

[<__main__.Main instance at 0xb777056c>, <__main__.Main instance at 0xb7770a8c>, <__main__.Main instance at 0xb7770b4c>, <__main__.Main instance at 0xb7770bec>, <__main__.Main instance at 0xb777918c>, <__main__.Main instance at 0xb77794ac>, <__main__.Main instance at 0xb7523cec>, <__main__.Main instance at 0xb7523d0c>, <__main__.Main instance at 0xb7523d2c>] - they all look like different references — opensourcegeek
– opensourcegeek, Commented Nov 11, 2013 at 15:24
My understanding is that basicConfig must be called only once. Once it is configured, the logging module will have handlers, and the doc says This function does nothing if the root logger already has handlers configured for it.. You probably need to call logging.shutdown after your logging to a file is done, in order to flush the handlers and allow logging to be reconfigured with an other file. — njzk2
– njzk2, Commented Nov 11, 2013 at 15:29
(bottom line: call logging.shutdown() as the last line of _start) — njzk2
– njzk2, Commented Nov 11, 2013 at 15:30
Hmmm that didn't do anything either. I still get all the logging calls in one file. I can use some other config if basicConfig won't do the job. Cheers — opensourcegeek
– opensourcegeek, Commented Nov 11, 2013 at 15:36

Paco · Accepted Answer · 2013-11-13 09:43:51Z

This will work:

import logging


class MultipleInstancesStarter:
    def __init__(self):
        pass

    def startMany(self):
        objs = [Main(str(i)) for i in xrange(1, 10)]
        print objs
        for obj in objs:
            obj._start()


class Main:
    def __init__(self, i):
        self.index = i

    def _start(self):
        name = self.index
        logger = logging.getLogger('%s-%s' % (__name__, self.index))
        handler = logging.FileHandler('log_%s.log' % name)
        log_format = ('%(asctime)s -  %(levelname)s - %(filename)s:'
                      '%(module)s:%(lineno)d - %(message)s')
        formatter = logging.Formatter(log_format)
        handler.setFormatter(formatter)
        logger.addHandler(handler)

        logger.debug("%s Message Debug" % name)
        logger.info("%s Message Info" % name)
        logger.error("%s Message Error" % name)


if __name__ == '__main__':
    MultipleInstancesStarter().startMany()

This it the output (I used tail on the log files):

==> log_1.log <== 2013-11-12 21:08:15,866 - ERROR - test.py:test:31 - 1 Message Error

==> log_2.log <== 2013-11-12 21:08:15,866 - ERROR - test.py:test:31 - 2 Message Error

==> log_3.log <== 2013-11-12 21:08:15,866 - ERROR - test.py:test:31 - 3 Message Error

==> log_4.log <== 2013-11-12 21:08:15,866 - ERROR - test.py:test:31 - 4 Message Error

==> log_5.log <== 2013-11-12 21:08:15,866 - ERROR - test.py:test:31 - 5 Message Error

==> log_6.log <== 2013-11-12 21:08:15,866 - ERROR - test.py:test:31 - 6 Message Error

==> log_7.log <== 2013-11-12 21:08:15,867 - ERROR - test.py:test:31 - 7 Message Error

==> log_8.log <== 2013-11-12 21:08:15,867 - ERROR - test.py:test:31 - 8 Message Error

==> log_9.log <== 2013-11-12 21:08:15,867 - ERROR - test.py:test:31 - 9 Message Error

Basically, if the logging part becomes a bit tricky, use different loggers, one for each instance in that case. A good article to read on Python documentation

EDIT: I forgot to add the log level. To do so, you can add this before adding the handler to the logger

handler.setLevel(logging.DEBUG)

Collectives™ on Stack Overflow

python 2.4 logging - dynamic log file name inside loop

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related