How can a create a log file to log every steps in python?

Question

I am running a python script and I want to log each steps of the python program.

for example, lets say I have a program

def main()
    if do:
       do the work
       if yes:
          do the work
          for list os.listdir(dir):
              sys.system("python " + dirOfPython + " " + dirOflists)

I want to log where my program is.. and what they are currently doing.

In my log I want something like

Inside of main()
inside of if do:
doing work
inside of if yes:
doing work
inside of for each files/dirs calling list
calling sys.system python
executing python with give dir path

not exactly what is above.. but some sort of log to see what the program is doing and if it fails this log will contain where if failed log with error message

I just want a formal log file

@Jeff I am not sure if this is the right example.. but something like nohup in linux.. where it records all the log in nohup.out. All I want to do is log each step of the coding in some output file.. it could txt or something else. — PETER
– PETER, Commented Mar 18, 2015 at 18:36
You say fail like a backtrace ? Or you want to check were it fails ? ( after many iterations ) — Cristiano Araujo
– Cristiano Araujo, Commented Mar 18, 2015 at 19:22

Adam Smith · Accepted Answer · 2015-03-18 19:29:02Z

3

You know, there is a logging module!

import logging
import os

logging.basicConfig(filename='tmp.log',
                    format='%(levelname)s %(asctime)s :: %(message)s',
                    level=logging.DEBUG)
# format is a formatter string, level shows what level of logs it will record
# in this case it is everything!
# Levels are as follows from most to least critical
#   CRITICAL
#   ERROR
#   WARNING
#   INFO
#   DEBUG

do = True
yes = True
do_the_work = lambda: None

def main():
    logging.debug("Inside of main()")
    if do:
        logging.debug("Inside of if do:")
        do_the_work()
        logging.debug("doing work")
        if yes:
            logging.debug("inside of if yes:")
            do_the_work()
            logging.debug("doing work")
            for list in os.listdir('.'): # there were three files in my folder
                logging.debug("inside of for each files/dirs calling list")
                print('python')
                logging.debug("calling sys.system python")
                logging.debug("executing python with give dir path")

Which will produce an output of:

DEBUG 2015-03-18 12:26:59,272 :: Inside of main()
DEBUG 2015-03-18 12:26:59,272 :: Inside of if do:
DEBUG 2015-03-18 12:26:59,272 :: doing work
DEBUG 2015-03-18 12:26:59,272 :: inside of if yes:
DEBUG 2015-03-18 12:26:59,272 :: doing work
DEBUG 2015-03-18 12:26:59,272 :: inside of for each files/dirs calling list
DEBUG 2015-03-18 12:26:59,272 :: calling sys.system python
DEBUG 2015-03-18 12:26:59,272 :: executing python with give dir path
DEBUG 2015-03-18 12:26:59,272 :: inside of for each files/dirs calling list
DEBUG 2015-03-18 12:26:59,272 :: calling sys.system python
DEBUG 2015-03-18 12:26:59,272 :: executing python with give dir path
DEBUG 2015-03-18 12:26:59,272 :: inside of for each files/dirs calling list
DEBUG 2015-03-18 12:26:59,272 :: calling sys.system python
DEBUG 2015-03-18 12:26:59,272 :: executing python with give dir path

You can easily catch exceptions and have them throw more critical events.

try:
    really_important_method()
except EndOfTheWorldError:
    logging.critical("Duck and cover boys, it's gonna blow.")

edited Mar 18, 2015 at 19:29

answered Mar 18, 2015 at 19:17

Adam Smith

54.6k13 gold badges84 silver badges120 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

PETER Over a year ago

where does the log get saved? does it store the output in some sort of txt file or? or is it just printing on the screen?

Adam Smith Over a year ago

logging.basicConfig(filename='path/to/the/log/file.txt', ...)

PETER Over a year ago

is there a way that you dont have to keep writing logging.debug(....) ? it just logs where it is and what it is doing?.. also.. when I run my original program it outputs some line.. how can I store than into a log? I tried your coding.. it doesnt store the output.

Adam Smith Over a year ago

@PETER How would your code possibly know on its own what it's supposed to be logging? Yes, you should be explicitly logging anything you feel needs logged. You can probably do some fancy complex redirection of stdout to tee to both stdout and a StringIO that will dump to the logger, but that seems...tiresome. Ultimately it's your job as a developer to know what needs to be logged, how it needs to be logged, and then to log it appropriately

Rama Over a year ago

@Adam Smith, can you suggest configuration in java logging class .?

|

Anand Tripathi · Accepted Answer · 2017-11-13 06:46:25Z

I think in program you need to know what is executing currently and what is the flow of the program and where it is breaking, this is where logs come into picture. For this you can use this pip library function-logger.

(INSTALLING) Install it with the following command:

pip install function-logger

(IMPORT STATEMENT) Inside your code import this library

from function_logger import function_logger

(USAGE) how to use Use it like the decorator on any function for which you want to see logs ex:

@function_logger(logger)
def log_function(name=None, age=None):
    logger.debug("Inside log function")
    return dict(bmi=19)

Output:

Inside Function log_function with parameters: (),{'name': 'hello', 'age': 23'}
Inside log function
Function: log_function returns 19

Collectives™ on Stack Overflow

How can a create a log file to log every steps in python?

2 Answers 2

6 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related