I have been coding a lot in Python of late. And I have been working with data that I haven't worked with before, using formulae never seen before and dealing with huge files. All this made me write a lot of print statements to verify if it's all going right and identify the points of failure. But, generally, outputting so much information is not a good practice. How do I use the print statements only when I want to debug and let them be skipped when I don't want them to be printed?
-
Related: How to step through Python code to help debug issues?wjandrea– wjandrea2024-09-30 20:05:26 +00:00Commented Sep 30, 2024 at 20:05
Add a comment
|
6 Answers
The logging module in the standard library has everything you could want. It may seem excessive at first, but only use the parts you need. I'd recommend using logging.basicConfig to set the logging level then using the simple log methods: debug, info, warning, error and critical.
import logging
logging.basicConfig(level=logging.DEBUG)
logging.debug('A debug message!')
logging.info('We processed %d records', len(processed_records))
6 Comments
Eduardo Pignatelli
How does one set the level of the logging such that it only prints errors and not debug messages?
Matt Joiner
@EduardoPignatelli set
level, in the basicConfig call, to logging.ERROR.Eduardo Pignatelli
I am afraid this does not work on jupyter lab 1.2.6. You can set the logging level once, and re-setting is using
logging.basicConfig(stream=sys.stderr, level=logging.ERROR) will have no effect. Restarting the kernel and setting the new level works, but that's a workaround for me.Matt Joiner
@EduardoPignatelli you should ask another question for this. But likely you'll need to directly change the level on the root logger, jupyter is probably calling basicConfig before you.
smcs
@MattJoiner Correct, use
logging.getLogger().setLevel(logging.ERROR) in Jupyter |
A simple way to do this is to call a logging function:
DEBUG = True
def log(s):
if DEBUG:
print s
log("hello world")
Then you can change the value of DEBUG and run your code with or without logging.
The standard logging module has a more elaborate mechanism for this.
5 Comments
Greg Hewgill
True, but it's worthwhile understanding how one could roll their own.
mgiuca
Indeed. The above is a good idea of how
logging works (at a very simple level).
crsuarezf
This is the one that I use for my aws lambdas.
Warf
I would replace
s with *s so you can pass comma delimited values like in print() function.Use the logging built-in library module instead of printing.
You create a Logger object (say logger), and then after that, whenever you insert a debug print, you just put:
logger.debug("Some string")
You can use logger.setLevel at the start of the program to set the output level. If you set it to DEBUG, it will print all the debugs. Set it to INFO or higher and immediately all of the debugs will disappear.
You can also use it to log more serious things, at different levels (INFO, WARNING and ERROR).
Comments
First off, I will second the nomination of python's logging framework. Be a little careful about how you use it, however. Specifically: let the logging framework expand your variables, don't do it yourself. For instance, instead of:
logging.debug("datastructure: %r" % complex_dict_structure)
make sure you do:
logging.debug("datastructure: %r", complex_dict_structure)
because while they look similar, the first version incurs the repr() cost even if it's disabled. The second version avoid this. Similarly, if you roll your own, I'd suggest something like:
def debug_stdout(sfunc):
print(sfunc())
debug = debug_stdout
called via:
debug(lambda: "datastructure: %r" % complex_dict_structure)
which will, again, avoid the overhead if you disable it by doing:
def debug_noop(*args, **kwargs):
pass
debug = debug_noop
The overhead of computing those strings probably doesn't matter unless they're either 1) expensive to compute or 2) the debug statement is in the middle of, say, an n^3 loop or something. Not that I would know anything about that.
3 Comments
Martin CR
There's more information on this important topic under 'optimisation' in the logging howto: docs.python.org/3/howto/logging.html#optimization
ViRuSTriNiTy
Because while they look similar, the first version incurs the repr() cost even if it's disabled. The second version avoid this. - As dummy I ask myself, where is this documented? What is a repr() cost?
pjz
It's not documented, it's implicit: doing
foo("%r" % bar) "obviously" executes the interpolation (% operator) before passing the result to foo. Hence, even if foo() does nothing with the result, the computational cost of doing the interpolation is still incurred.I don't know about others, but I was used to define a "global constant" (DEBUG) and then a global function (debug(msg)) that would print msg only if DEBUG == True.
Then I write my debug statements like:
debug('My value: %d' % value)
...then I pick up unit testing and never did this again! :)
7 Comments
crazyaboutliv
Unit testing ha. Okay, thats another thing to be picked up then :(
mgiuca
I don't want to discourage unit testing -- it is essential. But I don't think it's a substitute for logging, even as a debugging technique. I still do a lot of printing to quickly test things.
mac
@crazyaboutliv - Unit testing done properly is great. Have a look to this chapter of diving into python for a snappy, concise, easy-to-follow presentation
mac
@mgiuca - I do quick printing too, but it's just a couple or so of
print() while bringing up my code to the required level to pass the test. I never end up with huge amount of print() all over the place. Logging is cool too! :)
culix
@mac It looks like your link now requires an explicit 'www' - it is now hosted here.
|
A better way to debug the code is, by using module clrprint
It prints a color full output only when pass parameter debug=True
from clrprint import *
clrprint('ERROR:', information,clr=['r','y'], debug=True)
1 Comment
Abhay Nainan
Generally good practice to add a disclaimer if you're advocating for software that you've built yourself.