How can I time-profile unittests in Python?

Question

I have a test suite and code it is testing. If I put from memory_profiler import profile at the tops of the appropriate files, decorate the functions I want profiled with @profile, and run in the standard way with python TestThing.py, I get great line-by-line results.

But line_profiler doesn't come in a package this way, and the only way I have been able to profile anything with it is with kernprof -l -v thing.py. If I use it on my unit tests, none of the tests are run (unsurprising, really), and no results are generated. How can I time-profile my tests and the code they use in a way analogous to memory_profiler?

georgexsh · Accepted Answer · 2017-10-04 18:51:42Z

3

of course, you could import line_profiler:

import line_profiler

profiler = line_profiler.LineProfiler()


@profiler
def foo():
    for i in range(10):
        print(i)


foo()

profiler.dump_stats('foo.lprof')

result stored in foo.lprof.

or, you could wrap a memory_profiler like decorator, print result after calling:

def profile_and_show(func):
    profiler = line_profiler.LineProfiler(func)
    def _(*args, **kwargs):
        try:
            return func(*args, **kwargs)
        finally:
            lstats = profiler.get_stats()
            line_profiler.show_text(lstats.timings, lstats.unit)
    return _

edited Oct 4, 2017 at 18:51

answered Oct 3, 2017 at 8:49

georgexsh

16.8k3 gold badges45 silver badges71 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Pavel Komarov Over a year ago

A modification of this is working: import line_profiler, then declare a static profiler = line_profiler.LineProfiler for the class. Then decorate functions with @profiler. Then call ClassName.profiler.dump_stats('method.lprof') at the ends of those functions. Then read them with python -m line_profiler method.lprof. But isn't this goofy? I don't want to add that many lines to my code that I'll have to remove later. Is there a cleaner way?

georgexsh Over a year ago

which part is not clean?

Pavel Komarov Over a year ago

For memory_profiler I don't need to instantiate an object or worry about calling its dump_stats method some time after a profiled function has been run. I can work with this method of time profiling, but it's more work than it needs to be.

georgexsh Over a year ago

@pvlkmrv you can always wrap around, I've written an example above.

tin_nqn · Accepted Answer · 2021-05-19 04:19:27Z

0

Please check pytest-line-profiler.

You only need to decorate (mark) one or more tests with the functions to be profiled, and will get the line-by-line report at the end.

PS: I'm the author of this plugin and I'd really appreciate your feedback.

answered May 19, 2021 at 4:19

tin_nqn

1064 bronze badges

Collectives™ on Stack Overflow

How can I time-profile unittests in Python?

2 Answers 2

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related