Test execution of callbacks in async or multi-threaded code

Question

I need to check that certain operations have occurred in a particular order in threaded/asynchronous code. Something along the lines of:

def test_threaded_stuff():
    # I can define the callbacks to the operation
    op = start_operation(callback1, callback2, cbargs...)
    op.checkpoint1.wait()
    # check that callback1 and callback2 have been invoked,
    # in that order
    op.checkpoint2.wait()
    # check that they have been invoked again, in reverse order

I can provide test callbacks that will be invoked by the operation, but I cannot place py.test assertions inside them because I need to test the overall order of their execution, not the state of any individual callback. Also, some of the callbacks are executed in separate threads which are not under the control of py.test.

To test such code, I came up with the following pattern:

def callback1(log):
    log(1)

def callback2(log):
    log(2)

def test_threaded_stuff():
    events = []
    op = start_operation(cb1, cb2, events.append)
    op.checkpoint1.wait()
    assert events == [1, 2]
    op.checkpoint2.wait()
    assert events == [1, 2, 2, 1]

Is there an idiomatic way to express this in py.test? For example, a callable fixture that automatically logs its invocations, so that I can query the invocations in my tests.

If concrete examples are needed, this file is an example, as are other files in the same directory.

user4815162342 · Accepted Answer · 2018-03-19 07:48:01Z

1

It might not be necessary for pytest to have a specific functionality as I think, the standard Python unittest module would suffice.

You can make use of Mock objects that track calls to themselves as well as to methods and attributes, reference.

You can combine it with assert_has_calls() by building the list of calls you expect and want to test. It also allows to test for the specific order of the calls by default through the any_order=False param.

So by patching your module adequately and passing Mock objects instead of callbacks in your tests you will mostly be able to create your tests.

edited Mar 19, 2018 at 7:48

user4815162342

159k22 gold badges350 silver badges418 bronze badges

answered Mar 11, 2018 at 19:56

Enrique Saez

2,73018 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user4815162342 Over a year ago

Mock objects are pretty nice, thanks. In my particular case, where I want to assert that exactly specific calls occurred, using Mock doesn't seem to result in less code. In the example from the question, the assertion would need to be either log.call_args_list == [((1,),), ((2,),), ((2,),), ((1,),)]` or to log.call_args_list == [call(1), call(2), call(3)]`, both being more verbose than what I have now.

Collectives™ on Stack Overflow

Test execution of callbacks in async or multi-threaded code

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related