How do I selectively hide no-data-collected when using coverage.py?

Question

I have a test suite that runs a bunch of system tests on an API. Some system tests interact with the API by using Flask's test_client, whereas a few specific tests interact with the API by making HTTP queries to a Gunicorn running the API.

When I use coverage, as soon as it hits those special tests that use Gunicorn, a warning is shown:

CoverageWarning: No data was collected. (no-data-collected)

The warning makes perfect sense—coverage cannot track the actual coverage in tests when the tested API runs through Gunicorn, instead of being called directly. However, it's annoying to see this warning at every test run.

How do I hide it for those specific tests, while ensuring it will be shown if the coverage for other tests starts misbehaving?

A few additional details:

All tests (both the ones that rely on Flask's test_client and the ones that do HTTP queries to Gunicorn) are run in one go with the command:

coverage run --rcfile=coverage-options -m unittest discover tests/ --failfast

The contents of the aforementioned coverage-options file are:

[run]
branch = True
concurrency = multiprocessing
omit =
    /usr/*/dist-packages/*
    */.local/lib/*/site-packages/*
    tests/*

The tests/ directory contains several sub-directories:

tests/api/: about one hundred and thirty tests that rely on test_client.
tests/system/: four tests that perform HTTP queries to Gunicorn.
tests/unit/: seven very basic tests.

The output is:

(.venv) ~/src/example$ coverage run --rcfile=coverage-options -m unittest discover tests/ --failfast
...................................................................................................................................../home/u/src/example/.venv/lib/python3.13/site-packages/coverage/control.py:915: CoverageWarning: No data was collected. (no-data-collected)
  self._warn("No data was collected.", slug="no-data-collected")
.../home/u/src/example/.venv/lib/python3.13/site-packages/coverage/control.py:915: CoverageWarning: No data was collected. (no-data-collected)
  self._warn("No data was collected.", slug="no-data-collected")
.......
----------------------------------------------------------------------
Ran 143 tests in 45.853s

OK

As tests are ran in alphabetical order, the first one hundred and thirty something dots match the tests in tests/api/. The last seven correspond to the unit tests. The warnings are shown once the very first test from tests/system/ gets executed, and then once again when all four are executed.

By comparison, if I remove tests/system/ directory, this is what I get:

(.venv) ~/src/example$ coverage run --rcfile=coverage-options -m unittest discover tests/ --failfast
...........................................................................................................................................
----------------------------------------------------------------------
Ran 139 tests in 38.022s

OK

How are you running the tests? That warning should only appear when coverage is stopped, which makes it sound like you have a number of different runs of parts of the test suite? — Ned Batchelder
– Ned Batchelder, Commented May 26 at 0:50
@NedBatchelder: From my understanding, there's just one run. Anyway, I'm running it by doing: coverage run --rcfile=coverage-options -m unittest discover tests --failfast, and my coverage-options file contains branch = True, concurrency = multiprocessing, and a bunch of directories to exclude from coverage. — Arseni Mourzenko
– Arseni Mourzenko, Commented May 26 at 8:57
You said, "as soon as it hits those special tests that use Gunicorn, a warning is shown". Can you show us what you are seeing? — Ned Batchelder
– Ned Batchelder, Commented May 26 at 12:21
If the warning happens during the test execution, then it sounds like the tests themselves are running coverage. That would be unusual, and maybe unintended. — Ned Batchelder
– Ned Batchelder, Commented May 26 at 17:31
@NedBatchelder: found it. Check my answer. It's trickier than I thought, as the coverage gets collected when multiprocessing.Process exits (which was happening in two of the tests). Now, why does it do that, and what is the cleanest way to deal with it, is another subject. — Arseni Mourzenko
– Arseni Mourzenko, Commented May 26 at 22:36

Arseni Mourzenko · Accepted Answer · 2025-05-26 22:34:24Z

Trying to create a minimal reproducible example, it appears that the warning is emitted every time there is a multiprocessing.Process in the test, and more precisely when this process terminates.

Here are the steps to get the warning.

mkdir /tmp/ndc
cd /tmp/ndc
python3 -m venv .venv
source .venv/bin/activate
pip install coverage

Create the following tree in /tmp/ndc:

tests/
  __init__.py (empty)
  test_demo.py ①
coverage-options ②

Contents of /tmp/ndc/tests/test_demo.py ①:

import multiprocessing
import time
import unittest

class DemoTests(unittest.TestCase):
    def test_with_multiprocessing(self):
        p = multiprocessing.Process(target=t)
        p.start()
        p.join()
        self.assertTrue(2 * 2 == 4)


def t():
    time.sleep(0.1)

Contents of /tmp/ndc/coverage-options ②:

[run]
branch = True
concurrency = multiprocessing
omit =
    /usr/*/dist-packages/*
    */.local/lib/*/site-packages/*
    tests/*

Then run the tests:

coverage run --rcfile=coverage-options -m unittest discover tests

The output would be:

/tmp/ndc/.venv/lib/python3.13/site-packages/coverage/control.py:915: CoverageWarning: No data was collected. (no-data-collected)
  self._warn("No data was collected.", slug="no-data-collected")
.
----------------------------------------------------------------------
Ran 1 test in 0.121s

OK
/tmp/ndc/.venv/lib/python3.13/site-packages/coverage/control.py:915: CoverageWarning: No data was collected. (no-data-collected)
  self._warn("No data was collected.", slug="no-data-collected")

The second warning is expected—actually, the test didn't get any coverage. The first warning, however, happens at the moment p (that is, multiprocessing.Process) terminates.

One possible solution to get rid of this warning is to remove concurrency = multiprocessing from the coverage options file, or to change the value to a different library, such as concurrency = eventlet.

Collectives™ on Stack Overflow

How do I selectively hide no-data-collected when using coverage.py?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related