Added helper to test.support for parameterizing tests

[serhiy-storchaka]

This is a simple decorator modeled after pytest.mark.parametrize. It allows to "parametrize" tests, each parametrized test is run as a subtest.

The decorator specifies the name(s) of parameters (as an iterable or a comma separated string) and their values (an iterable of values for parameters):

@subTests('a,b', [('foo', 2), ('bar', 5)])
def test_foo(self, a, b):
    ...

is equivalent to

def test_foo(self):
    for a, b in [('foo', 2), ('bar', 5)]:
        with self.subTest(a=a, b=b):
            ...

It allows to save 2 levels of indentation.

Multiple decorators can be used, this generates a Descartes production of parameter values:

@subTests('a,b', [('foo', 2), ('bar', 5)])
@subTests('c', range(5))
def test_foo(self, a, b, c):
    ...

is equivalent to

def test_foo(self):
    for a, b in [('foo', 2), ('bar', 5)]:
        with self.subTest(a=a, b=b):
            for c in range(5):
                with self.subTest(c=c):
                    ...

It allows to save even more 2 levels of indentation.

It has also optional keyword-only parameter _do_cleanups. _do_cleanups=True tells to call doCleanups() after each subtest. This is temporary feature, only for use in test_ntpath. In general, it is wrong to do this because doCleanups() will also calls callbacks added in setUp. It will be replaced either by addSubTestCleanup() (see https://discuss.python.org/t/unittest-add-addcleanup-to-subtest/91827 and #134079) or by other methods (for example by adding fences for cleanup). Also, it will not be needed for real test parameterization.

There are differences from "real" parameterization:

• setUp and tearDown are only run once, before and after all subtests.
• It is impossible to select or filter out tests with specific parameters.
• It can only parametrize test methods, not classes.

I plan to implement real parameterization, this is why I only propose to add that decorator in test.support, not in unittest. In any case it would be backported in test.support. We may finally add both decorators in unittest if this does not create confusion.

The name subTests is intentionally chosen to not confuse with parametrize which is reserved for real test parameterization.

The PR contains also several examples of using that decorator.

Linked PRs

• gh-135120: Add test.support.subTests() #135121
• [3.14] gh-135120: Add test.support.subTests() (GH-135121) #135208
• [3.13] gh-135120: Add test.support.subTests() (GH-135121) #135210