基于 `pytest` 和 `yield` 的测试

Question

我正在尝试将一堆测试从 nose 迁移到 pytest，但我无法迁移一个验证整个过程的测试。

我已经简化了它来代表我的问题：

def is_equal(a, b):
    assert a == b


def inner():
    yield is_equal, 2, 2
    yield is_equal, 3, 3


def test_simple():
    yield is_equal, 0, 0
    yield is_equal, 1, 1
    for test in inner():
        yield test
    yield is_equal, 4, 4
    yield is_equal, 5, 5


def test_complex():
    integers = list()


    def update_integers():
        integers.extend([0, 1, 2, 3, 4, 5])

    yield update_integers

    for x in integers:
        yield is_equal, x, x

test_simple 在nose 和pytest 之间运行良好，但test_complex 只运行初始update_integers 测试：

~/projects/testbox$ nosetests -v
test_nose_tests.test_simple(0, 0) ... ok
test_nose_tests.test_simple(1, 1) ... ok
test_nose_tests.test_simple(2, 2) ... ok
test_nose_tests.test_simple(3, 3) ... ok
test_nose_tests.test_simple(4, 4) ... ok
test_nose_tests.test_simple(5, 5) ... ok
test_nose_tests.test_complex ... ok
test_nose_tests.test_complex(0, 0) ... ok
test_nose_tests.test_complex(1, 1) ... ok
test_nose_tests.test_complex(2, 2) ... ok
test_nose_tests.test_complex(3, 3) ... ok
test_nose_tests.test_complex(4, 4) ... ok
test_nose_tests.test_complex(5, 5) ... ok

----------------------------------------------------------------------
Ran 13 tests in 0.004s


~/projects/testbox$ pytest -v
====================================================================     test session starts         =====================================================================
platform linux2 -- Python 2.7.12, pytest-3.0.6, py-1.4.32, pluggy-0.4.0 -- /usr/bin/python
cachedir: .cache
rootdir: /home/local/ANT/cladam/projects/testbox, inifile: 
collected 7 items 

tests/test_nose_tests.py::test_simple::[0] PASSED
tests/test_nose_tests.py::test_simple::[1] PASSED
tests/test_nose_tests.py::test_simple::[2] PASSED
tests/test_nose_tests.py::test_simple::[3] PASSED
tests/test_nose_tests.py::test_simple::[4] PASSED
tests/test_nose_tests.py::test_simple::[5] PASSED
tests/test_nose_tests.py::test_complex::[0] PASSED

=================================================================== pytest-warning summary     ===================================================================
WC1 /home/local/ANT/cladam/projects/testbox/tests/test_nose_tests.py yield tests are deprecated, and scheduled to be removed in pytest 4.0
....
======================================================== 7 passed, 7     pytest-warnings in 0.01 seconds =========================================================

我假设这是因为在收集时整数列表为空，因此它不会收集另外 6 个 yields。

有什么方法可以在pytest 中复制这个测试结构？通过pytest_generate_tests?

这个测试代表了一个更大的事件序列来构建一个对象并对其进行操作，并在过程的每个阶段进行测试。

1. 建模
2. 验证一些模型属性
3. 根据模型创建和输出文件
4. 对比已知输出以查看是否有变化。

提前致谢

Answer 1

正如您的测试输出所示，基于yield 的测试已被弃用：

WC1 /home/local/ANT/cladam/projects/testbox/tests/test_nose_tests.py yield tests are deprecated, and scheduled to be removed in pytest 4.0

我建议你改用装饰器pytest.parametrize。您可以在以下位置查看更多信息：

• https://docs.pytest.org/en/latest/how-to/parametrize.html
• https://docs.pytest.org/en/latest/example/parametrize.html

从你的例子中，我会为测试创建这样的东西：

import pytest


def is_equal(a, b):
    return a == b


class TestComplexScenario:
    @pytest.mark.parametrize("my_integer", [0, 1, 2])
    def test_complex(self, my_integer):
        assert is_equal(my_integer, my_integer)

这是一个输出示例：

test_complex.py::TestComplexScenario::test_complex[0] PASSED
test_complex.py::TestComplexScenario::test_complex[1] PASSED
test_complex.py::TestComplexScenario::test_complex[2] PASSED

您可以在以下位置找到更多关于参数化的示例： http://layer0.authentise.com/pytest-and-parametrization.html

您还可以对测试输入进行排列，请查看以下示例： parameterized test with cartesian product of arguments in pytest

Answer 2

问题是 pytest 在运行任何测试之前收集所有测试，因此在 test_complex 内，直到收集过程结束后才会调用 update_integers 函数。

A您可以通过将is_generator 检查从收集阶段移动到测试运行阶段来运行测试，方法是将以下内容放入conftest.py。不幸的是，这些钩子不允许 pytest_runtest_protocol 作为生成器运行，因此 pytest-3.2.1 的 _pytest.main.pytest_runtestloop 的全部内容被复制和修改。

import pytest
from _pytest.compat import is_generator
def pytest_pycollect_makeitem(collector, name, obj):
    """
    Override the collector so that generators are saved as functions
    to be run during the test phase rather than the collection phase.
    """
    if collector.istestfunction(obj, name) and is_generator(obj):
        return [pytest.Function(name, collector, args=(), callobj=obj)]

def pytest_runtestloop(session):
    """
    Copy of _pytest.main.pytest_runtestloop with the session iteration
    modified to perform a subitem iteration.
    """
    if (session.testsfailed and
            not session.config.option.continue_on_collection_errors):
        raise session.Interrupted(
            "%d errors during collection" % session.testsfailed)

    if session.config.option.collectonly:
        return True

    for i, item in enumerate(session.items):
        nextitem = session.items[i + 1] if i + 1 < len(session.items) else None
        # The new functionality is here: treat all items as if they
        # might have sub-items, and run through them one by one.

        for subitem in get_subitems(item):
            subitem.config.hook.pytest_runtest_protocol(item=subitem, nextitem=nextitem)
            if getattr(session, "shouldfail", False):
                raise session.Failed(session.shouldfail)
            if session.shouldstop:
                raise session.Interrupted(session.shouldstop)
    return True

def get_subitems(item):
    """
    Return a sequence of subitems for the given item.  If the item is
    not a generator, then just yield the item itself as the sequence.
    """
    if not isinstance(item, pytest.Function):
        yield item
    obj = item.obj
    if is_generator(obj):
        for number, yielded in enumerate(obj()):
            index, call, args = interpret_yielded_test(yielded, number)
            test = pytest.Function(item.name+index, item.parent, args=args, callobj=call)
            yield test
    else:
        yield item


def interpret_yielded_test(obj, number):
    """
    Process an item yielded from a generator.  If the item is named,
    then set the index to "['name']", otherwise set it to "[number]".
    Return the index, the callable and the arguments to the callable.
    """
    if not isinstance(obj, (tuple, list)):
        obj = (obj,)
    if not callable(obj[0]):
        index = "['%s']"%obj[0]
        obj = obj[1:]
    else:
        index = "[%d]"%number
    call, args = obj[0], obj[1:]
    return index, call, args

如果 pytest 自 3.2.1 版以来变化太大，上述方法可能不起作用。而是酌情复制和修改最新版本的_pytest.main.pytest_runtestloop；这应该为您的项目提供时间逐步从基于产量的测试用例迁移，或者至少迁移到可以在收集时收集的基于产量的测试用例。