Skip to content

Test

invoke_tasklib.testing

Test, doctest, coverage and benchmark tasks (test.*).

The tests run with pytest, through the running interpreter (python -m pytest). The directory of each scope (unit, integration, functional and benchmarks) comes from the tasklib.paths config; a scope whose directory does not exist is skipped with a ⏭️ line, except test.benchmark, which fails. The options of pytest come from tasklib.test: the timeout of each scope, xdoctest, timeout, cov_fail_under and cov_reports.

The module is named testing so that it does not shadow the standard library package test; its namespace is test, see NAMESPACE.

invoke_tasklib.testing.NAMESPACE module-attribute

NAMESPACE = 'test'

The name of the namespace of the tasks of this module.

invoke_tasklib.testing.all_

all_(
    c: Context,
    *,
    cov: bool = False,
    args: str | None = None,
    paths: str | None = None,
) -> None

Run all tests (unit, integration, and functional).

The whole tests directory is run in one pytest call, so the timeout of each test is the largest of the three scopes (tasklib.test.unit_timeout, integration_timeout and functional_timeout). If the tests directory does not exist, nothing is run.

Parameters:

Name Type Description Default
c Context

The invoke context.

required
cov bool

If True, generate the coverage reports of tasklib.test.cov_reports (html,xml,term by default). Default is False.

False
args str | None

Extra arguments passed to pytest, e.g. "-k foo -x".

None
paths str | None

The files or directories to run instead of the tests directory.

None

invoke_tasklib.testing.benchmark

benchmark(
    c: Context,
    *,
    args: str | None = None,
    paths: str | None = None,
) -> None

Run performance benchmarks.

The command is pytest --benchmark-only --benchmark-enable: --benchmark-enable turns the benchmarks on even if the pytest config disables them (--benchmark-disable in addopts, to run each benchmark once in the normal test runs).

Parameters:

Name Type Description Default
c Context

The invoke context.

required
args str | None

Extra arguments passed to pytest.

None
paths str | None

The files or directories to run on instead of paths.benchmarks.

None

Raises:

Type Description
Exit

With the code 1 if paths.benchmarks does not exist and paths is not given. The task is run on purpose, so a nightly job does not pass without measuring anything.

invoke_tasklib.testing.coverage_report

coverage_report(
    c: Context,
    *,
    open_browser: bool = False,
    args: str | None = None,
) -> None

Generate an HTML coverage report.

The report is built from the last recorded coverage data, and the terminal summary is printed. The report is written to the directory of the html:<dir> value of tasklib.test.cov_reports if there is one, else to the directory of [tool.coverage.html] in pyproject.toml (htmlcov by default).

Parameters:

Name Type Description Default
c Context

The invoke context.

required
open_browser bool

If True, open the generated HTML report in the default web browser. Default is False.

False
args str | None

Extra arguments passed to coverage report, e.g. "--fail-under=90".

None

invoke_tasklib.testing.doctest

doctest(c: Context) -> None

Run doctests on both source code and markdown files.

It runs test.doctest-src then test.doctest-markdown with their defaults, and stops at the first failure.

Parameters:

Name Type Description Default
c Context

The invoke context.

required

invoke_tasklib.testing.doctest_markdown

doctest_markdown(
    c: Context,
    *,
    args: str | None = None,
    paths: str | None = None,
) -> None

Run doctests on the Python examples of the markdown files.

The files are the *.md files of the project (the files that git tracks, or all the files outside the excluded directories when the project is not a git work tree), or paths. They run with python -m doctest and the options NORMALIZE_WHITESPACE, ELLIPSIS and REPORT_NDIFF. A project without markdown file passes.

Parameters:

Name Type Description Default
c Context

The invoke context.

required
args str | None

Extra arguments passed to python -m doctest.

None
paths str | None

The markdown files to check instead of every markdown file of the project.

None

invoke_tasklib.testing.doctest_src

doctest_src(
    c: Context,
    *,
    args: str | None = None,
    paths: str | None = None,
) -> None

Run doctests on source code.

The command always has --xdoctest, whatever the value of tasklib.test.xdoctest, which only applies to the test scopes. If the xdoctest package is missing, the task stops with a message that names it.

Parameters:

Name Type Description Default
c Context

The invoke context.

required
args str | None

Extra arguments passed to pytest.

None
paths str | None

The files or directories to run on instead of paths.src.

None

Raises:

Type Description
Exit

With the code 2 if paths is not set and paths.src does not exist.

invoke_tasklib.testing.functional

functional(
    c: Context,
    *,
    cov: bool = False,
    append: bool = False,
    args: str | None = None,
    paths: str | None = None,
) -> None

Run functional tests.

The timeout of each test is tasklib.test.functional_timeout (60 seconds by default). Nothing is run if the directory does not exist.

Parameters:

Name Type Description Default
c Context

The invoke context.

required
cov bool

If True, generate the coverage reports of tasklib.test.cov_reports. Default is False.

False
append bool

If True and cov is True, add to the coverage data of the previous runs (--cov-append) instead of replacing it, e.g. after test.unit --cov. Default is False.

False
args str | None

Extra arguments passed to pytest, e.g. "-k foo -x".

None
paths str | None

The files or directories to run instead of the tests of this scope.

None

invoke_tasklib.testing.integration

integration(
    c: Context,
    *,
    cov: bool = False,
    append: bool = False,
    args: str | None = None,
    paths: str | None = None,
) -> None

Run integration tests.

The timeout of each test is tasklib.test.integration_timeout (60 seconds by default). Nothing is run if the directory does not exist.

Parameters:

Name Type Description Default
c Context

The invoke context.

required
cov bool

If True, generate the coverage reports of tasklib.test.cov_reports. Default is False.

False
append bool

If True and cov is True, add to the coverage data of the previous runs (--cov-append) instead of replacing it, e.g. after test.unit --cov. Default is False.

False
args str | None

Extra arguments passed to pytest, e.g. "-k foo -x".

None
paths str | None

The files or directories to run instead of the tests of this scope.

None

invoke_tasklib.testing.unit

unit(
    c: Context,
    *,
    cov: bool = False,
    args: str | None = None,
    paths: str | None = None,
) -> None

Run unit tests.

The timeout of each test is tasklib.test.unit_timeout (10 seconds by default). If the unit tests directory does not exist (a flat tests layout), the whole tests directory is run instead; if neither exists, nothing is run.

Parameters:

Name Type Description Default
c Context

The invoke context.

required
cov bool

If True, generate the coverage reports of tasklib.test.cov_reports. Default is False.

False
args str | None

Extra arguments passed to pytest, e.g. "-k foo -x".

None
paths str | None

The files or directories to run instead of the tests of this scope.

None