The hooks other scenarios never reach¶
pytest 7.0.0 - 7.0.1 - Assertion comparison, deselection, string conditions and the debugger.
These 2 releases produced an identical flow, so they share one page. Identical as far as this scenario can measure - a scenario exercising more hooks may tell them apart.
A plain test run reaches about three quarters of pytest's hooks. The rest need
provoking: an assertion that compares rather than merely fails, a deselected
test, a skipif written as a string, the assertion-pass hook switched on, and
a debugger session.
This scenario provokes all of them in one run, which is why it observes more
hooks than any other. pytest.ini enables enable_assertion_pass_hook, and
autopdb.py supplies a Pdb that continues the moment it is entered - so the
debugger hooks fire without the run hanging.
Three hooks remain out of reach even here. pytest_cmdline_parse runs before
any plugin exists to watch it. pytest_report_to_serializable and
pytest_report_from_serializable are never called by pytest itself - they
belong to plugins that ship reports between processes, such as xdist.
How this was produced¶
- Source:
scenarios/edge-cases/- the project this diagram was traced from - Invocation:
pytest -p hook_atlas_pytest . -k not deselect_me --pdb --pdbcls=autopdb:AutoPdb - Traced against: pytest 7.0.1, pluggy 1.6.0, Python 3.9.25
- Covers: 2 releases - pytest 7.0.0 - 7.0.1, which produced an identical flow
- Observed: 217 hook calls, 43 distinct hooks
The project is autopdb.py, conftest.py, pytest.ini, test_edges.py. Browse it in the repository rather than inline here - scenarios grow directory trees, and GitHub renders them better than a collapsed block can.
The whole run¶
Phases left to right, steps top to bottom. Colour says which phase a hook belongs to; how dark a step is says how often it was called. Click a column heading to jump to that phase in full detail, or any hook to open its pytest documentation. Why it looks like this
Startup and configuration¶
Everything before collection begins: initial conftests are loaded, plugins register, and the session is configured.
Collection¶
Finding tests. Note that pytest_make_collect_report recurses - directories contain directories contain files.
The run-test protocol¶
pytest_runtestloop runs this once per collected test. Note the rhythm: setup, call and teardown are each followed by their own makereport and logreport pair - three reports per test, not one.
The path 1 of 4 took.
The others differed:
- 1 test also called
pytest_enter_pdb,pytest_exception_interact,pytest_leave_pdband skippedpytest_assertion_pass - 1 test also called
pytest_markeval_namespaceand skippedpytest_assertion_pass,pytest_assertrepr_compare - 1 test skipped
pytest_assertrepr_compare
Session finish¶
Summary reporting and teardown of the session.
Every hook observed¶
| Hook | Calls | Semantics | Implemented by, in call order |
|---|---|---|---|
pytest_assertion_pass |
2 | conftest.py |
|
pytest_assertrepr_compare |
2 | _pytest.assertion |
|
pytest_cmdline_main |
1 | firstresult | _pytest.setupplan [tryfirst]_pytest.setuponly [tryfirst]_pytest.mark [tryfirst]_pytest.python_pytest.helpconfig_pytest.main |
pytest_cmdline_preparse |
1 | - | |
pytest_collect_file |
7 | _pytest.doctest_pytest.python |
|
pytest_collection |
1 | firstresult | _pytest.logging.LoggingPlugin (logging-plugin) [hookwrapper, tryfirst]_pytest.warnings [hookwrapper, tryfirst]_pytest.config.Config (pytestconfig) [hookwrapper]_pytest.terminal.TerminalReporter_pytest.assertion_pytest.main |
pytest_collection_finish |
1 | _pytest.terminal.TerminalReporter |
|
pytest_collection_modifyitems |
1 | _pytest.fixtures.FixtureManager (funcmanage)_pytest.main_pytest.mark |
|
pytest_collectreport |
2 | _pytest.main.Session.pytest_runtest_logreport (session) [tryfirst]_pytest.terminal.TerminalReporter |
|
pytest_collectstart |
2 | _pytest.main.Session [tryfirst] |
|
pytest_configure |
1 | historic | _pytest.faulthandler_pytest.warnings_pytest.junitxml_pytest.tmpdir_pytest.legacypath_pytest.skipping_pytest.debugging_pytest.terminal_pytest.python_pytest.mark_pytest.config.PytestPluginManager (<anonymous>)_pytest.pastebin [trylast]_pytest.logging [trylast] |
pytest_deselected |
1 | _pytest.terminal.TerminalReporter |
|
pytest_enter_pdb |
1 | _pytest.faulthandler [tryfirst] |
|
pytest_exception_interact |
1 | _pytest.faulthandler [tryfirst]_pytest.debugging.PdbInvoke |
|
pytest_generate_tests |
5 | _pytest.fixtures.FixtureManager (funcmanage)_pytest.python |
|
pytest_ignore_collect |
7 | firstresult | _pytest.main |
pytest_itemcollected |
5 | - | |
pytest_leave_pdb |
1 | - | |
pytest_load_initial_conftests |
1 | _pytest.warnings [hookwrapper]_pytest.capture [hookwrapper]_pytest.python_path [tryfirst]_pytest.legacypath [tryfirst]_pytest.config.Config (pytestconfig) [trylast]1 |
|
pytest_make_collect_report |
2 | firstresult | _pytest.capture.CaptureManager [hookwrapper]_pytest.runner |
pytest_markeval_namespace |
1 | - | |
pytest_plugin_registered |
87 | historic | _pytest.legacypath_pytest.terminal.TerminalReporter_pytest.fixtures.FixtureManager (funcmanage) |
pytest_pycollect_makeitem |
9 | firstresult | _pytest.unittest_pytest.python [trylast] |
pytest_pycollect_makemodule |
1 | firstresult | _pytest.python |
pytest_pyfunc_call |
4 | firstresult | _pytest.python [trylast] |
pytest_report_collectionfinish |
1 | - | |
pytest_report_header |
1 | _pytest.terminal.TerminalReporter_pytest.helpconfig |
|
pytest_report_teststatus |
13 | firstresult | _pytest.skipping_pytest.runner_pytest.terminal [trylast] |
pytest_runtest_call |
4 | _pytest.threadexception [hookwrapper, tryfirst]_pytest.unraisableexception [hookwrapper, tryfirst]_pytest.logging.LoggingPlugin (logging-plugin) [hookwrapper]_pytest.capture.CaptureManager [hookwrapper]_pytest.skipping [hookwrapper]_pytest.runner |
|
pytest_runtest_logfinish |
4 | _pytest.logging.LoggingPlugin (logging-plugin)_pytest.terminal.TerminalReporter |
|
pytest_runtest_logreport |
12 | _pytest.main.Session [tryfirst]_pytest.logging.LoggingPlugin (logging-plugin)_pytest.terminal.TerminalReporter |
|
pytest_runtest_logstart |
4 | _pytest.logging.LoggingPlugin (logging-plugin)_pytest.terminal.TerminalReporter |
|
pytest_runtest_makereport |
12 | firstresult | _pytest.skipping [hookwrapper]_pytest.unittest [tryfirst]_pytest.runner |
pytest_runtest_protocol |
4 | firstresult | _pytest.warnings [hookwrapper, tryfirst]_pytest.assertion [hookwrapper, tryfirst]_pytest.unittest [hookwrapper]_pytest.faulthandler [hookwrapper, trylast]_pytest.runner |
pytest_runtest_setup |
4 | _pytest.unraisableexception [hookwrapper, tryfirst]_pytest.logging.LoggingPlugin (logging-plugin) [hookwrapper]_pytest.capture.CaptureManager [hookwrapper]_pytest.threadexception [hookwrapper, trylast]_pytest.skipping [tryfirst]_pytest.runner_pytest.nose [trylast] |
|
pytest_runtest_teardown |
4 | _pytest.threadexception [hookwrapper, tryfirst]_pytest.unraisableexception [hookwrapper, tryfirst]_pytest.logging.LoggingPlugin (logging-plugin) [hookwrapper]_pytest.capture.CaptureManager [hookwrapper]_pytest.runner |
|
pytest_runtestloop |
1 | firstresult | _pytest.logging.LoggingPlugin (logging-plugin) [hookwrapper]_pytest.main |
pytest_sessionfinish |
1 | _pytest.logging.LoggingPlugin (logging-plugin) [hookwrapper, tryfirst]_pytest.terminal.TerminalReporter [hookwrapper]_pytest.warnings [hookwrapper]_pytest.assertion_pytest.runner |
|
pytest_sessionstart |
1 | _pytest.logging.LoggingPlugin (logging-plugin) [hookwrapper, tryfirst]_pytest.fixtures_pytest.runner_pytest.terminal.TerminalReporter [trylast] |
|
pytest_terminal_summary |
1 | _pytest.terminal.TerminalReporter [hookwrapper]_pytest.warnings [hookwrapper]_pytest.pastebin_pytest.runner |
|
pytest_unconfigure |
1 | _pytest.logging.LoggingPlugin (logging-plugin)_pytest.terminal.TerminalReporter_pytest.faulthandler_pytest.doctest_pytest.junitxml_pytest.pastebin_pytest.mark_pytest.python_path [trylast]2 |
|
pytest_warning_captured |
1 | historic | - |
pytest_warning_recorded |
1 | historic | _pytest.terminal.TerminalReporter |
2 of these changed implementer somewhere in pytest 7.0.0 - 7.0.1 without changing the flow - pytest moves implementations between its own plugins. The table shows the newest; what changed along the way is below.