The hooks other scenarios never reach

Showing pytest 9.1.1 pytest 9.1.0 pytest 9.0.3 pytest 9.0.2 pytest 9.0.1 pytest 9.0.0 pytest 8.4.2 pytest 8.4.1 pytest 8.4.0 pytest 8.3.5 pytest 8.3.4 pytest 8.3.3 pytest 8.3.2 pytest 8.3.1 pytest 8.3.0 pytest 8.2.2 pytest 8.2.1 pytest 8.2.0 pytest 8.1.2 pytest 8.1.1 pytest 8.0.2 pytest 8.0.1 pytest 8.0.0 pytest 7.4.4 pytest 7.4.3 pytest 7.4.2 pytest 7.4.1 pytest 7.4.0 pytest 7.3.2 pytest 7.3.1 pytest 7.3.0 pytest 7.2.2 pytest 7.2.1 pytest 7.2.0 pytest 7.1.3 pytest 7.1.2 pytest 7.1.1 pytest 7.1.0 pytest 7.0.1 pytest 7.0.0 pytest 6.2.5 pytest 6.2.4 pytest 6.2.3 pytest 6.2.2 pytest 6.2.1 pytest 6.2.0 pytest 6.1.2 pytest 6.1.1 pytest 6.1.0 pytest 6.0.2 pytest 6.0.1 pytest 6.0.0

pytest 6.0.0 - 6.0.2 - Assertion comparison, deselection, string conditions and the debugger.

These 3 releases produced an identical flow, so they share one page. Identical as far as this scenario can measure - a scenario exercising more hooks may tell them apart.

A plain test run reaches about three quarters of pytest's hooks. The rest need provoking: an assertion that compares rather than merely fails, a deselected test, a skipif written as a string, the assertion-pass hook switched on, and a debugger session.

This scenario provokes all of them in one run, which is why it observes more hooks than any other. pytest.ini enables enable_assertion_pass_hook, and autopdb.py supplies a Pdb that continues the moment it is entered - so the debugger hooks fire without the run hanging.

Three hooks remain out of reach even here. pytest_cmdline_parse runs before any plugin exists to watch it. pytest_report_to_serializable and pytest_report_from_serializable are never called by pytest itself - they belong to plugins that ship reports between processes, such as xdist.

How this was produced

• Source: scenarios/edge-cases/ - the project this diagram was traced from
• Invocation: pytest -p hook_atlas_pytest . -k not deselect_me --pdb --pdbcls=autopdb:AutoPdb
• Traced against: pytest 6.0.2, pluggy 0.13.1, Python 3.9.25
• Covers: 3 releases - pytest 6.0.0 - 6.0.2, which produced an identical flow
• Observed: 242 hook calls, 42 distinct hooks

The project is autopdb.py, conftest.py, pytest.ini, test_edges.py. Browse it in the repository rather than inline here - scenarios grow directory trees, and GitHub renders them better than a collapsed block can.

The whole run

Phases left to right, steps top to bottom. Colour says which phase a hook belongs to; how dark a step is says how often it was called. Click a column heading to jump to that phase in full detail, or any hook to open its pytest documentation. Why it looks like this

Startup and configuration

Everything before collection begins: initial conftests are loaded, plugins register, and the session is configured.

Collection

Finding tests. Note that pytest_make_collect_report recurses - directories contain directories contain files.

The run-test protocol

pytest_runtestloop runs this once per collected test. Note the rhythm: setup, call and teardown are each followed by their own makereport and logreport pair - three reports per test, not one.

The path 1 of 4 took.

The others differed:

• 1 test also called pytest_enter_pdb, pytest_exception_interact, pytest_leave_pdb and skipped pytest_assertion_pass
• 1 test skipped pytest_assertion_pass, pytest_assertrepr_compare
• 1 test skipped pytest_assertrepr_compare

Session finish

Summary reporting and teardown of the session.

Every hook observed

Hook	Calls	Semantics	Implemented by, in call order
pytest_assertion_pass	2		conftest.py
pytest_assertrepr_compare	2		_pytest.assertion
pytest_cmdline_main	1	firstresult	_pytest.setupplan [tryfirst] _pytest.setuponly [tryfirst] _pytest.mark [tryfirst] _pytest.python _pytest.helpconfig _pytest.main
pytest_cmdline_preparse	1		-
pytest_collect_file	7		_pytest.doctest _pytest.python
pytest_collection	1	firstresult	_pytest.logging.LoggingPlugin (logging-plugin) [hookwrapper, tryfirst] _pytest.warnings [hookwrapper, tryfirst] _pytest.terminal.TerminalReporter _pytest.assertion _pytest.main
pytest_collection_finish	1		_pytest.terminal.TerminalReporter
pytest_collection_modifyitems	1		_pytest.fixtures.FixtureManager (funcmanage) _pytest.main _pytest.mark
pytest_collectreport	2		_pytest.main.Session.pytest_runtest_logreport (session) [tryfirst] _pytest.terminal.TerminalReporter
pytest_collectstart	2		_pytest.main.Session [tryfirst]
pytest_configure	2	historic	_pytest.faulthandler _pytest.warnings _pytest.resultlog _pytest.junitxml _pytest.tmpdir _pytest.skipping _pytest.debugging _pytest.terminal _pytest.python _pytest.mark _pytest.config.PytestPluginManager (<anonymous>) _pytest.pastebin [trylast] _pytest.logging [trylast] _pytest.faulthandler.FaultHandlerHooks (faulthandler-hooks)
pytest_deselected	1		_pytest.terminal.TerminalReporter
pytest_enter_pdb	1		_pytest.faulthandler.FaultHandlerHooks (faulthandler-hooks) [tryfirst]
pytest_exception_interact	1		_pytest.faulthandler.FaultHandlerHooks (faulthandler-hooks) [tryfirst] _pytest.debugging.PdbInvoke
pytest_generate_tests	5		_pytest.fixtures.FixtureManager (funcmanage) _pytest.python
pytest_ignore_collect	7	firstresult	_pytest.main
pytest_itemcollected	5		-
pytest_leave_pdb	1		-
pytest_load_initial_conftests	1		_pytest.capture [hookwrapper] _pytest.config.Config (pytestconfig) [trylast]
pytest_make_collect_report	2	firstresult	_pytest.capture.CaptureManager [hookwrapper] _pytest.runner
pytest_plugin_registered	81	historic	_pytest.terminal.TerminalReporter _pytest.fixtures.FixtureManager (funcmanage)
pytest_pycollect_makeitem	40	firstresult	_pytest.unittest _pytest.python [trylast]
pytest_pycollect_makemodule	1	firstresult	_pytest.python
pytest_pyfunc_call	4	firstresult	_pytest.python [trylast]
pytest_report_collectionfinish	1		-
pytest_report_header	1		_pytest.terminal.TerminalReporter _pytest.helpconfig
pytest_report_teststatus	13	firstresult	_pytest.skipping _pytest.runner _pytest.terminal [trylast]
pytest_runtest_call	4		_pytest.logging.LoggingPlugin (logging-plugin) [hookwrapper] _pytest.capture.CaptureManager [hookwrapper] _pytest.skipping [hookwrapper] _pytest.runner
pytest_runtest_logfinish	4		_pytest.logging.LoggingPlugin (logging-plugin) _pytest.terminal.TerminalReporter
pytest_runtest_logreport	12		_pytest.main.Session [tryfirst] _pytest.logging.LoggingPlugin (logging-plugin) _pytest.terminal.TerminalReporter
pytest_runtest_logstart	4		_pytest.logging.LoggingPlugin (logging-plugin) _pytest.terminal.TerminalReporter
pytest_runtest_makereport	12	firstresult	_pytest.skipping [hookwrapper] _pytest.unittest [tryfirst] _pytest.runner
pytest_runtest_protocol	4	firstresult	_pytest.warnings [hookwrapper, tryfirst] _pytest.assertion [hookwrapper, tryfirst] _pytest.unittest [hookwrapper] _pytest.faulthandler.FaultHandlerHooks (faulthandler-hooks) [hookwrapper, trylast] _pytest.runner
pytest_runtest_setup	4		_pytest.logging.LoggingPlugin (logging-plugin) [hookwrapper] _pytest.capture.CaptureManager [hookwrapper] _pytest.skipping [tryfirst] _pytest.runner _pytest.nose [trylast]
pytest_runtest_teardown	4		_pytest.logging.LoggingPlugin (logging-plugin) [hookwrapper] _pytest.capture.CaptureManager [hookwrapper] _pytest.runner
pytest_runtestloop	1	firstresult	_pytest.logging.LoggingPlugin (logging-plugin) [hookwrapper] _pytest.main
pytest_sessionfinish	1		_pytest.logging.LoggingPlugin (logging-plugin) [hookwrapper, tryfirst] _pytest.terminal.TerminalReporter [hookwrapper] _pytest.warnings [hookwrapper] _pytest.assertion _pytest.runner
pytest_sessionstart	1		_pytest.logging.LoggingPlugin (logging-plugin) [hookwrapper, tryfirst] _pytest.fixtures _pytest.runner _pytest.terminal.TerminalReporter [trylast]
pytest_terminal_summary	1		_pytest.terminal.TerminalReporter [hookwrapper] _pytest.warnings [hookwrapper] _pytest.pastebin _pytest.runner
pytest_unconfigure	1		_pytest.logging.LoggingPlugin (logging-plugin) _pytest.terminal.TerminalReporter _pytest.faulthandler.FaultHandlerHooks (faulthandler-hooks) _pytest.doctest _pytest.resultlog _pytest.junitxml _pytest.pastebin _pytest.mark
pytest_warning_captured	1	historic	-
pytest_warning_recorded	1	historic	_pytest.terminal.TerminalReporter