Skip to content

A conftest implementing every hook

pytest 8.1.1 - 9.1.1 - What actually happens when a conftest.py implements all 52 declared hooks.

These 20 releases produced an identical flow, so they share one page. Identical as far as this scenario can measure - a scenario exercising more hooks may tell them apart.

The conftest.py here is generated: it implements every hook pytest declares.

Deprecated hooks are implemented too, which pytest would normally refuse - the run suppresses its own removal warnings so that the conftest really does cover everything the version declares.

This is the scenario that shows which hooks a conftest.py can and cannot serve. Some hooks have already fired by the time a conftest is imported, so implementing them there does nothing at all - the implementation is registered, and never called.

How this was produced

  • Source: scenarios/every-hook-conftest/ - the project this diagram was traced from
  • Invocation: pytest . -q -W ignore::pytest.PytestDeprecationWarning
  • Traced against: pytest 9.1.1, pluggy 1.6.0, Python 3.12.3
  • Covers: 20 releases - pytest 8.1.1 - 9.1.1, which produced an identical flow
  • Observed: 324 hook calls, 40 distinct hooks

The project is test_dummy.py. Browse it in the repository rather than inline here - scenarios grow directory trees, and GitHub renders them better than a collapsed block can.

The whole run

Phases left to right, steps top to bottom. Colour says which phase a hook belongs to; how dark a step is says how often it was called. Click a column heading to jump to that phase in full detail, or any hook to open its pytest documentation. Why it looks like this

%3 cluster_phase_startup STARTUP AND CONFIGURATION cluster_startup_n1 pytest_load_initial_conftests cluster_startup_n6 pytest_configure historic cluster_startup_n8 pytest_sessionstart cluster_phase_collection COLLECTION cluster_collection_n1 pytest_collection firstresult cluster_phase_runtest THE RUN-TEST PROTOCOL cluster_runtest_n1 pytest_runtest_protocol firstresult · 8x in run cluster_phase_finish SESSION FINISH cluster_finish_n1 pytest_sessionfinish startup_n2 pytest_plugin_registered historic · 112x in run startup_n3 pytest_addhooks historic startup_n2->startup_n3 startup_n4 pytest_addoption historic startup_n3->startup_n4 startup_n5 pytest_plugin_registered historic · x33 startup_n4->startup_n5 startup_n7 pytest_plugin_registered historic · x38 startup_n5->startup_n7 startup_n9 pytest_plugin_registered historic · x38 startup_n7->startup_n9 collection_n2 pytest_collectstart 3x in run collection_n3 pytest_make_collect_report firstresult · 3x in run collection_n2->collection_n3 collection_n4 pytest_collectreport 3x in run collection_n3->collection_n4 collection_n5 pytest_collectstart 3x in run collection_n4->collection_n5 collection_n6 pytest_make_collect_report firstresult · 3x in run collection_n5->collection_n6 collection_n7 pytest_collectstart 3x in run collection_n6->collection_n7 collection_n8 pytest_make_collect_report firstresult · 3x in run collection_n7->collection_n8 collection_n9 pytest_itemcollected x8 collection_n8->collection_n9 collection_n10 pytest_collectreport x2 collection_n9->collection_n10 collection_n11 pytest_collection_modifyitems collection_n10->collection_n11 collection_n12 pytest_collection_finish collection_n11->collection_n12 collection_n13 pytest_warning_recorded historic · 2x in run collection_n12->collection_n13 runtest_n2 pytest_runtest_logstart 8x in run runtest_n3 pytest_runtest_setup 8x in run runtest_n2->runtest_n3 runtest_n4 pytest_runtest_makereport firstresult · 23x in run runtest_n3->runtest_n4 runtest_n5 pytest_runtest_logreport 23x in run runtest_n4->runtest_n5 runtest_n6 pytest_runtest_call 7x in run runtest_n5->runtest_n6 runtest_n7 pytest_runtest_makereport firstresult · 23x in run runtest_n6->runtest_n7 runtest_n8 pytest_runtest_logreport 23x in run runtest_n7->runtest_n8 runtest_n9 pytest_runtest_teardown 8x in run runtest_n8->runtest_n9 runtest_n10 pytest_runtest_makereport firstresult · 23x in run runtest_n9->runtest_n10 runtest_n11 pytest_runtest_logreport 23x in run runtest_n10->runtest_n11 runtest_n12 pytest_runtest_logfinish 8x in run runtest_n11->runtest_n12 finish_n2 pytest_terminal_summary finish_n3 pytest_unconfigure finish_n2->finish_n3

Startup and configuration

Everything before collection begins: initial conftests are loaded, plugins register, and the session is configured.

%3 cluster_n1 pytest_load_initial_conftests cluster_n6 pytest_configure historic cluster_n8 pytest_sessionstart n2 pytest_plugin_registered historic · 112x in run n3 pytest_addhooks historic n2->n3 n4 pytest_addoption historic n3->n4 n5 pytest_plugin_registered historic · x33 n4->n5 n7 pytest_plugin_registered historic · x38 n5->n7 n9 pytest_plugin_registered historic · x38 n7->n9

Collection

Finding tests. Note that pytest_make_collect_report recurses - directories contain directories contain files.

%3 cluster_n1 pytest_collection firstresult cluster_n3 pytest_make_collect_report firstresult · 3x in run cluster_n7 pytest_make_collect_report firstresult · 3x in run cluster_n17 pytest_collect_file 5x in run cluster_n20 pytest_make_collect_report firstresult · 3x in run cluster_n22 pytest_pycollect_makeitem firstresult · 10x in run cluster_n23 pytest_generate_tests 4x in run cluster_n25 pytest_pycollect_makeitem firstresult · x3 cluster_n30 pytest_collection_finish n2 pytest_collectstart 3x in run n4 pytest_collect_directory firstresult n2->n4 n5 pytest_collectreport 3x in run n4->n5 n6 pytest_collectstart 3x in run n5->n6 n8 pytest_ignore_collect firstresult · x2 n6->n8 n9 pytest_collect_file 5x in run n8->n9 n10 pytest_ignore_collect firstresult · 6x in run n9->n10 n11 pytest_collect_file 5x in run n10->n11 n12 pytest_ignore_collect firstresult · 6x in run n11->n12 n13 pytest_collect_file 5x in run n12->n13 n14 pytest_ignore_collect firstresult · 6x in run n13->n14 n15 pytest_collect_file 5x in run n14->n15 n16 pytest_ignore_collect firstresult · 6x in run n15->n16 n18 pytest_pycollect_makemodule firstresult n16->n18 n19 pytest_collectstart 3x in run n18->n19 n21 pytest_pycollect_makeitem firstresult · x6 n19->n21 n24 pytest_make_parametrize_id firstresult · x5 n21->n24 n26 pytest_generate_tests 4x in run n24->n26 n27 pytest_itemcollected x8 n26->n27 n28 pytest_collectreport x2 n27->n28 n29 pytest_collection_modifyitems n28->n29 n31 pytest_report_collectionfinish n29->n31 n32 pytest_warning_recorded historic · 2x in run n31->n32

The run-test protocol

pytest_runtestloop runs this once per collected test. Note the rhythm: setup, call and teardown are each followed by their own makereport and logreport pair - three reports per test, not one.

%3 cluster_n1 pytest_runtest_protocol firstresult · 8x in run cluster_n3 pytest_runtest_setup 8x in run cluster_n6 pytest_runtest_logreport 23x in run cluster_n8 pytest_runtest_call 7x in run cluster_n11 pytest_runtest_logreport 23x in run cluster_n13 pytest_runtest_teardown 8x in run cluster_n16 pytest_runtest_logreport 23x in run n2 pytest_runtest_logstart 8x in run n4 pytest_fixture_setup firstresult · x2 n2->n4 n5 pytest_runtest_makereport firstresult · 23x in run n4->n5 n7 pytest_report_teststatus firstresult · 24x in run n5->n7 n9 pytest_pyfunc_call firstresult · 7x in run n7->n9 n10 pytest_runtest_makereport firstresult · 23x in run n9->n10 n12 pytest_report_teststatus firstresult · 24x in run n10->n12 n14 pytest_fixture_post_finalizer x2 n12->n14 n15 pytest_runtest_makereport firstresult · 23x in run n14->n15 n17 pytest_report_teststatus firstresult · 24x in run n15->n17 n18 pytest_runtest_logfinish 8x in run n17->n18

The path 5 of 8 took.

The others differed:

  • 1 test also called pytest_exception_interact
  • 1 test skipped pytest_fixture_post_finalizer, pytest_fixture_setup, pytest_pyfunc_call, pytest_runtest_call
  • 1 test also called pytest_warning_recorded and skipped pytest_fixture_post_finalizer, pytest_fixture_setup

Session finish

Summary reporting and teardown of the session.

%3 cluster_n1 pytest_sessionfinish cluster_n2 pytest_terminal_summary n3 pytest_report_teststatus firstresult · 24x in run n4 pytest_unconfigure n3->n4

Hooks a conftest cannot serve

This scenario's conftest.py implements every hook pytest declares. The hooks below still fired without it: the implementation was registered and never invoked, because the conftest had not been imported yet when the hook ran, or sits below the level the hook applies to.

Every hook observed

Hook Calls Semantics Implemented by, in call order
pytest_addhooks 1 historic conftest.py
pytest_addoption 1 historic conftest.py
pytest_cmdline_main 1 firstresult _pytest.setupplan [tryfirst]
_pytest.setuponly [tryfirst]
_pytest.mark [tryfirst]
conftest.py
_pytest.helpconfig
_pytest.fixtures
_pytest.main
1
pytest_collect_directory 1 firstresult _pytest.python
_pytest.main
pytest_collect_file 5 conftest.py
_pytest.doctest
_pytest.python
pytest_collection 1 firstresult _pytest.logging.LoggingPlugin (logging-plugin) [wrapper, tryfirst]
_pytest.warnings [wrapper, tryfirst]
_pytest.config.Config (pytestconfig) [wrapper]
_pytest.terminal.TerminalReporter
conftest.py
_pytest.assertion
_pytest.main
pytest_collection_finish 1 _pytest.fixtures.FixtureManager (funcmanage)
_pytest.terminal.TerminalReporter
conftest.py
2
pytest_collection_modifyitems 1 _pytest.fixtures.FixtureManager (funcmanage)
conftest.py
_pytest.main
_pytest.mark
pytest_collectreport 3 _pytest.main.Session.pytest_runtest_logreport (session) [tryfirst]
_pytest.terminal.TerminalReporter
conftest.py
pytest_collectstart 3 _pytest.main.Session [tryfirst]
conftest.py
pytest_configure 1 historic conftest.py
_pytest.subtests
_pytest.faulthandler
_pytest.warnings
_pytest.threadexception
_pytest.unraisableexception
_pytest.junitxml
_pytest.assertion
_pytest.tmpdir
_pytest.legacypath
_pytest.skipping
_pytest.unittest
_pytest.debugging
_pytest.terminal
_pytest.python
_pytest.mark
_pytest.config.PytestPluginManager (<anonymous>)
_pytest.pastebin [trylast]
_pytest.logging [trylast]
3
pytest_exception_interact 1 _pytest.faulthandler [tryfirst]
conftest.py
pytest_fixture_post_finalizer 11 conftest.py
_pytest.setuponly
pytest_fixture_setup 11 firstresult _pytest.setuponly [wrapper]
_pytest.setupplan [tryfirst]
conftest.py
_pytest.fixtures
pytest_generate_tests 4 _pytest.fixtures.FixtureManager (funcmanage)
conftest.py
_pytest.python
pytest_ignore_collect 6 firstresult conftest.py
_pytest.main
pytest_itemcollected 8 conftest.py
pytest_load_initial_conftests 1 _pytest.warnings [wrapper]
_pytest.capture [wrapper]
_pytest.legacypath [tryfirst]
_pytest.config.Config (pytestconfig) [trylast]
4
pytest_make_collect_report 3 firstresult _pytest.fixtures.FixtureManager (funcmanage) [wrapper]
_pytest.capture.CaptureManager [wrapper]
conftest.py
_pytest.runner
5
pytest_make_parametrize_id 5 firstresult conftest.py
pytest_plugin_registered 112 historic _pytest.legacypath
conftest.py
_pytest.terminal.TerminalReporter
_pytest.fixtures.FixtureManager (funcmanage)
pytest_pycollect_makeitem 10 firstresult conftest.py
_pytest.unittest
_pytest.python [trylast]
pytest_pycollect_makemodule 1 firstresult conftest.py
_pytest.python
pytest_pyfunc_call 7 firstresult conftest.py
_pytest.python [trylast]
pytest_report_collectionfinish 1 conftest.py
pytest_report_teststatus 24 firstresult _pytest.subtests [tryfirst]
conftest.py
_pytest.skipping
_pytest.runner
_pytest.terminal [trylast]
6
pytest_runtest_call 7 _pytest.logging.LoggingPlugin (logging-plugin) [wrapper]
_pytest.capture.CaptureManager [wrapper]
_pytest.skipping [wrapper]
conftest.py
_pytest.runner
_pytest.unraisableexception [trylast]
_pytest.threadexception [trylast]
pytest_runtest_logfinish 8 _pytest.logging.LoggingPlugin (logging-plugin)
conftest.py
7
pytest_runtest_logreport 23 _pytest.main.Session [tryfirst]
_pytest.logging.LoggingPlugin (logging-plugin)
_pytest.terminal.TerminalReporter
conftest.py
pytest_runtest_logstart 8 _pytest.logging.LoggingPlugin (logging-plugin)
_pytest.terminal.TerminalReporter
conftest.py
pytest_runtest_makereport 23 firstresult _pytest.tmpdir [wrapper, tryfirst]
_pytest.skipping [wrapper]
_pytest.unittest [tryfirst]
conftest.py
_pytest.runner
pytest_runtest_protocol 8 firstresult _pytest.warnings [wrapper, tryfirst]
_pytest.assertion [wrapper, tryfirst]
_pytest.unittest [wrapper]
_pytest.faulthandler [wrapper, trylast]
conftest.py
_pytest.runner
pytest_runtest_setup 8 _pytest.logging.LoggingPlugin (logging-plugin) [wrapper]
_pytest.capture.CaptureManager [wrapper]
_pytest.skipping [tryfirst]
conftest.py
_pytest.runner
_pytest.unraisableexception [trylast]
_pytest.threadexception [trylast]
pytest_runtest_teardown 8 _pytest.logging.LoggingPlugin (logging-plugin) [wrapper]
_pytest.capture.CaptureManager [wrapper]
conftest.py
_pytest.runner
_pytest.unraisableexception [trylast]
_pytest.threadexception [trylast]
pytest_runtestloop 1 firstresult _pytest.logging.LoggingPlugin (logging-plugin) [wrapper]
_pytest.terminal.TerminalReporter [wrapper]
conftest.py
_pytest.main
8
pytest_sessionfinish 1 _pytest.logging.LoggingPlugin (logging-plugin) [wrapper, tryfirst]
_pytest.terminal.TerminalReporter [wrapper]
_pytest.warnings [wrapper]
conftest.py
_pytest.assertion
_pytest.tmpdir
_pytest.runner
pytest_sessionstart 1 _pytest.logging.LoggingPlugin (logging-plugin) [wrapper, tryfirst]
conftest.py
_pytest.fixtures
_pytest.runner
_pytest.terminal.TerminalReporter [trylast]
pytest_terminal_summary 1 _pytest.terminal.TerminalReporter [wrapper]
_pytest.warnings [wrapper]
conftest.py
_pytest.pastebin
_pytest.runner
pytest_unconfigure 1 _pytest.logging.LoggingPlugin (logging-plugin)
_pytest.terminal.TerminalReporter
conftest.py
_pytest.faulthandler
_pytest.unraisableexception
_pytest.doctest
_pytest.junitxml
_pytest.pastebin
_pytest.mark
9
pytest_warning_recorded 2 historic _pytest.terminal.TerminalReporter
conftest.py

9 of these changed implementer somewhere in pytest 8.1.1 - 9.1.1 without changing the flow - pytest moves implementations between its own plugins. The table shows the newest; what changed along the way is below.

  1. 8.2.0 gained _pytest.fixtures and lost _pytest.python ↩

  2. 9.1.0 gained _pytest.fixtures.FixtureManager (funcmanage) ↩

  3. 8.4.0 gained _pytest.threadexception, _pytest.unraisableexception; 8.4.1 gained _pytest.unittest; 9.0.0 gained _pytest.subtests; 9.1.0 gained _pytest.assertion ↩

  4. 8.4.0 lost _pytest.python_path ↩

  5. 9.1.0 gained _pytest.fixtures.FixtureManager (funcmanage) ↩

  6. 9.0.0 gained _pytest.subtests ↩

  7. 8.3.0 lost _pytest.terminal.TerminalReporter ↩

  8. 8.3.0 gained _pytest.terminal.TerminalReporter ↩

  9. 8.4.0 lost _pytest.python_path; 9.1.0 gained _pytest.unraisableexception ↩