tests/run.sh always passes the whole tests directory to pytest, so naming a test file in its arguments runs the whole suite, not the file #274

Found by lane openRepoTools-2's independent review of #228's pull request 3, now #268 (its verdict is #228's comment 6273189381: the bullet "The run.sh trap", and run b0). Lane openRepoTools-4 (#228, comment 6268952914) and lane 2's interim findings (comment 6268030648) had met the same thing. Lane 3 named the fix a v1.2.0 candidate on #228 (comment 6269261641): "Making run.sh honour a path is a v1.2.0 candidate, which lane 3 proposes to Brett Heap."

What happens

• tests/run.sh always runs python3 -m pytest tests -q -p no:cacheprovider --basetemp=… and then appends the person's arguments. tests is in the command whatever the arguments are, so tests/run.sh tests/test_release_workflow.py runs the whole suite, with the file named a second time.
• Measured by lane 2's review (run b0): the literal run.sh tests/test_release_workflow.py --collect-only collects all 33 modules, 2,332 items, at PR 3's 29cc1b3. With -x it stops at the first failure anywhere in the suite (the review's bullet; lane 2's interim comment says the same).
• Measured by lane 4 on The register's rewrites refuse a copy or write they cannot prove and put a cut-short register back, retire-rows refuses an unpushed log rename before it writes and calls a lane closed only from its published log, and add-row refuses an archive row it cannot read and takes back a row refused after its rebase (#120, #122) #263's tree: a named file "runs alongside all 1903 tests". The register's rewrites refuse a copy or write they cannot prove and put a cut-short register back, retire-rows refuses an unpushed log rename before it writes and calls a lane closed only from its published log, and add-row refuses an archive row it cannot read and takes back a row refused after its rebase (#120, #122) #263's writer found it with a collect-only probe before its first real run.
• The failure scenario: a writer told to run only the files they touch goes through tests/run.sh tests/<file>.py -x. The run is the whole suite, which the wrapper's own header calls "minutes of bash and hundreds of git processes" (tests/run.sh:9-10), held under the workstation's lock the whole time (tests/run.sh:360-371), and it stops at the first failure in any module, including one the writer never touched.
• The workaround is -k. run.sh -k test_release_workflow ran 312 passed and 2,020 deselected (run b1, 312 + 2,020 = 2,332). Lane 4's advice: then confirm with --collect-only that the selection is the file meant. Lane 3 corrected the board's test rule to -k (Releases: annotated tags, release/vX.Y branches, and openrepotools on TestPyPI and PyPI from v1.1.0 #228, comment 6269261641), and lane openRepoTools-5's T025 run used each step's -k selection for the same reason (Releases: annotated tags, release/vX.Y branches, and openrepotools on TestPyPI and PyPI from v1.1.0 #228, comment 6269608275).
• The documentation says the opposite of what happens: tests/run.sh:5-6 ("…and anything else pytest takes") and AGENTS.md:233 ("the suite, serialized — pass any pytest argument").

Where

• tests/run.sh:342 and :348, on main 00d52ae: the two python3 -m pytest tests -q -p no:cacheprovider --basetemp="$RUN_ROOT/basetemp" ${1+"$@"} lines (the first runs the suite in the foreground when stdin is a terminal, the second in the background). :338 echoes the same line.
• Pinned by: tests/test_run_wrapper.py:197 (rec["arg"][:5] == ["-m", "pytest", "tests", "-q", "-p"]), tests/test_repo_hygiene.py:4265 ("python3 -m pytest tests -q" in text), and AGENTS.md:266, which writes the form out.

How it was found

• Lane 2 reported it in its interim findings on PR 3 (Releases: annotated tags, release/vX.Y branches, and openrepotools on TestPyPI and PyPI from v1.1.0 #228, comment 6268030648), and lane 4 reported it against the board's rule "run only the files you touch" (comment 6268952914). Lane 2's review of PR 3 measured it again with run b0 (comment 6273189381).

The fix

• When an argument names a path under tests/, pass it in place of tests. Lane 4 put the same as "run.sh could stop adding tests when a path is given" (Releases: annotated tags, release/vX.Y branches, and openrepotools on TestPyPI and PyPI from v1.1.0 #228, comment 6268952914).
• What a fix has to settle (nothing here is decided):
  • Which arguments count as naming a path. A node id such as tests/test_x.py::test_name does. A value that an option takes and that names a path under tests/ without selecting it, as pytest's --ignore and --deselect do, is not a selection.
  • The three places that pin the old form: tests/test_run_wrapper.py:197, tests/test_repo_hygiene.py:4265 and AGENTS.md:266. With no path given, the command must stay as they have it.
  • The test: tests/test_run_wrapper.py:187, test_a_run_relocates_bytecode_caches_and_temp_and_leaves_none_of_it, already runs the wrapper against a stub that records its argv. A case with a path argument fits beside it.

Why it is not in v1.1.0

• It changes how the wrapper every lane runs its tests through treats its arguments, and it came out of Releases: annotated tags, release/vX.Y branches, and openrepotools on TestPyPI and PyPI from v1.1.0 #228's review rounds. The workaround, -k, works, and the board's rule was corrected to it. Lane 3 named the change a v1.2.0 candidate (Releases: annotated tags, release/vX.Y branches, and openrepotools on TestPyPI and PyPI from v1.1.0 #228, comment 6269261641).

Filed in milestone v1.2.0 on Brett Heap's word (2026-10-19T18:3xZ, "yes, file the v1.2.0 issues"). It is unclaimed.

Lane: openRepoTools-3

🤖 Generated with Claude Code