T056, the standalone generate path, end to end (5.1 part) (plan 034)

[brettheap]

Lane: openxfactory-4 (openXfactory-4-openDox_extraction)

Plan 034 T056 [US2] [oDc], the standalone generate path, end to end (#1144's 5.1, in part), from specs/034-opendox-standalone-operation/tasks.md at openxFactory main 91e4685f:

python -m opendox.cli generate and generate-and-open --no-open run on the fixture with neither sibling importable, and the server STARTS (the limit measured in research R7 is lifted).

• Falsifier: F5.3; F10.1's generate-and-open run through python -m opendox.cli, with a plain install and no --local, which arrives in phase 3 (T070); and the verb's half of spec.md's stage: edge case. python -m opendox.cli generate, over a copy of T050's fixture in which one document declares a stage: value outside the six role keys, reports it, naming the document, the value and the six keys, and the snapshot it writes reads that document as a source. T054 tests the projection's half in process. F10.1 as batch H amends it is T077's.
• Ruled: R1Q22 (a), 5817152735.
• After: T055.

Claimed on openxFactory#656 in comment 5901343950.

Was stacked on #59 (T055), and is now on main

• The PR was cut at T055, serve and generate standalone (5.5, 4.3 part) (plan 034) #59's head e3ef506a. T055, serve and generate standalone (5.5, 4.3 part) (plan 034) #59's heads 814516b7, c2a8ad9f and 183b40fc were merged in at 5a26532e, e3574774 and 38761c76.
• T055, serve and generate standalone (5.5, 4.3 part) (plan 034) #59 has landed as fa140875, a squash of 0c946f4e, whose tree fa140875 equals. The base is now main, and main is merged in at a6e953ce.
• T055, serve and generate standalone (5.5, 4.3 part) (plan 034) #59 had moved after 183b40fc only in default_registry.py and tests/test_projection_seams.py. T056 touches neither, so the two add/add conflicts took main's side.
• The merge carries exactly T056's delta. The stable patch-id of git diff 183b40fc 38761c76 equals the patch-id of git diff main a6e953ce: eee3f01c both times.
• This PR's diff is T056's alone, four files. T054, openDox's small neutral projection (5.1-5.3) (plan 034) #57 (a691e4e4), T057, openDox's own validator and its input set (7.1, 7.1a, 7.1b, 7.2) (plan 034) #58 (8ec08e91) and T055, serve and generate standalone (5.5, 4.3 part) (plan 034) #59 (fa140875) have all landed, so nothing gates it but its own review.
• F10.1's plain-install run is not here. It needs the console script with no --local, which arrives in phase 3 (T070, and T077 as batch H amends it).

What T056 found, and the one fix it needed

At #59's head the verbs already run standalone. T055's writer measured the verb's half of F5.3 there, and this PR confirms it. The server also starts. But on a pipe, it never says where it started.

• cli.cmd_generate_and_open and serve.serve print the URL and then block in serve_forever(). Neither flushed first.

• When standard output is a pipe or a file, Python buffers it by block. So a wrapper reading the URL never sees it while the server runs. It cannot learn an ephemeral port, or tell that the server started.

• Measured at e3ef506a, with the siblings blocked and --port 0:

  entry point	stdout on a pipe, default buffering	with PYTHONUNBUFFERED=1
  python -m opendox.cli generate-and-open --no-open	0 lines in 20 s	8 lines, the URL in 0.4 s
  python -m opendox.serve --snapshot … --checkout-root …	0 lines in 15 s	the URL in 0.2 s

• The fix is flush=True on the URL line and on "serving until interrupted" in cli.py, and on serve.serve's announcement in serve.py. Each carries a comment saying why.

The tests: tests/test_standalone_generate_path.py (new)

Each case runs a real child process, python -m …, the way F5.3 is written. The child is built by tests/standalone_child.py, a new helper module that holds no case. T058's F7.2 case uses the same helper.

How "neither sibling is importable" is made true. A sitecustomize sits on a directory put first on PYTHONPATH.

• It installs a meta-path finder that refuses openxdox, ideation_dashboard, doc_health and corpus_adapter_openxfactory, whatever is installed. It also drops any of them that a .pth file imported before it ran.
• It logs every name it refuses, and each case asserts that the log is empty. A refused import that an except ImportError swallowed would otherwise pass as a degraded run.

The child's stdout is a pipe with default buffering. PYTHONUNBUFFERED is taken out of its environment on purpose, because that is what a wrapper sees.

case	what it holds
test_F5_3_generate_through_the_module_writes_the_neutral_snapshot	#1144's F5.3: generate over a fresh copy of T050's fixture. The snapshot is non-empty and of kind opendox-snapshot, and none of F5.3's 14 declared words is in a string value.
test_the_verb_reports_a_stage_outside_the_six_and_reads_it_as_a_source	candidate-toolshed-rebuild.md is edited to stage: someday. There is exactly one notice:, naming the document and 'someday', and carrying the six keys as one rendered list, (source, grouping, candidate, selection, submission, completion). The document's entry is stage: source, and no string value in the snapshot contains someday.
test_the_unedited_fixture_declares_that_document_a_candidate	The control: as T050 ships it, the same document is a candidate and draws no notice.
test_generate_and_open_starts_a_server_that_answers_with_no_sibling	generate-and-open --no-open --port 0, with no --no-serve. The URL is read off the pipe while the child runs. /index.html is 200 and HTML. /snapshot.json is 200 and equals the written file. /capabilities is 200, with the regenerate binding. /source/notes-toolshed-inventory.md is 200, byte-equal to the document. /source/.git/config is 404. SIGINT exits 0, and the port is closed afterwards.
test_serve_main_starts_a_server_that_answers_with_no_sibling	The same checks for python -m opendox.serve, over a snapshot generate wrote.
test_a_child_that_ignores_the_interrupt_is_killed_at_the_deadline	The harness itself: a child that ignores SIGINT is killed at the deadline, and the timeout is raised, so a server that will not stop is reported rather than waited out.
test_a_child_stops_on_the_interrupt_even_when_the_runner_ignores_it	The harness itself: a child stops on the interrupt with status 0 even when the runner ignores SIGINT, as a suite started with nohup … & does. The parent's handler is back afterwards. See the section below.

What the merge of main exposed: the runner's ignored SIGINT

After merging main, I ran the whole suite as nohup pytest … &. Cases 3 and 4 failed with TimeoutExpired at the interrupt. Run in the foreground, the same module passed 6 of 6.

The cause is the runner, not the server:

• POSIX starts an asynchronous command with SIGINT ignored when job control is off. Under nohup … &, a probe reads signal.getsignal(SIGINT) == 1, which is SIG_IGN. In the foreground it reads default_int_handler.
• An ignored signal survives exec, and Python installs its KeyboardInterrupt handler only where SIGINT was not ignored. So every server the harness started ignored the interrupt, and the two cases reported how the suite had been launched.
• The estate runs long suites exactly this way. Any lane running this module under nohup would have read a false red.

The fix is in the harness, not the product. A program started with SIGINT ignored should keep it ignored. The child's sitecustomize in tests/standalone_child.py now sets SIGINT back to signal.default_int_handler. That models what the cases claim, a Ctrl-C at a terminal. A child that ignores SIGINT itself, after startup, still does, and case 5 still kills it at the deadline.

The falsifier is test_a_child_stops_on_the_interrupt_even_when_the_runner_ignores_it. It builds that runner in process: it ignores SIGINT while the child starts, and restores it at once.

• Before the fix it is red (TimeoutExpired at 10 s), and that is in a foreground run.
• After the fix it is green.
• The module under nohup … & went from 2 failed, 4 passed to 7 passed.
• Mutant M14, the reset removed, is killed by it.

The falsifiers, failing before and passing after

F5.3, verbatim from #1144 (a fresh venv, pip install ".[test]", the sibling-absence assertion, python -m opendox.cli generate, then the vocabulary scan):

• At openDox-code main fa8862cc, before T055 routed the verb, it exits 1:
  opendox.consumer_reach.ConsumerReachUnavailable: 'openxdox.corpus_root' belongs to openXdox, the layer that PINS this one, …
• At this head it exits 0:

  wrote …/snap.json
    repository=fixture kind=opendox-snapshot
    documents=8 clusters=2 possibles=1 staged_topics=1 changes=2 keywords=28
  documents: 8
  

The verb's half of the stage: edge case, over a copy of the fixture with one document edited to stage: someday:

notice: candidate-toolshed-rebuild.md: its stage: value 'someday' is not one of the six station role keys (source, grouping, candidate, selection, submission, completion), so it is not a declaration; the document is read as a source

The snapshot reads that document as ['source'].

This file:

• At main fa8862cc: 5 failed. Every verb refuses with ModuleNotFoundError: No module named 'openxdox'.
• At T055, serve and generate standalone (5.5, 4.3 part) (plan 034) #59's head e3ef506a, without the two flushes: 2 failed, 3 passed. The two server-start cases fail with python -m opendox.cli generate-and-open never printed a line matching '^(http://…)/index\.html$' on its (buffered) standard output while it ran: exit status None, standard output '', and the same for opendox.serve.
• At 38761c76: 6 passed (3.4 s). It also passes under --noconftest.
• At 149d7295, and at this head a7bda066: 7 passed, both in the foreground and under nohup … & (4.6 s).

Copilot:

• At 65c943ff, approval recommended, with no findings.
• At 1597511d, one finding, r4139607689: an interrupt the child ignores held the caller while its pipe readers were joined. Fixed in 42a08a31: the child is killed before the join. The new case fails without the fix and passes with it. The thread is answered with that evidence and resolved.
• At 5a26532e, one finding, r4139809410: the stage: case rejected only a string EQUAL to someday, not one containing it. Fixed in e939c31f. A mutant that sets the summary to stage: someday passes the old check and fails the new one. Answered and resolved.
• At e3574774, one finding, r4146289331: "serving until interrupted" was checked only after the interrupt, when Python's exit flush delivers it anyway. Fixed in caa01caf: the case reads the line while the child runs. Mutant M11, that line unflushed, now fails. Answered and resolved.
• At 38761c76, approval recommended, with no findings.
• At 149d7295, one finding, r4147767447: the stage: case checked each role key as a word anywhere in the notice. But candidate is in the document's name and source is in "read as a source", so a list that left both out still passed. Mutant M15, that list, SURVIVED the old case. Fixed in a7bda066: the whole rendered list is one substring. M15 is now killed. Answered and resolved.
• At this head a7bda066, a review is re-requested through the reviewer API.

Mutation check: 15 of 15 killed, re-run at this head

Each mutant was applied to the source, the named cases were run, and the sources were restored and checked by sha256.

mutant	killed by
M1: generate-and-open flushes neither line (the pre-T056 code)	the generate-and-open case
M2: serve.serve's URL line is not flushed	the serve case
M3: /source serves a hidden path (both hidden-name checks dropped)	both server cases
M4: a sibling probe is swallowed on the generate path (try: import openxdox / except ImportError: pass)	F5.3, stage:, generate-and-open (the refused-import log)
M5: an out-of-six stage: is read as a candidate	stage:
M6: the notice omits the six keys	stage:
M7: the verb does not print the notice	stage:
M8: a governed word (draft) leaks into a title	F5.3
M9: no stage: value is a declaration	the control
M10: generate-and-open exits non-zero on an interrupt	generate-and-open
M11: the serving line is not flushed (the URL line still is)	generate-and-open, which reads that line while the child runs
M12: the undeclared stage: value is carried inside a larger string	stage:
M13: an ignored interrupt is joined before the child is killed (the harness)	the ignored-interrupt case
M14: the child inherits the runner's ignored SIGINT (the sitecustomize reset removed)	the runner-ignores-SIGINT case
M15: the notice's key list leaves out candidate and source, which both appear elsewhere in the notice	stage: (survived the word-by-word check, which is why that check was replaced)

Two first spellings were equivalent, and each was re-expressed as the real regression:

• Dropping only the URL line's flush is covered by the flush on the next line under --no-open.
• Dropping only the first hidden-name check is covered by the second, which still refuses .git/config.

The repository's own checks

The whole suite ran locally as CI runs it: CI=true, LANG=C.UTF-8, PostgreSQL 16, -e ".[runtime,test]" with the constraints file, at the committed head, with a clean tree.

tree	passed	skipped
#59's head e3ef506a	2724	11
65c943ff	2729	11
1597511d (the harness moved into tests/standalone_child.py)	2729	11
42a08a31 (an ignored interrupt is killed before the pipes are joined)	2730	11
5a26532e (#59's 814516b7 merged in)	2753	11
e939c31f (the stage: case looks inside every string)	2753	11
e3574774 (#59's c2a8ad9f, with #58's landing, merged in)	2982	11
38761c76 (the serving line read while running; #59's 183b40fc merged in)	2983	11
a6e953ce (main merged in, #59 landed), run under nohup … &	2982, 2 failed	11
149d7295 (the harness resets SIGINT), run under nohup … &	2985	11
this head a7bda066 (the six keys checked as one rendered list), run under nohup … &	2985	11

At each head, the case-by-case junit diff against the base shows only this file's cases added, 0 removed and 0 changed outcomes, and the same 11 skips. Against 38761c76, this head adds exactly two cases, both passing. One is test_a_child_stops_on_the_interrupt_even_when_the_runner_ignores_it. The other is test_a_regenerate_promotes_no_session_and_moves_no_active_key, which #59 gained after 183b40fc. pyflakes finds nothing in the two new files. No floor, workflow, pyproject.toml, conftest.py or pin is touched.

Files

• src/opendox/cli.py: flush=True on the URL line and on "serving until interrupted" in cmd_generate_and_open.
• src/opendox/serve.py: flush=True on serve.serve's announcement.
• tests/test_standalone_generate_path.py: new, the seven cases.
• tests/standalone_child.py: new, the child-process harness (a helper module, no case). Its sitecustomize refuses the siblings and sets SIGINT back to Python's handler.

Both are created files, with no carve-manifest row (RULED OQ-C).

For the holder

• The overlap. With T055, serve and generate standalone (5.5, 4.3 part) (plan 034) #59 landed, this PR's edits in cli.py and serve.py are three print calls in two functions. It does not touch default_registry.py or serve_project.py. But cases 3 and 4 GET /source/notes-toolshed-inventory.md, expecting 200 and byte-equal, and /source/.git/config, expecting 404, and both requests run through serve_project and default_registry.resolve_source. So the follow-up to T055, serve and generate standalone (5.5, 4.3 part) (plan 034) #59 on resolve_source's second lookup should run this module.
• An observation, not changed here. Standalone, /capabilities answers actions.gate: true: it is local_human, meaning a resolved actor, a real checkout and loopback. I did not check whether any gate route answers without openXdox's contributed verbs.

Arc: neutral-product-standalone-operability
Lane: openxfactory-4 (openXfactory-4-openDox_extraction)

🤖 Generated with Claude Code

[Copilot]

Copilot review overview

🟡 Changes recommended

One newly flushed status line is checked only after process exit, leaving its real-time buffering behavior untested.

Review effort: Balanced
Findings: 1

Open (1)

• Test buffering before SIGINT instead of after process termination · New

[Copilot]

Copilot review overview

🟢 Approval recommended

The buffering fix is targeted and comprehensively covered by subprocess-level regression tests.

Review effort: Balanced
Findings: None

Resolved since last review (1)

• Test buffering before SIGINT instead of after process termination

[Copilot]

Copilot review overview

🟡 Changes recommended

The stage-notice test can pass when two required role keys are omitted from the advertised key list.

Review effort: Balanced
Findings: 1

Open (1)

• Assert the complete rendered role key list · New

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Copilot]

Copilot review overview

🟢 Approval recommended

The buffering fix is correct and thoroughly covered by focused end-to-end regression tests.

Review effort: Balanced
Findings: None

Resolved since last review (1)

• Assert the complete rendered role key list

[brettheap]

READY at a7bda06 — Lane: openxfactory-4 (openXfactory-4-openDox_extraction)

[sourcery-ai]

Sorry @brettheap, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 1 day and 19 hours by commenting @sourcery-ai review. Upgrade to get a review now.

[brettheap]

Lane: openxfactory-4 (openXfactory-4-openDox_extraction)

LANDED — lane openxfactory-4, 2026-09-30T18:28:29Z, PR #66 → a23e422 (opensoft/openDox-code main; plain gate)

Brett: land the phase 1 PRs when green

[brettheap]

Record correction (no code change). The landed squash commit a23e422's message cites three 8-character shas with a wrong 8th character, taken from 7-character oneline output: 42a08a3a should be 42a08a3, caa01ca7 should be caa01ca, and e939c315 should be e939c31. The PR body and review replies were corrected before landing; main is not rewritten.

Lane: openxfactory-4 (openXfactory-4-openDox_extraction)