From d5d1aa4e638c21ff20df76633bd1cde047b9f539 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sun, 27 Sep 2026 16:05:30 +0000 Subject: [PATCH 01/38] T050: the plain-documents fixture, spread across the six stations (plan 034) tests/fixtures/plain-documents/ carries eight .md documents: three sources (no `stage:` header, two sharing the phrase "rain barrel" so a future topic-based grouping pass has a pair to find) and one document each declaring `stage: grouping`, `stage: candidate`, `stage: selection`, `stage: submission` and `stage: completion` (RULED R1Q13 (a) with (c), openxFactory#656 comment 5850003126). Every document carries the small neutral field set the default adapter will require regardless of station (`title`, `summary`). tests/test_plain_documents_fixture.py is the vocabulary test T050 names as its falsifier: it asserts the fixture carries none of the eight controlled `Status:` words or the change/spec/delta nouns, that it covers all six stations with exactly one document per explicit station, and that at least two (but not all) sources share the topic fixture code will need to group on. Not yet wired into .github/workflows/validate.yml's explicit pytest list: phase2-ahead scope keeps this PR out of that file, the pin files and conftest.py/pyproject.toml/README.md, which the phase-1 chain still edits. This suite runs by node id today. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- .../candidate-toolshed-rebuild.md | 11 + .../completion-path-resurfacing.md | 11 + .../grouping-compost-corner.md | 12 + .../plain-documents/notes-rain-barrel-leak.md | 9 + .../notes-rain-barrel-overflow.md | 9 + .../notes-toolshed-inventory.md | 10 + .../selection-spring-planting-plan.md | 11 + .../submission-shed-door-repair.md | 11 + tests/test_plain_documents_fixture.py | 219 ++++++++++++++++++ 9 files changed, 303 insertions(+) create mode 100644 tests/fixtures/plain-documents/candidate-toolshed-rebuild.md create mode 100644 tests/fixtures/plain-documents/completion-path-resurfacing.md create mode 100644 tests/fixtures/plain-documents/grouping-compost-corner.md create mode 100644 tests/fixtures/plain-documents/notes-rain-barrel-leak.md create mode 100644 tests/fixtures/plain-documents/notes-rain-barrel-overflow.md create mode 100644 tests/fixtures/plain-documents/notes-toolshed-inventory.md create mode 100644 tests/fixtures/plain-documents/selection-spring-planting-plan.md create mode 100644 tests/fixtures/plain-documents/submission-shed-door-repair.md create mode 100644 tests/test_plain_documents_fixture.py diff --git a/tests/fixtures/plain-documents/candidate-toolshed-rebuild.md b/tests/fixtures/plain-documents/candidate-toolshed-rebuild.md new file mode 100644 index 0000000..67bbd95 --- /dev/null +++ b/tests/fixtures/plain-documents/candidate-toolshed-rebuild.md @@ -0,0 +1,11 @@ +stage: candidate +title: Rebuilding the toolshed door and frame +summary: A short write-up on replacing the toolshed's sagging door frame, put forward as one option worth picking. + +# Rebuilding the toolshed door and frame + +The toolshed door frame has sagged enough that the door no longer latches +without a shove. One option on the table is a full frame rebuild over a +weekend, using pressure-treated lumber and new hinges, rather than another +temporary shim. It would cost a Saturday and a modest supply run, and it +would settle a problem that keeps coming back every few months. diff --git a/tests/fixtures/plain-documents/completion-path-resurfacing.md b/tests/fixtures/plain-documents/completion-path-resurfacing.md new file mode 100644 index 0000000..2fb75da --- /dev/null +++ b/tests/fixtures/plain-documents/completion-path-resurfacing.md @@ -0,0 +1,11 @@ +stage: completion +title: The gravel path resurfacing is done +summary: A short note closing out the gravel path work: what was laid, how much it took, and what is left over. + +# The gravel path resurfacing is done + +The path from the gate to the shared beds has a fresh layer of gravel now, +about two inches deep over the old base, and the low spot that used to +hold water after storms is filled and level. It took four wheelbarrow +loads more than expected, and the leftover gravel is stacked by the +toolshed for whoever tackles the side path next. diff --git a/tests/fixtures/plain-documents/grouping-compost-corner.md b/tests/fixtures/plain-documents/grouping-compost-corner.md new file mode 100644 index 0000000..bdce97e --- /dev/null +++ b/tests/fixtures/plain-documents/grouping-compost-corner.md @@ -0,0 +1,12 @@ +stage: grouping +title: Compost corner ideas, gathered in one place +summary: Three separate notes about turning food scraps into usable soil, pulled into a single set. + +# Compost corner ideas, gathered in one place + +Several notes about a compost corner behind the shared beds turned out to +be circling the same handful of questions: how often to turn the pile, +whether food scraps need to be covered with leaves right away, and where a +second bin would fit without blocking the wheelbarrow path. Gathering them +here makes it easier to see what they have in common before picking one to +work on next. diff --git a/tests/fixtures/plain-documents/notes-rain-barrel-leak.md b/tests/fixtures/plain-documents/notes-rain-barrel-leak.md new file mode 100644 index 0000000..88c6499 --- /dev/null +++ b/tests/fixtures/plain-documents/notes-rain-barrel-leak.md @@ -0,0 +1,9 @@ +title: A slow leak at the rain barrel +summary: The seal under the downspout drips after every storm and the ground underneath stays soft. + +# A slow leak at the rain barrel + +Water pools under the rain barrel near the garden gate after every storm. +The rubber seal where the downspout meets the lid looks cracked on one +side. Someone should check whether a new seal or a different downspout +fitting would keep the ground from staying soft all week. diff --git a/tests/fixtures/plain-documents/notes-rain-barrel-overflow.md b/tests/fixtures/plain-documents/notes-rain-barrel-overflow.md new file mode 100644 index 0000000..f464ddb --- /dev/null +++ b/tests/fixtures/plain-documents/notes-rain-barrel-overflow.md @@ -0,0 +1,9 @@ +title: The rain barrel overflows before the storm ends +summary: A second barrel or an overflow hose would carry extra water to the garden beds instead of the path. + +# The rain barrel overflows before the storm ends + +The single rain barrel by the garden gate fills within twenty minutes of a +hard storm and then spills straight onto the gravel path. A short hose run +to the garden beds, or a second barrel chained to the first, would put that +extra water to use instead of letting it wash out the path edge. diff --git a/tests/fixtures/plain-documents/notes-toolshed-inventory.md b/tests/fixtures/plain-documents/notes-toolshed-inventory.md new file mode 100644 index 0000000..6eae560 --- /dev/null +++ b/tests/fixtures/plain-documents/notes-toolshed-inventory.md @@ -0,0 +1,10 @@ +title: What is actually in the toolshed +summary: A walk-through count of hand tools, hoses and pots, so nobody buys a third trowel. + +# What is actually in the toolshed + +A slow Saturday walk-through of the toolshed turned up four trowels, two +working wheelbarrows and one with a flat tire, a tangle of hoses that +mostly still hold water, and more clay pots stacked in the back corner than +anyone remembered buying. Worth doing again before the next round of seed +starting. diff --git a/tests/fixtures/plain-documents/selection-spring-planting-plan.md b/tests/fixtures/plain-documents/selection-spring-planting-plan.md new file mode 100644 index 0000000..613088f --- /dev/null +++ b/tests/fixtures/plain-documents/selection-spring-planting-plan.md @@ -0,0 +1,11 @@ +stage: selection +title: What we are planting behind the shared beds this spring +summary: The picked list of vegetables and flowers for the shared beds, chosen from several ideas floated over the winter. + +# What we are planting behind the shared beds this spring + +After looking at several ideas floated over the winter, the shared beds +this spring will carry tomatoes, bush beans, a border of marigolds, and a +short row of sunflowers along the back fence. Herbs stay in the raised +boxes near the gate, where they did well last year. This is the picked +list the bed captains are working from until the ground is turned. diff --git a/tests/fixtures/plain-documents/submission-shed-door-repair.md b/tests/fixtures/plain-documents/submission-shed-door-repair.md new file mode 100644 index 0000000..fcde0ac --- /dev/null +++ b/tests/fixtures/plain-documents/submission-shed-door-repair.md @@ -0,0 +1,11 @@ +stage: submission +title: Shed door repair, sent in for the work weekend +summary: A ready-to-go write-up of the shed door repair, sent in for the next work weekend's list. + +# Shed door repair, sent in for the work weekend + +This write-up covers the shed door repair end to end: new hinges, a +replacement latch, and enough pressure-treated lumber for the frame pieces +that have started to soften. It is sent in as one item for the next work +weekend's list, with the tools and the rough time it will take already +worked out, so the crew can pick it up without more back and forth first. diff --git a/tests/test_plain_documents_fixture.py b/tests/test_plain_documents_fixture.py new file mode 100644 index 0000000..6a0f23d --- /dev/null +++ b/tests/test_plain_documents_fixture.py @@ -0,0 +1,219 @@ +"""The plain-documents fixture: spread across the six stations, and clean of +the publishing repository's own vocabulary (plan 034, T050). + +`tests/fixtures/plain-documents/` is 5.0's fixture (plan 034 tasks.md § Phase +2, slice P2-F): a handful of `.md` documents a plain, ungoverned git +repository could contain, read by AT-R1 (spec.md § "AT-R1 — the release-1 +acceptance test", step 3(a)) and by F5.3, F7.2, F10.1 and F13.1 once those +falsifiers exist. None of it is wired to any `opendox` code yet — T052 and +T054 (the generator seam and the neutral projection) have not landed — so +this suite tests the fixture's own two guarantees rather than a projection +over it. + +THE TWO GUARANTEES, AND WHY. + +1. NEUTRAL VOCABULARY (T050's task line: *"It carries none of the declared + vocabulary: the eight `Status:` words and the change/spec/delta + nouns."*). A neutral product shipping its own publisher's governance + words in ITS TEST DATA is the same defect requirement 3's second scenario + refuses in RENDERED words and names + (`tests/test_default_profile.py`); this is the shipped-fixture half of + that same property, swept on its own footing because a fixture is prose + no engine renders and the rendering sweep never sees it. +2. SIX STATIONS, ONE GROUP (RULED R1Q13 (a) with (c), `openxFactory#656` + comment `5850003126`; spec.md line ~170, ~454-463). A neutral + front-matter key, `stage: `, names a document's station; a + document that declares no `stage:` line at all is a SOURCE. This fixture + carries one document per explicit station (`grouping`, `candidate`, + `selection`, `submission`, `completion`) and three sources, two of which + share the phrase "rain barrel" verbatim so a future topic-based grouping + pass (T054) has a pair to find — the fixture must yield at least one + group, so AT-R1 can open the chat pane from a grouping tile (spec.md + § AT-R1 steps 6-7). + +Every document also carries the small neutral field set the default adapter +will require regardless of station — `title` and `summary` — per T050's task +line and the answer's own example. + +NOT YET WIRED INTO `.github/workflows/validate.yml`'s explicit pytest list: +the phase-2 draft-ahead scope keeps this PR out of that file (conftest.py, +pyproject.toml, validate.yml and README.md are the phase-1 chain's), so this +suite runs by node id today, exactly as other narrowed-out suites in this +tree have (`tests/test_display_facet_leaves.py`'s own S7-residue history). +It joins the enumerated list whichever later task next touches it — most +likely F5.3/T056, or T049's own close. + +A CREATED file: no row in openxFactory's `docs/opendox-carve-manifest.yaml` +(RULED OQ-C: the manifest declares what LEAVES openxFactory, never what a +destination assembles). +""" + +from __future__ import annotations + +import re +import sys +from pathlib import Path + +import pytest + +ROOT = Path(__file__).resolve().parents[1] +FIXTURE_DIR = ROOT / "tests" / "fixtures" / "plain-documents" + +sys.path.insert(0, str(ROOT / "src")) +from opendox.display_profile import STAGE_ROLES # noqa: E402 + +#: Its STATUS TAXONOMY: the eight controlled `Status:` words this project's +#: own root `CLAUDE.md` names verbatim (*"brainstorm | staged | draft | +#: ratified | standard | superseded | retired | record"*), identical to the +#: eight `tests/test_default_profile.py`'s `STATUS_TAXONOMY`, +#: `tests/test_web_boundary.py`'s `_STATUS_WORDS` and +#: `tests/test_display_facet.py`'s `OPENXFACTORY_WORDS` all open with, at +#: openxFactory `133e37d9`. T050's own task line says "the eight", so this +#: suite does not also carry `test_default_profile.py`'s ninth, +#: `projection` — that word is `openxfactory-engineering.yaml`'s own +#: lifecycle addition, a different sweep than this one. +STATUS_WORDS = ( + "brainstorm", "staged", "draft", "ratified", "standard", + "superseded", "retired", "record", +) + +#: Its CHANGE/SPEC/DELTA NOUNS: the same family `tests/test_default_profile.py` +#: carries as `CHANGE_SPEC_DELTA_NOUNS` — the OpenSpec artifact a governed +#: change is, and the proposal that opens one. Copied rather than imported, +#: on the same footing as `STATUS_WORDS`: this fixture must be readable as +#: plain prose by something that never imports `opendox` at all (the +#: acceptance harness copies it into a bare `git init`), so the word list a +#: reader would check it against has to stand on its own too. +CHANGE_SPEC_DELTA_NOUNS = ( + "change", "changes", "spec", "specs", "delta", "deltas", + "openspec", "proposal", "proposals", +) + +DECLARED_VOCABULARY = frozenset(STATUS_WORDS + CHANGE_SPEC_DELTA_NOUNS) + +_WORD_PATTERN = re.compile( + r"\b(" + "|".join(re.escape(word) for word in DECLARED_VOCABULARY) + r")\b", + re.IGNORECASE, +) + +#: The neutral field set the default adapter will require of every document +#: regardless of station (T050's task line: *"e.g. title, summary"*). +REQUIRED_FIELDS = ("title", "summary") + +#: The phrase at least two SOURCE documents (no `stage:` header) share +#: verbatim, so a topic-based grouping pass has a pair to find (R1Q13 (a) +#: with (c)). +SHARED_SOURCE_TOPIC = "rain barrel" + +FIXTURE_FILES = sorted(FIXTURE_DIR.glob("*.md")) if FIXTURE_DIR.is_dir() else [] + + +def _header_and_body(text: str) -> tuple[dict[str, str], str]: + """The same convention `LocalGitCorpus._header_of` reads (`runtime/ + local_git_adapter.py`): the leading run of `Name: value` lines up to the + first blank line is the header, and everything after is the body. + Re-implemented rather than imported — this fixture must be readable as + plain text by something that never builds a corpus at all, exactly as + the acceptance harness (T095) and a human skimming the directory would + read it. + """ + lines = text.splitlines() + header: dict[str, str] = {} + body_start = len(lines) + for i, line in enumerate(lines): + if not line.strip(): + body_start = i + 1 + break + name, separator, value = line.partition(":") + if separator: + header[name.strip()] = value.strip() + return header, "\n".join(lines[body_start:]) + + +def _documents() -> dict[Path, tuple[dict[str, str], str]]: + return {path: _header_and_body(path.read_text(encoding="utf-8")) + for path in FIXTURE_FILES} + + +def test_fixture_directory_exists_and_is_not_empty() -> None: + assert FIXTURE_DIR.is_dir(), f"expected {FIXTURE_DIR} to exist" + assert FIXTURE_FILES, f"no .md fixture files found under {FIXTURE_DIR}" + + +@pytest.mark.parametrize("path", FIXTURE_FILES, ids=lambda p: p.name) +def test_no_declared_vocabulary_word(path: Path) -> None: + """Neither the header nor the body of any fixture document carries one + of the eight `Status:` words or a change/spec/delta noun.""" + text = path.read_text(encoding="utf-8") + hits = sorted({m.group(1).lower() for m in _WORD_PATTERN.finditer(text)}) + assert not hits, ( + f"{path.name} carries declared vocabulary word(s) {hits}; a neutral " + "fixture must not spell its publisher's governance words" + ) + + +@pytest.mark.parametrize("path", FIXTURE_FILES, ids=lambda p: p.name) +def test_required_fields_present(path: Path) -> None: + header, _ = _header_and_body(path.read_text(encoding="utf-8")) + missing = [field for field in REQUIRED_FIELDS if field not in header] + assert not missing, f"{path.name} is missing required field(s) {missing}" + + +def test_spread_across_the_six_stations() -> None: + """One document per explicit station, and every undeclared document is + a source (R1Q13 (a) with (c)).""" + by_role: dict[str, list[Path]] = {role: [] for role in STAGE_ROLES} + for path, (header, _) in _documents().items(): + stage = header.get("stage") + if stage is None: + by_role["source"].append(path) + continue + assert stage in STAGE_ROLES, ( + f"{path.name} declares stage {stage!r}, not one of {STAGE_ROLES}" + ) + by_role[stage].append(path) + + empty = [role for role, paths in by_role.items() if not paths] + assert not empty, f"no fixture document occupies station(s) {empty}" + + # "source" is where every undeclared document lands, so it alone may + # hold more than one; the five explicit stations carry exactly one each. + for role in STAGE_ROLES: + if role == "source": + continue + assert len(by_role[role]) == 1, ( + f"station {role!r} has {len(by_role[role])} document(s), want 1" + ) + + +def test_at_least_two_sources_share_a_topic() -> None: + """R1Q13 (a) with (c): groups derive from the topics sources share, and + the fixture must yield at least one group.""" + sharing = [ + path for path, (header, body) in _documents().items() + if "stage" not in header + and SHARED_SOURCE_TOPIC in " ".join( + (header.get("title", ""), header.get("summary", ""), body) + ).lower() + ] + assert len(sharing) >= 2, ( + f"expected >= 2 source documents sharing {SHARED_SOURCE_TOPIC!r}, " + f"found {[p.name for p in sharing]}" + ) + + +def test_at_least_one_source_shares_no_topic() -> None: + """Keeps the assertion above honest: a fixture where every source shared + one word would not exercise topic-based selection at all.""" + singleton = [ + path for path, (header, body) in _documents().items() + if "stage" not in header + and SHARED_SOURCE_TOPIC not in " ".join( + (header.get("title", ""), header.get("summary", ""), body) + ).lower() + ] + assert singleton, "every source shares the same topic phrase" + + +if __name__ == "__main__": + sys.exit(pytest.main([__file__, "-v"])) From 834f8ead4e4f333f39d07f740009d7bf85f47597 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sun, 27 Sep 2026 16:39:22 +0000 Subject: [PATCH 02/38] T052: 5.4, declare the generator seam, and register openDox's own generator (plan 034) Box 5.4 of openxFactory's add-neutral-product-standalone-operability: "DECLARE THE GENERATOR SEAM - it does not exist and CorpusAdapter is not it." This declares it, in src/opendox/generator_seam.py, beside domain_profile.register(): - the operation handed over: generate(repo_root, repository, *, source_revision=None, generated_at=None, **inputs) -> the snapshot; - the registration point: register() for a host, and register_default(...) for an entry point, which registers only where nothing is registered; - the conformance a contributed generator must meet. It declares the contract it writes and every further input it reads, its operation takes the seam's call, and it answers a snapshot whose kind is that contract and whose schema_version is an integer. The seam checks the declaration when it is made and each snapshot when it comes back, and it refuses, never drops, an undeclared input. The conformance clause names T053's neutral snapshot kind, "opendox-snapshot" (openDox-spec#16), for openDox's own generator: src/opendox/default_generator.py's GENERATOR. The entry points (cli.build_parser, cli.main, serve.build_server, serve.main) register it where no host has (R1Q10 (a), openxFactory#656 comment 5850003126, in R1Q3 (a)'s pattern). A bare process still refuses, naming the seam and the call. A host replaces the default only before a snapshot has been generated from it. CorpusAdapter stays closed at six members. openDox's own generator refuses until T054 builds its projection. Plan 034 orders T052 before T054, and T054 edits neither cli.py nor serve.py (their single-writer order runs T052, then T055). No verb reaches the seam before T055 routes the generate verbs, which comes after T054. Falsifier: the seam tests, tests/test_generator_seam.py (59 cases). F5.2 is quoted by T059 and T061. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/cli.py | 9 + src/opendox/default_generator.py | 76 ++++ src/opendox/generator_seam.py | 515 +++++++++++++++++++++++++ src/opendox/serve.py | 9 + tests/test_generator_seam.py | 632 +++++++++++++++++++++++++++++++ 5 files changed, 1241 insertions(+) create mode 100644 src/opendox/default_generator.py create mode 100644 src/opendox/generator_seam.py create mode 100644 tests/test_generator_seam.py diff --git a/src/opendox/cli.py b/src/opendox/cli.py index 994111d..cf4b9ed 100644 --- a/src/opendox/cli.py +++ b/src/opendox/cli.py @@ -128,6 +128,10 @@ # in (R1Q3 (a)). Importing either registers nothing: `build_parser()` and # `main()` below make the registration, and only where no host has made one. from opendox import default_profile, domain_profile # noqa: E402 +# openDox's OWN snapshot generator, and the generator seam an entry point +# registers it at (5.4; plan 034 T052). Importing either registers nothing: +# `build_parser()` and `main()` below register it, only where no host has. +from opendox import default_generator, generator_seam # noqa: E402 is_rfc3339_datetime = consumer_reach.is_rfc3339_datetime # noqa: E402 # ...and `SCANNED_ROOTS` keeps its NAME and its behaviour, not just its value: # :239 iterates it (`for root in SCANNED_ROOTS`) and that line is not one the @@ -951,6 +955,9 @@ def build_parser(*, subcommand_extensions: tuple = ()) -> argparse.ArgumentParse # `SUBCOMMAND_EXTENSIONS` is read a few lines below; the corpus # adapter's registration is read later, from `authoring.py`). corpus_adapter.register_default_home(_default_home_factory) + # AND openDox's OWN snapshot generator (5.4, T052; R1Q10 (a), in the same + # R1Q3 (a) pattern), registered only where no host has contributed one. + generator_seam.register_default(default_generator.GENERATOR) parser = argparse.ArgumentParser(prog="ideation-dashboard", description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) sub = parser.add_subparsers(dest="command", required=True) @@ -1054,6 +1061,8 @@ def main(argv: list[str] | None = None, *, # own match here: the OUTERMOST entry point states the contract on its # own, independent of what `build_parser()` does inside. corpus_adapter.register_default_home(_default_home_factory) + # AND openDox's own snapshot generator (5.4, T052), the same way. + generator_seam.register_default(default_generator.GENERATOR) args = build_parser( subcommand_extensions=subcommand_extensions).parse_args(argv) try: diff --git a/src/opendox/default_generator.py b/src/opendox/default_generator.py new file mode 100644 index 0000000..f2b9cb6 --- /dev/null +++ b/src/opendox/default_generator.py @@ -0,0 +1,76 @@ +"""openDox's OWN snapshot generator, declared as the generator seam's default. + +WHY THIS FILE EXISTS. openDox's entry points register openDox's own generator +where no host has contributed one (R1Q10 (a), `openxFactory#656` comment +`5850003126`, in R1Q3 (a)'s pattern). They do so beside the default profile and +the default home corpus, which they register the same way. This module is what +they register: `GENERATOR`, a `generator_seam.SnapshotGenerator` that writes +the neutral snapshot contract, `generator_seam.NEUTRAL_SNAPSHOT_KIND` (plan +034's T053; R1Q11 (a)), and takes no input beyond the operation's own four. + +ITS PROJECTION IS NOT BUILT YET, AND IT SAYS SO. Plan 034 orders the seam +before the projection. T054, openDox's small neutral projection over +`CorpusAdapter` (#1144's 5.1-5.3), comes after this task (T052). The entry +points' registration calls are this task's, because T054 edits neither +`cli.py` nor `serve.py`, whose single-writer order runs T052, then T055. So +`generate()` below refuses, naming itself and the task that builds it, and it +generates nothing. It never answers an empty snapshot, which would read exactly +like an honest one. + +NO VERB REACHES IT YET. The generate verbs still call the consumer's generator, +through `consumer_reach`, until T055 routes them through the seam, and T055 +comes after T054. The refusal is therefore reachable only by a library caller +that generates through the seam directly. T054 replaces it with the +projection, and the declaration below keeps its contract and its inputs. + +HOW IT IS REGISTERED: by the entry points, and never at import (R1Q3 (a)'s +pattern). `cli.build_parser()`, `cli.main()`, `serve.build_server()` and +`serve.main()` call `generator_seam.register_default(GENERATOR)`, which +registers it only where nothing is registered. Importing this module registers +nothing. + +IMPORT WEIGHT. `opendox.generator_seam` and the standard library. So this +module imports with no extra installed and no sibling present. + +A CREATED FILE: it has no row in openxFactory's +`docs/opendox-carve-manifest.yaml`, because the manifest declares what LEAVES +openxFactory and never what a destination assembles (RULED OQ-C). +""" + +from __future__ import annotations + +from pathlib import Path +from typing import Any + +from opendox import generator_seam + +__all__ = ["GENERATOR", "NeutralProjectionNotBuilt", "generate"] + + +class NeutralProjectionNotBuilt(generator_seam.GeneratorSeamError): + """openDox's own generator is declared, and its projection is not built. + + A subclass of the seam's own refusal, so a verb that reports the seam's + refusals reports this one too. It goes when plan 034's T054 lands the + projection.""" + + +def generate(repo_root: Path, repository: str, *, + source_revision: str | None = None, + generated_at: str | None = None) -> dict[str, Any]: + """The operation the seam hands over, for openDox's own generator. + + It REFUSES until plan 034's T054 builds the neutral projection. See the + module docstring for why the declaration lands first.""" + raise NeutralProjectionNotBuilt( + f"openDox's own generator is declared at the generator seam, writing " + f"{generator_seam.NEUTRAL_SNAPSHOT_KIND!r}, but its projection is not " + f"built at this commit (plan 034's T054 builds it), so nothing was " + f"generated for {repository!r} at {str(repo_root)!r}. A host that has " + f"a generator of its own registers it at process start with " + f"{generator_seam.REGISTRATION_CALL}.") + + +#: openDox's OWN generator, as the entry points register it where no host has. +GENERATOR = generator_seam.SnapshotGenerator( + contract=generator_seam.NEUTRAL_SNAPSHOT_KIND, generate=generate) diff --git a/src/opendox/generator_seam.py b/src/opendox/generator_seam.py new file mode 100644 index 0000000..0e8edcb --- /dev/null +++ b/src/opendox/generator_seam.py @@ -0,0 +1,515 @@ +"""THE GENERATOR SEAM: the one snapshot generator a process generates with, a +host's contribution or openDox's own default, held for late resolution. + +WHY THIS FILE EXISTS. Box 5.4 of openxFactory's +`add-neutral-product-standalone-operability` (RATIFIED, `openxFactory#656` +comment `5815412869`) reads *"DECLARE THE GENERATOR SEAM — it does not exist +and `CorpusAdapter` is not it"*. Its requirement 4 has openDox grow a neutral +generator of its own, while the consumer KEEPS its governed generator and +contributes it *"through a DECLARED GENERATOR SEAM"*. That seam *"SHALL BE +DECLARED BY THIS ARC, naming the operation it hands over, the registration +point, and what a conformant implementation must satisfy"*. `consumer_reach.py` +names the same gap from the other side. The injection that would retire its +`generator` reach, *"openDox naming a protocol and being handed an +implementation"*, *"does not exist yet and is BUILD-arc work"*. This module is +that declaration (plan 034's T052). + +IT ROUTES NOTHING YET. The generate verbs still call the consumer's generator +through `consumer_reach` until plan 034's T055 routes them through `generate()` +below. T055 comes after T054, which builds openDox's own projection. So from +the day a verb generates through this seam, openDox's own generator can +generate. + +`CorpusAdapter` IS NOT IT, AND STAYS CLOSED AT SIX MEMBERS. The corpus-read +interface (`corpus_adapter.py`) declares `resolve`, `list_documents`, `read`, +`classify`, `check` and `write_back`, and its own docstring says *"Nothing +else, ever"*. None of the six hands a generator over, and a seventh member +would be a seam that leaks. So a generator is handed over HERE, through a +registration of its own. A generator READS its corpus through that interface, +as openDox's own projection does (plan 034's T054). + +THE OPERATION HANDED OVER is one call. It answers the snapshot, a JSON-ready +`dict`: + + generate(repo_root, repository, *, + source_revision=None, generated_at=None, **inputs) + +* `repo_root` is the corpus checkout the generator scans, as a `Path`. +* `repository` is the repository id the snapshot is for. +* `source_revision` is the source anchor to pin, or `None`, which leaves the + generator to read the checkout itself. +* `generated_at` is the generation anchor, recorded verbatim when it is given. +* `**inputs` are the further inputs a generator DECLARES that it takes. Each is + passed only when the caller has a value for it. + +The four arguments are the ones openDox's generate verbs already hand a +generator. A governed generator's own extras, such as a project register, +travel as declared inputs. So the seam never drops one, and never names one +either. + +THE REGISTRATION POINT sits beside `domain_profile.register()`, in the shape +the product already uses (5.4's own words). A host calls +`opendox.generator_seam.register()` ONCE, at process +start. openDox's own entry points call `register_default(...)`, which registers +openDox's own generator only where nothing is registered. + +THE CONFORMANCE A CONTRIBUTED GENERATOR MUST MEET. `SnapshotGenerator` declares +it, and `generate()` enforces it. + +1. It DECLARES THE CONTRACT IT WRITES: the `kind` of every snapshot it answers. + openDox's own generator writes `NEUTRAL_SNAPSHOT_KIND`, the neutral + snapshot contract whose schema openDox's own spec leg owns (plan 034's + T053; R1Q11 (a), `openxFactory#656` comment `5850003126`). A consumer's + governed generator declares its own. So two generators that write two + contracts can never both claim to be the one conformant implementation, + which is the failure 5.4 was raised against. +2. Its operation TAKES THE SEAM'S CALL: the four arguments, and every input it + declares. This is checked when the declaration is made, wherever the + callable's signature can be read. +3. It DECLARES EVERY FURTHER INPUT it reads. An input that it does not declare + is refused before the generator is called, and never dropped. +4. It ANSWERS A SNAPSHOT OF ITS DECLARED CONTRACT: a `dict` whose `kind` is + that contract and whose `schema_version` is an integer, which is what a + validator picks a schema by. Anything else is refused, not handed on. +5. It is DETERMINISTIC over its inputs, as the generate verbs promise (*"the + deterministic snapshot"*): the same corpus at the same anchors answers the + same snapshot. Each generator's own suite proves that. The seam does not + check it again. + +REFUSAL, NOT A DEFAULT (4.2's discipline, and `domain_profile`'s). With nothing +registered, `current()` and `generate()` refuse, naming this seam and the +registration call. They never fall back to openDox's own generator, and they +never answer an empty snapshot, which would read exactly like an honest one. + +THE ENTRY POINTS REGISTER openDox's OWN GENERATOR WHERE NO HOST HAS (R1Q10 +(a), comment `5850003126`, in R1Q3 (a)'s pattern, comment `5817152735`). +`cli.build_parser()`, `cli.main()`, `serve.build_server()` and `serve.main()` +call `register_default(default_generator.GENERATOR)`, beside their +registrations of the default profile and the default home corpus. So, as for +the profile: + +* a process that builds nothing still meets `GeneratorNotRegistered`, which is + the library caller's case; +* a host registration made BEFORE anything is generated from the default + replaces it; +* AFTER a snapshot has been generated from the default, a host's registration + is refused as `GeneratorAlreadyRegistered`. One process would otherwise + write two contracts, and a reader could not tell which snapshot was which. + This is the same reason `domain_profile` refuses a swap after a build + (R1Q3 (ii); RN-1 (a), comment `5850003126`). + +ONE REGISTRATION. Registering the same declaration again is a no-op, so an +idempotent host start is not punished. A different declaration over a host's +is refused as `GeneratorAlreadyRegistered`. `unregister()` makes a deliberate +swap explicit. + +RESOLVE PER CALL. A caller generates through `generate()`, and does not hold +`current()`'s answer across calls. So every caller in a process answers from +the one registration that exists at the time. + +IMPORT WEIGHT. The standard library only. So this module imports with no extra +installed and no sibling present, and it names no sibling. + +A CREATED FILE: it has no row in openxFactory's +`docs/opendox-carve-manifest.yaml`, because the manifest declares what LEAVES +openxFactory and never what a destination assembles (RULED OQ-C). +""" + +from __future__ import annotations + +import inspect +import keyword +from dataclasses import dataclass +from pathlib import Path +from typing import Any, Callable + +__all__ = [ + "GeneratorAlreadyRegistered", + "GeneratorInputRefused", + "GeneratorNotConformant", + "GeneratorNotRegistered", + "GeneratorSeamError", + "NEUTRAL_SNAPSHOT_KIND", + "OPERATION_ARGUMENTS", + "REGISTRATION_CALL", + "SnapshotGenerator", + "current", + "generate", + "is_registered", + "name_of", + "register", + "register_default", + "unregister", +] + +#: The contract openDox's OWN generator writes: the `kind` of the neutral +#: snapshot, whose schema openDox's spec leg owns (plan 034's T053; R1Q11 (a), +#: `openxFactory#656` comment `5850003126`). It is the `kind` const of that +#: schema, `contracts/schemas/opendox-snapshot.schema.yaml` (openDox-spec#16), +#: and it must stay equal to it. The value is a string in every neutral +#: snapshot, so F5.3's sweep of the snapshot's string values reads it too, and +#: it carries none of the words that sweep forbids. +NEUTRAL_SNAPSHOT_KIND = "opendox-snapshot" + +#: The operation's own arguments, in its call order. The first two are +#: positional, and the last two are keywords. A generator's declared inputs may +#: not reuse these names. +OPERATION_ARGUMENTS: tuple[str, ...] = ( + "repo_root", "repository", "source_revision", "generated_at") + +#: The ONE call a host makes, quoted verbatim in every refusal, so that a +#: refusal names its remedy rather than its symptom. +REGISTRATION_CALL = ( + "opendox.generator_seam.register()") + +#: How much of a `repr` a refusal quotes before it stops being read. +_NAME_LIMIT = 120 + + +class GeneratorSeamError(RuntimeError): + """A refusal of the generator seam. + + One base for every refusal below, so a verb that generates can report the + seam's refusals in one clause, as `cli.main()` reports its own.""" + + +class GeneratorNotRegistered(GeneratorSeamError): + """Nothing is registered: no host's generator, and no entry point's default. + + Raised instead of falling back to openDox's own generator. The default is a + registration an ENTRY POINT makes (R1Q10 (a), in R1Q3 (a)'s pattern), so + this is what a process meets when it generates without having built + anything through an entry point, which is the library caller's case. It is + also what a process meets after `unregister()`, because the refusal reports + what is registered NOW and keeps no history.""" + + +class GeneratorAlreadyRegistered(GeneratorSeamError): + """A second, different generator was registered over a first. + + ONE registration is the contract, as it is for the profile. A process whose + snapshots came from two generators would write two contracts under one + name. The entry point's default is replaceable only until a snapshot has + been generated from it. `unregister()` makes a deliberate swap explicit.""" + + +class GeneratorInputRefused(GeneratorSeamError): + """The registered generator was handed an input it does not declare. + + Refused before the generator is called. A generator that silently ignored + the input would write a snapshot that looks as though the input had been + read.""" + + +class GeneratorNotConformant(GeneratorSeamError): + """A generator broke the conformance this seam declares. + + It answered something other than a snapshot of its declared contract, or + it was offered as openDox's own default while declaring another contract. + What it answered is refused, not handed on: the validator and the views + read a snapshot by its `kind`.""" + + +def _qualified(value: Any) -> str: + """`module.qualname` for a callable, else a truncated `repr`. Never raises.""" + try: + module = getattr(value, "__module__", None) + qualname = getattr(value, "__qualname__", None) + if isinstance(module, str) and isinstance(qualname, str) and module and qualname: + return f"{module}.{qualname}" + except Exception: # noqa: BLE001 - naming must never out-raise + pass + try: + text = repr(value) + except Exception: # noqa: BLE001 + text = "" + if not text: + return "an unnameable callable" + return text if len(text) <= _NAME_LIMIT else text[:_NAME_LIMIT - 1] + "…" + + +@dataclass(frozen=True, eq=False) +class SnapshotGenerator: + """ONE generator's declaration: the contract it writes, its operation, and + the further inputs it takes. It is what `register()` and + `register_default()` hold. + + `contract` is the `kind` of every snapshot the generator answers. + `generate` is the operation this module's docstring declares. `inputs` + names the keyword inputs it reads beyond the operation's own four, and + defaults to none. + + Construction refuses a declaration the seam could not honour, rather than + registering one that fails at its first generation. That covers a contract + that is not a non-empty string, an operation that is not callable, an input + name that is not an identifier (or is a keyword, a duplicate, or one of the + operation's own four), and an operation whose signature cannot take the + seam's call. Frozen, and compared by identity, as `register()` compares + registrations.""" + + contract: str + generate: Callable[..., dict] + inputs: tuple[str, ...] = () + + def __post_init__(self) -> None: + if not isinstance(self.contract, str): + raise TypeError( + "a SnapshotGenerator declares the contract it writes as a " + f"string, the kind of every snapshot it answers, not " + f"{type(self.contract).__name__}") + if not self.contract or self.contract != self.contract.strip(): + raise ValueError( + "a SnapshotGenerator's contract is the kind its snapshots carry, " + f"a non-empty name with no surrounding space, not {self.contract!r}") + if not callable(self.generate): + raise TypeError( + "a SnapshotGenerator's generate is the operation the seam hands " + f"over, a callable, not {type(self.generate).__name__}") + if not isinstance(self.inputs, tuple) or not all( + isinstance(name, str) for name in self.inputs): + raise TypeError( + "a SnapshotGenerator declares its further inputs as a tuple of " + f"names, not {self.inputs!r}") + bad = sorted({name for name in self.inputs + if not name.isidentifier() or keyword.iskeyword(name)}) + reused = sorted(set(self.inputs) & set(OPERATION_ARGUMENTS)) + repeated = sorted({name for name in self.inputs + if self.inputs.count(name) > 1}) + if bad or reused or repeated: + raise ValueError( + "a SnapshotGenerator's inputs name keyword arguments its " + "operation takes beyond the operation's own four " + f"({', '.join(OPERATION_ARGUMENTS)}). Refused: " + f"not an identifier {bad}, one of the four {reused}, " + f"repeated {repeated}") + self._refuse_an_operation_that_cannot_take_the_call() + + def _refuse_an_operation_that_cannot_take_the_call(self) -> None: + try: + signature = inspect.signature(self.generate) + except (TypeError, ValueError): + # The signature cannot be read (some builtins). The seam's call is + # then checked when it is first made, and not here. + return + try: + signature.bind(Path("."), "repository", source_revision=None, + generated_at=None, + **{name: None for name in self.inputs}) + except TypeError as exc: + raise TypeError( + f"{_qualified(self.generate)} cannot take the generator seam's " + "call, generate(repo_root, repository, *, source_revision=, " + "generated_at=" + "".join(f", {name}=" for name in self.inputs) + + f"): {exc}. A contributed generator takes the operation's own " + "four arguments and every input it declares") from None + + +#: THE one registration, a host's or the entry point's default, or `None`. +_registered: SnapshotGenerator | None = None + +#: Whether `_registered` is the default an ENTRY POINT registered +#: (`register_default()`), rather than a host's own `register()`. +_is_default: bool = False + +#: Whether a snapshot has been generated from that default. Set by +#: `generate()`, before the default's operation is called. +_generated_from_default: bool = False + + +def name_of(generator: Any) -> str: + """A generator's most nameable name, with its contract, for a refusal. + + Public, as `domain_profile.name_of()` is, so a host's own refusal can name a + generator the same way. It never raises: a refusal that fails while + formatting itself replaces the reader's problem with a worse one.""" + if isinstance(generator, SnapshotGenerator): + return f"{_qualified(generator.generate)} (writing {generator.contract!r})" + return _qualified(generator) + + +def _require_a_declaration(generator: Any, call: str) -> None: + if not isinstance(generator, SnapshotGenerator): + raise TypeError( + f"{call} takes a SnapshotGenerator, the declaration of a generator's " + "contract, operation and inputs, not " + f"{type(generator).__name__}. A host with no generator of its own " + "does not register one: the seam then answers with openDox's own " + "generator where an entry point registered it, and refuses where " + "none did.") + + +def register(generator: SnapshotGenerator) -> SnapshotGenerator: + """THE one registration. A host calls this at process start. + + Returns the generator, so a host can register it and hold it in one + expression. + + Registering the SAME declaration again is a no-op. A DIFFERENT one over a + host's registration raises `GeneratorAlreadyRegistered`. Over the entry + point's default it REPLACES the default while nothing has been generated + from it, and is refused once something has. Either way the host ends up + holding the one registration, or knows why it does not.""" + global _registered, _is_default, _generated_from_default + _require_a_declaration(generator, "register()") + if _registered is generator: + # The same object again: a no-op, before any bookkeeping is touched. + return generator + if _registered is not None: + if not _is_default: + raise GeneratorAlreadyRegistered( + f"a host's generator is already registered at openDox's " + f"generator seam ({name_of(_registered)}), and " + f"{name_of(generator)} would replace it. Registration happens " + "ONCE, at process start: one process generating through two " + "generators would write two contracts, and a reader could not " + "tell which snapshot was which. Call " + "opendox.generator_seam.unregister() first if the swap is " + "deliberate.") + if _generated_from_default: + raise GeneratorAlreadyRegistered( + f"openDox's own default generator ({name_of(_registered)}) is " + "registered, because an entry point registered it where no host " + "had, and a snapshot has already been generated from it, so " + f"{name_of(generator)} cannot replace it now. A swap would leave " + "one process writing two contracts. A host's generator replaces " + "the default only BEFORE anything is generated from it, as a " + "host's profile replaces the default profile only before a build " + "(R1Q3 (ii), openxFactory#656 comment 5817152735; RN-1 (a), " + "comment 5850003126). So register the host's generator at " + "process start, ahead of the first generation. Call " + "opendox.generator_seam.unregister() first if the swap is " + "deliberate.") + _registered = generator + _is_default = False + _generated_from_default = False + return generator + + +def register_default(generator: SnapshotGenerator) -> SnapshotGenerator: + """AN ENTRY POINT'S registration of openDox's OWN generator (R1Q10 (a)). + + `cli.build_parser()`, `cli.main()`, `serve.build_server()` and + `serve.main()` call this with `default_generator.GENERATOR`. It registers + `generator` ONLY where nothing is registered, and leaves a host's + registration, or a default already registered, exactly as it is. Returns + whatever is registered afterwards. + + It is NOT for hosts. A host calls `register()`. The conformance clause holds + openDox's own generator to the neutral contract, so a declaration that + writes another contract is refused as `GeneratorNotConformant`, whether or + not anything is registered.""" + global _registered, _is_default, _generated_from_default + _require_a_declaration(generator, "register_default()") + if generator.contract != NEUTRAL_SNAPSHOT_KIND: + raise GeneratorNotConformant( + "register_default() registers openDox's OWN generator, which writes " + f"the neutral snapshot contract {NEUTRAL_SNAPSHOT_KIND!r}, and " + f"{name_of(generator)} declares another. A host's generator is " + f"registered with {REGISTRATION_CALL}.") + if _registered is None: + _registered = generator + _is_default = True + _generated_from_default = False + return _registered + + +def unregister() -> None: + """Drop the registration, a host's or the entry point's default. + + For test isolation and for a host tearing down. The record of a generation + from the default goes with it.""" + global _registered, _is_default, _generated_from_default + _registered = None + _is_default = False + _generated_from_default = False + + +def is_registered() -> bool: + """Is a generator registered? Answers without resolving or refusing.""" + return _registered is not None + + +def current() -> SnapshotGenerator: + """The registered generator, or a refusal naming this seam and its call. + + It answers what is registered: a host's generator, or the default an entry + point registered. It never falls back to the default itself, and asking + records nothing: only a generation closes the default's window.""" + if _registered is None: + raise GeneratorNotRegistered( + "no snapshot generator is registered at openDox's generator seam " + "(opendox.generator_seam), so there is nothing to generate a " + "snapshot with, and nothing is generated. openDox ships a generator " + "of its own, opendox.default_generator, but it is a registration an " + "ENTRY POINT makes and never a fallback here (R1Q10 (a), " + "openxFactory#656 comment 5850003126, in R1Q3 (a)'s pattern). " + "`cli.build_parser()`, `cli.main()`, `serve.build_server()` and " + "`serve.main()` register it where no host has. Nothing is " + "registered now, so either nothing in this process has been built " + "through one of them, or opendox.generator_seam.unregister() has " + "dropped the registration since. A host that contributes its own " + "generator registers it at process start with\n\n " + + REGISTRATION_CALL + "\n\nbefore the first generation.") + return _registered + + +def generate(repo_root: Path | str, repository: str, *, + source_revision: str | None = None, + generated_at: str | None = None, + **inputs: Any) -> dict: + """Generate a snapshot through the registered generator. This is THE + operation the seam hands over, called and checked. + + An input given as `None` means "not given", and it is not passed, so a verb + can hand over its options whether or not they were set. Any other input the + registered generator does not declare is refused before the generator is + called (`GeneratorInputRefused`). What the generator answers is handed back + only if it is a snapshot of the contract the generator declared + (`GeneratorNotConformant`). With nothing registered this refuses as + `current()` does.""" + global _generated_from_default + generator = current() + given = {name: value for name, value in inputs.items() if value is not None} + undeclared = sorted(set(given) - set(generator.inputs)) + if undeclared: + declared = ", ".join(generator.inputs) or "none" + raise GeneratorInputRefused( + f"{name_of(generator)} declares the inputs ({declared}) beyond the " + f"operation's own four, and was handed {', '.join(undeclared)}. An " + "input a generator does not declare is refused rather than dropped: " + "a generator that silently ignored it would write a snapshot that " + "looks as though the input had been read.") + if _is_default: + # Before the call, so a registration racing a first generation cannot + # slip in between them. + _generated_from_default = True + snapshot = generator.generate( + Path(repo_root), repository, source_revision=source_revision, + generated_at=generated_at, **given) + _refuse_a_snapshot_that_does_not_conform(generator, snapshot) + return snapshot + + +def _refuse_a_snapshot_that_does_not_conform(generator: SnapshotGenerator, + snapshot: Any) -> None: + if not isinstance(snapshot, dict): + raise GeneratorNotConformant( + f"{name_of(generator)} answered {type(snapshot).__name__}, not a " + "snapshot. The seam hands on only a snapshot, a dict, and an answer " + "it cannot read is refused rather than taken for one.") + kind = snapshot.get("kind") + if kind != generator.contract: + raise GeneratorNotConformant( + f"{name_of(generator)} declares that it writes " + f"{generator.contract!r}, and it answered a snapshot whose kind is " + f"{kind!r}. The seam hands back only a snapshot of the contract its " + "generator declared. The validator and the views read a snapshot " + "by its kind, so a generator whose output drifted from its " + "declaration would be two contracts under one name.") + version = snapshot.get("schema_version") + if type(version) is not int: + raise GeneratorNotConformant( + f"{name_of(generator)} answered a {generator.contract!r} snapshot " + f"whose schema_version is {version!r}, not an integer. A validator " + "picks a schema by kind and version, so a snapshot that does not " + "say which version of its contract it is cannot be checked.") diff --git a/src/opendox/serve.py b/src/opendox/serve.py index 9e3a2c0..a0cd530 100644 --- a/src/opendox/serve.py +++ b/src/opendox/serve.py @@ -161,6 +161,10 @@ # `opendox.runtime.config` and `opendox.corpus_adapter` besides the stdlib. from opendox import corpus_adapter # noqa: E402 from opendox.runtime import local_git_adapter # noqa: E402 +# THE GENERATOR SEAM'S DEFAULT (5.4; plan 034 T052): openDox's own snapshot +# generator, which `build_server()` and `main()` register where no host has. +# Both modules are stdlib-only and name no sibling, so this adds no reach. +from opendox import default_generator, generator_seam # noqa: E402 registry_mod = consumer_reach.snapshot_registry # noqa: E402 # THE BY-FUNCTION SPLIT (`split-opendox-two-layer-product` § 2.4, PRs 2 and 3 @@ -1649,6 +1653,9 @@ def build_server( # registers ONLY where nothing already answers `corpus_adapter.home()`, # exactly as `domain_profile.register_default()` does for the profile. corpus_adapter.register_default_home(_default_home_factory) + # AND openDox's OWN snapshot generator (5.4, T052; R1Q10 (a), in the same + # R1Q3 (a) pattern), registered only where no host has contributed one. + generator_seam.register_default(default_generator.GENERATOR) from opendox import doxbench_turns # Imported HERE rather than at module scope, for the reason that is @@ -2147,6 +2154,8 @@ def main(argv: list[str] | None = None) -> int: # no-op once that runs; kept for the same reason T016 keeps its own # match here: the OUTERMOST entry point states the contract on its own. corpus_adapter.register_default_home(_default_home_factory) + # AND openDox's own snapshot generator (5.4, T052), the same way. + generator_seam.register_default(default_generator.GENERATOR) parser = argparse.ArgumentParser(prog="ideation-dashboard-serve", description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) parser.add_argument("--web-dir", default=str(Path(__file__).resolve().parent / "web"), diff --git a/tests/test_generator_seam.py b/tests/test_generator_seam.py new file mode 100644 index 0000000..d8e1bab --- /dev/null +++ b/tests/test_generator_seam.py @@ -0,0 +1,632 @@ +"""The generator seam, held to 5.4 of openxFactory's +`add-neutral-product-standalone-operability` (plan 034, T052). + +Box 5.4 reads *"DECLARE THE GENERATOR SEAM — it does not exist and +`CorpusAdapter` is not it"*. It asks for the operation handed over, the +registration point beside `domain_profile.register()`, and the conformance a +contributed generator must satisfy. Plan 034's T052 adds that the conformance +clause names T053's neutral snapshot kind for openDox's own generator, and that +the entry points register openDox's own generator where no host has (R1Q10 (a), +`openxFactory#656` comment `5850003126`, in R1Q3 (a)'s pattern). These are +T052's seam tests. + +WHAT IT ASSERTS, AND WHY EACH IS HERE + +1. NOTHING IS REGISTERED BY AN IMPORT, AND THE SEAM MAKES NO REACH. A fresh + process that imports both modules and both entry points registers nothing, + and it meets the refusal. The two modules import with every sibling + blocked, pull in no third-party module, and name no sibling in any import, + late or not. +2. NOTHING REGISTERED REFUSES, NAMING THE SEAM AND THE CALL (4.2's + discipline). It never falls back to openDox's own generator. +3. THE DECLARATION IS CHECKED WHEN IT IS MADE. A contract that is not a name, + an operation that is not callable or cannot take the seam's call, and an + input that is not a name beyond the operation's own four are refused before + anything is registered. +4. THE OPERATION IS HANDED OVER AND ITS ANSWER CHECKED. The registered + generator receives the four arguments and its declared inputs. An + undeclared input is refused before the call. A `None` input is not passed. + Only a snapshot of the declared contract comes back. +5. ONE REGISTRATION. The same declaration twice is a no-op, and a second host + is refused. +6. THE ENTRY POINTS' DEFAULT, in R1Q3 (a)'s pattern. openDox's own generator + writes the neutral kind and takes no input. `register_default()` registers + it only where nothing is, and holds it to the neutral contract. A host + replaces it before a generation, and is refused after one. Until T054 lands, + the default refuses, naming itself. +7. EACH ENTRY POINT REGISTERS IT. `cli.build_parser()` and `cli.main()` run for + real. `serve.build_server()` and `serve.main()` still cannot run in a lone + checkout (research R7), so their registration is executed from their own + source lines, as `tests/test_authoring_seam.py` does for the home corpus. +8. `CorpusAdapter` STAYS CLOSED AT SIX MEMBERS. + +`--noconftest` SAFE. The autouse fixture below saves and restores the three +registries the entry points write, so no case depends on the root conftest or +leaves a registration behind. + +A CREATED file: no carve-manifest row (RULED OQ-C). +""" + +from __future__ import annotations + +import ast +import inspect +import re +import subprocess +import sys +import textwrap +from pathlib import Path + +import pytest + +from opendox import corpus_adapter, default_generator, domain_profile +from opendox import generator_seam as gs + +ROOT = Path(__file__).resolve().parent.parent +SRC = ROOT / "src" +CLI = SRC / "opendox" / "cli.py" +SERVE = SRC / "opendox" / "serve.py" + +#: The seam's two modules. +SEAM_MODULES = {"opendox.generator_seam": SRC / "opendox" / "generator_seam.py", + "opendox.default_generator": SRC / "opendox" / "default_generator.py"} + +#: The four packages a neutral openDox must import without (#1144's F2.1). +SIBLINGS = ("openxdox", "ideation_dashboard", "doc_health", + "corpus_adapter_openxfactory") + +#: F5.3's declared vocabulary, verbatim from #1144's `tasks.md`. F5.3 reads +#: every string value of the neutral snapshot, and the snapshot's `kind` is one. +F5_3_WORDS = ("brainstorm", "staged", "draft", "ratified", "standard", + "superseded", "retired", "record", "openspec", "proposal.md", + "tasks.md", "design.md", "added requirements", + "modified requirements") + + +@pytest.fixture(autouse=True) +def _isolated_registries(): + """Every case starts with NO generator registered, and PUTS BACK what it found. + + The three registries an entry point writes are process-global by design: + the generator seam, the profile and the home corpus. Each is saved whole, + because each keeps more than its registration (whether it is the entry + point's default, and whether anything was generated or built from it), and + a restore through `register()` would hand an entry point's default back as + a host's. + """ + seam = (gs._registered, gs._is_default, gs._generated_from_default) + profile = (domain_profile._registered, domain_profile._is_default, + domain_profile._built_from_default) + home = corpus_adapter._home_factory + gs.unregister() + yield + (gs._registered, gs._is_default, gs._generated_from_default) = seam + (domain_profile._registered, domain_profile._is_default, + domain_profile._built_from_default) = profile + corpus_adapter._home_factory = home + + +def _declared(contract: str = "stand-in-snapshot", *, inputs: tuple = (), + answer=None): + """A stand-in generator's declaration, and the list its operation records. + + The operation takes exactly the seam's call. It answers `answer` when one + is given, and otherwise a minimal snapshot of `contract`.""" + calls: list[dict] = [] + + def operation(repo_root, repository, *, source_revision=None, + generated_at=None, **extra): + calls.append({"repo_root": repo_root, "repository": repository, + "source_revision": source_revision, + "generated_at": generated_at, **extra}) + if answer is not None: + return answer + return {"schema_version": 1, "kind": contract, "repository": repository} + + return gs.SnapshotGenerator(contract=contract, generate=operation, + inputs=inputs), calls + + +def _fresh_process(program: str) -> subprocess.CompletedProcess: + """`program` in a fresh interpreter, with this checkout's `src` first. The + registry is process-global, so a case about what a PROCESS meets runs in + one of its own.""" + return subprocess.run( + [sys.executable, "-c", f"import sys; sys.path.insert(0, {str(SRC)!r})\n" + + textwrap.dedent(program)], + capture_output=True, text=True, cwd=str(ROOT)) + + +# -------------------------------------------------------------------------- +# 1 — nothing is registered by an import, and the seam makes no reach +# -------------------------------------------------------------------------- + +def test_importing_the_seam_and_the_entry_points_registers_nothing() -> None: + """The library caller's case. A process that builds nothing still refuses, + because openDox's own generator is a registration an entry point MAKES.""" + done = _fresh_process(""" + import opendox.cli, opendox.serve, opendox.default_generator + from opendox import generator_seam as gs + assert gs.is_registered() is False, "an import registered a generator" + for call in (gs.current, lambda: gs.generate(".", "fixture")): + try: + call() + except gs.GeneratorNotRegistered: + continue + raise AssertionError(f"{call} answered with nothing registered") + print("refused") + """) + assert done.returncode == 0, done.stderr + assert done.stdout.strip() == "refused" + + +def test_the_seam_imports_with_no_sibling_and_no_third_party_module() -> None: + """Both modules import with the four siblings blocked at the finder, and + what they add to `sys.modules` is openDox's own or the standard library's.""" + block = "".join(f"sys.modules[{name!r}] = None\n" for name in SIBLINGS) + done = _fresh_process(block + textwrap.dedent(""" + before = set(sys.modules) + import opendox.generator_seam, opendox.default_generator + added = {name.split(".")[0] for name in set(sys.modules) - before} + print(sorted(added - set(sys.stdlib_module_names) - {"opendox"})) + """)) + assert done.returncode == 0, done.stderr + assert done.stdout.strip() == "[]", ( + f"the seam pulls in a module that is neither openDox's nor the " + f"standard library's: {done.stdout.strip()}") + + +def _named_imports(node: ast.AST) -> list[str]: + """The module names an import node, or an `import_module`/`__import__` + call with a literal name, reaches. This is #1144's F4.1 scan's own reading.""" + if isinstance(node, ast.Import): + return [alias.name for alias in node.names] + if isinstance(node, ast.ImportFrom): + return [node.module] if node.level == 0 and node.module else [] + if (isinstance(node, ast.Call) and node.args + and isinstance(node.args[0], ast.Constant) + and isinstance(node.args[0].value, str) + and getattr(node.func, "attr", getattr(node.func, "id", "")) + in ("import_module", "__import__")): + return [node.args[0].value] + return [] + + +@pytest.mark.parametrize("module", sorted(SEAM_MODULES)) +def test_the_seam_names_no_sibling_in_any_import_late_or_not(module: str) -> None: + """No import anywhere in either module, at module scope or inside a + function body, names a sibling. A late one is still a reach, and it is the + class F4.1's scan exists for.""" + tree = ast.parse(SEAM_MODULES[module].read_text(encoding="utf-8")) + reaches = sorted({name for node in ast.walk(tree) + for name in _named_imports(node) + if any(name == s or name.startswith(s + ".") + for s in SIBLINGS)}) + assert reaches == [], f"{module} reaches {reaches}" + + +# -------------------------------------------------------------------------- +# 2 — nothing registered: the seam refuses, naming itself and the call (4.2) +# -------------------------------------------------------------------------- + +def test_nothing_registered_refuses_naming_the_seam_and_the_call() -> None: + with pytest.raises(gs.GeneratorNotRegistered) as caught: + gs.current() + message = str(caught.value) + for expected in (gs.REGISTRATION_CALL, "opendox.generator_seam", + "opendox.default_generator", "ENTRY POINT", + "cli.build_parser()", "cli.main()", + "serve.build_server()", "serve.main()", + "R1Q10 (a)", "5850003126", "unregister()"): + assert expected in message, f"the refusal no longer says {expected!r}" + + +def test_generate_with_nothing_registered_refuses_as_the_seam(tmp_path) -> None: + """Never a fallback to openDox's own generator, and never an empty snapshot.""" + with pytest.raises(gs.GeneratorNotRegistered) as caught: + gs.generate(tmp_path, "fixture") + assert isinstance(caught.value, gs.GeneratorSeamError) + assert gs.is_registered() is False + + +# -------------------------------------------------------------------------- +# 3 — the declaration is checked when it is made +# -------------------------------------------------------------------------- + +@pytest.mark.parametrize("contract,error", ( + (None, TypeError), (3, TypeError), ("", ValueError), (" padded ", ValueError))) +def test_a_declaration_names_the_contract_it_writes(contract, error) -> None: + with pytest.raises(error): + gs.SnapshotGenerator(contract=contract, generate=default_generator.generate) + + +@pytest.mark.parametrize("operation", (None, "not callable", 3)) +def test_a_declaration_carries_a_callable_operation(operation) -> None: + with pytest.raises(TypeError): + gs.SnapshotGenerator(contract="stand-in-snapshot", generate=operation) + + +@pytest.mark.parametrize("inputs,error", ( + (["project_register_source"], TypeError), + ((1,), TypeError), + (("not an identifier",), ValueError), + (("class",), ValueError), + (("repository",), ValueError), + (("generated_at",), ValueError), + (("register", "register"), ValueError))) +def test_declared_inputs_are_names_beyond_the_operations_own_four(inputs, + error) -> None: + def operation(repo_root, repository, **kw): + return {} + + with pytest.raises(error): + gs.SnapshotGenerator(contract="stand-in-snapshot", generate=operation, + inputs=inputs) + + +def test_an_operation_that_cannot_take_the_seams_call_is_refused() -> None: + """Conformance clause 2, checked where the signature can be read.""" + with pytest.raises(TypeError) as caught: + gs.SnapshotGenerator(contract="stand-in-snapshot", + generate=lambda repo_root, repository: {}) + assert "cannot take the generator seam's call" in str(caught.value) + + def takes_the_four(repo_root, repository, *, source_revision=None, + generated_at=None): + return {} + + with pytest.raises(TypeError): + gs.SnapshotGenerator(contract="stand-in-snapshot", + generate=takes_the_four, + inputs=("project_register_source",)) + + +def test_a_governed_generators_shape_is_declarable() -> None: + """A consumer's generator keeps its own signature and declares its extras. + + This is the call shape of openXdox's `generator.generate_snapshot`, as T059 + will declare it, restated here because openDox may not import it. Its + keyword-only test hooks stay undeclared, and so the seam never passes them. + """ + def generate_snapshot(repo_root, repository, *, source_revision=None, + generated_at=None, git=None, generator_version="v", + project_register_source=None, possibles_source=None, + excluded_documents=None): + return {} + + declared = gs.SnapshotGenerator( + contract="governed-stand-in", generate=generate_snapshot, + inputs=("project_register_source", "possibles_source")) + assert declared.inputs == ("project_register_source", "possibles_source") + + +@pytest.mark.parametrize("candidate", (None, "a generator", default_generator.generate)) +def test_only_a_declaration_is_registered(candidate) -> None: + for call in (gs.register, gs.register_default): + with pytest.raises(TypeError): + call(candidate) + assert gs.is_registered() is False + + +# -------------------------------------------------------------------------- +# 4 — the operation handed over, and its answer checked +# -------------------------------------------------------------------------- + +def test_the_registered_generator_is_handed_the_operation(tmp_path) -> None: + declared, calls = _declared() + assert gs.register(declared) is declared + assert gs.current() is declared + snapshot = gs.generate(str(tmp_path), "fixture", source_revision="abc123", + generated_at="2026-09-27T00:00:00Z") + assert calls == [{"repo_root": tmp_path, "repository": "fixture", + "source_revision": "abc123", + "generated_at": "2026-09-27T00:00:00Z"}] + assert isinstance(calls[0]["repo_root"], Path), "repo_root is handed over as a Path" + assert snapshot == {"schema_version": 1, "kind": "stand-in-snapshot", + "repository": "fixture"} + + +def test_a_declared_input_is_passed_and_an_undeclared_one_refused(tmp_path) -> None: + declared, calls = _declared(inputs=("project_register_source",)) + gs.register(declared) + gs.generate(tmp_path, "fixture", + project_register_source=Path("register.yaml")) + assert calls[-1]["project_register_source"] == Path("register.yaml") + + with pytest.raises(gs.GeneratorInputRefused) as caught: + gs.generate(tmp_path, "fixture", possibles_source=Path("possibles.yaml")) + assert len(calls) == 1, "the generator was called with an undeclared input" + message = str(caught.value) + assert "possibles_source" in message and "project_register_source" in message + + +def test_an_input_given_as_none_is_not_passed(tmp_path) -> None: + """So a verb can hand over its options whether or not they were set.""" + declared, calls = _declared() + gs.register(declared) + gs.generate(tmp_path, "fixture", project_register_source=None, + possibles_source=None) + assert set(calls[-1]) == {"repo_root", "repository", "source_revision", + "generated_at"} + + +@pytest.mark.parametrize("answer", ( + ["not", "a", "snapshot"], + "a snapshot", + {"schema_version": 1, "kind": "another-snapshot"}, + {"schema_version": 1}, + {"kind": "stand-in-snapshot"}, + {"schema_version": "1", "kind": "stand-in-snapshot"}, + {"schema_version": True, "kind": "stand-in-snapshot"}, + {"schema_version": 1.0, "kind": "stand-in-snapshot"})) +def test_only_a_snapshot_of_the_declared_contract_comes_back(answer, + tmp_path) -> None: + """Conformance clause 4. Two generators writing two contracts can then + never both answer for one of them.""" + declared, calls = _declared(answer=answer) + gs.register(declared) + with pytest.raises(gs.GeneratorNotConformant) as caught: + gs.generate(tmp_path, "fixture") + assert calls, "the generator was never called" + assert "stand-in-snapshot" in str(caught.value) + + +# -------------------------------------------------------------------------- +# 5 — one registration +# -------------------------------------------------------------------------- + +def test_registering_the_same_declaration_twice_is_a_no_op() -> None: + declared, _ = _declared() + assert gs.register(declared) is declared + assert gs.register(declared) is declared + assert gs.current() is declared + + +def test_a_second_different_host_generator_is_refused() -> None: + first, _ = _declared("first-snapshot") + second, _ = _declared("second-snapshot") + gs.register(first) + with pytest.raises(gs.GeneratorAlreadyRegistered) as caught: + gs.register(second) + message = str(caught.value) + for expected in ("first-snapshot", "second-snapshot", "ONCE", + "unregister()"): + assert expected in message, f"the refusal no longer says {expected!r}" + assert gs.current() is first + + +def test_unregister_makes_a_deliberate_swap_explicit() -> None: + first, _ = _declared("first-snapshot") + second, _ = _declared("second-snapshot") + gs.register(first) + gs.unregister() + assert gs.is_registered() is False + assert gs.register(second) is second + + +# -------------------------------------------------------------------------- +# 6 — the entry points' default (R1Q10 (a), in R1Q3 (a)'s pattern) +# -------------------------------------------------------------------------- + +def test_openDoxs_own_generator_writes_the_neutral_contract() -> None: + """The conformance clause names T053's neutral kind for openDox's own.""" + declared = default_generator.GENERATOR + assert isinstance(declared, gs.SnapshotGenerator) + assert declared.contract == gs.NEUTRAL_SNAPSHOT_KIND + assert declared.inputs == () + assert declared.generate is default_generator.generate + assert list(inspect.signature(default_generator.generate).parameters) == \ + list(gs.OPERATION_ARGUMENTS) + + +def test_the_neutral_kind_carries_no_word_f5_3_forbids() -> None: + """The kind is a string value of every neutral snapshot, and F5.3 sweeps + every string value, lowercased, for its declared words.""" + pattern = re.compile(r"\b(" + "|".join(re.escape(w) for w in F5_3_WORDS) + r")\b") + assert not pattern.search(gs.NEUTRAL_SNAPSHOT_KIND.lower()), \ + gs.NEUTRAL_SNAPSHOT_KIND + + +def test_register_default_registers_only_where_nothing_is_registered() -> None: + host, _ = _declared("host-snapshot") + gs.register(host) + assert gs.register_default(default_generator.GENERATOR) is host + gs.unregister() + assert gs.register_default(default_generator.GENERATOR) is \ + default_generator.GENERATOR + other_default, _ = _declared(gs.NEUTRAL_SNAPSHOT_KIND) + assert gs.register_default(other_default) is default_generator.GENERATOR + + +def test_register_default_holds_the_default_to_the_neutral_contract() -> None: + """An entry point's default is openDox's own, so it writes the neutral kind. + Another contract is refused, whether or not anything is registered.""" + governed, _ = _declared("governed-stand-in") + with pytest.raises(gs.GeneratorNotConformant) as caught: + gs.register_default(governed) + assert gs.NEUTRAL_SNAPSHOT_KIND in str(caught.value) + assert gs.is_registered() is False + host, _ = _declared("host-snapshot") + gs.register(host) + with pytest.raises(gs.GeneratorNotConformant): + gs.register_default(governed) + assert gs.current() is host + + +def test_a_host_replaces_a_default_nothing_was_generated_from() -> None: + assert gs.register_default(default_generator.GENERATOR) is \ + default_generator.GENERATOR + host, _ = _declared("host-snapshot") + assert gs.register(host) is host + assert gs.current() is host + + +def test_a_host_after_a_generation_from_the_default_is_refused(tmp_path) -> None: + """R1Q3 (ii)'s reason, for the generator: one process would then write two + contracts. A stand-in writes the neutral kind here, because openDox's own + projection is T054's.""" + stand_in_default, _ = _declared(gs.NEUTRAL_SNAPSHOT_KIND) + gs.register_default(stand_in_default) + gs.generate(tmp_path, "fixture") + host, _ = _declared("host-snapshot") + with pytest.raises(gs.GeneratorAlreadyRegistered) as caught: + gs.register(host) + message = str(caught.value) + for expected in ("default generator", "generated", "BEFORE", "R1Q3 (ii)", + "RN-1 (a)", "5850003126", "unregister()"): + assert expected in message, f"the refusal no longer says {expected!r}" + assert gs.current() is stand_in_default + + +def test_asking_or_being_refused_is_not_a_generation(tmp_path) -> None: + """Only a generation closes the default's window. Asking which generator is + registered does not, and nor does an input refused before the call.""" + stand_in_default, calls = _declared(gs.NEUTRAL_SNAPSHOT_KIND) + gs.register_default(stand_in_default) + assert gs.is_registered() is True + assert gs.current() is stand_in_default + gs.name_of(gs.current()) + with pytest.raises(gs.GeneratorInputRefused): + gs.generate(tmp_path, "fixture", project_register_source=Path("r.yaml")) + assert calls == [] + host, _ = _declared("host-snapshot") + assert gs.register(host) is host + + +def test_unregister_clears_the_default_and_its_generation(tmp_path) -> None: + stand_in_default, _ = _declared(gs.NEUTRAL_SNAPSHOT_KIND) + gs.register_default(stand_in_default) + gs.generate(tmp_path, "fixture") + gs.unregister() + host, _ = _declared("host-snapshot") + assert gs.register(host) is host + + +def test_a_host_that_registers_the_default_itself_holds_a_hosts_registration() -> None: + """Which kind a registration is, is set by the call that MADE it.""" + gs.register(default_generator.GENERATOR) + assert gs.register_default(default_generator.GENERATOR) is \ + default_generator.GENERATOR + host, _ = _declared("host-snapshot") + with pytest.raises(gs.GeneratorAlreadyRegistered) as caught: + gs.register(host) + assert "a host's generator is already registered" in str(caught.value) + + +def test_openDoxs_own_generator_refuses_until_its_projection_lands(tmp_path) -> None: + """Plan 034 orders the seam (T052) before the projection (T054). So the + default refuses, naming itself and T054, and generates nothing. It never + answers an empty snapshot. T054 replaces this case with its own tests.""" + with pytest.raises(default_generator.NeutralProjectionNotBuilt) as caught: + default_generator.generate(tmp_path, "fixture") + message = str(caught.value) + for expected in (gs.NEUTRAL_SNAPSHOT_KIND, "T054", gs.REGISTRATION_CALL): + assert expected in message, f"the refusal no longer says {expected!r}" + assert isinstance(caught.value, gs.GeneratorSeamError) + gs.register_default(default_generator.GENERATOR) + with pytest.raises(default_generator.NeutralProjectionNotBuilt): + gs.generate(tmp_path, "fixture") + + +# -------------------------------------------------------------------------- +# 7 — each entry point registers openDox's own generator +# -------------------------------------------------------------------------- + +def _is_the_registration(stmt: ast.stmt) -> bool: + """`generator_seam.register_default(default_generator.GENERATOR)`, alone.""" + call = stmt.value if isinstance(stmt, ast.Expr) else None + return (isinstance(call, ast.Call) and isinstance(call.func, ast.Attribute) + and call.func.attr == "register_default" + and isinstance(call.func.value, ast.Name) + and call.func.value.id == "generator_seam" + and [ast.unparse(arg) for arg in call.args] + == ["default_generator.GENERATOR"] + and not call.keywords) + + +def _binds_the_seam(stmt: ast.stmt) -> bool: + """`from opendox import default_generator, generator_seam`.""" + return (isinstance(stmt, ast.ImportFrom) and stmt.level == 0 + and stmt.module == "opendox" + and {"default_generator", "generator_seam"} + <= {alias.asname or alias.name for alias in stmt.names}) + + +def _module_and_function(path: Path, function: str) -> tuple[list, list]: + """`path`'s module-level statements, and the body of its top-level `function`.""" + tree = ast.parse(path.read_text(encoding="utf-8")) + for node in tree.body: + if isinstance(node, ast.FunctionDef) and node.name == function: + return tree.body, node.body + raise AssertionError(f"{path.name} no longer defines {function}()") + + +@pytest.mark.parametrize("path,function", ( + (CLI, "build_parser"), (CLI, "main"), (SERVE, "build_server"), (SERVE, "main"))) +def test_each_entry_point_registers_openDoxs_own_generator_once(path: Path, + function: str) -> None: + module_body, body = _module_and_function(path, function) + registrations = [stmt for stmt in body if _is_the_registration(stmt)] + assert len(registrations) == 1, ( + f"{path.name}:{function}() makes {len(registrations)} registrations of " + "openDox's own generator, where R1Q10 (a) asks for exactly one " + "`generator_seam.register_default(default_generator.GENERATOR)`") + assert [stmt for stmt in module_body if _binds_the_seam(stmt)], ( + f"{path.name} registers openDox's own generator without importing it") + + +def test_build_parser_registers_openDoxs_own_generator_where_no_host_has() -> None: + from opendox import cli + + cli.build_parser() + assert gs.current() is default_generator.GENERATOR + + +def test_cli_main_registers_openDoxs_own_generator() -> None: + from opendox import cli + + with pytest.raises(SystemExit) as exited: + cli.main(["--help"]) + assert exited.value.code == 0 + assert gs.current() is default_generator.GENERATOR + + +def test_a_host_registered_first_is_kept_by_the_cli_entry_points() -> None: + from opendox import cli + + host, _ = _declared("host-snapshot") + gs.register(host) + cli.build_parser() + with pytest.raises(SystemExit): + cli.main(["--help"]) + assert gs.current() is host + + +@pytest.mark.parametrize("function", ("build_server", "main")) +def test_the_server_entry_points_register_openDoxs_own_generator(function: str) -> None: + """`serve.build_server()` and `serve.main()` cannot run in a lone checkout + until phase 2 routes their snapshot source (research R7). So the import + and the registration are lifted out of `serve.py` and executed: the tree's + own statements, not a paraphrase of them.""" + module_body, body = _module_and_function(SERVE, function) + lifted = [stmt for stmt in module_body if _binds_the_seam(stmt)] + \ + [stmt for stmt in body if _is_the_registration(stmt)] + module = ast.Module(body=lifted, type_ignores=[]) + exec(compile(ast.fix_missing_locations(module), str(SERVE), "exec"), {}) # noqa: S102 + assert gs.current() is default_generator.GENERATOR + host, _ = _declared("host-snapshot") + assert gs.register(host) is host, ( + "registering the default generated nothing, so a host still replaces it") + + +# -------------------------------------------------------------------------- +# 8 — `CorpusAdapter` stays closed at six members +# -------------------------------------------------------------------------- + +def test_corpus_adapter_stays_closed_at_six_members() -> None: + """The generator is handed over at its own seam, not as a seventh member.""" + assert corpus_adapter.OPERATIONS == ( + "resolve", "list_documents", "read", "classify", "check", "write_back") + assert set(corpus_adapter.CorpusAdapter.__protocol_attrs__) == \ + set(corpus_adapter.OPERATIONS) + assert not hasattr(corpus_adapter.CorpusAdapter, "generate") From 4ca45d27ac17a573cf0f5e5e304532b116b0b402 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sun, 27 Sep 2026 17:35:47 +0000 Subject: [PATCH 03/38] T052: each declared input optional; a default's generation recorded once it answers Copilot's review of this PR at 834f8ead raised two findings. Both are addressed here. 1. A declared input the operation cannot do without was accepted when it was declared. SnapshotGenerator bound the seam's call with every declared input given. But generate() passes an input only when its caller has a value for it, so such an operation failed with a raw TypeError the first time the option was unset. The declaration now binds the call twice, once with every declared input given and once with none given, and every call the seam can make lies between the two. Conformance clause 2 now says that each declared input is optional. 2. The default's window closed before its operation ran. generate() recorded a generation from the entry point's default before the call. So a generation that failed shut every host out although it wrote nothing, and that included openDox's own refusal before T054. The record is now made once a conformant snapshot comes back. A generation that fails, or whose answer the seam refuses, records nothing. While a generation from the default is under way, a host's registration is refused, because that snapshot would come back after the swap. The early record gave that property. A count of the generations under way now holds it. The registry's bookkeeping moves under one threading.Lock, because serve.py's ThreadingHTTPServer answers each request on a thread of its own. generate() holds the lock only to resolve and mark, and to record the end, and never across the generator's call. So a generator may register, unregister or generate from inside its own call. Tests: 59 -> 66 cases. Against 834f8ead's seam logic, 8 of the 66 go red: the new declared-input case, the pre-T054 default case, the two wrote-nothing cases, the three under-way cases and the re-entrant case. The concurrency and re-entrancy cases run in a process of their own with a time limit, so a lock held across a call fails them rather than hanging the suite. 13 of 13 mutations are caught. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/generator_seam.py | 232 +++++++++++++++++++++---------- tests/test_generator_seam.py | 253 ++++++++++++++++++++++++++++++++-- 2 files changed, 394 insertions(+), 91 deletions(-) diff --git a/src/opendox/generator_seam.py b/src/opendox/generator_seam.py index 0e8edcb..a2c8184 100644 --- a/src/opendox/generator_seam.py +++ b/src/opendox/generator_seam.py @@ -64,8 +64,11 @@ contracts can never both claim to be the one conformant implementation, which is the failure 5.4 was raised against. 2. Its operation TAKES THE SEAM'S CALL: the four arguments, and every input it - declares. This is checked when the declaration is made, wherever the - callable's signature can be read. + declares, EACH ONE OPTIONAL, because the seam passes a declared input only + when its caller has a value for it. This is checked when the declaration is + made, wherever the callable's signature can be read. The call is bound once + with every declared input given and once with none given, and between them + the two binds cover every call the seam can make. 3. It DECLARES EVERY FURTHER INPUT it reads. An input that it does not declare is refused before the generator is called, and never dropped. 4. It ANSWERS A SNAPSHOT OF ITS DECLARED CONTRACT: a `dict` whose `kind` is @@ -96,13 +99,24 @@ is refused as `GeneratorAlreadyRegistered`. One process would otherwise write two contracts, and a reader could not tell which snapshot was which. This is the same reason `domain_profile` refuses a swap after a build - (R1Q3 (ii); RN-1 (a), comment `5850003126`). + (R1Q3 (ii); RN-1 (a), comment `5850003126`); +* WHILE a snapshot is being generated from the default, a host's registration + is refused too, because that snapshot would come back after the swap; +* a generation that fails, or whose answer the seam refuses, wrote nothing, + so it records nothing, and a host still replaces the default after it. ONE REGISTRATION. Registering the same declaration again is a no-op, so an idempotent host start is not punished. A different declaration over a host's is refused as `GeneratorAlreadyRegistered`. `unregister()` makes a deliberate swap explicit. +ONE LOCK. The registration and its records are kept under one lock. `serve.py` +answers each request on a thread of its own (a `ThreadingHTTPServer`), so a +generation can run beside a registration. `generate()` holds the lock only for +its bookkeeping, and never across the generator's own call. So a slow +generator holds up nobody else, and a generator that itself registers or +generates cannot deadlock the seam. + RESOLVE PER CALL. A caller generates through `generate()`, and does not hold `current()`'s answer across calls. So every caller in a process answers from the one registration that exists at the time. @@ -119,6 +133,7 @@ import inspect import keyword +import threading from dataclasses import dataclass from pathlib import Path from typing import Any, Callable @@ -190,7 +205,8 @@ class GeneratorAlreadyRegistered(GeneratorSeamError): ONE registration is the contract, as it is for the profile. A process whose snapshots came from two generators would write two contracts under one name. The entry point's default is replaceable only until a snapshot has - been generated from it. `unregister()` makes a deliberate swap explicit.""" + been generated from it, and not while one is being generated. + `unregister()` makes a deliberate swap explicit.""" class GeneratorInputRefused(GeneratorSeamError): @@ -244,8 +260,8 @@ class SnapshotGenerator: that is not a non-empty string, an operation that is not callable, an input name that is not an identifier (or is a keyword, a duplicate, or one of the operation's own four), and an operation whose signature cannot take the - seam's call. Frozen, and compared by identity, as `register()` compares - registrations.""" + seam's call, whether its declared inputs are given or not. Frozen, and + compared by identity, as `register()` compares registrations.""" contract: str generate: Callable[..., dict] @@ -291,17 +307,27 @@ def _refuse_an_operation_that_cannot_take_the_call(self) -> None: # The signature cannot be read (some builtins). The seam's call is # then checked when it is first made, and not here. return - try: - signature.bind(Path("."), "repository", source_revision=None, - generated_at=None, - **{name: None for name in self.inputs}) - except TypeError as exc: - raise TypeError( - f"{_qualified(self.generate)} cannot take the generator seam's " - "call, generate(repo_root, repository, *, source_revision=, " - "generated_at=" + "".join(f", {name}=" for name in self.inputs) - + f"): {exc}. A contributed generator takes the operation's own " - "four arguments and every input it declares") from None + # `generate()` passes a declared input only when its caller has a value + # for it. So the call is bound with every declared input given, which + # proves that each is taken, and with none given, which proves that + # each is optional. Every call the seam can make lies between the two. + calls = [("", dict.fromkeys(self.inputs))] + if self.inputs: + calls.append((" with none of its declared inputs given", {})) + for when, given in calls: + try: + signature.bind(Path("."), "repository", source_revision=None, + generated_at=None, **given) + except TypeError as exc: + raise TypeError( + f"{_qualified(self.generate)} cannot take the generator " + f"seam's call{when}, generate(repo_root, repository, *, " + "source_revision=, generated_at=" + + "".join(f", {name}=" for name in given) + + f"): {exc}. A contributed generator takes the operation's " + "own four arguments and every input it declares, and each " + "declared input is optional, because the seam passes one " + "only when its caller has a value for it") from None #: THE one registration, a host's or the entry point's default, or `None`. @@ -311,10 +337,23 @@ def _refuse_an_operation_that_cannot_take_the_call(self) -> None: #: (`register_default()`), rather than a host's own `register()`. _is_default: bool = False -#: Whether a snapshot has been generated from that default. Set by -#: `generate()`, before the default's operation is called. +#: Whether a snapshot has been generated from that default, meaning that its +#: operation has answered a snapshot the seam handed back. `generate()` sets it +#: once that snapshot has come back. A generation that failed, or whose answer +#: the seam refused, wrote nothing, and so records nothing. _generated_from_default: bool = False +#: How many generations from that default are under way: begun, and not yet +#: answered or failed. While one is, a host's registration is refused, as it is +#: after one, because the snapshot being generated would come back after the +#: swap. `unregister()` leaves it alone: it counts calls that are still +#: running, and each one takes itself off when it ends. +_default_generations_under_way: int = 0 + +#: Guards the four above. It is held only for bookkeeping, and never across a +#: generator's own call. +_lock = threading.Lock() + def name_of(generator: Any) -> str: """A generator's most nameable name, with its contract, for a refusal. @@ -346,43 +385,53 @@ def register(generator: SnapshotGenerator) -> SnapshotGenerator: Registering the SAME declaration again is a no-op. A DIFFERENT one over a host's registration raises `GeneratorAlreadyRegistered`. Over the entry - point's default it REPLACES the default while nothing has been generated - from it, and is refused once something has. Either way the host ends up - holding the one registration, or knows why it does not.""" + point's default it REPLACES the default while no snapshot has been + generated from it and none is being generated. It is refused once one has + been, or while one is. Either way the host ends up holding the one + registration, or knows why it does not.""" global _registered, _is_default, _generated_from_default _require_a_declaration(generator, "register()") - if _registered is generator: - # The same object again: a no-op, before any bookkeeping is touched. - return generator - if _registered is not None: - if not _is_default: - raise GeneratorAlreadyRegistered( - f"a host's generator is already registered at openDox's " - f"generator seam ({name_of(_registered)}), and " - f"{name_of(generator)} would replace it. Registration happens " - "ONCE, at process start: one process generating through two " - "generators would write two contracts, and a reader could not " - "tell which snapshot was which. Call " - "opendox.generator_seam.unregister() first if the swap is " - "deliberate.") - if _generated_from_default: - raise GeneratorAlreadyRegistered( - f"openDox's own default generator ({name_of(_registered)}) is " - "registered, because an entry point registered it where no host " - "had, and a snapshot has already been generated from it, so " - f"{name_of(generator)} cannot replace it now. A swap would leave " - "one process writing two contracts. A host's generator replaces " - "the default only BEFORE anything is generated from it, as a " - "host's profile replaces the default profile only before a build " - "(R1Q3 (ii), openxFactory#656 comment 5817152735; RN-1 (a), " - "comment 5850003126). So register the host's generator at " - "process start, ahead of the first generation. Call " - "opendox.generator_seam.unregister() first if the swap is " - "deliberate.") - _registered = generator - _is_default = False - _generated_from_default = False - return generator + with _lock: + held = _registered + if held is generator: + # The same object again: a no-op, before any bookkeeping is touched. + return generator + over_a_host = held is not None and not _is_default + why = "" + if held is not None and _is_default: + if _default_generations_under_way: + why = "a snapshot is being generated from it now" + elif _generated_from_default: + why = "a snapshot has already been generated from it" + if not over_a_host and not why: + _registered = generator + _is_default = False + _generated_from_default = False + return generator + # Named outside the lock: naming a generator can run its own code. + if over_a_host: + raise GeneratorAlreadyRegistered( + f"a host's generator is already registered at openDox's " + f"generator seam ({name_of(held)}), and " + f"{name_of(generator)} would replace it. Registration happens " + "ONCE, at process start: one process generating through two " + "generators would write two contracts, and a reader could not " + "tell which snapshot was which. Call " + "opendox.generator_seam.unregister() first if the swap is " + "deliberate.") + raise GeneratorAlreadyRegistered( + f"openDox's own default generator ({name_of(held)}) is " + "registered, because an entry point registered it where no host " + f"had, and {why}, so " + f"{name_of(generator)} cannot replace it now. A swap would leave " + "one process writing two contracts. A host's generator replaces " + "the default only BEFORE anything is generated from it, as a " + "host's profile replaces the default profile only before a build " + "(R1Q3 (ii), openxFactory#656 comment 5817152735; RN-1 (a), " + "comment 5850003126). So register the host's generator at " + "process start, ahead of the first generation. Call " + "opendox.generator_seam.unregister() first if the swap is " + "deliberate.") def register_default(generator: SnapshotGenerator) -> SnapshotGenerator: @@ -406,22 +455,26 @@ def register_default(generator: SnapshotGenerator) -> SnapshotGenerator: f"the neutral snapshot contract {NEUTRAL_SNAPSHOT_KIND!r}, and " f"{name_of(generator)} declares another. A host's generator is " f"registered with {REGISTRATION_CALL}.") - if _registered is None: - _registered = generator - _is_default = True - _generated_from_default = False - return _registered + with _lock: + if _registered is None: + _registered = generator + _is_default = True + _generated_from_default = False + return _registered def unregister() -> None: """Drop the registration, a host's or the entry point's default. For test isolation and for a host tearing down. The record of a generation - from the default goes with it.""" + from the default goes with it. A generation still under way is not stopped. + When it answers, it records its snapshot only if the same default is + registered at that moment.""" global _registered, _is_default, _generated_from_default - _registered = None - _is_default = False - _generated_from_default = False + with _lock: + _registered = None + _is_default = False + _generated_from_default = False def is_registered() -> bool: @@ -434,7 +487,8 @@ def current() -> SnapshotGenerator: It answers what is registered: a host's generator, or the default an entry point registered. It never falls back to the default itself, and asking - records nothing: only a generation closes the default's window.""" + records nothing. Only a generation closes the default's window, while it + runs and once it has answered.""" if _registered is None: raise GeneratorNotRegistered( "no snapshot generator is registered at openDox's generator seam " @@ -466,11 +520,22 @@ def generate(repo_root: Path | str, repository: str, *, called (`GeneratorInputRefused`). What the generator answers is handed back only if it is a snapshot of the contract the generator declared (`GeneratorNotConformant`). With nothing registered this refuses as - `current()` does.""" - global _generated_from_default - generator = current() + `current()` does. + + A generation from the entry point's default holds the default's window shut + while it runs. Once its snapshot has come back, it closes the window for + good (see `register()`). A generation that fails, or whose answer is + refused, reopens it.""" + global _default_generations_under_way given = {name: value for name, value in inputs.items() if value is not None} - undeclared = sorted(set(given) - set(generator.inputs)) + with _lock: + # One hold for the resolve and the mark, so that no registration can + # land between the generator this call resolves and its call. + generator = current() + undeclared = sorted(set(given) - set(generator.inputs)) + from_default = _is_default and not undeclared + if from_default: + _default_generations_under_way += 1 if undeclared: declared = ", ".join(generator.inputs) or "none" raise GeneratorInputRefused( @@ -479,17 +544,32 @@ def generate(repo_root: Path | str, repository: str, *, "input a generator does not declare is refused rather than dropped: " "a generator that silently ignored it would write a snapshot that " "looks as though the input had been read.") - if _is_default: - # Before the call, so a registration racing a first generation cannot - # slip in between them. - _generated_from_default = True - snapshot = generator.generate( - Path(repo_root), repository, source_revision=source_revision, - generated_at=generated_at, **given) - _refuse_a_snapshot_that_does_not_conform(generator, snapshot) + answered = False + try: + snapshot = generator.generate( + Path(repo_root), repository, source_revision=source_revision, + generated_at=generated_at, **given) + _refuse_a_snapshot_that_does_not_conform(generator, snapshot) + answered = True + finally: + if from_default: + _end_a_generation_from_the_default(generator, answered) return snapshot +def _end_a_generation_from_the_default(generator: SnapshotGenerator, + answered: bool) -> None: + """Take a generation from the default off the count of those under way. If + it answered a snapshot the seam handed back, record that one was generated. + The record is made only while the default it came from is still the one + registered, so after `unregister()` it records nothing against a host.""" + global _default_generations_under_way, _generated_from_default + with _lock: + _default_generations_under_way -= 1 + if answered and _registered is generator and _is_default: + _generated_from_default = True + + def _refuse_a_snapshot_that_does_not_conform(generator: SnapshotGenerator, snapshot: Any) -> None: if not isinstance(snapshot, dict): diff --git a/tests/test_generator_seam.py b/tests/test_generator_seam.py index d8e1bab..9f9ca4e 100644 --- a/tests/test_generator_seam.py +++ b/tests/test_generator_seam.py @@ -20,9 +20,10 @@ 2. NOTHING REGISTERED REFUSES, NAMING THE SEAM AND THE CALL (4.2's discipline). It never falls back to openDox's own generator. 3. THE DECLARATION IS CHECKED WHEN IT IS MADE. A contract that is not a name, - an operation that is not callable or cannot take the seam's call, and an - input that is not a name beyond the operation's own four are refused before - anything is registered. + an operation that is not callable or cannot take the seam's call (with its + declared inputs given, or with none of them given), and an input that is not + a name beyond the operation's own four are refused before anything is + registered. 4. THE OPERATION IS HANDED OVER AND ITS ANSWER CHECKED. The registered generator receives the four arguments and its declared inputs. An undeclared input is refused before the call. A `None` input is not passed. @@ -32,8 +33,10 @@ 6. THE ENTRY POINTS' DEFAULT, in R1Q3 (a)'s pattern. openDox's own generator writes the neutral kind and takes no input. `register_default()` registers it only where nothing is, and holds it to the neutral contract. A host - replaces it before a generation, and is refused after one. Until T054 lands, - the default refuses, naming itself. + replaces it before a generation, and after one that wrote nothing. A host is + refused while a generation runs, and after one that wrote a snapshot. A + generator may register or generate from inside its own call without + deadlocking the seam. Until T054 lands, the default refuses, naming itself. 7. EACH ENTRY POINT REGISTERS IT. `cli.build_parser()` and `cli.main()` run for real. `serve.build_server()` and `serve.main()` still cannot run in a lone checkout (research R7), so their registration is executed from their own @@ -90,17 +93,19 @@ def _isolated_registries(): The three registries an entry point writes are process-global by design: the generator seam, the profile and the home corpus. Each is saved whole, because each keeps more than its registration (whether it is the entry - point's default, and whether anything was generated or built from it), and - a restore through `register()` would hand an entry point's default back as - a host's. + point's default, and whether anything was generated or built from it, or + is being generated), and a restore through `register()` would hand an entry + point's default back as a host's. """ - seam = (gs._registered, gs._is_default, gs._generated_from_default) + seam = (gs._registered, gs._is_default, gs._generated_from_default, + gs._default_generations_under_way) profile = (domain_profile._registered, domain_profile._is_default, domain_profile._built_from_default) home = corpus_adapter._home_factory gs.unregister() yield - (gs._registered, gs._is_default, gs._generated_from_default) = seam + (gs._registered, gs._is_default, gs._generated_from_default, + gs._default_generations_under_way) = seam (domain_profile._registered, domain_profile._is_default, domain_profile._built_from_default) = profile corpus_adapter._home_factory = home @@ -130,11 +135,12 @@ def operation(repo_root, repository, *, source_revision=None, def _fresh_process(program: str) -> subprocess.CompletedProcess: """`program` in a fresh interpreter, with this checkout's `src` first. The registry is process-global, so a case about what a PROCESS meets runs in - one of its own.""" + one of its own. So does a case that could deadlock the seam's lock: the + time limit then fails it, where in this process it would hang the suite.""" return subprocess.run( [sys.executable, "-c", f"import sys; sys.path.insert(0, {str(SRC)!r})\n" + textwrap.dedent(program)], - capture_output=True, text=True, cwd=str(ROOT)) + capture_output=True, text=True, cwd=str(ROOT), timeout=120) # -------------------------------------------------------------------------- @@ -281,23 +287,66 @@ def takes_the_four(repo_root, repository, *, source_revision=None, inputs=("project_register_source",)) -def test_a_governed_generators_shape_is_declarable() -> None: +def test_a_declared_input_the_operation_cannot_do_without_is_refused() -> None: + """Conformance clause 2: each declared input is optional. The seam passes a + declared input only when its caller has a value for it. So an operation + that REQUIRES one would take the call while the option is set, and fail + with a raw `TypeError` the first time it is not. Such an operation is + refused when it is declared, whether it requires the input by keyword or by + position.""" + def requires_it_by_keyword(repo_root, repository, *, project_register_source, + source_revision=None, generated_at=None): + return {} + + def requires_it_by_position(repo_root, repository, project_register_source, + *, source_revision=None, generated_at=None): + return {} + + for operation in (requires_it_by_keyword, requires_it_by_position): + with pytest.raises(TypeError) as caught: + gs.SnapshotGenerator(contract="stand-in-snapshot", generate=operation, + inputs=("project_register_source",)) + message = str(caught.value) + for expected in ("with none of its declared inputs given", + "project_register_source", "optional"): + assert expected in message, ( + f"{operation.__name__}: the refusal no longer says {expected!r}") + + def takes_it_optionally(repo_root, repository, project_register_source=None, + *, source_revision=None, generated_at=None): + return {} + + gs.SnapshotGenerator(contract="stand-in-snapshot", generate=takes_it_optionally, + inputs=("project_register_source",)) + + +def test_a_governed_generators_shape_is_declarable(tmp_path) -> None: """A consumer's generator keeps its own signature and declares its extras. This is the call shape of openXdox's `generator.generate_snapshot`, as T059 will declare it, restated here because openDox may not import it. Its keyword-only test hooks stay undeclared, and so the seam never passes them. + It generates with its declared inputs unset and with one of them set. """ def generate_snapshot(repo_root, repository, *, source_revision=None, generated_at=None, git=None, generator_version="v", project_register_source=None, possibles_source=None, excluded_documents=None): - return {} + return {"schema_version": 1, "kind": "governed-stand-in", + "register": project_register_source, + "possibles": possibles_source, "git": git} declared = gs.SnapshotGenerator( contract="governed-stand-in", generate=generate_snapshot, inputs=("project_register_source", "possibles_source")) assert declared.inputs == ("project_register_source", "possibles_source") + gs.register(declared) + unset = gs.generate(tmp_path, "fixture", project_register_source=None, + possibles_source=None) + assert (unset["register"], unset["possibles"], unset["git"]) == (None, None, None) + one_set = gs.generate(tmp_path, "fixture", + project_register_source=Path("register.yaml")) + assert (one_set["register"], one_set["possibles"]) == (Path("register.yaml"), None) @pytest.mark.parametrize("candidate", (None, "a generator", default_generator.generate)) @@ -502,6 +551,174 @@ def test_unregister_clears_the_default_and_its_generation(tmp_path) -> None: assert gs.register(host) is host +@pytest.mark.parametrize("failure", ("raises", "answers another kind")) +def test_a_generation_from_the_default_that_wrote_nothing_is_not_a_generation( + failure: str, tmp_path) -> None: + """A generation that fails, or whose answer the seam refuses, wrote no + snapshot, so it leaves the default's window open. A host still replaces the + default after it, and nothing is left counted as under way.""" + if failure == "raises": + def operation(repo_root, repository, *, source_revision=None, + generated_at=None): + raise OSError("the checkout is not there") + + stand_in_default = gs.SnapshotGenerator( + contract=gs.NEUTRAL_SNAPSHOT_KIND, generate=operation) + expected = OSError + else: + stand_in_default, _ = _declared( + gs.NEUTRAL_SNAPSHOT_KIND, + answer={"schema_version": 1, "kind": "another-snapshot"}) + expected = gs.GeneratorNotConformant + gs.register_default(stand_in_default) + with pytest.raises(expected): + gs.generate(tmp_path, "fixture") + assert gs._default_generations_under_way == 0 + host, _ = _declared("host-snapshot") + assert gs.register(host) is host + + +#: A program for a fresh process. A generation from the default runs on a +#: thread of its own, as a request does in `serve.py`'s `ThreadingHTTPServer`. +#: A host's registration is tried while it runs and again once it has ended. +#: `OUTCOME` is how the generation ends. +_UNDER_WAY = """ + import threading + from opendox import generator_seam as gs + OUTCOME = OUTCOME_VALUE + started, release = threading.Event(), threading.Event() + + def operation(repo_root, repository, *, source_revision=None, generated_at=None): + started.set() + release.wait(30) + if OUTCOME == "raises": + raise RuntimeError("the checkout went away mid-generation") + kind = gs.NEUTRAL_SNAPSHOT_KIND if OUTCOME == "answers" else "another-snapshot" + return {"schema_version": 1, "kind": kind} + + def host_operation(repo_root, repository, *, source_revision=None, generated_at=None): + return {"schema_version": 1, "kind": "host-snapshot"} + + host = gs.SnapshotGenerator(contract="host-snapshot", generate=host_operation) + gs.register_default(gs.SnapshotGenerator(contract=gs.NEUTRAL_SNAPSHOT_KIND, + generate=operation)) + ended = [] + + def request(): + try: + gs.generate(".", "fixture") + ended.append("answered") + except Exception as exc: + ended.append(type(exc).__name__) + + def try_the_host(): + try: + gs.register(host) + return "registered" + except gs.GeneratorAlreadyRegistered as exc: + if "a snapshot is being generated from it now" in str(exc): + return "refused-under-way" + if "a snapshot has already been generated from it" in str(exc): + return "refused-generated" + return "refused-otherwise" + + worker = threading.Thread(target=request) + worker.start() + assert started.wait(30), "the generation never started" + during = try_the_host() + release.set() + worker.join(30) + assert not worker.is_alive(), "the generation never ended" + assert gs._default_generations_under_way == 0, gs._default_generations_under_way + print(during, ended[0], try_the_host()) +""" + + +@pytest.mark.parametrize("outcome,ends,host_after", ( + ("answers", "answered", "refused-generated"), + ("raises", "RuntimeError", "registered"), + ("answers another kind", "GeneratorNotConformant", "registered"))) +def test_a_host_is_refused_while_a_generation_from_the_default_is_under_way( + outcome: str, ends: str, host_after: str) -> None: + """While a snapshot is being generated from the default, a host's + registration is refused, because that snapshot would come back after the + swap. Whether the window stays shut afterwards depends on whether the + generation WROTE a snapshot. A conformant answer shuts it for good. A + failure, or an answer the seam refuses, reopens it. The case runs in a + process of its own (see `_fresh_process`).""" + done = _fresh_process(_UNDER_WAY.replace("OUTCOME_VALUE", repr(outcome))) + assert done.returncode == 0, done.stderr + assert done.stdout.split() == ["refused-under-way", ends, host_after] + + +def test_a_generator_may_register_or_generate_from_inside_its_own_call() -> None: + """The seam's lock is held only for its bookkeeping, and never across a + generator's call. So a generator that registers, unregisters or generates + from inside its own call cannot deadlock the seam, and the seam's records + stay true. A default swapped out from inside its own call records nothing + against the default that replaced it. The case runs in a process of its own + (see `_fresh_process`).""" + done = _fresh_process(""" + from opendox import generator_seam as gs + KIND = gs.NEUTRAL_SNAPSHOT_KIND + seen = [] + + def snapshot(repository): + return {"schema_version": 1, "kind": KIND, "repository": repository} + + def host_operation(repo_root, repository, *, source_revision=None, + generated_at=None): + return {"schema_version": 1, "kind": "host-snapshot"} + + host = gs.SnapshotGenerator(contract="host-snapshot", generate=host_operation) + + def nesting(repo_root, repository, *, source_revision=None, generated_at=None): + if repository == "outer": + try: + gs.register(host) + seen.append("host-registered-inside") + except gs.GeneratorAlreadyRegistered as exc: + seen.append("refused-inside" + if "is being generated from it now" in str(exc) + else "refused-otherwise") + seen.append(gs.generate(repo_root, "inner")["repository"]) + return snapshot(repository) + + gs.register_default(gs.SnapshotGenerator(contract=KIND, generate=nesting)) + seen.append(gs.generate(".", "outer")["repository"]) + try: + gs.register(host) + seen.append("host-registered-after") + except gs.GeneratorAlreadyRegistered: + seen.append("refused-after") + + def replacing(repo_root, repository, *, source_revision=None, + generated_at=None): + return snapshot(repository) + + replacement = gs.SnapshotGenerator(contract=KIND, generate=replacing) + + def swapping(repo_root, repository, *, source_revision=None, + generated_at=None): + gs.unregister() + gs.register_default(replacement) + return snapshot(repository) + + gs.unregister() + gs.register_default(gs.SnapshotGenerator(contract=KIND, generate=swapping)) + gs.generate(".", "swapped") + assert gs.current() is replacement + seen.append("host-registered-over-the-replacement" + if gs.register(host) is host else "not-registered") + assert gs._default_generations_under_way == 0 + print(" ".join(seen)) + """) + assert done.returncode == 0, done.stderr + assert done.stdout.split() == ["refused-inside", "inner", "outer", + "refused-after", + "host-registered-over-the-replacement"] + + def test_a_host_that_registers_the_default_itself_holds_a_hosts_registration() -> None: """Which kind a registration is, is set by the call that MADE it.""" gs.register(default_generator.GENERATOR) @@ -516,7 +733,9 @@ def test_a_host_that_registers_the_default_itself_holds_a_hosts_registration() - def test_openDoxs_own_generator_refuses_until_its_projection_lands(tmp_path) -> None: """Plan 034 orders the seam (T052) before the projection (T054). So the default refuses, naming itself and T054, and generates nothing. It never - answers an empty snapshot. T054 replaces this case with its own tests.""" + answers an empty snapshot. Because it wrote nothing, its refusal shuts no + host out: a host's generator still replaces it afterwards. T054 replaces + this case with its own tests.""" with pytest.raises(default_generator.NeutralProjectionNotBuilt) as caught: default_generator.generate(tmp_path, "fixture") message = str(caught.value) @@ -526,6 +745,10 @@ def test_openDoxs_own_generator_refuses_until_its_projection_lands(tmp_path) -> gs.register_default(default_generator.GENERATOR) with pytest.raises(default_generator.NeutralProjectionNotBuilt): gs.generate(tmp_path, "fixture") + host, _ = _declared("host-snapshot") + assert gs.register(host) is host, ( + "a refused generation from openDox's own generator wrote nothing, so " + "a host still replaces it") # -------------------------------------------------------------------------- From f52b11178c129f84508d7fb493496d953dda5abc Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sun, 27 Sep 2026 18:20:50 +0000 Subject: [PATCH 04/38] T051: the malformed fixture, one title-and-summary-are-text violation (plan 034) tests/fixtures/malformed/ carries two .md documents: notes-bee-boxes.md is an ordinary valid source (non-empty title and summary), and notes-empty-title.md declares `title:` with nothing after the colon -- present in the header, but an empty string once parsed -- while its `summary:` stays ordinary. That is exactly one violation of the neutral snapshot schema T053 adds (opensoft/openDox-spec#16 at cd49eb25): `documents[].title` and `.summary` are each `type: ["string", "null"], minLength: 1` (x-rule `title-and-summary-are-text`, "each non-empty text, or null"). A document that declares no title/summary line at all yields null, which is valid (openDox-spec's own no-front-matter example); an empty string is the one value that is neither null nor non-empty text. Holder decision: T054 copies title/summary verbatim, without coercing an empty declared value to null or excluding the document, so the violation survives unchanged into the generated snapshot. tests/fixtures/malformed/EXPECTED_RULE holds the violated rule's identifier verbatim, `title-and-summary-are-text`, for the future `opendox generate --strict` to name (spec.md AT-R1 scenario 2). T051 carries no falsifier of its own (tasks.md: "used by F7.2"); this PR adds only the fixture and the sentinel file. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/fixtures/malformed/EXPECTED_RULE | 1 + tests/fixtures/malformed/notes-bee-boxes.md | 8 ++++++++ tests/fixtures/malformed/notes-empty-title.md | 7 +++++++ 3 files changed, 16 insertions(+) create mode 100644 tests/fixtures/malformed/EXPECTED_RULE create mode 100644 tests/fixtures/malformed/notes-bee-boxes.md create mode 100644 tests/fixtures/malformed/notes-empty-title.md diff --git a/tests/fixtures/malformed/EXPECTED_RULE b/tests/fixtures/malformed/EXPECTED_RULE new file mode 100644 index 0000000..b9af122 --- /dev/null +++ b/tests/fixtures/malformed/EXPECTED_RULE @@ -0,0 +1 @@ +title-and-summary-are-text diff --git a/tests/fixtures/malformed/notes-bee-boxes.md b/tests/fixtures/malformed/notes-bee-boxes.md new file mode 100644 index 0000000..c530fb7 --- /dev/null +++ b/tests/fixtures/malformed/notes-bee-boxes.md @@ -0,0 +1,8 @@ +title: Setting up two new bee boxes +summary: Where the new hives will go, and how far from the path. + +# Setting up two new bee boxes + +Two new bee boxes go in along the back fence, angled away from the main +path so foragers do not cross where people walk. Each box gets a paving +slab underneath to keep it level and dry through the winter. diff --git a/tests/fixtures/malformed/notes-empty-title.md b/tests/fixtures/malformed/notes-empty-title.md new file mode 100644 index 0000000..4cf6410 --- /dev/null +++ b/tests/fixtures/malformed/notes-empty-title.md @@ -0,0 +1,7 @@ +title: +summary: A short summary is here; only the title line is declared and left empty. + +# A note with no title + +The title line above is present but carries no text after the colon. Its +summary is ordinary. Nothing else about this note is unusual. From 25fe7526f94342a022a1d36a1044bf3542284113 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sun, 27 Sep 2026 21:19:08 +0000 Subject: [PATCH 05/38] T054 (work in progress): openDox's small neutral projection over CorpusAdapter (plan 034) A checkpoint commit, pushed on the coordinator's usage stop. It is not the finished task: four node-harness cases in tests/test_display_facet.py still carry the governed snapshot values in their fixtures and fail against the new defaults, and T054's own test file is not written yet. What this commit holds: - src/opendox/neutral_projection.py (new): the projection. It writes T053's opendox-snapshot, places a document by its neutral stage: key (reading a value outside the six role keys as a source and reporting it), copies title and summary verbatim, and applies the topic rule and the group rule its docstring names. - src/opendox/default_generator.py: generate() projects the home corpus instead of refusing. NeutralProjectionNotBuilt is retired, and so is its refusal case in tests/test_generator_seam.py. - src/opendox/display_profile.py and web/views/display.js: SNAPSHOT_VALUES' defaults become the neutral snapshot's values. display.js keeps its line count, so the web census row is unchanged. - src/opendox/runtime/local_git_adapter.py: leading_header() is public, NEUTRAL_FIELDS is declared, WorkingTreeCorpus defaults required_fields to it, and the suffix branch of classify reports missing fields when fields are required. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/default_generator.py | 102 +++-- src/opendox/display_profile.py | 19 +- src/opendox/neutral_projection.py | 490 +++++++++++++++++++++++ src/opendox/runtime/local_git_adapter.py | 98 ++++- src/opendox/web/views/display.js | 42 +- tests/test_generator_seam.py | 24 +- 6 files changed, 678 insertions(+), 97 deletions(-) create mode 100644 src/opendox/neutral_projection.py diff --git a/src/opendox/default_generator.py b/src/opendox/default_generator.py index f2b9cb6..3d516ca 100644 --- a/src/opendox/default_generator.py +++ b/src/opendox/default_generator.py @@ -8,20 +8,33 @@ the neutral snapshot contract, `generator_seam.NEUTRAL_SNAPSHOT_KIND` (plan 034's T053; R1Q11 (a)), and takes no input beyond the operation's own four. -ITS PROJECTION IS NOT BUILT YET, AND IT SAYS SO. Plan 034 orders the seam -before the projection. T054, openDox's small neutral projection over -`CorpusAdapter` (#1144's 5.1-5.3), comes after this task (T052). The entry -points' registration calls are this task's, because T054 edits neither -`cli.py` nor `serve.py`, whose single-writer order runs T052, then T055. So -`generate()` below refuses, naming itself and the task that builds it, and it -generates nothing. It never answers an empty snapshot, which would read exactly -like an honest one. +WHAT IT GENERATES. `generate()` reads the home corpus through the home-corpus +seam, `corpus_adapter.home()`, and projects it with openDox's own small +neutral projection, `opendox.neutral_projection` (plan 034's T054; #1144's +5.1-5.3). Standalone, the home corpus is the default the entry points register, +`local_git_adapter.WorkingTreeCorpus`, which is how the projection is bound to +`LocalGitCorpus` (5.2). With no home corpus registered, the seam's own refusal +names the call that is missing, and nothing is generated. + +ITS TWO ANCHORS. `source_revision` is recorded as the caller pins it, and +otherwise it is the revision the corpus resolved at, its git HEAD. The +projection refuses where there is neither. `generated_at` is recorded as the +caller gives it, and otherwise it is the committer date of that revision, +which `git` reads from the checkout (`git show -s --format=%cI`, the stamp the +generate verbs have always used). It is looked up only for a revision spelled +as a hexadecimal object id, so a revision is never handed to `git` where it +could read as an option. It is left out where the lookup fails. Nothing reads +the clock, so the same tree at the same anchors answers the same snapshot. + +WHAT IT REPORTS. The projection's notices go to standard error, one line each, +`notice: : `. A `stage:` value outside the +six role keys is one (R1Q13 (a)): the line names the document, the value and +the six keys, and the snapshot reads that document as a source. NO VERB REACHES IT YET. The generate verbs still call the consumer's generator, -through `consumer_reach`, until T055 routes them through the seam, and T055 -comes after T054. The refusal is therefore reachable only by a library caller -that generates through the seam directly. T054 replaces it with the -projection, and the declaration below keeps its contract and its inputs. +through `consumer_reach`, until T055 routes them through the seam (T055 comes +after T054, in `cli.py`'s and `serve.py`'s single-writer order). Until then a +library caller reaches it through the seam, `generator_seam.generate()`. HOW IT IS REGISTERED: by the entry points, and never at import (R1Q3 (a)'s pattern). `cli.build_parser()`, `cli.main()`, `serve.build_server()` and @@ -29,8 +42,11 @@ registers it only where nothing is registered. Importing this module registers nothing. -IMPORT WEIGHT. `opendox.generator_seam` and the standard library. So this -module imports with no extra installed and no sibling present. +IMPORT WEIGHT. `opendox.generator_seam`, `opendox.corpus_adapter`, +`opendox.neutral_projection` (which adds `opendox.display_profile`, +`opendox.path_slug` and `opendox.runtime.local_git_adapter`), and the standard +library. So this module imports with no extra installed and no sibling +present. A CREATED FILE: it has no row in openxFactory's `docs/opendox-carve-manifest.yaml`, because the manifest declares what LEAVES @@ -39,36 +55,58 @@ from __future__ import annotations +import re +import sys from pathlib import Path from typing import Any -from opendox import generator_seam +from opendox import corpus_adapter, generator_seam, neutral_projection +from opendox.runtime.local_git_adapter import GitCommandFailed, GitRunner -__all__ = ["GENERATOR", "NeutralProjectionNotBuilt", "generate"] +__all__ = ["GENERATOR", "generate"] +#: A revision spelled as a hexadecimal object id, abbreviated or whole. Only +#: such a revision is handed to `git` to read its date. +_OBJECT_ID = re.compile(r"[0-9a-fA-F]{4,64}") -class NeutralProjectionNotBuilt(generator_seam.GeneratorSeamError): - """openDox's own generator is declared, and its projection is not built. +#: `git show -s --format=%cI`'s shape: a strict ISO 8601 date-time with an +#: explicit offset, as the neutral schema's `generated-at-is-rfc3339` rule +#: admits it. +_COMMIT_DATE = re.compile( + r"(?!0000)[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-5][0-9]" + r"(?:Z|[+-][0-9]{2}:[0-9]{2})") - A subclass of the seam's own refusal, so a verb that reports the seam's - refusals reports this one too. It goes when plan 034's T054 lands the - projection.""" + +def _commit_date(location: str, revision: str) -> str | None: + """The committer date of `revision` in the checkout at `location`, or + None where `git` cannot answer one.""" + if not _OBJECT_ID.fullmatch(revision): + return None + try: + stamp = GitRunner(Path(location)).out( + "show", "-s", "--format=%cI", f"{revision}^{{commit}}", "--") + except (GitCommandFailed, OSError): + return None + text = stamp.decode("utf-8", "replace").strip() + return text if _COMMIT_DATE.fullmatch(text) else None def generate(repo_root: Path, repository: str, *, source_revision: str | None = None, generated_at: str | None = None) -> dict[str, Any]: - """The operation the seam hands over, for openDox's own generator. - - It REFUSES until plan 034's T054 builds the neutral projection. See the - module docstring for why the declaration lands first.""" - raise NeutralProjectionNotBuilt( - f"openDox's own generator is declared at the generator seam, writing " - f"{generator_seam.NEUTRAL_SNAPSHOT_KIND!r}, but its projection is not " - f"built at this commit (plan 034's T054 builds it), so nothing was " - f"generated for {repository!r} at {str(repo_root)!r}. A host that has " - f"a generator of its own registers it at process start with " - f"{generator_seam.REGISTRATION_CALL}.") + """The operation the seam hands over, for openDox's own generator: the + neutral snapshot of the home corpus at `repo_root`.""" + adapter, ref = corpus_adapter.home()(str(repo_root)) + corpus = adapter.resolve(ref) + anchor = source_revision if source_revision is not None else corpus.revision + if generated_at is None and anchor is not None: + generated_at = _commit_date(corpus.location, anchor) + projection = neutral_projection.project( + adapter, corpus, repository, + source_revision=anchor, generated_at=generated_at) + for notice in projection.notices: + print(f"notice: {notice}", file=sys.stderr) + return projection.snapshot #: openDox's OWN generator, as the entry points register it where no host has. diff --git a/src/opendox/display_profile.py b/src/opendox/display_profile.py index 18b2b2f..821a2c7 100644 --- a/src/opendox/display_profile.py +++ b/src/opendox/display_profile.py @@ -224,12 +224,25 @@ class C — *"replace every governance word in it with a placeholder and the fil #: A HOST MAY STILL OVERRIDE THEM, on the `values` block of the facet, for the #: descendant whose generator writes different words into the same schema. The #: default is openDox's declaration, not a domain's silence. +#: +#: THE DEFAULTS ARE openDox's OWN NEUTRAL SNAPSHOT'S VALUES (RULED R1Q11 (a), +#: openxFactory#656 comment `5850003126`; plan 034 T054, as T007's batch G +#: amends #1144's 5.3). openDox generates that snapshot itself +#: (`neutral_projection`), and T053's schema closes both enums: a document's +#: stage is one of the six station role keys, and a candidate's state is one +#: of `unselected`, `selected`, `declined` and `replaced`. So the product's own +#: views match its own snapshot with no facet declared. A source document's +#: stage is `source`, and a document that gathers others is at `grouping`, so +#: those two are the captured and the organized stage. The candidate's four +#: are NEUTRAL_DISPLAY's own candidate words below, role for role. No word is +#: re-authored. The governed snapshot's own values move to openXdox's facet, +#: on its `values` block (T060). SNAPSHOT_VALUES: dict[str, dict[str, str]] = { # `documents[].stage` — which pipeline column a source document sits in. - "document_stage": {"captured": "brainstorm", "organized": "staged"}, + "document_stage": {"captured": "source", "organized": "grouping"}, # `possibles[].state` — the candidate register's own four-state enum. - "register_state": {"captured": "latent", "proposed": "picked", - "retired": "rejected", "superseded": "superseded"}, + "register_state": {"captured": "unselected", "proposed": "selected", + "retired": "declined", "superseded": "replaced"}, } #: THE STAGING TEMPLATE'S CANONICAL HEADING ORDER — the last member of § 2.2 diff --git a/src/opendox/neutral_projection.py b/src/opendox/neutral_projection.py new file mode 100644 index 0000000..edb75a8 --- /dev/null +++ b/src/opendox/neutral_projection.py @@ -0,0 +1,490 @@ +"""openDox's OWN small neutral projection: a plain corpus, read through +`CorpusAdapter`, written as the neutral snapshot (plan 034 T054). + +WHAT IT REALIZES. #1144's boxes 5.1, 5.2 and 5.3, as plan 034's T054 reads +them. + +* 5.1. This is new code over the `CorpusAdapter` protocol, written to openDox's + own needs. It is not openXdox's generator re-expressed, and it imports + nothing of openXdox's or openxFactory's. +* 5.2. It reads whatever adapter the home-corpus seam hands it. Standalone, + that is openDox's own default, `local_git_adapter.WorkingTreeCorpus`, a + `LocalGitCorpus` (T022). `default_generator` makes that call. +* 5.3. It writes T053's neutral snapshot contract, `opendox-snapshot` + (openDox-spec's `contracts/schemas/opendox-snapshot.schema.yaml`; R1Q11 + (a), openxFactory#656 comment `5850003126`). Its values are the six station + role keys and the four candidate states, and `display_profile + .SNAPSHOT_VALUES` defaults to those same values, so openDox's views place + every card by them and render the six words. + +WHERE A DOCUMENT LANDS (RULED R1Q13 (a) with (c), same comment). A document +names its station with the neutral `stage:` key of its leading `Name: value` +header, which is `local_git_adapter.leading_header`'s block. That is the same +reader the default adapter's `classify` uses. The value must be one of the six +station role keys, `display_profile.STAGE_ROLES`. A document that declares no +`stage:` is a source. A `stage:` value outside the six, the empty value +included, is not a declaration either. The document is read as a source, and +the projection reports it, naming the document, the value and the six keys. So +no other value can reach the snapshot, whose schema admits only the six. + +THE TOPIC RULE. Every document carries `topics`. A document that declares a +`topics:` header carries the ones it lists, split on commas, with whitespace +collapsed and letters case-folded. A document that declares none carries the +words of its NAME, plus the words of the name of every other document it +MENTIONS. Its name is its `title:`, else its first `#` heading, else its file +name without the suffix. It mentions another document where its text holds +that document's name, or that document's file name, as a run of whole words. +A word is a run of three or more letters, case-folded, that is not one of +`STOPWORDS`. So the rule needs no front matter at all. AT-R1's repository (b) +(quickstart.md § 2) is three notes named by their headings, and a note that +mentions another shares that note's name as a topic. An entry the adapter +cannot classify is listed but never read, so it carries no topic and nothing +mentions it. + +GROUPS (the grouping station). A group forms wherever two or more sources +share a topic. Topics that the same sources share form one group, named by +those topics. A document that declares `stage: grouping` is a group of its +own: it gathers itself and every source that shares one of its topics. Each +group holds one edge per member document, naming the topics that matched. + +THE OTHER STATIONS. A declared candidate is a candidate. It is `unselected`, +because only an act selects, declines or replaces one, and it is claimed by +every group it shares a topic with. A declared selection is a selection, and +a declared submission or completion is a changes entry, `active` or +`archived` as `display_profile.STAGE_FIELDS` declares them. Each of the three +lists the document as its file. So a plain repository fills every station its +documents name, and no other. + +TITLE AND SUMMARY are copied from the header EXACTLY as the header gives them, +an empty value included, and `null` only where the document does not give the +key. They are never coerced, and no document is left out for lacking them +(the holder's ruling on T051: the malformed fixture's empty `title:` reaches +the snapshot and fails `title-and-summary-are-text`). + +DETERMINISTIC. The same tree at the same anchors answers the same snapshot. +Every list is ordered by path or sorted, and nothing reads the clock. +`generation.source_revision` is the revision the caller pins, else the one the +corpus resolved. `generation.generated_at` is recorded only as the caller +hands it, and `default_generator` hands the source revision's own commit date. + +A CREATED FILE: it has no row in openxFactory's +`docs/opendox-carve-manifest.yaml`, because the manifest declares what LEAVES +openxFactory and never what a destination assembles (RULED OQ-C). +""" + +from __future__ import annotations + +import re +from collections import Counter +from dataclasses import dataclass +from pathlib import PurePosixPath +from typing import Any + +from opendox import generator_seam +from opendox.corpus_adapter import CorpusAdapter, ResolvedCorpus +from opendox.display_profile import STAGE_FIELDS, STAGE_ROLES +from opendox.path_slug import slug +from opendox.runtime.local_git_adapter import leading_header + +__all__ = [ + "GENERATOR_VERSION", + "Notice", + "Projection", + "ProjectionRefused", + "SCHEMA_VERSION", + "STOPWORDS", + "project", +] + +#: The neutral contract's version, which its schema holds at `const: 1`. +SCHEMA_VERSION = 1 + +#: This projection's own name and version, written to `generation`. +GENERATOR_VERSION = "opendox-neutral-1" + +#: The neutral header keys this projection reads. `title` and `summary` are the +#: default adapter's small field set (`local_git_adapter.NEUTRAL_FIELDS`). +STAGE_KEY = "stage" +TOPICS_KEY = "topics" +TITLE_KEY = "title" +SUMMARY_KEY = "summary" + +#: The station a document lands in when it declares none (R1Q13 (c)). +SOURCE = "source" +GROUPING = "grouping" +CANDIDATE = "candidate" +SELECTION = "selection" + +#: A candidate's state until an act selects, declines or replaces it. +UNSELECTED = "unselected" + +#: `changes[].status` for the two stations that share the `changes` section, +#: read off `display_profile.STAGE_FIELDS` rather than restated. +CHANGE_STATUS: dict[str, str] = { + role: status for role, _section, status in STAGE_FIELDS if status} + +#: Common English words the topic rule never takes as a topic. Words shorter +#: than three letters are never topics anyway, so none of those is listed. +STOPWORDS: frozenset[str] = frozenset(""" + about above across actually after again against all along already also + always among and another any are around back been before behind being + below beside between beyond both but can cannot could did does doing done + down during each either else even ever every few for from further get + gets got had has have having her here hers herself him himself his how + inside into its itself just last less let like made make many may maybe + might more most much must myself near neither never new next none nor + not nothing now off often once one only onto other others ought our ours + ourselves out outside over own per perhaps quite rather really same say + see she should since some still such than that the their theirs them + themselves then there these they this those though three through thus + till too toward towards two under until upon use very via want was way + well were what whatever when where whether which while who whom whose why + will with within without would yet you your yours yourself yourselves +""".split()) + +#: A run of letters or digits: what the topic rule reads as one token. +_TOKEN = re.compile(r"[^\W_]+") + +#: An ATX heading (`# Roadmap`, `## Budget ##`), its text in group 1. +_ATX_HEADING = re.compile(r"^ {0,3}#{1,6}(?:[ \t]+(.*?))?(?:[ \t]+#+)?[ \t]*$") + +#: A fenced code block's opening or closing line. A `#` line inside one is code. +_FENCE = re.compile(r"^ {0,3}(`{3,}|~{3,})") + +#: The characters the neutral schema refuses in a path or a topic +#: (`path-is-repo-relative`, `topic-is-trimmed-text`). +_CONTROL = re.compile("[\u0000-\u001f\u007f-\u009f]") + + +class ProjectionRefused(generator_seam.GeneratorSeamError): + """The projection cannot write a snapshot the neutral contract admits. + + A subclass of the generator seam's own refusal, so a verb that reports the + seam's refusals reports this one too.""" + + +@dataclass(frozen=True) +class Notice: + """One thing the projection read otherwise than the document declared it. + + `document` is the document's path as the adapter lists it. The generate + verbs report each notice, and the snapshot never carries one.""" + + document: str + message: str + + def __str__(self) -> str: + return f"{self.document}: {self.message}" + + +@dataclass(frozen=True) +class Projection: + """The neutral snapshot, and what the projection had to report making it.""" + + snapshot: dict[str, Any] + notices: tuple[Notice, ...] + + +@dataclass +class _Document: + """One listed document, as the projection reads it.""" + + path: str + stage: str = SOURCE + title: str | None = None + summary: str | None = None + topics: tuple[str, ...] = () + read: bool = False + name: str = "" + tokens: tuple[str, ...] = () + declared_topics: tuple[str, ...] | None = None + + @property + def stem(self) -> str: + return PurePosixPath(self.path).stem or self.path + + +def _tokens(text: str) -> tuple[str, ...]: + return tuple(token.casefold() for token in _TOKEN.findall(text)) + + +def _is_word(token: str) -> bool: + return len(token) >= 3 and token.isalpha() and token not in STOPWORDS + + +def _words(text: str) -> set[str]: + return {token for token in _tokens(text) if _is_word(token)} + + +def _first_heading(text: str) -> str | None: + """The text of the first ATX heading outside a fenced code block.""" + fence: str | None = None + for line in text.splitlines(): + marker = _FENCE.match(line) + if fence is not None: + if (marker and marker.group(1)[0] == fence[0] + and len(marker.group(1)) >= len(fence)): + fence = None + continue + if marker: + fence = marker.group(1) + continue + heading = _ATX_HEADING.match(line) + if heading and (heading.group(1) or "").strip(): + return heading.group(1).strip() + return None + + +def _declared_topics(value: str) -> tuple[str, ...]: + """The topics a `topics:` header lists: comma-separated, each with its + whitespace collapsed and its letters case-folded, each once, sorted. A + topic holding a control character is dropped, since no snapshot can carry + it (`topic-is-trimmed-text`).""" + topics: set[str] = set() + for part in value.split(","): + topic = " ".join(part.replace("", " ").split()).casefold() + if topic and not _CONTROL.search(topic): + topics.add(topic) + return tuple(sorted(topics)) + + +def _unwritable(path: str) -> str | None: + """Why the neutral schema could not carry `path`, or None where it can. + + The schema's `path-is-repo-relative` rule, stated as reasons: git can list + a path that holds any of these, and the snapshot must not.""" + if not path: + return "it is empty" + if path.startswith("/"): + return "it starts with a slash" + if re.match(r"[A-Za-z]:", path): + return "it starts with a drive letter and a colon" + if "\\" in path: + return "it holds a backslash" + if _CONTROL.search(path): + return "it holds a control character" + if ".." in path.split("/"): + return "it has a .. segment" + if any("\ud800" <= char <= "\udfff" for char in path): + return "its name is not valid UTF-8" + return None + + +def _unique(base: str, taken: set[str]) -> str: + """`base`, or `base-2`, `base-3`, ... where an earlier entry holds it.""" + candidate, number = base, 2 + while candidate in taken: + candidate, number = f"{base}-{number}", number + 1 + taken.add(candidate) + return candidate + + +def _read_documents(adapter: CorpusAdapter, corpus: ResolvedCorpus, + notices: list[Notice]) -> list[_Document]: + """Every listed document the snapshot can carry, in path order, with its + station, its two copied fields, its name and its tokens.""" + stations = set(STAGE_ROLES) + documents: list[_Document] = [] + listed = sorted(adapter.list_documents(corpus), key=lambda d: d.key) + for identity in listed: + reason = _unwritable(identity.key) + if reason is not None: + # NAMED BY ITS repr: the path is one no snapshot can carry, and a + # control character or a lone surrogate in it must not reach a + # terminal raw either. + notices.append(Notice( + repr(identity.key), + f"is listed, but the neutral snapshot cannot carry its path " + f"({reason}), so it is left out")) + continue + document = _Document(path=identity.key) + documents.append(document) + if adapter.classify(corpus, identity).kind is None: + # UNCLASSIFIABLE: listed, never read. It is a source with nothing + # to copy, no topic, and no name another document can mention. + continue + text = adapter.read(corpus, identity).content.decode("utf-8", "replace") + header = leading_header(text) + document.read = True + document.title = header.get(TITLE_KEY) + document.summary = header.get(SUMMARY_KEY) + declared = header.get(STAGE_KEY) + if declared is not None: + if declared in stations: + document.stage = declared + else: + notices.append(Notice( + identity.key, + f"its stage: value {declared!r} is not one of the six " + f"station role keys ({', '.join(STAGE_ROLES)}), so it is " + "not a declaration; the document is read as a source")) + listed_topics = _declared_topics(header.get(TOPICS_KEY) or "") + document.declared_topics = listed_topics or None + document.name = (document.title or _first_heading(text) + or document.stem) + document.tokens = _tokens(text) + return documents + + +def _assign_topics(documents: list[_Document]) -> None: + """The topic rule (this module's docstring), over the documents read.""" + read = [document for document in documents if document.read] + own = [_words(document.name) for document in read] + # Each name is looked for by its first WORD, at that word's offset, so a + # text is scanned once and a common short token ("the") costs nothing. + by_first_word: dict[str, list[tuple[tuple[str, ...], int, int]]] = {} + for index, document in enumerate(read): + for alias in {_tokens(document.name), _tokens(document.stem)}: + # A name with no word in it contributes no topic, so it is not + # looked for: `a.md` would otherwise be mentioned by every "a". + offset = next((at for at, token in enumerate(alias) + if _is_word(token)), None) + if offset is not None: + by_first_word.setdefault(alias[offset], []).append( + (alias, offset, index)) + for index, document in enumerate(read): + if document.declared_topics is not None: + document.topics = document.declared_topics + continue + mentioned: set[int] = set() + tokens = document.tokens + for position, token in enumerate(tokens): + for alias, offset, other in by_first_word.get(token, ()): + start = position - offset + if (other != index and other not in mentioned and start >= 0 + and tokens[start:start + len(alias)] == alias): + mentioned.add(other) + topics = set(own[index]) + for other in mentioned: + topics |= own[other] + document.topics = tuple(sorted(topics)) + + +def _groups(documents: list[_Document], notices: list[Notice]) -> list[dict]: + """The grouping station: each declared group, then each derived one.""" + sources = [document for document in documents if document.stage == SOURCE] + taken: set[str] = set() + clusters: list[dict] = [] + for document in documents: + if document.stage != GROUPING: + continue + if not document.topics: + notices.append(Notice( + document.path, + "declares stage grouping but carries no topic, so no group " + "forms around it")) + continue + mine = set(document.topics) + members = [document] + [source for source in sources + if mine & set(source.topics)] + members.sort(key=lambda member: member.path) + clusters.append({ + "id": _unique(slug(document.stem), taken), + "name": document.name, + "topics": list(document.topics), + "document_edges": [ + {"document": member.path, + "matched_topics": sorted(mine & set(member.topics))} + for member in members], + }) + carriers: dict[str, list[str]] = {} + for source in sources: + for topic in source.topics: + carriers.setdefault(topic, []).append(source.path) + shared: dict[tuple[str, ...], list[str]] = {} + for topic, paths in carriers.items(): + if len(paths) >= 2: + shared.setdefault(tuple(paths), []).append(topic) + for topics, paths in sorted((tuple(sorted(topics)), paths) + for paths, topics in shared.items()): + name = " · ".join(topics) + clusters.append({ + "id": _unique(slug(name), taken), + "name": name, + "topics": list(topics), + "document_edges": [ + {"document": path, "matched_topics": list(topics)} + for path in sorted(paths)], + }) + return clusters + + +def project(adapter: CorpusAdapter, corpus: ResolvedCorpus, repository: str, + *, source_revision: str | None = None, + generated_at: str | None = None) -> Projection: + """The neutral snapshot of `corpus`, read through `adapter`. + + `source_revision`, where given, is recorded verbatim as the determinism + anchor; otherwise the revision `corpus` was resolved at is. With neither, + the projection refuses, since the contract requires an anchor. + `generated_at`, where given, is recorded verbatim, and it is otherwise + absent. `repository` is recorded as given.""" + revision = source_revision if source_revision is not None else corpus.revision + if revision is None: + raise ProjectionRefused( + f"the neutral snapshot of {repository!r} needs a source revision, " + f"and none was given, and the corpus at {corpus.location!r} " + "resolved to none (a repository with no commit yet). Commit the " + "documents, or pass the revision to anchor them to.") + notices: list[Notice] = [] + documents = _read_documents(adapter, corpus, notices) + _assign_topics(documents) + clusters = _groups(documents, notices) + + possibles: list[dict] = [] + staged_topics: list[dict] = [] + changes: list[dict] = [] + taken: dict[str, set[str]] = {"possibles": set(), "staged_topics": set(), + "changes": set()} + for document in documents: + if document.stage == CANDIDATE: + mine = set(document.topics) + candidate: dict[str, Any] = { + "id": _unique(slug(document.stem), taken["possibles"]), + "title": document.name, + } + if document.summary: + candidate["claim"] = document.summary + candidate["state"] = UNSELECTED + candidate["claiming_clusters"] = [ + cluster["id"] for cluster in clusters + if mine & set(cluster["topics"])] + possibles.append(candidate) + elif document.stage == SELECTION: + staged_topics.append({ + "staging_id": _unique(slug(document.stem), + taken["staged_topics"]), + "files": [document.path], + }) + elif document.stage in CHANGE_STATUS: + changes.append({ + "id": _unique(slug(document.stem), taken["changes"]), + "status": CHANGE_STATUS[document.stage], + "files": [document.path], + }) + + counts = Counter(topic for document in documents + for topic in document.topics) + generation: dict[str, Any] = {"source_revision": revision} + if generated_at is not None: + generation["generated_at"] = generated_at + generation["generator_version"] = GENERATOR_VERSION + snapshot: dict[str, Any] = { + "schema_version": SCHEMA_VERSION, + "kind": generator_seam.NEUTRAL_SNAPSHOT_KIND, + "repository": repository, + "generation": generation, + "documents": [ + {"id": document.path, "path": document.path, + "stage": document.stage, "title": document.title, + "summary": document.summary, "topics": list(document.topics)} + for document in documents], + "clusters": clusters, + "possibles": possibles, + "staged_topics": staged_topics, + "changes": changes, + "keyword_index": [ + {"keyword": keyword, "declared_doc_count": counts[keyword]} + for keyword in sorted(counts)], + } + return Projection(snapshot=snapshot, notices=tuple(notices)) diff --git a/src/opendox/runtime/local_git_adapter.py b/src/opendox/runtime/local_git_adapter.py index 2afdd9f..de37dd3 100644 --- a/src/opendox/runtime/local_git_adapter.py +++ b/src/opendox/runtime/local_git_adapter.py @@ -140,9 +140,11 @@ DEFAULT_BRANCH = "main" #: What this corpus can say about a document's SHAPE, which is all a plain git -#: repository knows. Every kind obliges NO fields: obligations are governance, -#: and a pre-governed repository has none — that is the whole of what makes -#: this the trivial implementation rather than a small governed one. +#: repository knows. By default every kind obliges NO fields: obligations are +#: governance, and a pre-governed repository has none — that is the whole of +#: what makes this the trivial implementation rather than a small governed one. +#: A corpus constructed with `required_fields` obliges those; openDox's own +#: standalone default obliges `NEUTRAL_FIELDS` below, and nothing more. KINDS_BY_SUFFIX: dict[str, str] = { ".md": "text", ".markdown": "text", @@ -160,6 +162,16 @@ #: near its top, so classifying one is never a scan of its whole body. MAX_HEADER_LINES = 64 +#: THE SMALL NEUTRAL FIELD SET (RULED R1Q13 (a), openxFactory#656 comment +#: `5850003126`, in the answer's own example; plan 034 T054): what openDox's own +#: standalone default adapter, `WorkingTreeCorpus`, obliges of a document it +#: recognizes, so `authoring.required_header_fields()` answers it. They are +#: the `title` and `summary` of T053's neutral snapshot, which openDox's own +#: projection copies from the same header (`leading_header`). A document +#: without them is still listed and read, as a source: a missing field is +#: reported, never a reason to refuse or drop the document. +NEUTRAL_FIELDS: tuple[str, ...] = ("title", "summary") + #: The corpus's ONE verdict of its own, and it is a fact about git rather than #: a rule about documents: RULING C3 says "documents are always git-backed", so #: a tracked file whose checkout differs from the resolved commit is a document @@ -202,6 +214,30 @@ def _leading_lines(text: str, limit: int) -> Iterator[str]: start = boundary.end() +def leading_header(text: str) -> dict[str, str]: + """The leading `Name: value` block of `text`: this corpus's one header + convention, as a mapping from each name to its stripped value. + + The block is the run of lines up to the first blank one, and at most + `MAX_HEADER_LINES` lines are looked at. A line with no colon is passed + over, a later line repeating a name wins, and a name given with nothing + after its colon maps to the empty string. So a key that is present with + no value is still told apart from one that is absent. + + PUBLIC because openDox's own neutral projection reads the same block + (`opendox.neutral_projection`, plan 034 T054). The fields it copies are + the ones `classify` below reports as missing, so the two cannot disagree + about what a document gives: both call this function.""" + header: dict[str, str] = {} + for line in _leading_lines(text, MAX_HEADER_LINES): + if not line.strip(): + break + name, separator, value = line.partition(":") + if separator: + header[name.strip()] = value.strip() + return header + + #: Environment variables that SELECT A REPOSITORY or inject configuration, and #: which are therefore stripped from every `git` this package runs. #: @@ -1906,10 +1942,17 @@ def classify(self, corpus: ResolvedCorpus, (RULED openxFactory#656 comment 5714365086, Q-F2 (a) — see `__init__`). WITHOUT `kind_field` — the default, and what this class has always done - — the shape is the file SUFFIX and `required_fields` is empty for every - kind. That is the honest answer for a plain local git repository rather - than an unfinished one: obligations are governance, and a pre-governed - repository has none. + — the shape is the file SUFFIX, and with the default `required_fields` + of `()` no kind obliges any field. That is the honest answer for a + plain local git repository rather than an unfinished one: obligations + are governance, and a pre-governed repository has none. A corpus built + WITH `required_fields` obliges those of every kind it recognizes, and + reports as missing each one the document's header does not give, or + gives with nothing after its colon (an empty value is no field, as + `authoring.missing_required_headers` has it). openDox's own standalone + default, `WorkingTreeCorpus`, is built with `NEUTRAL_FIELDS` (plan 034 + T054, R1Q13 (a)). A missing field is reported and nothing more: the + document is still listed, still readable, and still classified. WITH `kind_field` the corpus declares its own shape in a document HEADER, which is how the neutral conformance corpus is written @@ -1923,7 +1966,6 @@ def classify(self, corpus: ResolvedCorpus, """ if self._kind_field is not None: return self._classify_by_header(corpus, document) - del corpus suffix = Path(document.key).suffix.lower() kind = KINDS_BY_SUFFIX.get(suffix) if kind is None: @@ -1933,8 +1975,16 @@ def classify(self, corpus: ResolvedCorpus, f"{document.key!r} has no shape this corpus recognizes " f"(suffix {suffix or '(none)'!r}); it is still listed and " "still readable")) - return Classification(id=document, kind=kind, required_fields=(), - missing_fields=()) + if not self._required_fields: + # NOTHING OBLIGED, NOTHING READ: the bare default answers from the + # suffix alone, exactly as it always has. + return Classification(id=document, kind=kind, required_fields=(), + missing_fields=()) + header = self._header_of(corpus, document) + return Classification( + id=document, kind=kind, required_fields=self._required_fields, + missing_fields=tuple(field for field in self._required_fields + if not header.get(field))) def _classify_by_header(self, corpus: ResolvedCorpus, document: DocumentId) -> Classification: @@ -1975,14 +2025,7 @@ def _header_of(self, corpus: ResolvedCorpus, yields the same lines and stops. """ text = self.read(corpus, document).content.decode("utf-8", "replace") - header: dict[str, str] = {} - for line in _leading_lines(text, MAX_HEADER_LINES): - if not line.strip(): - break - name, separator, value = line.partition(":") - if separator: - header[name.strip()] = value.strip() - return header + return leading_header(text) # -- check ------------------------------------------------------------ @@ -3010,7 +3053,24 @@ class WorkingTreeCorpus(LocalGitCorpus): visible on the very next one. The factory that registers this default (`cli.py`'s and `serve.py`'s `_default_home_factory`) also constructs a fresh instance on every call, so no state survives across registrations - either (plan 034, T022).""" + either (plan 034, T022). + + THE SMALL NEUTRAL FIELD SET IS THIS CLASS'S DEFAULT (plan 034 T054; RULED + R1Q13 (a), openxFactory#656 comment `5850003126`). `required_fields` + defaults to `NEUTRAL_FIELDS`, `title` and `summary`, where + `LocalGitCorpus` defaults to `()`. So the factory above, which builds + `WorkingTreeCorpus()`, hands every standalone caller an adapter whose + `classify` obliges them, and `authoring.required_header_fields()` answers + them. Nothing else about the class changes: a document without them is + still listed and read, and `classify` reports what is missing.""" + + def __init__(self, *, executable: str = "git", + write_path: str | None = WRITE_PATH, + kind_field: str | None = None, + required_fields: tuple[str, ...] = NEUTRAL_FIELDS) -> None: + super().__init__(executable=executable, write_path=write_path, + kind_field=kind_field, + required_fields=required_fields) def _list_documents_bound(self, git: GitRunner, corpus: ResolvedCorpus, scope: str) -> tuple[DocumentId, ...]: diff --git a/src/opendox/web/views/display.js b/src/opendox/web/views/display.js index 309ab24..7b4576f 100644 --- a/src/opendox/web/views/display.js +++ b/src/opendox/web/views/display.js @@ -220,29 +220,29 @@ export const SESSION_BRANCH_NAMESPACES = { [SCOPE_KINDS.candidate]: "possible", }; -// REGISTER_STATES — the candidate register's own four-state enum, by status -// role. The fifth SEAM KEY table on § 2.2 rule 3's footing: these are VALUES -// the snapshot carries on a candidate (`possibles[].state`), written by the -// generator and compared against here, never rendered — what a human reads is -// `display.status(VOCABULARY.CANDIDATE, role)`. openxFactory's own profile -// declares the same four against the same roles -// (`contracts/domain-profiles/openxfactory-engineering.yaml`:294-297), which is -// what makes the role the right key and the word the wrong one. -// THE SNAPSHOT'S CLOSED ENUM VALUES, by role — the rest of § 2.2 rule 3's -// schema half, mirrored from `display_profile.SNAPSHOT_VALUES` and carried on -// the payload (`values`) so a host may override them. -// -// `documents[].stage` and `possibles[].state` are values the renderer MATCHES — +// THE SNAPSHOT'S CLOSED ENUM VALUES, by role — the fifth SEAM KEY table on +// § 2.2 rule 3's footing and the rest of its schema half, mirrored from +// `display_profile.SNAPSHOT_VALUES` and carried on the payload (`values`) so a +// host may override them. `documents[].stage` and `possibles[].state`, the +// candidate register's four-state enum, are values the renderer MATCHES — // which column a card belongs in, which dot a candidate wears — and never -// values it RENDERS: every word a human reads comes from `display.status(...)`. -// That is § 2.2 rule 3 read over a closed enum rather than over a field name, -// and it is what keeps the product working: every profile in the estate today -// registers WITHOUT a `DISPLAY` facet, so a board that filtered on the facet's -// neutral word would show an empty column against the very snapshot it renders. +// values it RENDERS: every word a human reads comes from `display.status(...)`, +// by role. That is § 2.2 rule 3 read over a closed enum rather than over a +// field name, and it is what keeps the product working: a board that filtered +// on a facet's word would show an empty column against the snapshot it renders. +// +// The defaults are openDox's own neutral snapshot's values (R1Q11 (a); plan +// 034 T054), exactly as `display_profile.SNAPSHOT_VALUES` declares them: a +// source document's stage is `source` and a gathering one's is `grouping`, and +// a candidate is `unselected`, `selected`, `declined` or `replaced`, which are +// the candidate words NEUTRAL_DISPLAY renders, role for role. A host whose +// generator writes other values into the same schema declares them on its +// facet's `values` block, as openXdox's facet does for the governed snapshot +// (T060). export const SNAPSHOT_VALUES = { - document_stage: { captured: "brainstorm", organized: "staged" }, - register_state: { captured: "latent", proposed: "picked", - retired: "rejected", superseded: "superseded" }, + document_stage: { captured: "source", organized: "grouping" }, + register_state: { captured: "unselected", proposed: "selected", + retired: "declined", superseded: "replaced" }, }; // DRILL_KINDS — the explorer's own tile-kind keys, by stage role. The sixth and diff --git a/tests/test_generator_seam.py b/tests/test_generator_seam.py index 9f9ca4e..40face7 100644 --- a/tests/test_generator_seam.py +++ b/tests/test_generator_seam.py @@ -36,7 +36,8 @@ replaces it before a generation, and after one that wrote nothing. A host is refused while a generation runs, and after one that wrote a snapshot. A generator may register or generate from inside its own call without - deadlocking the seam. Until T054 lands, the default refuses, naming itself. + deadlocking the seam. What the default generates is T054's neutral + projection, and `tests/test_neutral_projection.py` holds it. 7. EACH ENTRY POINT REGISTERS IT. `cli.build_parser()` and `cli.main()` run for real. `serve.build_server()` and `serve.main()` still cannot run in a lone checkout (research R7), so their registration is executed from their own @@ -730,27 +731,6 @@ def test_a_host_that_registers_the_default_itself_holds_a_hosts_registration() - assert "a host's generator is already registered" in str(caught.value) -def test_openDoxs_own_generator_refuses_until_its_projection_lands(tmp_path) -> None: - """Plan 034 orders the seam (T052) before the projection (T054). So the - default refuses, naming itself and T054, and generates nothing. It never - answers an empty snapshot. Because it wrote nothing, its refusal shuts no - host out: a host's generator still replaces it afterwards. T054 replaces - this case with its own tests.""" - with pytest.raises(default_generator.NeutralProjectionNotBuilt) as caught: - default_generator.generate(tmp_path, "fixture") - message = str(caught.value) - for expected in (gs.NEUTRAL_SNAPSHOT_KIND, "T054", gs.REGISTRATION_CALL): - assert expected in message, f"the refusal no longer says {expected!r}" - assert isinstance(caught.value, gs.GeneratorSeamError) - gs.register_default(default_generator.GENERATOR) - with pytest.raises(default_generator.NeutralProjectionNotBuilt): - gs.generate(tmp_path, "fixture") - host, _ = _declared("host-snapshot") - assert gs.register(host) is host, ( - "a refused generation from openDox's own generator wrote nothing, so " - "a host still replaces it") - - # -------------------------------------------------------------------------- # 7 — each entry point registers openDox's own generator # -------------------------------------------------------------------------- From 0f42f678ec2de9bacf3eec5fbf385e826a29d76a Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sun, 27 Sep 2026 22:17:32 +0000 Subject: [PATCH 06/38] T054: the projection's tests, the holder's two rulings, and the display-facet cases (plan 034) This finishes the work 25fe7526 checkpointed. The holder ruled on two of T054's open questions, relayed by the coordinator: - An entry the adapter cannot classify (a Makefile, an image) is not a document. The projection now leaves it out unread, and nothing mentions it. - The wheel's grouping tile counts a group's edges, and the neutral schema is not widened with a tally. views/wheel-model.js now counts document_edges where a group carries no tallies.document_links, and uses the tally where one exists, as the governed snapshot's groups do. The file keeps its 1358 lines, so its census row is unchanged. tests/test_neutral_projection.py (new) holds T054's falsifier: - in a fresh process with every sibling blocked, the CLI entry point's defaults generate over T050's fixture through the seam, and the result validates against T053's schema with no F5.3 word; - the topic rule groups a copy of AT-R1's repository (b), which has no front matter; - a stage: value outside the six is reported, naming the document, the value and the six keys, and is read as a source. Around them it tests: - the holder's T051 rule (the malformed fixture breaks only its EXPECTED_RULE); - what a document is, the stations, and the anchors; - the default adapter's field set through the entry point; - the neutral display values in Python and in display.js, and the wheel's edge count; - that the projection makes no reach. The schema is a byte-identical copy of openDox-spec#16's at cd49eb25, held to its sha256, until T057 ships the packaged one. tests/test_display_facet.py: four node-harness cases carried the governed snapshot values in their fixtures. Each now reads the value openDox ships (SNAPSHOT_VALUES), and the property each one tests is unchanged. The canvas case's neutral assertion no longer compares against a value the neutral install can never write. It now asserts openDox's own word. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/neutral_projection.py | 33 +- src/opendox/web/views/wheel-model.js | 10 +- tests/fixtures/opendox-snapshot.schema.yaml | 443 ++++++++++ tests/test_display_facet.py | 50 +- tests/test_neutral_projection.py | 908 ++++++++++++++++++++ 5 files changed, 1403 insertions(+), 41 deletions(-) create mode 100644 tests/fixtures/opendox-snapshot.schema.yaml create mode 100644 tests/test_neutral_projection.py diff --git a/src/opendox/neutral_projection.py b/src/opendox/neutral_projection.py index edb75a8..b4e5cc1 100644 --- a/src/opendox/neutral_projection.py +++ b/src/opendox/neutral_projection.py @@ -17,6 +17,12 @@ .SNAPSHOT_VALUES` defaults to those same values, so openDox's views place every card by them and render the six words. +WHAT IS A DOCUMENT. An entry the adapter lists AND classifies. An entry it +cannot classify (a `Makefile`, an image) is not a document: it is left out +unread, and nothing mentions it (the holder's ruling on T054). So is an entry +whose path the neutral schema cannot carry (`path-is-repo-relative`), and the +projection reports that one, since the adapter did classify it. + WHERE A DOCUMENT LANDS (RULED R1Q13 (a) with (c), same comment). A document names its station with the neutral `stage:` key of its leading `Name: value` header, which is `local_git_adapter.leading_header`'s block. That is the same @@ -37,9 +43,7 @@ A word is a run of three or more letters, case-folded, that is not one of `STOPWORDS`. So the rule needs no front matter at all. AT-R1's repository (b) (quickstart.md § 2) is three notes named by their headings, and a note that -mentions another shares that note's name as a topic. An entry the adapter -cannot classify is listed but never read, so it carries no topic and nothing -mentions it. +mentions another shares that note's name as a topic. GROUPS (the grouping station). A group forms wherever two or more sources share a topic. Topics that the same sources share form one group, named by @@ -194,7 +198,6 @@ class _Document: title: str | None = None summary: str | None = None topics: tuple[str, ...] = () - read: bool = False name: str = "" tokens: tuple[str, ...] = () declared_topics: tuple[str, ...] | None = None @@ -281,12 +284,14 @@ def _unique(base: str, taken: set[str]) -> str: def _read_documents(adapter: CorpusAdapter, corpus: ResolvedCorpus, notices: list[Notice]) -> list[_Document]: - """Every listed document the snapshot can carry, in path order, with its - station, its two copied fields, its name and its tokens.""" + """Every document the snapshot can carry, in path order, with its station, + its two copied fields, its name and its tokens.""" stations = set(STAGE_ROLES) documents: list[_Document] = [] listed = sorted(adapter.list_documents(corpus), key=lambda d: d.key) for identity in listed: + if adapter.classify(corpus, identity).kind is None: + continue # not a document (this module's docstring) reason = _unwritable(identity.key) if reason is not None: # NAMED BY ITS repr: the path is one no snapshot can carry, and a @@ -299,13 +304,8 @@ def _read_documents(adapter: CorpusAdapter, corpus: ResolvedCorpus, continue document = _Document(path=identity.key) documents.append(document) - if adapter.classify(corpus, identity).kind is None: - # UNCLASSIFIABLE: listed, never read. It is a source with nothing - # to copy, no topic, and no name another document can mention. - continue text = adapter.read(corpus, identity).content.decode("utf-8", "replace") header = leading_header(text) - document.read = True document.title = header.get(TITLE_KEY) document.summary = header.get(SUMMARY_KEY) declared = header.get(STAGE_KEY) @@ -327,14 +327,13 @@ def _read_documents(adapter: CorpusAdapter, corpus: ResolvedCorpus, def _assign_topics(documents: list[_Document]) -> None: - """The topic rule (this module's docstring), over the documents read.""" - read = [document for document in documents if document.read] - own = [_words(document.name) for document in read] + """The topic rule (this module's docstring), over every document.""" + own = [_words(document.name) for document in documents] # Each name is looked for by its first WORD, at that word's offset, so a # text is scanned once and a common short token ("the") costs nothing. by_first_word: dict[str, list[tuple[tuple[str, ...], int, int]]] = {} - for index, document in enumerate(read): - for alias in {_tokens(document.name), _tokens(document.stem)}: + for index, document in enumerate(documents): + for alias in sorted({_tokens(document.name), _tokens(document.stem)}): # A name with no word in it contributes no topic, so it is not # looked for: `a.md` would otherwise be mentioned by every "a". offset = next((at for at, token in enumerate(alias) @@ -342,7 +341,7 @@ def _assign_topics(documents: list[_Document]) -> None: if offset is not None: by_first_word.setdefault(alias[offset], []).append( (alias, offset, index)) - for index, document in enumerate(read): + for index, document in enumerate(documents): if document.declared_topics is not None: document.topics = document.declared_topics continue diff --git a/src/opendox/web/views/wheel-model.js b/src/opendox/web/views/wheel-model.js index 0a40314..99a68c5 100644 --- a/src/opendox/web/views/wheel-model.js +++ b/src/opendox/web/views/wheel-model.js @@ -47,6 +47,9 @@ const CANDIDATE_ROLE = STAGE_ROLES[2]; const cap = (s) => (s ? s.charAt(0).toUpperCase() + s.slice(1) : s); const MAX_DEMO_POSSIBLES = 6; +// A group's document count: its tally where the snapshot carries one, and its +// edges otherwise (openDox's neutral snapshot, whose groups carry no tallies). +const docLinks = (c) => c?.tallies?.document_links ?? (c?.document_edges || []).length; function basename(path) { return String(path).split("/").at(-1) || String(path); @@ -68,9 +71,7 @@ export function isUndisposedDerived(possible) { export function synthesizeDemoPossibles(clusters, display) { const d = display || neutralDisplay(); const ranked = [...(clusters || [])].sort((a, b) => { - const ta = a.tallies?.document_links || 0; - const tb = b.tallies?.document_links || 0; - return tb - ta || String(a.id).localeCompare(String(b.id)); + return docLinks(b) - docLinks(a) || String(a.id).localeCompare(String(b.id)); }); return ranked.slice(0, MAX_DEMO_POSSIBLES).map((c) => ({ id: "demo-" + c.id, @@ -117,8 +118,7 @@ function buildItems({ display, sources, groups, candidates, selections, })), [GROUPING]: groups.map((c) => ({ id: c.id, label: c.name || c.id, - sub: (c.tallies?.document_links || 0) + " " - + display.count(SOURCE, c.tallies?.document_links || 0), ref: c, + sub: docLinks(c) + " " + display.count(SOURCE, docLinks(c)), ref: c, })), [CANDIDATE]: candidates.map((p) => ({ id: p.id, label: p.title || p.id, diff --git a/tests/fixtures/opendox-snapshot.schema.yaml b/tests/fixtures/opendox-snapshot.schema.yaml new file mode 100644 index 0000000..2be5018 --- /dev/null +++ b/tests/fixtures/opendox-snapshot.schema.yaml @@ -0,0 +1,443 @@ +# openDox's own neutral snapshot contract (plan 034, T053). +# +# WHY THIS FILE IS JSON. It is YAML whose body is one JSON object, and that is +# deliberate. The leg's required `validate` check installs pytest and nothing +# else, so `tests/test_opendox_snapshot_contract.py` reads this file with +# Python's built-in `json` module once these comment lines are set aside. +# JSON is YAML, so every YAML loader in the family reads the same object, and +# the same test proves that PyYAML agrees wherever PyYAML is installed. This +# leg's negative chat-turn examples already take this form. +# +# SECTIONS. openDox's views render six stations from five sections, as +# openDox's own STAGE_FIELDS declares them: source reads documents, grouping +# reads clusters, candidate reads possibles, selection reads staged_topics, +# and the submission and completion stations read the changes entries whose +# status is active and archived. Every station section is required, and it +# is an empty list when its station holds nothing. keyword_index is optional: +# without it, a reader derives the keyword rail from the documents' topics. +# +# CLOSED VALUES, all neutral. A document's stage is one of the six station +# role keys, a candidate's state is one of unselected, selected, declined and +# replaced, and a changes entry's status is active or archived. openDox's +# views match them through SNAPSHOT_VALUES, whose defaults T054 moves to +# these values. +# +# DETERMINISTIC, which is the generator's to keep and no schema can check: the +# same tree yields a byte-identical snapshot, so nothing in it records when +# the generator ran. generated_at, when present, is fixed by the source +# revision (its commit date, or a date recorded with it), never read from the +# clock. FORWARD-COMPATIBLE. A reader ignores unknown properties, so no object +# sets additionalProperties false, and an additive field needs no +# schema_version bump. +# +# RULES. Every rule has an identifier. A subschema names, in x-rule, the rule +# its keywords enforce; x-rules lists every rule with its class; and a +# validator's refusal names the rule it broke. A shape rule is enforced by +# this schema's own keywords. A reference rule is a cross-reference that no +# JSON Schema keyword can state, so a validator enforces it. +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "opendox-snapshot.schema.yaml", + "title": "openDox neutral snapshot", + "contract_schema_version": 1, + "description": "openDox's own snapshot: what its neutral generator writes over a plain repository, and what its views read, with no consumer installed (plan 034 T053, on R1Q11 (a) and R1Q12 (a): opensoft/openxFactory issue 656, comment 5850003126). openXdox's governed generator keeps its own contract, openXdox-spec's ideation-dashboard-snapshot, which this schema leaves unchanged; a contributed generator declares which of the two kinds it writes (T052). The file's comment header, the section descriptions and the x-rules catalog say the rest.", + "x-rule": "envelope-keys", + "type": "object", + "required": [ + "schema_version", + "kind", + "repository", + "generation", + "documents", + "clusters", + "possibles", + "staged_topics", + "changes" + ], + "properties": { + "schema_version": {"x-rule": "schema-version-is-1", "const": 1}, + "kind": {"x-rule": "kind-is-opendox-snapshot", "const": "opendox-snapshot"}, + "repository": {"x-rule": "repository-is-text", "type": "string", "minLength": 1}, + "generation": {"$ref": "#/$defs/generation"}, + "documents": { + "description": "The source station, and the product's whole document list: every document the corpus adapter lists, in the station its stage names. Every entry carries its stage, and no reader supplies one: the generator writes source for a document that declares no stage.", + "x-rule": "section-is-a-list", + "type": "array", + "items": {"$ref": "#/$defs/document"} + }, + "clusters": { + "description": "The grouping station: the groups that form around topics documents share, each with one edge per member document.", + "x-rule": "section-is-a-list", + "type": "array", + "items": {"$ref": "#/$defs/group"} + }, + "possibles": { + "description": "The candidate station.", + "x-rule": "section-is-a-list", + "type": "array", + "items": {"$ref": "#/$defs/candidate"} + }, + "staged_topics": { + "description": "The selection station.", + "x-rule": "section-is-a-list", + "type": "array", + "items": {"$ref": "#/$defs/selection"} + }, + "changes": { + "description": "The submission station (status active) and the completion station (status archived), which share this one section.", + "x-rule": "section-is-a-list", + "type": "array", + "items": {"$ref": "#/$defs/submission"} + }, + "keyword_index": { + "description": "Optional. The keyword rail's seed. When present, every topic the documents carry has one entry, and each entry counts the documents that carry its keyword.", + "x-rule": "section-is-a-list", + "type": "array", + "items": {"$ref": "#/$defs/keyword_entry"} + } + }, + "$defs": { + "id": {"x-rule": "id-is-text", "type": "string", "minLength": 1}, + "path": { + "description": "A path relative to the repository root, as the corpus adapter lists it.", + "x-rule": "path-is-repo-relative", + "type": "string", + "pattern": "^(?!/)(?![A-Za-z]:)(?![\\s\\S]*[\\\\\\u0000-\\u001f\\u007f-\\u009f])(?!(?:[\\s\\S]*/)?\\.\\.(?:/|$))[\\s\\S]+$" + }, + "topic": { + "x-rule": "topic-is-trimmed-text", + "type": "string", + "pattern": "^(?![\\s\\ufeff])(?![\\s\\S]*[\\s\\ufeff]$)[^\\u0000-\\u001f\\u007f-\\u009f]+$" + }, + "stage_role": { + "description": "The station a document sits in: one of the six station role keys, in spine order, exactly openDox's display_profile.STAGE_ROLES. The set is closed. A declared value outside it is not a declaration: the generator reads that document as a source and writes stage source, so the value never reaches a snapshot.", + "x-rule": "stage-is-a-station-role", + "enum": ["source", "grouping", "candidate", "selection", "submission", "completion"] + }, + "candidate_state": { + "description": "A candidate's state, in openDox's own words for the candidate station (NEUTRAL_DISPLAY's candidate vocabulary). A candidate is unselected until an act selects, declines or replaces it.", + "x-rule": "candidate-state-is-known", + "enum": ["unselected", "selected", "declined", "replaced"] + }, + "submission_status": { + "description": "The station a changes entry sits in: active for submission, archived for completion, as openDox's STAGE_FIELDS declares them.", + "x-rule": "submission-status-is-known", + "enum": ["active", "archived"] + }, + "generation": { + "description": "The generation stamp. source_revision is the determinism anchor: the tree revision the snapshot projects.", + "x-rule": "generation-anchored", + "type": "object", + "required": ["source_revision"], + "properties": { + "source_revision": {"x-rule": "generation-anchored", "type": "string", "minLength": 1}, + "generated_at": { + "x-rule": "generated-at-is-rfc3339", + "type": "string", + "format": "date-time", + "pattern": "^(?![\\s\\S]*[\\u0000-\\u001f\\u007f-\\u009f])(?!0000)(?:[0-9]{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12][0-9]|3[01])|(?:0[469]|11)-(?:0[1-9]|[12][0-9]|30)|02-(?:0[1-9]|1[0-9]|2[0-8]))|(?:[0-9]{2}(?:0[48]|[2468][048]|[13579][26])|(?:[02468][048]|[13579][26])00)-02-29)[Tt](?:[01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9](?:\\.[0-9]+)?(?:[Zz]|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])$" + }, + "generator_version": {"x-rule": "generation-anchored", "type": "string", "minLength": 1} + } + }, + "document": { + "description": "One document the corpus adapter lists. title and summary are the default adapter's small neutral field set (R1Q13 (a)). Neither is required, and the generator writes null for one the document does not give. topics, which every entry carries, are the topics the generator assigns: the ones the document declares, or the ones its topic rule derives when it declares none, and an empty list when there are none.", + "x-rule": "document-keys", + "type": "object", + "required": ["id", "path", "stage", "topics"], + "properties": { + "id": {"$ref": "#/$defs/id"}, + "path": {"$ref": "#/$defs/path"}, + "stage": {"$ref": "#/$defs/stage_role"}, + "title": { + "x-rule": "title-and-summary-are-text", + "type": ["string", "null"], + "minLength": 1 + }, + "summary": { + "x-rule": "title-and-summary-are-text", + "type": ["string", "null"], + "minLength": 1 + }, + "topics": { + "x-rule": "topics-are-unique", + "type": "array", + "uniqueItems": true, + "items": {"$ref": "#/$defs/topic"} + } + } + }, + "group": { + "description": "One group in the grouping station. document_edges holds one edge per member document, naming the topics that matched; the funnel draws them.", + "x-rule": "group-keys", + "type": "object", + "required": ["id", "name", "topics", "document_edges"], + "properties": { + "id": {"$ref": "#/$defs/id"}, + "name": {"x-rule": "group-keys", "type": "string", "minLength": 1}, + "topics": { + "x-rule": "group-has-a-topic", + "type": "array", + "minItems": 1, + "uniqueItems": true, + "items": {"$ref": "#/$defs/topic"} + }, + "document_edges": { + "x-rule": "group-keys", + "type": "array", + "items": {"$ref": "#/$defs/edge"} + } + } + }, + "edge": { + "x-rule": "edge-keys", + "type": "object", + "required": ["document", "matched_topics"], + "properties": { + "document": {"$ref": "#/$defs/id"}, + "matched_topics": { + "x-rule": "edge-keys", + "type": "array", + "minItems": 1, + "uniqueItems": true, + "items": {"$ref": "#/$defs/topic"} + } + } + }, + "candidate": { + "description": "One candidate in the candidate station. claiming_clusters holds the groups that claim it; pick holds the selection a selected candidate went to.", + "x-rule": "candidate-keys", + "type": "object", + "required": ["id", "title", "state"], + "properties": { + "id": {"$ref": "#/$defs/id"}, + "title": {"x-rule": "candidate-keys", "type": "string", "minLength": 1}, + "claim": {"x-rule": "candidate-keys", "type": "string", "minLength": 1}, + "state": {"$ref": "#/$defs/candidate_state"}, + "claiming_clusters": { + "x-rule": "candidate-keys", + "type": "array", + "uniqueItems": true, + "items": {"$ref": "#/$defs/id"} + }, + "pick": { + "x-rule": "selected-candidate-has-pick", + "type": "object", + "required": ["staging_id"], + "properties": {"staging_id": {"$ref": "#/$defs/id"}} + }, + "reason": {"x-rule": "closed-candidate-has-reason", "type": "string", "minLength": 1} + }, + "allOf": [ + { + "if": {"required": ["state"], "properties": {"state": {"const": "selected"}}}, + "then": {"x-rule": "selected-candidate-has-pick", "required": ["pick"]} + }, + { + "if": {"required": ["state"], "properties": {"state": {"enum": ["declined", "replaced"]}}}, + "then": {"x-rule": "closed-candidate-has-reason", "required": ["reason"]} + } + ] + }, + "selection": { + "description": "One selection in the selection station. files lists what it is made of, and target_change names the changes entry it went on to.", + "x-rule": "selection-keys", + "type": "object", + "required": ["staging_id"], + "properties": { + "staging_id": {"$ref": "#/$defs/id"}, + "files": { + "x-rule": "selection-keys", + "type": "array", + "uniqueItems": true, + "items": {"$ref": "#/$defs/path"} + }, + "target_change": {"$ref": "#/$defs/id"} + } + }, + "submission": { + "description": "One entry of the submission or the completion station. files lists what it is made of, as a selection's files do, and the tile opens onto them.", + "x-rule": "submission-keys", + "type": "object", + "required": ["id", "status"], + "properties": { + "id": {"$ref": "#/$defs/id"}, + "status": {"$ref": "#/$defs/submission_status"}, + "files": { + "x-rule": "submission-keys", + "type": "array", + "uniqueItems": true, + "items": {"$ref": "#/$defs/path"} + } + } + }, + "keyword_entry": { + "description": "declared_doc_count counts the documents whose topics carry the keyword.", + "x-rule": "keyword-entry-keys", + "type": "object", + "required": ["keyword", "declared_doc_count"], + "properties": { + "keyword": {"$ref": "#/$defs/topic"}, + "declared_doc_count": {"x-rule": "keyword-entry-keys", "type": "integer", "minimum": 0} + } + } + }, + "x-rules": [ + { + "id": "envelope-keys", + "class": "shape", + "says": "A snapshot is an object carrying schema_version, kind, repository, generation and the five station sections: documents, clusters, possibles, staged_topics and changes." + }, + {"id": "schema-version-is-1", "class": "shape", "says": "schema_version is 1."}, + { + "id": "kind-is-opendox-snapshot", + "class": "shape", + "says": "kind is opendox-snapshot. The governed generator's ideation-dashboard-snapshot is a different contract, and this schema refuses it." + }, + { + "id": "repository-is-text", + "class": "shape", + "says": "repository, the canonical id of the repository the snapshot projects, is non-empty text." + }, + { + "id": "generation-anchored", + "class": "shape", + "says": "generation is an object carrying source_revision, the revision of the tree the snapshot projects, as non-empty text; generator_version, when present, is non-empty text too." + }, + { + "id": "generated-at-is-rfc3339", + "class": "shape", + "says": "generation.generated_at, when present, is an RFC 3339 date-time on a day the calendar has (the pattern knows each month's length and the leap years), with no control character. It is narrower than RFC 3339 in two places. Its year is never 0000, which Python's datetime cannot hold. Its seconds run from 00 to 59 and are never a leap second's 60, which a git commit date cannot hold and neither Python's datetime nor a browser's Date can read. jsonschema's date-time checker refuses both. It is fixed by the source revision, never read from the clock." + }, + { + "id": "section-is-a-list", + "class": "shape", + "says": "Each section is a list: documents, clusters, possibles, staged_topics, changes and, when present, keyword_index. A station with nothing in it is an empty list." + }, + { + "id": "id-is-text", + "class": "shape", + "says": "Every entry id, and every reference to one, is non-empty text." + }, + { + "id": "document-keys", + "class": "shape", + "says": "A document is an object carrying id, path, stage and topics." + }, + { + "id": "path-is-repo-relative", + "class": "shape", + "says": "A path is relative to the repository root: it does not start with a slash or with a drive letter and a colon (C:/x or C:x, which Windows joins onto a root as a path outside it), holds no backslash and no control character (U+0000 to U+001F, U+007F to U+009F), and has no .. segment." + }, + { + "id": "stage-is-a-station-role", + "class": "shape", + "says": "A document's stage is one of the six station role keys: source, grouping, candidate, selection, submission, completion." + }, + { + "id": "title-and-summary-are-text", + "class": "shape", + "says": "A document's title and summary are each non-empty text, or null." + }, + { + "id": "topics-are-unique", + "class": "shape", + "says": "A document's topics are a list that names each topic once." + }, + { + "id": "topic-is-trimmed-text", + "class": "shape", + "says": "A topic is non-empty text with no leading or trailing whitespace and no control character (U+0000 to U+001F, U+007F to U+009F). Whitespace is what both Python and a browser count as whitespace, U+FEFF included, so both refuse the same topics." + }, + { + "id": "group-keys", + "class": "shape", + "says": "A group (a clusters entry) is an object carrying id, name, topics and document_edges. Its name is non-empty text, and its edges are a list." + }, + { + "id": "group-has-a-topic", + "class": "shape", + "says": "A group's topics name at least one topic, each once: a group forms around topics that its documents share." + }, + { + "id": "edge-keys", + "class": "shape", + "says": "A group's document edge is an object that names the document and at least one matched topic, each once." + }, + { + "id": "candidate-keys", + "class": "shape", + "says": "A candidate (a possibles entry) is an object carrying id, title and state. Its title, and its claim when present, are non-empty text; claiming_clusters, when present, names each group once." + }, + { + "id": "candidate-state-is-known", + "class": "shape", + "says": "A candidate's state is one of unselected, selected, declined and replaced." + }, + { + "id": "selected-candidate-has-pick", + "class": "shape", + "says": "A selected candidate carries pick, an object whose staging_id names the selection it went to." + }, + { + "id": "closed-candidate-has-reason", + "class": "shape", + "says": "A declined or replaced candidate carries reason, non-empty text that says why." + }, + { + "id": "selection-keys", + "class": "shape", + "says": "A selection (a staged_topics entry) is an object carrying staging_id; files, when present, lists repository-relative paths, each once." + }, + { + "id": "submission-keys", + "class": "shape", + "says": "A changes entry, a submission or a completed item, is an object carrying id and status; files, when present, lists repository-relative paths, each once." + }, + { + "id": "submission-status-is-known", + "class": "shape", + "says": "A changes entry's status is active (the submission station) or archived (the completion station)." + }, + { + "id": "keyword-entry-keys", + "class": "shape", + "says": "A keyword_index entry is an object carrying keyword and declared_doc_count, a whole number no less than 0." + }, + { + "id": "ids-are-unique", + "class": "reference", + "says": "Within each section, entry ids are unique: documents, clusters, possibles and changes by id, and staged_topics by staging_id." + }, + { + "id": "edge-names-a-document", + "class": "reference", + "says": "Every group edge names a document in documents." + }, + { + "id": "one-edge-per-document", + "class": "reference", + "says": "Within one group, the edges name each document once: one edge per member document. A document may feed several groups." + }, + { + "id": "candidate-names-a-group", + "class": "reference", + "says": "Every group that a candidate's claiming_clusters names is in clusters." + }, + { + "id": "pick-names-a-selection", + "class": "reference", + "says": "A candidate's pick.staging_id names a selection in staged_topics." + }, + { + "id": "target-names-a-submission", + "class": "reference", + "says": "A selection's target_change names an entry in changes." + }, + { + "id": "keyword-index-matches-topics", + "class": "reference", + "says": "keyword_index, when present, agrees with the documents: every topic a document carries has an entry, no keyword has two, and each entry's declared_doc_count is the number of documents that carry its keyword, 0 for a keyword that none carries." + } + ] +} diff --git a/tests/test_display_facet.py b/tests/test_display_facet.py index d376ffd..8270642 100644 --- a/tests/test_display_facet.py +++ b/tests/test_display_facet.py @@ -298,8 +298,9 @@ def test_two_roles_may_not_share_one_snapshot_enum_value(): display_manifest({"values": {"register_state": { "captured": "same", "proposed": "same"}}}) # the partial-override case: one declared value colliding with a shipped one + shipped = SNAPSHOT_VALUES["register_state"]["captured"] with pytest.raises(DisplayFacetError, match="BOTH"): - display_manifest({"values": {"register_state": {"proposed": "latent"}}}) + display_manifest({"values": {"register_state": {"proposed": shipped}}}) # …and distinct values are taken whole ok = display_manifest({"values": {"register_state": { "captured": "new", "proposed": "chosen"}}}) @@ -653,7 +654,8 @@ def test_the_board_really_emits_a_stripe_class_the_stylesheet_matches(tmp_path): const {{ renderBoard }} = await import(base + "board.js"); const snap = {{ documents: [{{ id: "a.md", path: "ideation/brainstorm/a.md", kind: "document", - stage: "brainstorm", summary: "a note", dates: {{ captured: "2026-01-01" }} }}], + stage: D.SNAPSHOT_VALUES.document_stage.captured, summary: "a note", + dates: {{ captured: "2026-01-01" }} }}], clusters: [], possibles: [], staged_topics: [{{ staging_id: "topic-x", files: ["a.md"] }}], changes: [{{ id: "ch-1", status: "active" }}, @@ -688,8 +690,8 @@ def test_the_board_really_emits_a_stripe_class_the_stylesheet_matches(tmp_path): f"declares no `.card.{emitted}` rule — the card renders with no " "stripe, which no assertion over its words would notice") # THE CLASS DOES NOT MOVE WITH THE HOST'S WORDS. `declared` renames the - # source station and keeps openxFactory's `brainstorm` enum; the classes are - # identical because they were never that word. + # source station and keeps openDox's shipped enum values; the classes are + # identical because they were never a word. assert sorted(out["declared"]) == sorted(out["neutral"]) @@ -1075,24 +1077,27 @@ def test_the_explorer_file_rows_show_the_declared_stage_word(tmp_path): const base = {views} + "/"; const D = await import(base + "display.js"); const {{ mountExplorer }} = await import(base + "explorer.js"); -const snap = {{ +// each facet is shown a document at ITS OWN captured enum value: the host's +// declared `brainstorm`, and the value openDox ships +const snapAt = (stage) => ({{ staged_topics: [{{ staging_id: "topic-x", files: ["a.md"] }}], - documents: [{{ path: "a.md", stage: "brainstorm", kind: "note" }}], + documents: [{{ path: "a.md", stage, kind: "note" }}], changes: [], -}}; +}}); const declared = D.readDisplay({{ display: {{ schema_version: 1, kind: "opendox.display-facet", host_facet: "declared", values: {{ document_stage: {{ captured: "brainstorm" }} }}, statuses: {{ document: {{ captured: "jotted" }} }}, }} }}); -function rowTexts(display) {{ +function rowTexts(display, stage) {{ const host = new Node("div"); - const explorer = mountExplorer(host, snap, {{ display }}); + const explorer = mountExplorer(host, snapAt(stage), {{ display }}); explorer.openTile(D.DRILL_KINDS.selection, "topic-x"); return flatten(host).filter((n) => n.className === "where").map((n) => n.textContent); }} console.log(JSON.stringify({{ - neutral: rowTexts(D.neutralDisplay()), declared: rowTexts(declared), + neutral: rowTexts(D.neutralDisplay(), D.SNAPSHOT_VALUES.document_stage.captured), + declared: rowTexts(declared, "brainstorm"), }})); """, tmp_path) assert out["declared"], "the tile rendered no file row at all" @@ -1146,9 +1151,13 @@ def test_the_canvas_confirmation_shows_the_declared_seed_word(tmp_path): declared_state = [t for t in out["declared"] if t.startswith("state: ")] assert declared_state, out["declared"] assert declared_state[0] == "state: parked", declared_state + # The neutral install seeds openDox's own shipped value, and the line reads + # openDox's own word for that role (the two are spelled alike since T054, + # which made the neutral snapshot's values the defaults). neutral_state = [t for t in out["neutral"] if t.startswith("state: ")] - assert neutral_state[0] != "state: latent", ( - "the confirmation still spells the register's enum value") + assert neutral_state == [ + "state: " + NEUTRAL_DISPLAY["statuses"]["candidate"]["captured"]], ( + "the neutral confirmation does not read openDox's word for the state") @pytest.mark.skipif(NODE is None, reason="node is not installed") @@ -1168,27 +1177,30 @@ def test_the_funnel_search_matches_the_word_a_human_can_see(tmp_path): const base = {views} + "/"; const D = await import(base + "display.js"); const {{ renderFunnel }} = await import(base + "funnel.js"); -const snap = {{ - documents: [{{ id: "a.md", path: "a.md", stage: "brainstorm", topics: [] }}], +// each facet is shown a document at ITS OWN captured enum value +const snapAt = (stage) => ({{ + documents: [{{ id: "a.md", path: "a.md", stage, topics: [] }}], clusters: [], possibles: [], staged_topics: [], changes: [], -}}; +}}); const declared = D.readDisplay({{ display: {{ schema_version: 1, kind: "opendox.display-facet", host_facet: "declared", values: {{ document_stage: {{ captured: "brainstorm" }} }}, statuses: {{ document: {{ captured: "jotted" }} }}, }} }}); -function hays(display) {{ +function hays(display, stage) {{ const root = new Node("div"); - renderFunnel(root, snap, {{ display }}); + renderFunnel(root, snapAt(stage), {{ display }}); return flatten(root).filter((n) => n.dataset && n.dataset.hay) .map((n) => n.dataset.hay); }} console.log(JSON.stringify({{ - declared: hays(declared), neutral: hays(D.neutralDisplay()), + declared: hays(declared, "brainstorm"), + neutral: hays(D.neutralDisplay(), D.SNAPSHOT_VALUES.document_stage.captured), + shipped: D.SNAPSHOT_VALUES.document_stage.captured, }})); """, tmp_path) assert out["declared"] == ["a.md brainstorm jotted"] - assert out["neutral"] == ["a.md brainstorm captured"] + assert out["neutral"] == [f"a.md {out['shipped']} captured"] # --------------------------------------------------------------------------- diff --git a/tests/test_neutral_projection.py b/tests/test_neutral_projection.py new file mode 100644 index 0000000..2c61670 --- /dev/null +++ b/tests/test_neutral_projection.py @@ -0,0 +1,908 @@ +"""openDox's own small neutral projection, held to plan 034's T054. + +T054 realizes #1144's 5.1, 5.2 and 5.3: `opendox.neutral_projection`, new code +over `CorpusAdapter`, bound to `LocalGitCorpus` through the home-corpus seam, +writing T053's neutral snapshot. Its falsifier, from plan 034's tasks.md: + + an in-process test that the projection over T050's fixture, with neither + sibling importable, validates against T053's schema and carries none of + F5.3's declared words; the topic-rule test over a copy of repository (b); + and a test that a `stage:` value outside the six is reported and read as a + source. + +Those are the first three cases below, in that order. F5.3 itself runs +`python -m opendox.cli generate`, whose verb reaches the consumer's generator +until T055 routes it, so T056 and T063 quote it. + +THE SCHEMA. `tests/fixtures/opendox-snapshot.schema.yaml` is openDox-spec's +neutral snapshot contract, copied byte for byte from openDox-spec#16 at +`cd49eb25` (T053), and held here to that file's sha256. T057 ships the packaged +copy, and that copy replaces this one when it lands. The evaluator below is a +port of openDox-spec's own (`tests/test_opendox_snapshot_contract.py` there): +the JSON Schema keywords the contract uses, as draft 2020-12 defines them, and +its seven reference rules. The leg's test extra installs no `jsonschema`, so +none is imported. + +WHAT ELSE IT HOLDS. + +1. THE HOLDER'S RULE ON T051: the projection copies `title` and `summary` + without coercing or excluding them, so T051's malformed fixture reaches the + snapshot and breaks exactly its `EXPECTED_RULE`. +2. WHAT A DOCUMENT IS: an entry the adapter classifies. One it cannot classify + is left out (the holder's ruling on T054), and so is a path the schema + cannot carry, which is reported. +3. THE STATIONS: a declared group gathers the sources that share its topics, + one edge per document; candidates are claimed by the groups they share a + topic with; ids are unique within each section. +4. THE ANCHORS: `source_revision` is the corpus HEAD unless pinned, + `generated_at` is that commit's own date unless given, and the same tree + answers the same bytes. +5. THE FIELD SET: openDox's default adapter obliges `title` and `summary` + (R1Q13 (a)), so `authoring.required_header_fields()` answers them through + the entry point, and a document without them is still read. +6. THE VALUES: `SNAPSHOT_VALUES`' defaults, in Python and in `display.js`, are + the neutral schema's values, so openDox's views match openDox's snapshot. + And the wheel's grouping tile counts a group's edges where the group carries + no tally, as the neutral snapshot's groups do not (the holder's ruling: + the schema is not widened). +7. NO REACH: the projection imports with every sibling blocked and pulls in no + third-party module. + +`--noconftest` SAFE. The autouse fixture saves and restores the three +registries an entry point writes, so no case leaves a registration behind. + +A CREATED file: no carve-manifest row (RULED OQ-C). +""" + +from __future__ import annotations + +import hashlib +import json +import os +import re +import shutil +import subprocess +import sys +import textwrap +from pathlib import Path +from typing import Any, Callable, Iterator, NamedTuple + +import pytest + +from opendox import ( + authoring, + corpus_adapter, + default_generator, + display_profile, + domain_profile, +) +from opendox import generator_seam as gs +from opendox import neutral_projection as projection +from opendox.runtime import local_git_adapter as lga + +ROOT = Path(__file__).resolve().parents[1] +SRC = ROOT / "src" +FIXTURES = ROOT / "tests" / "fixtures" +PLAIN = FIXTURES / "plain-documents" # T050, openDox-code#53 +MALFORMED = FIXTURES / "malformed" # T051, openDox-code#56 +SCHEMA_PATH = FIXTURES / "opendox-snapshot.schema.yaml" +DISPLAY_JS = SRC / "opendox" / "web" / "views" / "display.js" +WHEEL_MODEL_JS = SRC / "opendox" / "web" / "views" / "wheel-model.js" +NODE = shutil.which("node") + +#: The sha256 of `contracts/schemas/opendox-snapshot.schema.yaml` at +#: openDox-spec#16's head, `cd49eb25` (T053; its PR records the same digest). +SCHEMA_SHA256 = "f9e3e111af1d4bd4c377c933027d81b582ae2b0a395b66f4e4621992454a584a" + +#: The four packages a neutral openDox must import without (#1144's F2.1). +SIBLINGS = ("openxdox", "ideation_dashboard", "doc_health", + "corpus_adapter_openxfactory") + +#: F5.3's declared vocabulary, verbatim from #1144's `tasks.md`. +F5_3_WORDS = ("brainstorm", "staged", "draft", "ratified", "standard", + "superseded", "retired", "record", "openspec", "proposal.md", + "tasks.md", "design.md", "added requirements", + "modified requirements") + +#: AT-R1's repository (b), byte for byte as plan 034's quickstart.md § 2 +#: writes it with `printf`: ordinary Markdown with no front matter at all. +REPOSITORY_B = { + "roadmap.md": "# Roadmap\n\nThe roadmap links to the [budget](budget.md) " + "and the [notes](notes.md).\n", + "budget.md": "# Budget\n\nBudget figures for the roadmap.\n", + "notes.md": "# Meeting notes\n\nWe discussed the roadmap and the budget.\n", +} + +#: Every fixture commit is made at this date, so `generated_at` is known. +ANCHOR_DATE = "2026-09-27T12:00:00+00:00" + + +# --------------------------------------------------------------------------- +# the registries, and plain git repositories to project +# --------------------------------------------------------------------------- + +def _home(root: str): + """The entry points' default home factory, in its own shape (T022).""" + return (lga.WorkingTreeCorpus(), + corpus_adapter.CorpusRef(name="home", location=str(root))) + + +@pytest.fixture(autouse=True) +def _isolated_registries(): + """Each case starts with no generator registered and openDox's default + home corpus registered, and PUTS BACK the three registries it found.""" + seam = (gs._registered, gs._is_default, gs._generated_from_default, + gs._default_generations_under_way) + profile = (domain_profile._registered, domain_profile._is_default, + domain_profile._built_from_default) + home = corpus_adapter._home_factory + gs.unregister() + corpus_adapter.register_home(_home) + yield + (gs._registered, gs._is_default, gs._generated_from_default, + gs._default_generations_under_way) = seam + (domain_profile._registered, domain_profile._is_default, + domain_profile._built_from_default) = profile + corpus_adapter._home_factory = home + + +def _git(root: Path, *args: str) -> str: + """`git` in `root`, with no inherited `GIT_*` variable and no user or + system configuration, as the fixture's own identity at a fixed date.""" + env = {k: v for k, v in os.environ.items() if not k.startswith("GIT_")} + env.update({ + "GIT_AUTHOR_NAME": "fixture", "GIT_AUTHOR_EMAIL": "fixture@example.invalid", + "GIT_COMMITTER_NAME": "fixture", + "GIT_COMMITTER_EMAIL": "fixture@example.invalid", + "GIT_AUTHOR_DATE": ANCHOR_DATE, "GIT_COMMITTER_DATE": ANCHOR_DATE, + "GIT_CONFIG_GLOBAL": os.devnull, "GIT_CONFIG_SYSTEM": os.devnull, + }) + return subprocess.run(["git", "-C", str(root), *args], check=True, + capture_output=True, env=env).stdout.decode() + + +def _repository(tmp_path: Path, *, copy: Path | None = None, + files: dict[str, str | bytes] | None = None, + commit: bool = True, name: str = "repository") -> Path: + """A FRESH plain git repository: `copy`'s files and then `files`, added, + and committed unless `commit` is false.""" + root = tmp_path / name + if copy is not None: + shutil.copytree(copy, root) + else: + root.mkdir(parents=True) + for relative, body in (files or {}).items(): + path = root / relative + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(body if isinstance(body, bytes) else body.encode()) + _git(root, "-c", "init.defaultBranch=main", "init", "-q") + _git(root, "add", "-A") + if commit: + _git(root, "commit", "-qm", "fixture") + return root + + +def _generate(root: Path, **anchors: Any) -> dict[str, Any]: + """openDox's own generator over `root`, exactly as the seam calls it.""" + return default_generator.generate(root, "fixture", **anchors) + + +def _by_path(snapshot: dict[str, Any]) -> dict[str, dict[str, Any]]: + return {document["path"]: document for document in snapshot["documents"]} + + +def _values(node: Any) -> Iterator[str]: + """Every string VALUE, as F5.3 reads them: keys are the product's own.""" + if isinstance(node, dict): + for value in node.values(): + yield from _values(value) + elif isinstance(node, list): + for value in node: + yield from _values(value) + elif isinstance(node, str): + yield node + + +_F5_3 = re.compile(r"\b(" + "|".join(re.escape(w) for w in F5_3_WORDS) + r")\b") + + +def _leaks(snapshot: dict[str, Any]) -> list[str]: + return sorted({m.group(1) for value in _values(snapshot) + for m in _F5_3.finditer(value.lower())}) + + +# --------------------------------------------------------------------------- +# the neutral contract, and an evaluator for it (openDox-spec's own, ported) +# --------------------------------------------------------------------------- + +def _no_duplicate_keys(pairs: list[tuple[str, Any]]) -> dict[str, Any]: + out: dict[str, Any] = {} + for key, value in pairs: + if key in out: + raise ValueError(f"the key {key!r} appears twice in one object") + out[key] = value + return out + + +def _no_constants(name: str) -> Any: + raise ValueError(f"{name} is not JSON") + + +def _read_schema(path: Path) -> Any: + """The schema file's body: one JSON object after its `#` comment lines.""" + lines = path.read_text(encoding="utf-8").splitlines(keepends=True) + while lines and (lines[0].startswith("#") or not lines[0].strip()): + lines.pop(0) + return json.loads("".join(lines), object_pairs_hook=_no_duplicate_keys, + parse_constant=_no_constants) + + +SCHEMA = _read_schema(SCHEMA_PATH) + + +class Violation(NamedTuple): + rule: str # the broken rule's id, from `x-rule` or a reference check + where: str # a JSON pointer into the snapshot ("" is the root) + keyword: str # the schema keyword that failed, or "reference" + detail: str + + +def _pointer(parts: Any) -> str: + return "".join("/" + str(p).replace("~", "~0").replace("/", "~1") + for p in parts) + + +def _is_type(value: Any, name: str) -> bool: + if name == "object": + return isinstance(value, dict) + if name == "array": + return isinstance(value, list) + if name == "string": + return isinstance(value, str) + if name == "null": + return value is None + if name == "boolean": + return isinstance(value, bool) + if isinstance(value, bool): # JSON true is not the number 1 + return False + if name == "integer": + return isinstance(value, int) or ( + isinstance(value, float) and value.is_integer()) + if name == "number": + return isinstance(value, (int, float)) + raise AssertionError(f"the contract names an unknown type: {name!r}") + + +def _canon(value: Any) -> Any: + """JSON equality: `true` is not `1`, `1` is `1.0`, and key order is noise.""" + if isinstance(value, bool): + return ("boolean", value) + if isinstance(value, (int, float)): + return ("number", value) + if isinstance(value, str): + return ("string", value) + if value is None: + return ("null",) + if isinstance(value, list): + return ("array", tuple(_canon(v) for v in value)) + return ("object", tuple(sorted((k, _canon(v)) for k, v in value.items()))) + + +def _resolve(ref: str) -> dict[str, Any]: + assert ref.startswith("#/"), f"only local references are in the contract: {ref!r}" + node: Any = SCHEMA + for part in ref[2:].split("/"): + node = node[part.replace("~1", "/").replace("~0", "~")] + return node + + +def _check(value: Any, schema: dict[str, Any], where: str) -> Iterator[Violation]: + if "$ref" in schema: + yield from _check(value, _resolve(schema["$ref"]), where) + rule = schema.get("x-rule", "") + + def broken(keyword: str, detail: str) -> Violation: + return Violation(rule, where, keyword, detail) + + if "type" in schema: + names = schema["type"] if isinstance(schema["type"], list) else [schema["type"]] + if not any(_is_type(value, n) for n in names): + yield broken("type", f"{value!r} is not of type {names}") + if "const" in schema and _canon(value) != _canon(schema["const"]): + yield broken("const", f"{value!r} is not {schema['const']!r}") + if "enum" in schema and _canon(value) not in {_canon(v) for v in schema["enum"]}: + yield broken("enum", f"{value!r} is not one of {schema['enum']}") + if isinstance(value, str): + if len(value) < schema.get("minLength", 0): + yield broken("minLength", f"{value!r} is shorter than {schema['minLength']}") + if "pattern" in schema and not re.search(schema["pattern"], value): + yield broken("pattern", f"{value!r} does not match the rule's pattern") + if _is_type(value, "number") and "minimum" in schema and value < schema["minimum"]: + yield broken("minimum", f"{value!r} is less than {schema['minimum']}") + if isinstance(value, list): + if len(value) < schema.get("minItems", 0): + yield broken("minItems", f"{len(value)} items, fewer than {schema['minItems']}") + if schema.get("uniqueItems") and len({_canon(v) for v in value}) != len(value): + yield broken("uniqueItems", f"{value!r} repeats an item") + if "items" in schema: + for i, item in enumerate(value): + yield from _check(item, schema["items"], f"{where}/{i}") + if isinstance(value, dict): + for key in schema.get("required", ()): + if key not in value: + yield broken("required", f"{key!r} is required") + for key, sub in schema.get("properties", {}).items(): + if key in value: + yield from _check(value[key], sub, where + _pointer([key])) + for sub in schema.get("allOf", ()): + yield from _check(value, sub, where) + if "if" in schema and "then" in schema: # `if` is a test, never a failure + if not any(True for _ in _check(value, schema["if"], where)): + yield from _check(value, schema["then"], where) + + +def _entries(snap: Any, section: str) -> list[tuple[int, dict[str, Any]]]: + items = snap.get(section) if isinstance(snap, dict) else None + if not isinstance(items, list): + return [] + return [(i, e) for i, e in enumerate(items) if isinstance(e, dict)] + + +def _ids(snap: Any, section: str, key: str = "id") -> set[str]: + return {e[key] for _i, e in _entries(snap, section) if isinstance(e.get(key), str)} + + +def _ids_are_unique(snap: Any) -> Iterator[Violation]: + for section, key in (("documents", "id"), ("clusters", "id"), + ("possibles", "id"), ("staged_topics", "staging_id"), + ("changes", "id")): + seen: set[str] = set() + for i, entry in _entries(snap, section): + value = entry.get(key) + if isinstance(value, str): + if value in seen: + yield Violation("ids-are-unique", f"/{section}/{i}/{key}", + "reference", f"{value!r} is already an id") + seen.add(value) + + +def _edges(group: dict[str, Any]) -> list[tuple[int, Any]]: + edges = group.get("document_edges") + return list(enumerate(edges if isinstance(edges, list) else [])) + + +def _edge_names_a_document(snap: Any) -> Iterator[Violation]: + known = _ids(snap, "documents") + for gi, group in _entries(snap, "clusters"): + for ei, edge in _edges(group): + ref = edge.get("document") if isinstance(edge, dict) else None + if isinstance(ref, str) and ref not in known: + yield Violation("edge-names-a-document", + f"/clusters/{gi}/document_edges/{ei}/document", + "reference", f"no document has the id {ref!r}") + + +def _one_edge_per_document(snap: Any) -> Iterator[Violation]: + for gi, group in _entries(snap, "clusters"): + seen: set[str] = set() + for ei, edge in _edges(group): + ref = edge.get("document") if isinstance(edge, dict) else None + if isinstance(ref, str): + if ref in seen: + yield Violation("one-edge-per-document", + f"/clusters/{gi}/document_edges/{ei}/document", + "reference", f"{ref!r} already has an edge") + seen.add(ref) + + +def _keyword_index_matches_topics(snap: Any) -> Iterator[Violation]: + index = snap.get("keyword_index") if isinstance(snap, dict) else None + if not isinstance(index, list): + return + carried: dict[str, int] = {} + for _i, document in _entries(snap, "documents"): + topics = document.get("topics") + for topic in ({t for t in topics if isinstance(t, str)} + if isinstance(topics, list) else ()): + carried[topic] = carried.get(topic, 0) + 1 + listed: set[str] = set() + rule = "keyword-index-matches-topics" + for ki, entry in enumerate(index): + keyword = entry.get("keyword") if isinstance(entry, dict) else None + if not isinstance(keyword, str): + continue + if keyword in listed: + yield Violation(rule, f"/keyword_index/{ki}/keyword", "reference", + f"{keyword!r} already has an entry") + continue + listed.add(keyword) + count = entry.get("declared_doc_count") + if _is_type(count, "integer") and count != carried.get(keyword, 0): + yield Violation(rule, f"/keyword_index/{ki}/declared_doc_count", + "reference", f"{count} documents, but " + f"{carried.get(keyword, 0)} carry {keyword!r}") + unlisted = sorted(set(carried) - listed) + if unlisted: + yield Violation(rule, "/keyword_index", "reference", + f"topics the documents carry and the index lacks: {unlisted}") + + +def _candidate_names_a_group(snap: Any) -> Iterator[Violation]: + known = _ids(snap, "clusters") + for pi, candidate in _entries(snap, "possibles"): + refs = candidate.get("claiming_clusters") + for ri, ref in enumerate(refs if isinstance(refs, list) else []): + if isinstance(ref, str) and ref not in known: + yield Violation("candidate-names-a-group", + f"/possibles/{pi}/claiming_clusters/{ri}", + "reference", f"no group has the id {ref!r}") + + +def _pick_names_a_selection(snap: Any) -> Iterator[Violation]: + known = _ids(snap, "staged_topics", "staging_id") + for pi, candidate in _entries(snap, "possibles"): + pick = candidate.get("pick") + ref = pick.get("staging_id") if isinstance(pick, dict) else None + if isinstance(ref, str) and ref not in known: + yield Violation("pick-names-a-selection", + f"/possibles/{pi}/pick/staging_id", "reference", + f"no selection has the staging_id {ref!r}") + + +def _target_names_a_submission(snap: Any) -> Iterator[Violation]: + known = _ids(snap, "changes") + for ti, selection in _entries(snap, "staged_topics"): + ref = selection.get("target_change") + if isinstance(ref, str) and ref not in known: + yield Violation("target-names-a-submission", + f"/staged_topics/{ti}/target_change", "reference", + f"no changes entry has the id {ref!r}") + + +REFERENCE_CHECKS: dict[str, Callable[[Any], Iterator[Violation]]] = { + "ids-are-unique": _ids_are_unique, + "edge-names-a-document": _edge_names_a_document, + "one-edge-per-document": _one_edge_per_document, + "candidate-names-a-group": _candidate_names_a_group, + "pick-names-a-selection": _pick_names_a_selection, + "target-names-a-submission": _target_names_a_submission, + "keyword-index-matches-topics": _keyword_index_matches_topics, +} + + +def violations(snapshot: Any) -> list[Violation]: + """Every rule of the neutral contract that `snapshot` breaks.""" + found = list(_check(snapshot, SCHEMA, "")) + for check in REFERENCE_CHECKS.values(): + found.extend(check(snapshot)) + return found + + +def test_the_schema_copy_is_openDox_specs_contract_and_matches_the_product() -> None: + """The copy is T053's file, and the product's own declarations are the + contract's closed values (T053's writer asked for this cross-check).""" + digest = hashlib.sha256(SCHEMA_PATH.read_bytes()).hexdigest() + assert digest == SCHEMA_SHA256, ( + f"tests/fixtures/opendox-snapshot.schema.yaml is {digest}, not " + "openDox-spec#16's contract at cd49eb25. Copy the spec leg's file " + "again and update SCHEMA_SHA256 with it, in one commit") + defs = SCHEMA["$defs"] + assert SCHEMA["properties"]["kind"]["const"] == gs.NEUTRAL_SNAPSHOT_KIND + assert SCHEMA["properties"]["schema_version"]["const"] == projection.SCHEMA_VERSION + assert defs["stage_role"]["enum"] == list(display_profile.STAGE_ROLES) + sections = {section for _role, section, _status in display_profile.STAGE_FIELDS} + assert sections <= set(SCHEMA["required"]) + assert set(defs["submission_status"]["enum"]) == set( + projection.CHANGE_STATUS.values()) + assert sorted(REFERENCE_CHECKS) == sorted( + rule["id"] for rule in SCHEMA["x-rules"] if rule["class"] == "reference") + + +# --------------------------------------------------------------------------- +# the falsifier, part 1: T050's fixture, with neither sibling importable +# --------------------------------------------------------------------------- + +_FALSIFIER = """ +import importlib, json, sys +from pathlib import Path +blocked = [] +for name in SIBLINGS: + sys.modules[name] = None + try: + importlib.import_module(name) + except ImportError: + blocked.append(name) +from opendox import cli, generator_seam +cli.build_parser() # the entry point registers openDox's own defaults +snapshot = generator_seam.generate(Path(ROOT_OF_REPOSITORY), "fixture") +print(json.dumps({"blocked": blocked, "snapshot": snapshot})) +""" + + +def test_the_projection_over_the_plain_documents_fixture_is_a_neutral_snapshot( + tmp_path: Path) -> None: + """T054's falsifier, first part. A fresh process blocks every sibling, + lets the CLI entry point register openDox's own defaults (the default + profile, `WorkingTreeCorpus` as the home corpus, openDox's own generator), + and generates through the seam over a fresh repository of T050's fixture. + The snapshot validates against T053's schema, carries none of F5.3's + declared words, and fills the station each document names.""" + root = _repository(tmp_path, copy=PLAIN) + program = (f"import sys; sys.path.insert(0, {str(SRC)!r})\n" + f"SIBLINGS = {SIBLINGS!r}\nROOT_OF_REPOSITORY = {str(root)!r}\n" + + textwrap.dedent(_FALSIFIER)) + done = subprocess.run([sys.executable, "-c", program], capture_output=True, + text=True, cwd=str(ROOT), timeout=300) + assert done.returncode == 0, done.stderr + assert "notice:" not in done.stderr, done.stderr + out = json.loads(done.stdout.strip().splitlines()[-1]) + assert out["blocked"] == list(SIBLINGS), out["blocked"] + snapshot = out["snapshot"] + + assert violations(snapshot) == [] + assert _leaks(snapshot) == [], ( + f"openxFactory's vocabulary leaked into the neutral projection: " + f"{_leaks(snapshot)}") + assert snapshot["kind"] == gs.NEUTRAL_SNAPSHOT_KIND + assert snapshot["documents"], "the snapshot is empty" + + # EVERY DOCUMENT IN THE STATION IT NAMES, read here off the fixture's own + # `stage:` lines rather than through the projection's reader. + expected = {} + for path in sorted(PLAIN.glob("*.md")): + declared = re.search(r"^stage: (\S+)$", + path.read_text(encoding="utf-8"), re.M) + expected[path.name] = declared.group(1) if declared else "source" + assert {p: d["stage"] for p, d in _by_path(snapshot).items()} == expected + assert len(set(expected.values())) == 6, "T050 spreads over six stations" + assert snapshot["possibles"] and snapshot["staged_topics"] + assert {c["status"] for c in snapshot["changes"]} == {"active", "archived"} + + # AT LEAST ONE GROUP FORMS FROM SOURCES ALONE, which is what lets AT-R1 + # open the chat pane: the two rain-barrel notes share their name's words. + sources = {p for p, stage in expected.items() if stage == "source"} + derived = [c for c in snapshot["clusters"] + if {e["document"] for e in c["document_edges"]} <= sources] + assert derived, snapshot["clusters"] + assert any(set(c["topics"]) >= {"rain", "barrel"} for c in derived), derived + + +# --------------------------------------------------------------------------- +# the falsifier, part 2: the topic rule over repository (b), no front matter +# --------------------------------------------------------------------------- + +def test_the_topic_rule_groups_repository_b_which_has_no_front_matter( + tmp_path: Path) -> None: + """T054's falsifier, second part. THE TOPIC RULE: a document that declares + no `topics:` carries the words of its name (its `title:`, else its first + `#` heading, else its file name) and the name words of every other + document it mentions by name or file name. Repository (b)'s three notes + have no header at all, and they name each other, so they group.""" + for body in REPOSITORY_B.values(): + assert lga.leading_header(body) == {}, "repository (b) has front matter" + root = _repository(tmp_path, files=REPOSITORY_B, name="plain-notes") + snapshot = _generate(root) + assert violations(snapshot) == [] + assert _leaks(snapshot) == [] + + documents = _by_path(snapshot) + assert set(documents) == set(REPOSITORY_B) + for document in documents.values(): + assert document["stage"] == "source" + assert document["title"] is None and document["summary"] is None + assert documents["budget.md"]["topics"] == ["budget", "roadmap"] + assert documents["notes.md"]["topics"] == ["budget", "meeting", "notes", "roadmap"] + + assert snapshot["clusters"], "repository (b) yields no grouping tile (AT-R1 FAILS)" + shared = {tuple(c["topics"]): sorted(e["document"] for e in c["document_edges"]) + for c in snapshot["clusters"]} + assert shared[("budget", "roadmap")] == sorted(REPOSITORY_B) + for cluster in snapshot["clusters"]: + assert len(cluster["document_edges"]) >= 2, cluster + + +# --------------------------------------------------------------------------- +# the falsifier, part 3: a `stage:` value outside the six +# --------------------------------------------------------------------------- + +@pytest.mark.parametrize("value", ["shelved", "Grouping", ""]) +def test_a_stage_value_outside_the_six_is_reported_and_read_as_a_source( + value: str, tmp_path: Path, capsys) -> None: + """T054's falsifier, third part (spec.md's edge case, the projection's + half; T056 holds the verb's). A value outside the six role keys, the empty + value and another casing included, is no declaration: the generator + reports it, naming the document, the value and the six keys, and the + snapshot reads the document as a source, so no other value reaches it.""" + target = "grouping-compost-corner.md" + root = _repository(tmp_path, copy=PLAIN) + path = root / target + path.write_text(path.read_text(encoding="utf-8").replace( + "stage: grouping\n", f"stage: {value}\n" if value else "stage:\n", 1), + encoding="utf-8") + _git(root, "commit", "-qam", "an undeclared stage") + + snapshot = _generate(root) + notices = [line for line in capsys.readouterr().err.splitlines() + if line.startswith("notice: ")] + assert len(notices) == 1, notices + assert notices[0].startswith(f"notice: {target}: "), notices[0] + assert repr(value) in notices[0] + for role in display_profile.STAGE_ROLES: + assert role in notices[0], f"the report does not name {role!r}" + + assert _by_path(snapshot)[target]["stage"] == "source" + assert "grouping-compost-corner" not in {c["id"] for c in snapshot["clusters"]} + assert violations(snapshot) == [] + + +# --------------------------------------------------------------------------- +# 1 — title and summary, exactly as the document gives them (the holder rule) +# --------------------------------------------------------------------------- + +def test_the_malformed_fixture_keeps_its_one_violation(tmp_path: Path) -> None: + """The holder's ruling on T051: the projection neither coerces nor drops + an empty `title:`, so T051's fixture breaks exactly its EXPECTED_RULE, at + that document's title. EXPECTED_RULE itself has no shape the adapter + recognizes, so it is not a document.""" + expected_rule = (MALFORMED / "EXPECTED_RULE").read_text(encoding="utf-8").strip() + snapshot = _generate(_repository(tmp_path, copy=MALFORMED)) + paths = [d["path"] for d in snapshot["documents"]] + assert paths == ["notes-bee-boxes.md", "notes-empty-title.md"] + found = violations(snapshot) + assert [(v.rule, v.where) for v in found] == [ + (expected_rule, f"/documents/{paths.index('notes-empty-title.md')}/title")] + assert _by_path(snapshot)["notes-empty-title.md"]["title"] == "" + + +def test_title_and_summary_are_copied_exactly_and_null_only_when_absent( + tmp_path: Path) -> None: + root = _repository(tmp_path, files={ + "both.md": "title: Both fields\nsummary: kept as written \n\n# Other\n", + "none.md": "# Only a heading\n\nNo header here.\n", + "empty.md": "title:\nsummary:\n\nBody.\n", + "quoted.md": 'title: "Quoted"\n\nBody.\n', + }) + documents = _by_path(_generate(root)) + assert (documents["both.md"]["title"], documents["both.md"]["summary"]) == ( + "Both fields", "kept as written") + assert (documents["none.md"]["title"], documents["none.md"]["summary"]) == (None, None) + assert (documents["empty.md"]["title"], documents["empty.md"]["summary"]) == ("", "") + assert (documents["quoted.md"]["title"], documents["quoted.md"]["summary"]) == ( + '"Quoted"', None) + + +# --------------------------------------------------------------------------- +# 2 — what a document is +# --------------------------------------------------------------------------- + +def test_an_entry_the_adapter_cannot_classify_is_not_a_document(tmp_path: Path) -> None: + """The holder's ruling: a `Makefile` or an image is left out, and is never + read, so nothing mentions it either.""" + root = _repository(tmp_path, files={ + "Makefile": "all:\n\techo build\n", + "garden-layout.png": b"\x89PNG\r\n\x1a\n\x00binary", + "notes.md": "# Notes\n\nSee the Makefile and the garden layout png.\n", + }) + snapshot = _generate(root) + assert [d["path"] for d in snapshot["documents"]] == ["notes.md"] + assert _by_path(snapshot)["notes.md"]["topics"] == ["notes"] + assert violations(snapshot) == [] + + +@pytest.mark.parametrize(("name", "reason"), [ + ("back\\slash.md", "backslash"), ("tab\tin-name.md", "control character")]) +def test_a_path_the_schema_cannot_carry_is_left_out_and_reported( + name: str, reason: str, tmp_path: Path, capsys) -> None: + root = _repository(tmp_path, files={name: "# Odd\n", "plain.md": "# Plain\n"}) + snapshot = _generate(root) + assert [d["path"] for d in snapshot["documents"]] == ["plain.md"] + notices = [line for line in capsys.readouterr().err.splitlines() + if line.startswith("notice: ")] + assert len(notices) == 1 and repr(name) in notices[0] and reason in notices[0], notices + assert violations(snapshot) == [] + + +# --------------------------------------------------------------------------- +# 3 — topics, groups and the other stations +# --------------------------------------------------------------------------- + +def test_declared_topics_replace_the_derived_ones(tmp_path: Path) -> None: + root = _repository(tmp_path, files={ + "a.md": "title: Rain notes\ntopics: Rain, rain barrel , COMPOST, , " + "tab\there, bell\x07\n\nText.\n", + }) + snapshot = _generate(root) + assert _by_path(snapshot)["a.md"]["topics"] == [ + "compost", "rain", "rain barrel", "tab here"] + assert violations(snapshot) == [] + + +def test_a_declared_group_gathers_the_sources_that_share_its_topics( + tmp_path: Path) -> None: + root = _repository(tmp_path, files={ + "g.md": "stage: grouping\ntitle: Water, gathered\ntopics: water\n\n.\n", + "a.md": "topics: water, soil\n\n.\n", + "b.md": "topics: soil\n\n.\n", + "c.md": "stage: candidate\ntitle: Harvest the rain\nsummary: Gutters " + "to barrels.\ntopics: water\n\n.\n", + "d.md": "stage: selection\ntopics: water\n\n.\n", + }) + snapshot = _generate(root) + assert violations(snapshot) == [] + clusters = {c["id"]: c for c in snapshot["clusters"]} + assert list(clusters) == ["g", "soil"], "declared groups first, then derived" + assert clusters["g"]["name"] == "Water, gathered" + assert clusters["g"]["document_edges"] == [ + {"document": "a.md", "matched_topics": ["water"]}, + {"document": "g.md", "matched_topics": ["water"]}] + assert [e["document"] for e in clusters["soil"]["document_edges"]] == ["a.md", "b.md"] + [candidate] = snapshot["possibles"] + assert candidate == {"id": "c", "title": "Harvest the rain", + "claim": "Gutters to barrels.", "state": "unselected", + "claiming_clusters": ["g"]} + assert snapshot["staged_topics"] == [{"staging_id": "d", "files": ["d.md"]}] + + +def test_a_declared_group_with_no_topic_forms_no_group_and_is_reported( + tmp_path: Path, capsys) -> None: + snapshot = _generate(_repository(tmp_path, files={"g.md": "stage: grouping\ntitle: 2026\n"})) + assert snapshot["clusters"] == [] + assert _by_path(snapshot)["g.md"]["stage"] == "grouping" + assert "notice: g.md: declares stage grouping but carries no topic" in \ + capsys.readouterr().err + + +def test_station_ids_are_unique_within_a_section(tmp_path: Path) -> None: + snapshot = _generate(_repository(tmp_path, files={ + "a/plan.md": "stage: selection\n", "b/plan.md": "stage: selection\n", + "x/ship.md": "stage: submission\n", "y/ship.md": "stage: completion\n", + })) + assert [t["staging_id"] for t in snapshot["staged_topics"]] == ["plan", "plan-2"] + assert [(c["id"], c["status"]) for c in snapshot["changes"]] == [ + ("ship", "active"), ("ship-2", "archived")] + assert violations(snapshot) == [] + + +# --------------------------------------------------------------------------- +# 4 — the anchors, and determinism +# --------------------------------------------------------------------------- + +def test_the_anchors_come_from_the_source_revision_and_the_bytes_repeat( + tmp_path: Path) -> None: + root = _repository(tmp_path, copy=PLAIN) + first, second = _generate(root), _generate(root) + assert json.dumps(first) == json.dumps(second) + head = _git(root, "rev-parse", "HEAD").strip() + assert first["generation"] == { + "source_revision": head, + "generated_at": _git(root, "show", "-s", "--format=%cI", "HEAD").strip(), + "generator_version": projection.GENERATOR_VERSION} + assert first["generation"]["generated_at"] == ANCHOR_DATE + + pinned = _generate(root, source_revision="release-candidate", + generated_at="2026-01-02T03:04:05Z")["generation"] + assert (pinned["source_revision"], pinned["generated_at"]) == ( + "release-candidate", "2026-01-02T03:04:05Z") + unknown = _generate(root, source_revision="0" * 40)["generation"] + assert "generated_at" not in unknown + + +def test_a_repository_with_no_commit_needs_a_pinned_revision(tmp_path: Path) -> None: + root = _repository(tmp_path, files={"a.md": "# A\n"}, commit=False) + with pytest.raises(projection.ProjectionRefused) as caught: + _generate(root) + assert isinstance(caught.value, gs.GeneratorSeamError) + assert "source revision" in str(caught.value) + snapshot = _generate(root, source_revision="pinned-by-hand") + assert snapshot["generation"]["source_revision"] == "pinned-by-hand" + assert "generated_at" not in snapshot["generation"] + assert violations(snapshot) == [] + + +# --------------------------------------------------------------------------- +# 5 — the small neutral field set on openDox's default adapter +# --------------------------------------------------------------------------- + +def test_the_default_adapter_obliges_the_small_neutral_field_set(tmp_path: Path) -> None: + assert lga.NEUTRAL_FIELDS == ("title", "summary") + assert lga.WorkingTreeCorpus()._required_fields == lga.NEUTRAL_FIELDS + assert lga.LocalGitCorpus()._required_fields == () + root = _repository(tmp_path, files={ + "both.md": "title: T\nsummary: S\n", "none.md": "# N\n", + "empty.md": "title:\nsummary: S\n", "Makefile": "all:\n"}) + adapter, ref = _home(str(root)) + corpus = adapter.resolve(ref) + missing = {d.key: adapter.classify(corpus, d).missing_fields + for d in adapter.list_documents(corpus)} + assert missing == {"both.md": (), "none.md": ("title", "summary"), + "empty.md": ("title",), "Makefile": ()} + # a document without the fields is still read, as a source + assert _by_path(_generate(root))["none.md"]["stage"] == "source" + + +def test_required_header_fields_answers_the_neutral_set_through_the_entry_point() -> None: + from opendox import cli + + corpus_adapter._home_factory = corpus_adapter._UNSET + cli.build_parser() + assert authoring.required_header_fields() == ("title", "summary") + assert authoring.missing_required_headers("title: A note\n") == ["summary"] + + +def test_the_header_reader_keeps_an_empty_value_and_stops_at_a_blank_line() -> None: + assert lga.leading_header("a: 1\nno colon here\nb:\na: 2\n\nc: 3\n") == { + "a": "2", "b": ""} + + +# --------------------------------------------------------------------------- +# 6 — the product's views match the product's snapshot +# --------------------------------------------------------------------------- + +def _js_snapshot_values() -> dict[str, dict[str, str]]: + text = DISPLAY_JS.read_text(encoding="utf-8") + body = re.search(r"export const SNAPSHOT_VALUES = \{(.*?)\n\};", text, re.S).group(1) + return {name: dict(re.findall(r'(\w+): "([^"]*)"', table)) + for name, table in re.findall(r"(\w+): \{([^}]*)\}", body)} + + +def test_the_display_defaults_are_the_neutral_snapshots_values() -> None: + """R1Q11 (a), batch G's 5.3: the defaults are what openDox's own generator + writes, role for role, on both sides of the process boundary. The + candidate's four are NEUTRAL_DISPLAY's own words: no word is re-authored.""" + values = display_profile.SNAPSHOT_VALUES + assert values == { + "document_stage": {"captured": "source", "organized": "grouping"}, + "register_state": display_profile.NEUTRAL_DISPLAY["statuses"]["candidate"], + } + assert _js_snapshot_values() == values + defs = SCHEMA["$defs"] + assert set(values["document_stage"].values()) <= set(defs["stage_role"]["enum"]) + assert set(values["register_state"].values()) == set(defs["candidate_state"]["enum"]) + assert projection.SOURCE == values["document_stage"]["captured"] + assert projection.GROUPING == values["document_stage"]["organized"] + assert projection.UNSELECTED == values["register_state"]["captured"] + + +@pytest.mark.skipif(NODE is None, reason="node is not installed") +def test_the_wheel_counts_a_groups_edges_where_it_carries_no_tally(tmp_path: Path) -> None: + """The holder's ruling: the neutral schema is not widened with a tally, so + the wheel's grouping tile counts the group's edges. A group that carries a + tally, as the governed snapshot's do, still shows the tally.""" + snapshot = _generate(_repository(tmp_path, files=REPOSITORY_B)) + snapshot["clusters"].append({"id": "tallied", "name": "tallied", "topics": ["t"], + "document_edges": [], "tallies": {"document_links": 7}}) + script = tmp_path / "wheel.mjs" + script.write_text(textwrap.dedent(f""" + const W = await import({json.dumps(str(WHEEL_MODEL_JS))}); + const snap = {json.dumps(snapshot)}; + const model = W.buildWheelModel(snap); + const reel = model.wheels.find((w) => w.key === "grouping"); + console.log(JSON.stringify(reel.items.map((i) => [i.id, i.sub]))); + """), encoding="utf-8") + done = subprocess.run([NODE, str(script)], capture_output=True, text=True, timeout=120) + assert done.returncode == 0, done.stderr + subs = dict(json.loads(done.stdout.strip().splitlines()[-1])) + for cluster in snapshot["clusters"][:-1]: + assert subs[cluster["id"]].startswith(f"{len(cluster['document_edges'])} "), subs + assert subs["tallied"].startswith("7 "), subs + + +# --------------------------------------------------------------------------- +# 7 — no reach +# --------------------------------------------------------------------------- + +def test_the_projection_imports_with_no_sibling_and_no_third_party_module() -> None: + block = "".join(f"sys.modules[{name!r}] = None\n" for name in SIBLINGS) + program = (f"import sys; sys.path.insert(0, {str(SRC)!r})\n" + block + + textwrap.dedent(""" + before = set(sys.modules) + import opendox.neutral_projection, opendox.default_generator + added = {name.split(".")[0] for name in set(sys.modules) - before} + print(sorted(added - set(sys.stdlib_module_names) - {"opendox"})) + """)) + done = subprocess.run([sys.executable, "-c", program], capture_output=True, + text=True, cwd=str(ROOT), timeout=120) + assert done.returncode == 0, done.stderr + assert done.stdout.strip() == "[]", done.stdout + assert not hasattr(default_generator, "NeutralProjectionNotBuilt"), ( + "T052's refusal outlived the projection that retires it") From b2f92226714aab7173b563df3c182010a35b3105 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sun, 27 Sep 2026 22:34:23 +0000 Subject: [PATCH 07/38] T054: compare the commit date as a time, as git spells it either way (plan 034) validate failed at 0f42f678 in one case, test_the_anchors_come_from_the_source_revision_and_the_bytes_repeat. CI's git prints `%cI` for a zero offset as `2026-09-27T12:00:00Z`, and the git this was written against (2.43.0) prints `2026-09-27T12:00:00+00:00`. The projection was right: it records the stamp exactly as git gives it, and the neutral schema admits both spellings. The test compared that stamp against the fixture's date as a string. It now compares the two as instants. The case's first assertion still holds generated_at to the checkout's own `git show` output, so a date read from anywhere else is still caught. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/test_neutral_projection.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/test_neutral_projection.py b/tests/test_neutral_projection.py index 2c61670..2ccad05 100644 --- a/tests/test_neutral_projection.py +++ b/tests/test_neutral_projection.py @@ -64,6 +64,7 @@ import subprocess import sys import textwrap +from datetime import datetime from pathlib import Path from typing import Any, Callable, Iterator, NamedTuple @@ -777,7 +778,11 @@ def test_the_anchors_come_from_the_source_revision_and_the_bytes_repeat( "source_revision": head, "generated_at": _git(root, "show", "-s", "--format=%cI", "HEAD").strip(), "generator_version": projection.GENERATOR_VERSION} - assert first["generation"]["generated_at"] == ANCHOR_DATE + # The same instant as the commit's date. git spells a zero offset `Z` or + # `+00:00` depending on its version, and the stamp is recorded as git + # spells it, so the two are compared as times, not as strings. + assert datetime.fromisoformat(first["generation"]["generated_at"]) == \ + datetime.fromisoformat(ANCHOR_DATE) pinned = _generate(root, source_revision="release-candidate", generated_at="2026-01-02T03:04:05Z")["generation"] From c0747e965af9887a0ebe537d4ee0a70f7b879764 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sun, 27 Sep 2026 22:41:20 +0000 Subject: [PATCH 08/38] T054: an empty topics header is a declaration; T050's docstring says what is true now (plan 034) Copilot's review of 0f42f678 left three threads. - r4117298988 (neutral_projection.py): a `topics:` header that listed nothing fell back to the derived topics, which contradicted the rule the module's own docstring states. The header is now a declaration even when it is empty. The document carries no topic, nothing is derived for it, and it joins no group. That follows the holder's literal-copy principle for title and summary. New case: test_an_empty_topics_header_declares_no_topic. With the old rule restored, it fails. - r4117299013 and r4117299033 (tests/test_plain_documents_fixture.py, T050's file, merged here from #53): its docstring said T052 and T054 had not landed. It also said the suite was run by node id outside validate's explicit list. T054 makes the first false. The second has been false since T036 (#52), when validate began running the whole suite. Both paragraphs, and the two phrases beside them ("a future topic-based grouping pass", "will require"), now say what is true. Only the docstring changes. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/neutral_projection.py | 11 +++++----- tests/test_neutral_projection.py | 16 ++++++++++++++ tests/test_plain_documents_fixture.py | 30 +++++++++++++-------------- 3 files changed, 36 insertions(+), 21 deletions(-) diff --git a/src/opendox/neutral_projection.py b/src/opendox/neutral_projection.py index b4e5cc1..a4eab45 100644 --- a/src/opendox/neutral_projection.py +++ b/src/opendox/neutral_projection.py @@ -33,9 +33,10 @@ the projection reports it, naming the document, the value and the six keys. So no other value can reach the snapshot, whose schema admits only the six. -THE TOPIC RULE. Every document carries `topics`. A document that declares a -`topics:` header carries the ones it lists, split on commas, with whitespace -collapsed and letters case-folded. A document that declares none carries the +THE TOPIC RULE. Every document carries `topics`. A document with a `topics:` +header carries exactly the topics it lists, split on commas, with whitespace +collapsed and letters case-folded, and none where it lists none: the header is +a declaration even when it is empty. A document without the header carries the words of its NAME, plus the words of the name of every other document it MENTIONS. Its name is its `title:`, else its first `#` heading, else its file name without the suffix. It mentions another document where its text holds @@ -318,8 +319,8 @@ def _read_documents(adapter: CorpusAdapter, corpus: ResolvedCorpus, f"its stage: value {declared!r} is not one of the six " f"station role keys ({', '.join(STAGE_ROLES)}), so it is " "not a declaration; the document is read as a source")) - listed_topics = _declared_topics(header.get(TOPICS_KEY) or "") - document.declared_topics = listed_topics or None + if TOPICS_KEY in header: # a declaration, even an empty one + document.declared_topics = _declared_topics(header[TOPICS_KEY]) document.name = (document.title or _first_heading(text) or document.stem) document.tokens = _tokens(text) diff --git a/tests/test_neutral_projection.py b/tests/test_neutral_projection.py index 2ccad05..72c9866 100644 --- a/tests/test_neutral_projection.py +++ b/tests/test_neutral_projection.py @@ -718,6 +718,22 @@ def test_declared_topics_replace_the_derived_ones(tmp_path: Path) -> None: assert violations(snapshot) == [] +def test_an_empty_topics_header_declares_no_topic(tmp_path: Path) -> None: + """A `topics:` header is a declaration even when it lists nothing: the + document carries no topic, and none is derived for it, so it joins no + group although its name shares words with two sources that do group.""" + root = _repository(tmp_path, files={ + "a.md": "title: Rain barrel\ntopics:\n\n.\n", + "b.md": "title: Rain barrel notes\n\n.\n", + "c.md": "title: Rain barrel log\n\n.\n", + }) + snapshot = _generate(root) + assert _by_path(snapshot)["a.md"]["topics"] == [] + [group] = snapshot["clusters"] + assert [e["document"] for e in group["document_edges"]] == ["b.md", "c.md"] + assert violations(snapshot) == [] + + def test_a_declared_group_gathers_the_sources_that_share_its_topics( tmp_path: Path) -> None: root = _repository(tmp_path, files={ diff --git a/tests/test_plain_documents_fixture.py b/tests/test_plain_documents_fixture.py index 6a0f23d..5a51349 100644 --- a/tests/test_plain_documents_fixture.py +++ b/tests/test_plain_documents_fixture.py @@ -5,10 +5,10 @@ 2, slice P2-F): a handful of `.md` documents a plain, ungoverned git repository could contain, read by AT-R1 (spec.md § "AT-R1 — the release-1 acceptance test", step 3(a)) and by F5.3, F7.2, F10.1 and F13.1 once those -falsifiers exist. None of it is wired to any `opendox` code yet — T052 and -T054 (the generator seam and the neutral projection) have not landed — so -this suite tests the fixture's own two guarantees rather than a projection -over it. +falsifiers exist. T054's neutral projection reads it: +`tests/test_neutral_projection.py` projects it with every sibling blocked and +validates the snapshot against T053's schema. This suite tests the fixture's +own two guarantees, which hold whatever reads it. THE TWO GUARANTEES, AND WHY. @@ -26,22 +26,20 @@ document that declares no `stage:` line at all is a SOURCE. This fixture carries one document per explicit station (`grouping`, `candidate`, `selection`, `submission`, `completion`) and three sources, two of which - share the phrase "rain barrel" verbatim so a future topic-based grouping - pass (T054) has a pair to find — the fixture must yield at least one + share the phrase "rain barrel" verbatim so T054's topic rule has a pair + to find — the fixture must yield at least one group, so AT-R1 can open the chat pane from a grouping tile (spec.md § AT-R1 steps 6-7). Every document also carries the small neutral field set the default adapter -will require regardless of station — `title` and `summary` — per T050's task -line and the answer's own example. - -NOT YET WIRED INTO `.github/workflows/validate.yml`'s explicit pytest list: -the phase-2 draft-ahead scope keeps this PR out of that file (conftest.py, -pyproject.toml, validate.yml and README.md are the phase-1 chain's), so this -suite runs by node id today, exactly as other narrowed-out suites in this -tree have (`tests/test_display_facet_leaves.py`'s own S7-residue history). -It joins the enumerated list whichever later task next touches it — most -likely F5.3/T056, or T049's own close. +requires regardless of station — `title` and `summary` +(`local_git_adapter.NEUTRAL_FIELDS`, T054) — per T050's task line and the +answer's own example. + +COLLECTED BY THE REQUIRED CHECK. Since T036, `validate` runs the whole suite +(`python -m pytest -q` over the configured testpaths) instead of an explicit +list of files, so this suite is collected like every other and needs no entry +anywhere. A CREATED file: no row in openxFactory's `docs/opendox-carve-manifest.yaml` (RULED OQ-C: the manifest declares what LEAVES openxFactory, never what a From 5a6fe637b20123fe6a1190702874a7ff563f0591 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sun, 27 Sep 2026 22:53:27 +0000 Subject: [PATCH 09/38] T054: a word is a run of letters beside digits too; a pin labels the snapshot (plan 034) Copilot's review of c0747e96 raised two points. - Its overview said digit-adjacent topic names were tokenized wrongly, and they were. The docstring defines a word as a run of three or more letters, but the tokenizer took letter-and-digit runs and then dropped any run holding a digit, so `Q3planning` lost `planning`. Letter runs and digit runs are now separate tokens. New case: test_a_word_is_a_run_of_letters_even_beside_digits. With the old tokenizer restored, it fails. - r4117331489 asks that a supplied source_revision be passed into the CorpusRef. That is not taken, and the answer is on the product's own contract: - the seam defines source_revision as the source anchor to pin, and the CLI's help for --source-revision says "pin the source_revision anchor"; - openXdox's governed generator records a supplied revision and scans the tree it is handed; - the holder ruled for T054 that content comes from the working tree, per Brett's T022 ruling. default_generator's docstring now says so in a paragraph of its own. Only the docstring changes for it. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/default_generator.py | 9 +++++++++ src/opendox/neutral_projection.py | 5 +++-- tests/test_neutral_projection.py | 13 +++++++++++++ 3 files changed, 25 insertions(+), 2 deletions(-) diff --git a/src/opendox/default_generator.py b/src/opendox/default_generator.py index 3d516ca..1c81476 100644 --- a/src/opendox/default_generator.py +++ b/src/opendox/default_generator.py @@ -26,6 +26,15 @@ could read as an option. It is left out where the lookup fails. Nothing reads the clock, so the same tree at the same anchors answers the same snapshot. +A PIN LABELS THE SNAPSHOT; IT DOES NOT CHOOSE THE BYTES. The content is the +working tree's, on Brett's T022 ruling ("Working tree (Recommended)"), which the +holder confirmed for T054. A supplied `source_revision` is recorded as the +anchor the caller asserts, exactly as openXdox's governed generator records one +and scans the tree it is handed (its sealed-artifact lane pins a revision for a +tree that is not a checkout at all). So the pin never reaches the `CorpusRef`, +and a pin that names no commit here is still recorded rather than refused. A +caller that wants the tree at an older commit checks that commit out first. + WHAT IT REPORTS. The projection's notices go to standard error, one line each, `notice: : `. A `stage:` value outside the six role keys is one (R1Q13 (a)): the line names the document, the value and diff --git a/src/opendox/neutral_projection.py b/src/opendox/neutral_projection.py index a4eab45..d0addd8 100644 --- a/src/opendox/neutral_projection.py +++ b/src/opendox/neutral_projection.py @@ -147,8 +147,9 @@ will with within without would yet you your yours yourself yourselves """.split()) -#: A run of letters or digits: what the topic rule reads as one token. -_TOKEN = re.compile(r"[^\W_]+") +#: A run of letters, or a run of digits: what the topic rule reads as one +#: token. The two are split apart, so `Q3planning` holds the word `planning`. +_TOKEN = re.compile(r"[^\W\d_]+|\d+") #: An ATX heading (`# Roadmap`, `## Budget ##`), its text in group 1. _ATX_HEADING = re.compile(r"^ {0,3}#{1,6}(?:[ \t]+(.*?))?(?:[ \t]+#+)?[ \t]*$") diff --git a/tests/test_neutral_projection.py b/tests/test_neutral_projection.py index 72c9866..fb74cc7 100644 --- a/tests/test_neutral_projection.py +++ b/tests/test_neutral_projection.py @@ -734,6 +734,19 @@ def test_an_empty_topics_header_declares_no_topic(tmp_path: Path) -> None: assert violations(snapshot) == [] +def test_a_word_is_a_run_of_letters_even_beside_digits(tmp_path: Path) -> None: + """A word is a run of three or more letters: `Q3planning` holds `planning`, + so it groups with a source whose name spells the word on its own.""" + snapshot = _generate(_repository(tmp_path, files={ + "a.md": "title: Q3planning review\n\n.\n", + "b.md": "title: planning review\n\n.\n", + })) + assert _by_path(snapshot)["a.md"]["topics"] == ["planning", "review"] + [group] = snapshot["clusters"] + assert group["topics"] == ["planning", "review"] + assert violations(snapshot) == [] + + def test_a_declared_group_gathers_the_sources_that_share_its_topics( tmp_path: Path) -> None: root = _repository(tmp_path, files={ From bce09c50911686197c6910149bb8bf05ce149fc7 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sun, 27 Sep 2026 23:15:04 +0000 Subject: [PATCH 10/38] T052: each registration keeps its own records at the generator seam Copilot's overview of openDox-code#57 at c0747e96 said that the generator seam's "replacement-generator tracking can reject valid registrations". It opened no thread. The claim is real, and there are two cases, both reproduced at 4ca45d27: 1. The count of generations under way was global, not tied to the registration it counted. A generation from a default that unregister() had dropped kept counting against whatever default was registered next. So a host was refused over a fresh default with "a snapshot is being generated from it now", although nothing was. 2. A generation that outlived its registration recorded its late answer against a fresh registration of the same declaration. That shut the fresh window, against unregister()'s own promise that the record of a generation goes with the registration. Now every change of registration goes through one helper: register(), register_default() where it registers, and unregister(). The helper moves a registration serial on and starts the two records afresh. A generation carries the serial it began under. When it ends, it touches the records only if that registration is still current. The generation is not stopped, and its caller still gets its snapshot. Tests: 66 -> 70 cases. The new case test_a_generation_that_outlives_its_registration_touches_no_later_one runs in four variants, in a process of its own. The follower is another default or the same declaration, and the host registers either while the generation runs or after it answers. Three variants are red against 4ca45d27's seam logic, and the fourth guards the new scoping. test_unregister_clears_the_default_and_its_generation now registers the same default afresh before the host tries. 15 of 15 mutations are caught. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/generator_seam.py | 89 ++++++++++++++++++++++------------- tests/test_generator_seam.py | 87 ++++++++++++++++++++++++++++++++-- 2 files changed, 139 insertions(+), 37 deletions(-) diff --git a/src/opendox/generator_seam.py b/src/opendox/generator_seam.py index a2c8184..a15aecf 100644 --- a/src/opendox/generator_seam.py +++ b/src/opendox/generator_seam.py @@ -110,6 +110,12 @@ is refused as `GeneratorAlreadyRegistered`. `unregister()` makes a deliberate swap explicit. +EACH REGISTRATION KEEPS ITS OWN RECORDS. A generation belongs to the +registration it began under. So a generation still under way when +`unregister()` drops that registration neither holds shut, nor closes, the +window of the registration that follows. That holds even for a later +registration of the same declaration. + ONE LOCK. The registration and its records are kept under one lock. `serve.py` answers each request on a thread of its own (a `ThreadingHTTPServer`), so a generation can run beside a registration. `generate()` holds the lock only for @@ -337,24 +343,45 @@ def _refuse_an_operation_that_cannot_take_the_call(self) -> None: #: (`register_default()`), rather than a host's own `register()`. _is_default: bool = False -#: Whether a snapshot has been generated from that default, meaning that its -#: operation has answered a snapshot the seam handed back. `generate()` sets it -#: once that snapshot has come back. A generation that failed, or whose answer -#: the seam refused, wrote nothing, and so records nothing. +#: Whether a snapshot has been generated from THIS registration of that +#: default, meaning that its operation has answered a snapshot the seam handed +#: back. `generate()` sets it once that snapshot has come back. A generation +#: that failed, or whose answer the seam refused, wrote nothing, and so records +#: nothing. _generated_from_default: bool = False -#: How many generations from that default are under way: begun, and not yet -#: answered or failed. While one is, a host's registration is refused, as it is -#: after one, because the snapshot being generated would come back after the -#: swap. `unregister()` leaves it alone: it counts calls that are still -#: running, and each one takes itself off when it ends. +#: How many generations from THIS registration of that default are under way: +#: begun, and not yet answered or failed. While one is, a host's registration +#: is refused, as it is after one, because the snapshot being generated would +#: come back after the swap. _default_generations_under_way: int = 0 -#: Guards the four above. It is held only for bookkeeping, and never across a +#: Which registration is current. Every change of registration moves it on: +#: `register()`, `register_default()` where it registers, and `unregister()`. +#: A generation carries the serial it began under, and when it ends it touches +#: the two records above only if that registration is still the current one. +#: So a change of registration starts both records afresh, and a generation +#: that outlived its registration records nothing against the next. +_registration_serial: int = 0 + +#: Guards the five above. It is held only for bookkeeping, and never across a #: generator's own call. _lock = threading.Lock() +def _begin_a_registration(generator: SnapshotGenerator | None, + is_default: bool) -> None: + """Make `generator` the registration (or none), with records of its own. + The one place the registration changes. The caller holds `_lock`.""" + global _registered, _is_default, _generated_from_default + global _default_generations_under_way, _registration_serial + _registration_serial += 1 + _registered = generator + _is_default = is_default + _generated_from_default = False + _default_generations_under_way = 0 + + def name_of(generator: Any) -> str: """A generator's most nameable name, with its contract, for a refusal. @@ -389,7 +416,6 @@ def register(generator: SnapshotGenerator) -> SnapshotGenerator: generated from it and none is being generated. It is refused once one has been, or while one is. Either way the host ends up holding the one registration, or knows why it does not.""" - global _registered, _is_default, _generated_from_default _require_a_declaration(generator, "register()") with _lock: held = _registered @@ -404,9 +430,7 @@ def register(generator: SnapshotGenerator) -> SnapshotGenerator: elif _generated_from_default: why = "a snapshot has already been generated from it" if not over_a_host and not why: - _registered = generator - _is_default = False - _generated_from_default = False + _begin_a_registration(generator, is_default=False) return generator # Named outside the lock: naming a generator can run its own code. if over_a_host: @@ -447,7 +471,6 @@ def register_default(generator: SnapshotGenerator) -> SnapshotGenerator: openDox's own generator to the neutral contract, so a declaration that writes another contract is refused as `GeneratorNotConformant`, whether or not anything is registered.""" - global _registered, _is_default, _generated_from_default _require_a_declaration(generator, "register_default()") if generator.contract != NEUTRAL_SNAPSHOT_KIND: raise GeneratorNotConformant( @@ -457,24 +480,22 @@ def register_default(generator: SnapshotGenerator) -> SnapshotGenerator: f"registered with {REGISTRATION_CALL}.") with _lock: if _registered is None: - _registered = generator - _is_default = True - _generated_from_default = False + _begin_a_registration(generator, is_default=True) return _registered def unregister() -> None: """Drop the registration, a host's or the entry point's default. - For test isolation and for a host tearing down. The record of a generation - from the default goes with it. A generation still under way is not stopped. - When it answers, it records its snapshot only if the same default is - registered at that moment.""" - global _registered, _is_default, _generated_from_default + For test isolation and for a host tearing down. The registration's records + go with it: whether a snapshot was generated from the default, and how many + generations from it are under way. A generation still under way is not + stopped, and its caller still gets its snapshot. But it belongs to the + registration that was dropped, so when it ends it records nothing against + whatever is registered next, even a later registration of the same + declaration.""" with _lock: - _registered = None - _is_default = False - _generated_from_default = False + _begin_a_registration(None, is_default=False) def is_registered() -> bool: @@ -534,6 +555,7 @@ def generate(repo_root: Path | str, repository: str, *, generator = current() undeclared = sorted(set(given) - set(generator.inputs)) from_default = _is_default and not undeclared + serial = _registration_serial if from_default: _default_generations_under_way += 1 if undeclared: @@ -553,20 +575,23 @@ def generate(repo_root: Path | str, repository: str, *, answered = True finally: if from_default: - _end_a_generation_from_the_default(generator, answered) + _end_a_generation_from_the_default(serial, answered) return snapshot -def _end_a_generation_from_the_default(generator: SnapshotGenerator, - answered: bool) -> None: +def _end_a_generation_from_the_default(serial: int, answered: bool) -> None: """Take a generation from the default off the count of those under way. If it answered a snapshot the seam handed back, record that one was generated. - The record is made only while the default it came from is still the one - registered, so after `unregister()` it records nothing against a host.""" + Both happen only while the registration it began under (`serial`) is still + the current one. A registration dropped since took its records with it. So + a generation that outlived its registration touches nothing, and records + nothing against what was registered after it.""" global _default_generations_under_way, _generated_from_default with _lock: + if serial != _registration_serial: + return _default_generations_under_way -= 1 - if answered and _registered is generator and _is_default: + if answered: _generated_from_default = True diff --git a/tests/test_generator_seam.py b/tests/test_generator_seam.py index 9f9ca4e..62c9e43 100644 --- a/tests/test_generator_seam.py +++ b/tests/test_generator_seam.py @@ -34,9 +34,11 @@ writes the neutral kind and takes no input. `register_default()` registers it only where nothing is, and holds it to the neutral contract. A host replaces it before a generation, and after one that wrote nothing. A host is - refused while a generation runs, and after one that wrote a snapshot. A - generator may register or generate from inside its own call without - deadlocking the seam. Until T054 lands, the default refuses, naming itself. + refused while a generation runs, and after one that wrote a snapshot. Each + registration keeps its own records: a generation that outlives its + registration records nothing against the next. A generator may register or + generate from inside its own call without deadlocking the seam. Until T054 + lands, the default refuses, naming itself. 7. EACH ENTRY POINT REGISTERS IT. `cli.build_parser()` and `cli.main()` run for real. `serve.build_server()` and `serve.main()` still cannot run in a lone checkout (research R7), so their registration is executed from their own @@ -98,14 +100,14 @@ def _isolated_registries(): point's default back as a host's. """ seam = (gs._registered, gs._is_default, gs._generated_from_default, - gs._default_generations_under_way) + gs._default_generations_under_way, gs._registration_serial) profile = (domain_profile._registered, domain_profile._is_default, domain_profile._built_from_default) home = corpus_adapter._home_factory gs.unregister() yield (gs._registered, gs._is_default, gs._generated_from_default, - gs._default_generations_under_way) = seam + gs._default_generations_under_way, gs._registration_serial) = seam (domain_profile._registered, domain_profile._is_default, domain_profile._built_from_default) = profile corpus_adapter._home_factory = home @@ -543,10 +545,13 @@ def test_asking_or_being_refused_is_not_a_generation(tmp_path) -> None: def test_unregister_clears_the_default_and_its_generation(tmp_path) -> None: + """The record goes with the registration. So the same default, registered + afresh, has a window of its own, and a host still replaces it.""" stand_in_default, _ = _declared(gs.NEUTRAL_SNAPSHOT_KIND) gs.register_default(stand_in_default) gs.generate(tmp_path, "fixture") gs.unregister() + assert gs.register_default(stand_in_default) is stand_in_default host, _ = _declared("host-snapshot") assert gs.register(host) is host @@ -719,6 +724,78 @@ def swapping(repo_root, repository, *, source_revision=None, "host-registered-over-the-replacement"] +#: A program for a fresh process. A generation from one registration of the +#: default is still under way when `unregister()` drops that registration, and +#: a default is registered after it. `FOLLOWING` says which default: another, +#: or the same declaration again. `WHEN` says when a host then registers: while +#: that generation still runs, or after it has answered. +_OUTLIVED = """ + import threading + from opendox import generator_seam as gs + FOLLOWING, WHEN = FOLLOWING_VALUE, WHEN_VALUE + KIND = gs.NEUTRAL_SNAPSHOT_KIND + started, release = threading.Event(), threading.Event() + + def slow(repo_root, repository, *, source_revision=None, generated_at=None): + started.set() + release.wait(30) + return {"schema_version": 1, "kind": KIND, "repository": repository} + + def quick(repo_root, repository, *, source_revision=None, generated_at=None): + return {"schema_version": 1, "kind": KIND, "repository": repository} + + def host_operation(repo_root, repository, *, source_revision=None, + generated_at=None): + return {"schema_version": 1, "kind": "host-snapshot"} + + dropped = gs.SnapshotGenerator(contract=KIND, generate=slow) + host = gs.SnapshotGenerator(contract="host-snapshot", generate=host_operation) + answers = [] + gs.register_default(dropped) + worker = threading.Thread( + target=lambda: answers.append(gs.generate(".", "outlived")["repository"])) + worker.start() + assert started.wait(30), "the generation never started" + gs.unregister() + gs.register_default(dropped if FOLLOWING == "the same declaration" + else gs.SnapshotGenerator(contract=KIND, generate=quick)) + + def try_the_host(): + try: + return "registered" if gs.register(host) is host else "not-registered" + except gs.GeneratorAlreadyRegistered: + return "refused" + + if WHEN == "while it runs": + result = try_the_host() + release.set() + worker.join(30) + assert not worker.is_alive(), "the generation never ended" + if WHEN == "after it answers": + result = try_the_host() + print(result, answers[0] if answers else "no-answer", + gs._default_generations_under_way) +""" + + +@pytest.mark.parametrize("following", ("another default", "the same declaration")) +@pytest.mark.parametrize("when", ("while it runs", "after it answers")) +def test_a_generation_that_outlives_its_registration_touches_no_later_one( + following: str, when: str) -> None: + """Each registration keeps its own records. A generation still under way + when `unregister()` drops its registration neither holds shut, nor closes, + the window of the registration that follows. That holds whether the + follower is another default or the same declaration registered again, and + whether a host registers while that generation still runs or after it has + answered. The generation is not stopped, and its caller still gets its + snapshot. The case runs in a process of its own (see `_fresh_process`).""" + program = (_OUTLIVED.replace("FOLLOWING_VALUE", repr(following)) + .replace("WHEN_VALUE", repr(when))) + done = _fresh_process(program) + assert done.returncode == 0, done.stderr + assert done.stdout.split() == ["registered", "outlived", "0"] + + def test_a_host_that_registers_the_default_itself_holds_a_hosts_registration() -> None: """Which kind a registration is, is set by the call that MADE it.""" gs.register(default_generator.GENERATOR) From 97b5b01fb29ea18e1edb456724a34a3e62c0942d Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sun, 27 Sep 2026 23:16:50 +0000 Subject: [PATCH 11/38] T052: leave point 6 of the seam test's docstring as T054 edits it T054's branch (openDox-code#57) stacks on this one and rewrites the last sentence of point 6 of tests/test_generator_seam.py's docstring. The previous commit re-wrapped that same paragraph, so taking this branch would have conflicted there. Point 6 is restored to its 4ca45d27 text, and the new sentence moves to point 5, ONE REGISTRATION, where it belongs. A trial merge of this head into #57's head 5a6fe637 is clean, and 94 seam and projection cases pass in the merged tree. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/test_generator_seam.py | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/tests/test_generator_seam.py b/tests/test_generator_seam.py index 62c9e43..f3876e4 100644 --- a/tests/test_generator_seam.py +++ b/tests/test_generator_seam.py @@ -29,16 +29,15 @@ undeclared input is refused before the call. A `None` input is not passed. Only a snapshot of the declared contract comes back. 5. ONE REGISTRATION. The same declaration twice is a no-op, and a second host - is refused. + is refused. Each registration keeps its own records: a generation that + outlives its registration records nothing against the next. 6. THE ENTRY POINTS' DEFAULT, in R1Q3 (a)'s pattern. openDox's own generator writes the neutral kind and takes no input. `register_default()` registers it only where nothing is, and holds it to the neutral contract. A host replaces it before a generation, and after one that wrote nothing. A host is - refused while a generation runs, and after one that wrote a snapshot. Each - registration keeps its own records: a generation that outlives its - registration records nothing against the next. A generator may register or - generate from inside its own call without deadlocking the seam. Until T054 - lands, the default refuses, naming itself. + refused while a generation runs, and after one that wrote a snapshot. A + generator may register or generate from inside its own call without + deadlocking the seam. Until T054 lands, the default refuses, naming itself. 7. EACH ENTRY POINT REGISTERS IT. `cli.build_parser()` and `cli.main()` run for real. `serve.build_server()` and `serve.main()` still cannot run in a lone checkout (research R7), so their registration is executed from their own From 8d507c5be2bbe9cc37d15a1106c368afd64a6598 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Sun, 27 Sep 2026 23:33:34 +0000 Subject: [PATCH 12/38] T052: current() reads the registration once Copilot's review of this PR at bce09c50 (r4117421102) found that current() read _registered twice: once for its None check, and once for its answer. A thread that unregistered between the two reads made it answer None after passing the check, against its SnapshotGenerator-or- refusal contract. generate() calls it while holding the seam's lock, so that path was safe, but a bare caller was not. current() now reads the registration into a local once, and answers that local. It still takes no lock of its own, because generate() holds the seam's lock, which is not re-entrant, when it calls it. Test: test_current_answers_the_registration_it_checked forces the interleaving deterministically. A line tracer drops the registration before every line of current() after its first. The test is red against 97b5b01f's seam logic ("current() answered None after passing its check") and green here. 16 of 16 mutations are caught, the new M16 among them. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/generator_seam.py | 12 +++++++++--- tests/test_generator_seam.py | 36 +++++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+), 3 deletions(-) diff --git a/src/opendox/generator_seam.py b/src/opendox/generator_seam.py index a15aecf..a4d62b5 100644 --- a/src/opendox/generator_seam.py +++ b/src/opendox/generator_seam.py @@ -509,8 +509,14 @@ def current() -> SnapshotGenerator: It answers what is registered: a host's generator, or the default an entry point registered. It never falls back to the default itself, and asking records nothing. Only a generation closes the default's window, while it - runs and once it has answered.""" - if _registered is None: + runs and once it has answered. + + It reads the registration ONCE. So a registration another thread drops + while it answers never makes it answer `None`: it answers the registration + it checked. It takes no lock of its own, because `generate()` calls it + while holding the seam's lock.""" + registered = _registered + if registered is None: raise GeneratorNotRegistered( "no snapshot generator is registered at openDox's generator seam " "(opendox.generator_seam), so there is nothing to generate a " @@ -525,7 +531,7 @@ def current() -> SnapshotGenerator: "dropped the registration since. A host that contributes its own " "generator registers it at process start with\n\n " + REGISTRATION_CALL + "\n\nbefore the first generation.") - return _registered + return registered def generate(repo_root: Path | str, repository: str, *, diff --git a/tests/test_generator_seam.py b/tests/test_generator_seam.py index f3876e4..7b32ef1 100644 --- a/tests/test_generator_seam.py +++ b/tests/test_generator_seam.py @@ -236,6 +236,42 @@ def test_generate_with_nothing_registered_refuses_as_the_seam(tmp_path) -> None: assert gs.is_registered() is False +def test_current_answers_the_registration_it_checked() -> None: + """`current()` reads the registration once. `generate()` calls it holding + the seam's lock, but a host or a test may call it bare while another thread + unregisters. So a registration dropped between its check and its answer + never makes it answer `None`: it answers the registration it checked. + + The interleaving is forced, deterministically. A line tracer drops the + registration before every line of `current()` after its first, which is + where another thread's `unregister()` could land.""" + declared, _ = _declared() + gs.register(declared) + lines = [] + + def inside_current(frame, event, arg): + if event == "line": + lines.append(frame.f_lineno) + if len(lines) > 1: + gs.unregister() + return inside_current + + def tracer(frame, event, arg): + return inside_current if frame.f_code is gs.current.__code__ else None + + previous = sys.gettrace() + sys.settrace(tracer) + try: + answer = gs.current() + finally: + sys.settrace(previous) + assert len(lines) > 1, "the tracer never reached a second line of current()" + assert gs.is_registered() is False, "the interleaved unregister() never ran" + assert answer is declared, ( + f"current() answered {answer!r} after passing its check, not the " + "registration it checked") + + # -------------------------------------------------------------------------- # 3 — the declaration is checked when it is made # -------------------------------------------------------------------------- From c27eac356dc1e93f8ea5cdf2b76367c76e112956 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Mon, 28 Sep 2026 18:04:11 +0000 Subject: [PATCH 13/38] T055: serve and generate standalone (5.5, 4.3 part) (plan 034) The snapshot registry and source, the corpus-root predicate, the snapshot writer and the validator lookup each get a declared seam (src/opendox/projection_seams.py) and an openDox default of their own (default_registry.py, default_projection.py), per R1Q10 (a). The four entry points register the defaults where no host has, in R1Q3 (a)'s pattern. With nothing registered, a seam refuses and names itself and its call (4.2). A host's registration replaces a default until a consumer has read it, and is refused after that. The generate verbs go through the seams. cli.py's _generate_and_write and _gate_snapshot, and the default source's regenerate, call generator_seam.generate() and look the generator up on each call. They write through the registered writer. A snapshot is validated by the validator registered for its own kind. The core /snapshot.json arm's four handlers (_query_key, _read_snapshot, _serve_snapshot, _hosted_entry_refused) and hosted_ref_refused are serve.py's own now, read over the registry seam, so a lone openDox builds a server and answers /snapshot.json, /capabilities and /source/. branch_session's _change_rows goes through the corpus-root seam, and is_rfc3339_datetime is openDox's own (rfc3339.py). The seams do not carry either of them. consumer_reach retires snapshot_registry, snapshot, corpus_root, generator, find_validator, corpus_root_refusal, generate_snapshot, is_rfc3339_datetime, hosted_ref_refused, scanned_roots, and the function/constant stand-ins. LateProjectionRoutes now forwards only _serve_index. workbench.validate_manifest asks the validator lookup for 'ideation-workbench'. The _default_home_factory docstrings in cli.py and serve.py now name NEUTRAL_FIELDS as the adapter's default required_fields. _report prints the neutral kind in place of a project that snapshot does not carry. The validator lookup's default for openDox's own kinds is a stand-in that concludes nothing and names T057. openDox-code#58 carries T057's validator; T058 wires it in. F4.1's scan falls from 19 deferred reaches to 11, all T084's. The census of consumer_reach sites falls from 65 to 29. Whole suite: 2684 passed, 11 skipped. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/branch_session.py | 64 +- src/opendox/cli.py | 277 +++-- src/opendox/cli_project.py | 5 +- src/opendox/consumer_reach.py | 263 +---- src/opendox/default_generator.py | 8 +- src/opendox/default_projection.py | 171 +++ src/opendox/default_registry.py | 590 +++++++++++ src/opendox/generator_seam.py | 11 +- src/opendox/projection_seams.py | 604 +++++++++++ src/opendox/rfc3339.py | 52 + src/opendox/serve.py | 284 +++-- src/opendox/serve_wire.py | 10 +- src/opendox/serve_workbench.py | 10 +- src/opendox/workbench.py | 44 +- tests/test_authoring_seam.py | 28 +- tests/test_consumer_reach.py | 202 +--- tests/test_doxbench_entrypoint.py | 125 +-- tests/test_generator_seam.py | 17 +- tests/test_neutral_projection.py | 5 +- tests/test_profile_registration.py | 34 +- tests/test_projection_seams.py | 1233 ++++++++++++++++++++++ tests/test_reach_sweep.py | 13 +- tests/test_route_handler_contribution.py | 41 +- tests/test_source_core_arm.py | 74 +- 24 files changed, 3396 insertions(+), 769 deletions(-) create mode 100644 src/opendox/default_projection.py create mode 100644 src/opendox/default_registry.py create mode 100644 src/opendox/projection_seams.py create mode 100644 src/opendox/rfc3339.py create mode 100644 tests/test_projection_seams.py diff --git a/src/opendox/branch_session.py b/src/opendox/branch_session.py index 945368f..5604cc7 100644 --- a/src/opendox/branch_session.py +++ b/src/opendox/branch_session.py @@ -88,6 +88,13 @@ # instead (`defaults.py`), and openXdox-code's drift guard holds the two literals # together. from .defaults import DEFAULT_RECORDS_DIR +# THE SNAPSHOT REGISTRY AND THE CORPUS-ROOT PREDICATE, THROUGH THEIR SEAMS (plan +# 034 T055). A session's entry, its snapshot's regenerate and the corpus's +# change rows were deferred imports of openXdox's `snapshot_registry` and +# `generator`. Each is now read from the seam registered at the moment it is +# used, openDox's own where no host has contributed one. Stdlib only, so the +# import adds no edge. +from . import projection_seams # `SessionGitRefused` is re-exported (see __all__) so a caller catching session # refusals can catch both classes from one module: this module refuses on # identity/shape, `session_git` refuses on git discipline (a stage-everything @@ -1564,15 +1571,13 @@ def _repo_reference(root: Path, path: Path) -> str: def _change_rows(checkout_root: Path | str): - """The shared active/archive enumeration plus declared staged origin.""" - from openxdox import generator - - root = Path(checkout_root) - return tuple( - (change_id, status, folder, *generator.declared_origin_state(folder)) - for change_id, status, folder, _archive_date - in generator.iter_changes(root) - ) + """The shared active/archive enumeration plus declared staged origin: + `(change id, status, folder, origin state, origin)` per change, as the + REGISTERED corpus-root predicate enumerates the corpus's changes + (`projection_seams.corpus_root`, plan 034 T055). openDox's own corpus + declares none, so it answers no row.""" + return tuple(projection_seams.corpus_root.current().change_rows( + Path(checkout_root))) def _active_pick_fallbacks( @@ -2101,8 +2106,11 @@ def session_entry(repository: str, branch: str, worktree: Path | str, *, `tile` records WHOSE session this is (finding 6). The ref cannot answer it — a `-2` branch is one tile's first session and another's second — so the caller - that OPENED the session, the only place the answer exists, states it here.""" - from openxdox.snapshot_registry import SnapshotEntry # lazy: keeps the import graph flat + that OPENED the session, the only place the answer exists, states it here. + + The entry type is the REGISTERED registry's (`projection_seams.registry`, + plan 034 T055), so the entry fits the registry it is registered in.""" + SnapshotEntry = projection_seams.registry.current().SnapshotEntry # noqa: N806 return SnapshotEntry(repository=repository, ref=branch, source_root=Path(worktree), @@ -2148,7 +2156,8 @@ def register_session_entry(registry: Any, *, repository: str, branch: str, (or the bootstrap's marker read) is where the branch point is known, and the chat-turn binding check reads it back off the entry. None degrades to the original name-equality binding — advisory, never a refusal.""" - from openxdox.snapshot_registry import entry_from_snapshot_file + entry_from_snapshot_file = ( + projection_seams.registry.current().entry_from_snapshot_file) snapshot_path = session_snapshot_path(checkout_root, branch) snapshot_path.parent.mkdir(parents=True, exist_ok=True) @@ -2224,15 +2233,20 @@ def refresh_session_snapshot(registry: Any, *, repository: str, branch: str, The ACTIVE entry is restored afterwards: `_regenerate` promotes what it regenerates, and a session snapshot must never become what the wheel, the - funnel, and the pipeline board render (FR-014a).""" - from openxdox.snapshot_registry import BINDING_REGENERATE, SnapshotSource + funnel, and the pipeline board render (FR-014a). + + The source is the REGISTERED registry's (`projection_seams.registry`, plan + 034 T055), so its regenerate runs the registered generator and writes + through the registered writer, as the serve's own refresh does.""" + registry_mod = projection_seams.registry.current() - source = SnapshotSource(checkout_root=Path(worktree), generator=generator, - project_register=project_register) + source = registry_mod.SnapshotSource( + checkout_root=Path(worktree), generator=generator, + project_register=project_register) # the shared registry IS the source's registry: the entry regenerated is the # one liveness is keyed on, never a copy of it source.registry = registry - if source.refresh_binding != BINDING_REGENERATE: + if source.refresh_binding != registry_mod.BINDING_REGENERATE: raise SessionRefused( f"the session worktree {worktree} is not a directory this process can " "regenerate from, so the session snapshot cannot be generated from the " @@ -3569,20 +3583,20 @@ def refresh_main_view(registry: Any, *, repository: str, does for the worktree, so this is a CALLER of the one refresh mechanism and not a second one. Returns None when there is nothing regenerable (a served plane, a registry with no local `main` entry) — an absent shared snapshot is not an - error, and it must never be the reason a session cannot end.""" - from openxdox.snapshot_registry import ( - BINDING_REGENERATE, DEFAULT_REF, SnapshotSource, - ) + error, and it must never be the reason a session cannot end. The source is + the REGISTERED registry's (`projection_seams.registry`, plan 034 T055).""" + registry_mod = projection_seams.registry.current() - target = ref or DEFAULT_REF + target = ref or registry_mod.DEFAULT_REF getter = getattr(registry, "get", None) entry = getter(repository, target) if getter is not None else None if entry is None or getattr(entry, "snapshot_path", None) is None: return None - source = SnapshotSource(checkout_root=Path(checkout_root), generator=generator, - project_register=project_register) + source = registry_mod.SnapshotSource( + checkout_root=Path(checkout_root), generator=generator, + project_register=project_register) source.registry = registry - if source.refresh_binding != BINDING_REGENERATE: + if source.refresh_binding != registry_mod.BINDING_REGENERATE: return None return _preserving_active( registry, lambda: source.refresh(repository=repository, ref=target)) diff --git a/src/opendox/cli.py b/src/opendox/cli.py index cf4b9ed..116ac96 100644 --- a/src/opendox/cli.py +++ b/src/opendox/cli.py @@ -17,6 +17,7 @@ from __future__ import annotations import argparse +import json import os import sys import tempfile @@ -72,7 +73,6 @@ from opendox import consumer_reach # noqa: E402 gate_mod = consumer_reach.gate_console # noqa: E402 from opendox import serve as serve_mod # noqa: E402 -snapshot_mod = consumer_reach.snapshot # noqa: E402 from opendox import workbench as workbench_mod # noqa: E402 # THE HOME-CORPUS SEAM'S DEFAULT (4.1a; plan 034 T022) -- see # `_default_home_factory` and `corpus_adapter.register_default_home(...)` @@ -86,17 +86,18 @@ from opendox.boundary import ( # noqa: E402 BoundaryViolation, HumanGate, OutputBoundary, ) -# THE CORPUS-ROOT PREDICATE AND THE SNAPSHOT GENERATOR, NAMED LATE (BUILD slice -# 2b). Both statements named `openxdox` — the layer that PINS openDox — in an -# IMPORT, so `import opendox.cli` required the consumer to be installed, which -# `design.md`:243 refuses: *"what must not survive is the direction, not the -# calls."* The four names bind to `consumer_reach` stand-ins instead, in the -# idiom `gate_mod` and `snapshot_mod` above already use. Each resolves on first -# CALL and refuses naming the layering; every call site below is unchanged -# (`corpus_root_refusal` :141, `is_rfc3339_datetime` :167, `generate_snapshot` -# :195 and :512). -corpus_root_refusal = consumer_reach.corpus_root_refusal # noqa: E402 -generate_snapshot = consumer_reach.generate_snapshot # noqa: E402 +# THE CORPUS-ROOT PREDICATE, THE SNAPSHOT WRITER AND THE VALIDATOR LOOKUP, AS +# SEAMS (plan 034 T055; #1144 5.5 and 4.3 in part; R1Q10 (a), openxFactory#656 +# comment 5850003126). BUILD slice 2b bound them, and the snapshot generator, +# to late `consumer_reach` stand-ins over openXdox's `corpus_root`, `snapshot` +# and `generator`, so a lone openDox refused at the first generate. Each is now +# read from a declared seam (`opendox.projection_seams`, and the generator seam +# for the generate operation), at the moment it is used, where either openDox's +# own default or a host's contribution is registered. The entry points below +# register the defaults where no host has. `is_rfc3339_datetime` is not a seam: +# it is the neutral contract's own date-time rule, and openDox owns it. +from opendox import projection_seams # noqa: E402 +from opendox.rfc3339 import is_rfc3339_datetime # noqa: E402 # THE COMPOSITION POINT, BOUND AT LAST (§ 4.3; RULED ASK-2 option (2), # openxFactory#656 comment 5628886636). `build_parser()` below reads @@ -132,12 +133,6 @@ # registers it at (5.4; plan 034 T052). Importing either registers nothing: # `build_parser()` and `main()` below register it, only where no host has. from opendox import default_generator, generator_seam # noqa: E402 -is_rfc3339_datetime = consumer_reach.is_rfc3339_datetime # noqa: E402 -# ...and `SCANNED_ROOTS` keeps its NAME and its behaviour, not just its value: -# :239 iterates it (`for root in SCANNED_ROOTS`) and that line is not one the -# carve manifest declares, so the read cannot be respelled and the name has to -# go on behaving like the tuple it was. `_LateConsumerValue` is why it can. -SCANNED_ROOTS = consumer_reach.scanned_roots # noqa: E402 # THE MODULES THE § 2.4 SPLIT CREATED ARE NAMED RELATIVELY, and they are the # only imports in this file that are. @@ -172,7 +167,8 @@ class RepoRootRefused(Exception): - """`--repo-root` does not name a corpus checkout (`corpus_root.corpus_scan_defect`). + """`--repo-root` does not name a corpus checkout, as the REGISTERED + corpus-root predicate decides (`projection_seams.corpus_root`). Raised from the ONE shared generation chokepoint, BEFORE any scan and before the `OutputBoundary` write, so no snapshot file exists to be mistaken for a @@ -182,7 +178,8 @@ class RepoRootRefused(Exception): def _refuse_non_corpus_repo_root(args: argparse.Namespace) -> None: """Raise `RepoRootRefused` unless `--repo-root` could be a corpus checkout.""" - refusal = corpus_root_refusal(args.repo_root, shape=_GENERATE_SHAPE) + refusal = projection_seams.corpus_root.current().corpus_root_refusal( + args.repo_root, shape=_GENERATE_SHAPE) if refusal is not None: raise RepoRootRefused(refusal) @@ -197,16 +194,18 @@ class GeneratedAtRefused(Exception): scanned tree cannot supply it (the sealed source artifact `add-nightly-dashboard-refresh` hands the child is not a git checkout). So degrading here would drop the anchor in silence and produce the very - snapshot the flag exists to prevent: `generated_at` is OPTIONAL in - `contracts/schemas/ideation-dashboard-snapshot.schema.yaml`, so even - `--strict` would pass, the image would ship, and the served plane would lose - its freshness stamp with nothing anywhere saying why.""" + snapshot the flag exists to prevent: `generated_at` is OPTIONAL in the + snapshot contracts (the neutral `opendox-snapshot` schema's and the + governed one's alike), so even `--strict` would pass, the image would ship, + and the served plane would lose its freshness stamp with nothing anywhere + saying why.""" def _refuse_malformed_generated_at(args: argparse.Namespace) -> None: """Raise `GeneratedAtRefused` unless `--generated-at`, when given, is an - RFC 3339 date-time (`generator.is_rfc3339_datetime` — the shape the snapshot - schema declares for `generation.generated_at`).""" + RFC 3339 date-time (`opendox.rfc3339.is_rfc3339_datetime`, the neutral + snapshot contract's `generated-at-is-rfc3339` rule for + `generation.generated_at`).""" value = getattr(args, "generated_at", None) if value is None or is_rfc3339_datetime(value): return @@ -230,13 +229,22 @@ def _generate_and_write(args: argparse.Namespace, output: Path) -> tuple[dict, P A `--repo-root` that cannot be a corpus checkout is REFUSED here — the ONE guard both verbs pass through, ahead of the generation and the write, because an empty snapshot that exits 0 is indistinguishable from an honest one (T092; - see `corpus_root.corpus_scan_defect`). A malformed `--generated-at` is - refused in the same place and for the same reason, one anchor over: both - verbs, ahead of the write, so no snapshot file can survive a refused run.""" + the registered corpus-root predicate decides). A malformed `--generated-at` + is refused in the same place and for the same reason, one anchor over: both + verbs, ahead of the write, so no snapshot file can survive a refused run. + + THE GENERATION AND THE WRITE GO THROUGH THE SEAMS (plan 034 T055). The + snapshot is generated by whichever generator is registered NOW + (`generator_seam.generate`, which looks it up on each call), and it is + written by the registered writer. An option given as `None` is not passed, + so an unset `--project-register` or `--possibles` asks nothing of a + generator that declares no such input. A given one that the registered + generator does not declare is refused as `GeneratorInputRefused`, before + anything is generated or written, and `main` reports it.""" _refuse_non_corpus_repo_root(args) _refuse_malformed_generated_at(args) repo_root = Path(args.repo_root).resolve() - snapshot = generate_snapshot( + snapshot = generator_seam.generate( repo_root, args.repository, source_revision=args.source_revision, @@ -245,16 +253,25 @@ def _generate_and_write(args: argparse.Namespace, output: Path) -> tuple[dict, P possibles_source=Path(args.possibles).resolve() if args.possibles else None, ) boundary = OutputBoundary(output.parent, [output.name]) - written = snapshot_mod.write_snapshot(snapshot, output, boundary) + written = projection_seams.writer.current().write_snapshot( + snapshot, output, boundary) return snapshot, written def _report(snapshot: dict, written: Path, repo_root: Path) -> None: + """What a generate verb says it wrote. + + The neutral snapshot (`opendox-snapshot`) has no project and no project + group, so its line names its kind instead, rather than reporting a grouping + its contract does not carry. Every other kind's line is unchanged.""" stats = _stats(snapshot) print(f"wrote {written}") - print(f" repository={snapshot['repository']} " - f"project={snapshot.get('project', '')} " - f"project_group={snapshot.get('project_group', '')}") + if snapshot.get("kind") == generator_seam.NEUTRAL_SNAPSHOT_KIND: + print(f" repository={snapshot['repository']} kind={snapshot['kind']}") + else: + print(f" repository={snapshot['repository']} " + f"project={snapshot.get('project', '')} " + f"project_group={snapshot.get('project_group', '')}") print(f" source_revision={snapshot['generation']['source_revision']}") # Printed even when absent: a missing freshness stamp used to be invisible # (the schema makes it optional, so nothing downstream complains), and a run @@ -277,84 +294,107 @@ def _warn_on_empty_projection(stats: dict[str, int], repo_root: Path) -> None: `ideation/` exists but holds nothing passes it, and the dashboard then renders the same empty funnel it renders for an honest one, over copy that reads as a legitimate result. So the emptiness is stated on stderr, with the roots that - let the path through, and the human decides.""" + let the path through, and the human decides. + + The roots are the REGISTERED corpus-root predicate's (`SCANNED_ROOTS`). + openDox's own predicate names none, since its corpus is a whole repository, + and then the line says what it did accept instead.""" if stats["documents"]: return - present = ", ".join(f"{root}/" for root in SCANNED_ROOTS + roots = tuple(projection_seams.corpus_root.current().SCANNED_ROOTS) + present = ", ".join(f"{root}/" for root in roots if (repo_root / root).is_dir()) print(f" WARNING: ZERO documents were projected from {repo_root} — this " f"snapshot is EMPTY", file=sys.stderr) - print(f" it was accepted as a corpus checkout because it holds {present}, " - f"but nothing under those roots produced a governed document", - file=sys.stderr) + if present: + print(f" it was accepted as a corpus checkout because it holds {present}, " + f"but nothing under those roots produced a governed document", + file=sys.stderr) + else: + print(" it was accepted as a corpus checkout, but nothing in it was " + "read as a document", file=sys.stderr) print(" an empty corpus is legal, so this is a WARNING, not a failure — but " "the usual cause is a --repo-root naming the wrong tree, and the " "dashboard's empty funnel reads the same either way", file=sys.stderr) -def _locate_validator(written: Path, repo_root: Path) -> Path | None: - """The pinned validator, searched from the OUTPUT path and then from the - SERVED CHECKOUT (T092 acceptance sweep, defect 8). - - `snapshot.find_validator` walks UP from where it is started, and `_validate` - started it only at the output file's directory. `generate-and-open` defaults - its run dir to `tempfile.mkdtemp()`, so on the DOCUMENTED human launch the - search began in /tmp, no ancestor there ever holds an aggregation checkout, - and every such run printed "validation SKIPPED — no reachable openxFactory - checkout" and served an unvalidated snapshot. SC-002's fail-loud validation - therefore never fired for a real user, and the message blamed the one thing - that WAS present: `--repo-root` is by definition the openxFactory checkout - being rendered, and it carries the validator. - - The output path is still tried FIRST, so a run that deliberately writes - beside a different aggregation checkout keeps using that one; `--repo-root` - is the fallback that makes the ordinary launch validate.""" - return (snapshot_mod.find_validator(written.parent) - or snapshot_mod.find_validator(repo_root)) +def _written_kind(written: Path) -> str | None: + """The `kind` the written snapshot declares, which chooses its validator, + or None where the file declares none it can be read by.""" + try: + document = json.loads(written.read_text(encoding="utf-8")) + except (OSError, UnicodeDecodeError, ValueError): + return None + kind = document.get("kind") if isinstance(document, dict) else None + return kind if isinstance(kind, str) and kind else None + + +def _validate_by_kind(written: Path, kind: str, *, strict: bool, + search_from: tuple[Path, ...]): + """`(validator, result)` for the written snapshot, by its KIND + (`projection_seams.validators`, plan 034 T055). + + The validator registered for the snapshot's own kind runs it, and none + other: its generator's declared contract is that kind, so openDox's own + validator checks openDox's neutral snapshot and a host's checks the host's. + It is handed the two roots a search may start from, the OUTPUT path's + directory first and the SERVED CHECKOUT second (T092 acceptance sweep, + defect 8, which is why the order is kept). No validator registered for the + kind is VALIDATOR UNAVAILABLE, sub-case "nothing to run", and the lookup's + own refusal is the reason given.""" + try: + validator = projection_seams.validators.for_kind(kind) + except projection_seams.ValidatorNotRegistered as exc: + return None, projection_seams.ValidationResult( + False, -1, "", "", None, projection_seams.VALIDATOR_UNAVAILABLE, + str(exc).split("\n", 1)[0]) + return validator, validator.validate(written, strict=strict, + search_from=search_from) -def _warn_validator_not_found(written: Path, repo_root: Path) -> None: +def _warn_validator_not_found(written: Path, repo_root: Path, kind: str, + result) -> None: """VALIDATOR UNAVAILABLE, sub-case "nothing to run". - NOT routine, and — now that `_locate_validator` falls back to `--repo-root` - — no longer the ordinary launch's fate either. Both roots were searched, so - the message names BOTH and blames neither on its own: reaching here means no - aggregation checkout is reachable from the OUTPUT path OR from the served - checkout, and the snapshot went unvalidated however good the corpus was. The - old one-liner ("no reachable openxFactory checkout") read as routine while - quietly meaning "unvalidated", and pointed at a checkout that was present and - fine — which is exactly where it sent the T092 pass.""" + NOT routine. Both roots were offered to the search, so the message names + BOTH and blames neither on its own, and then gives the lookup's own reason: + reaching here means no validator for this snapshot's kind could be reached + from the OUTPUT path OR from the served checkout, and the snapshot went + unvalidated however good the corpus was. The old one-liner ("no reachable + openxFactory checkout") read as routine while quietly meaning + "unvalidated", and pointed at a checkout that was present and fine — which + is exactly where it sent the T092 pass.""" print(" validation SKIPPED — this snapshot was NOT checked against the " "pinned schema", file=sys.stderr) - print(f" no {snapshot_mod.VALIDATOR_RELPATH} exists above " - f"{written.parent} (the OUTPUT path, searched first) or above " + print(f" no validator for kind {kind!r} was reachable from " + f"{written.parent} (the OUTPUT path, searched first) or from " f"{repo_root} (--repo-root, the fallback)", file=sys.stderr) - print(" render a checkout that sits inside an aggregation checkout, " - "or write the snapshot into one (--output on generate, --run-dir " - "on generate-and-open), to have it validated", file=sys.stderr) + if result.unavailable_reason: + print(f" {result.unavailable_reason}", file=sys.stderr) -def _warn_validator_could_not_run(result) -> None: +def _warn_validator_could_not_run(result, validator) -> None: """VALIDATOR UNAVAILABLE, sub-case "found it, could not run it". The validator's OWN words are relayed verbatim rather than paraphrased: it is the thing that knows which dependency it wanted, and quoting it keeps this warning correct when that message changes. What we add is the part it cannot know — WHICH interpreter it was run under (a separate `sys.executable` - process, so the libraries have to exist wherever the dashboard runs, not - wherever openxFactory is developed), the remedy, and the reassurance that - the corpus is not the accused.""" + process, so the libraries have to exist wherever the dashboard runs), the + remedy the registered validator declares (`dependency_remedy`), and the + reassurance that the corpus is not the accused.""" print(" validation SKIPPED — this snapshot was NOT checked against the " "pinned schema", file=sys.stderr) print(f" the validator was found ({result.validator}) but could not run: " f"{result.unavailable_reason}", file=sys.stderr) for line in (result.stderr or result.stdout).strip().splitlines()[-10:]: print(f" {line}", file=sys.stderr) - print(f" it runs under {sys.executable} — a SEPARATE interpreter from " - f"whatever installed openxFactory — and the usual cause is that this " - f"one lacks its libraries. Remedy:", file=sys.stderr) - print(f" {sys.executable} -m {snapshot_mod.DEPENDENCY_REMEDY}", - file=sys.stderr) + remedy = getattr(validator, "dependency_remedy", None) + if remedy: + print(f" it runs under {sys.executable} — a SEPARATE interpreter " + f"from whatever installed the validator — and the usual cause is " + f"that this one lacks its libraries. Remedy:", file=sys.stderr) + print(f" {sys.executable} -m {remedy}", file=sys.stderr) def _report_non_conformance(written: Path, result) -> None: @@ -372,7 +412,9 @@ def _report_non_conformance(written: Path, result) -> None: def _validate(written: Path, args: argparse.Namespace, *, continues: str = "this command continues and exits 0") -> int: - """Post-render validation, with THREE outcomes (see `snapshot.VALIDATED`). + """Post-render validation, with THREE outcomes + (`projection_seams.VALIDATED`, `NOT_CONFORMANT`, `VALIDATOR_UNAVAILABLE`), + by the validator registered for the written snapshot's own KIND. A snapshot the validator REJECTS still fails the command. A validator that could not RUN warns loudly and returns 0 — for `generate-and-open` because a @@ -382,21 +424,26 @@ def _validate(written: Path, args: argparse.Namespace, *, file is written either way, the environment is what failed, and one policy across both verbs is one thing to explain. `--strict` overrides that in both — asking for strictness and getting "we skipped the check" would make the - flag a lie. `validate_or_raise` is untouched and still raises: that is the - generator's deliberate fail-loud path, and it is chosen by code, not by a - human waiting on a browser tab.""" + flag a lie. A written snapshot that declares no kind cannot choose a + validator, and it is not conformant either: every snapshot says which + contract it is.""" if args.no_validate: print(" validation skipped (--no-validate)") return 0 repo_root = Path(args.repo_root).resolve() - validator = _locate_validator(written, repo_root) - result = snapshot_mod.validate_snapshot(written, strict=args.strict, - validator=validator) + kind = _written_kind(written) + if kind is None: + print(f" validation FAILED — {written} declares no kind, so no " + f"validator can be chosen for it, and a snapshot that does not " + f"say which contract it is conforms to none.", file=sys.stderr) + return 1 + validator, result = _validate_by_kind( + written, kind, strict=args.strict, search_from=(written.parent, repo_root)) if not result.available: if result.validator is None: - _warn_validator_not_found(written, repo_root) + _warn_validator_not_found(written, repo_root, kind, result) else: - _warn_validator_could_not_run(result) + _warn_validator_could_not_run(result, validator) if args.strict: print(" --strict was given and it means what it says: a run that " "COULD NOT be validated FAILS rather than continuing " @@ -551,9 +598,10 @@ def _gate_actor(repo_root: Path, args: argparse.Namespace) -> str: def _gate_snapshot(args: argparse.Namespace) -> tuple[Path, dict]: """Regenerate the snapshot the gate action plans against (the same - deterministic generation the dashboard reads).""" + deterministic generation the dashboard reads), through the generator seam + as `_generate_and_write` does (plan 034 T055).""" repo_root = Path(args.repo_root).resolve() - snapshot = generate_snapshot( + snapshot = generator_seam.generate( repo_root, args.repository, source_revision=args.source_revision, project_register_source=Path(args.project_register).resolve() if args.project_register else None, possibles_source=Path(args.possibles).resolve() if args.possibles else None) @@ -628,10 +676,10 @@ def _session_registry(checkout_root: Path, repository: str | None): own remedy on stderr, and this function deletes nothing. This is the ONE place a CLI verb obtains a registry — every session-bearing - verb this feature adds goes through it.""" - from openxdox.snapshot_registry import SnapshotRegistry - - registry = SnapshotRegistry() + verb this feature adds goes through it. The registry is the REGISTERED + snapshot registry's (`projection_seams.registry`, plan 034 T055): openDox's + own where no host has contributed one.""" + registry = projection_seams.registry.current().SnapshotRegistry() report = branch_session_mod.bootstrap_sessions( registry, repository=repository or "", checkout_root=Path(checkout_root)) for note in report.stale: @@ -873,11 +921,12 @@ def _add_generate_args(sub: argparse.ArgumentParser) -> None: def _default_home_factory(root): """`home_corpus`'s shape (`adapter, ref = factory(root)`), over - `WorkingTreeCorpus` at its own bare defaults (`required_fields=()`; phase - 2's T054 sets the neutral fields R1Q13 decides). Its `__init__` takes no - root -- it is root-agnostic, and `resolve(ref)` reads `ref.location` -- - so one `CorpusRef` per call carries the root this factory was given, and - the adapter itself needs none. + `WorkingTreeCorpus` at its own defaults, whose `required_fields` is the + small neutral field set R1Q13 (a) decides (`NEUTRAL_FIELDS`, `title` and + `summary`, since plan 034 T054). Its `__init__` takes no root -- it is + root-agnostic, and `resolve(ref)` reads `ref.location` -- so one + `CorpusRef` per call carries the root this factory was given, and the + adapter itself needs none. READS THE WORKING TREE, uncommitted edits included -- RULING, Brett Heap, 2026-09-27, via the holder: "Working tree (Recommended)". A standalone @@ -958,6 +1007,11 @@ def build_parser(*, subcommand_extensions: tuple = ()) -> argparse.ArgumentParse # AND openDox's OWN snapshot generator (5.4, T052; R1Q10 (a), in the same # R1Q3 (a) pattern), registered only where no host has contributed one. generator_seam.register_default(default_generator.GENERATOR) + # AND openDox's OWN snapshot registry and source, corpus-root predicate, + # writer and validators (5.5, T055; the same ruling and pattern), each + # only where no host has registered its own. Registering reads nothing, so + # a host that registers after this parser is built still replaces them. + projection_seams.register_defaults() parser = argparse.ArgumentParser(prog="ideation-dashboard", description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) sub = parser.add_subparsers(dest="command", required=True) @@ -1063,6 +1117,8 @@ def main(argv: list[str] | None = None, *, corpus_adapter.register_default_home(_default_home_factory) # AND openDox's own snapshot generator (5.4, T052), the same way. generator_seam.register_default(default_generator.GENERATOR) + # AND openDox's own projection defaults (5.5, T055), the same way. + projection_seams.register_defaults() args = build_parser( subcommand_extensions=subcommand_extensions).parse_args(argv) try: @@ -1081,11 +1137,24 @@ def main(argv: list[str] | None = None, *, print(str(exc), file=sys.stderr) return 1 except RepoRootRefused as exc: - # The refusal is the whole message (`corpus_root.corpus_root_refusal`); - # stderr and a non-zero status, so a wrapper script cannot mistake a - # refused run for a generated snapshot. + # The refusal is the whole message (the registered corpus-root + # predicate's `corpus_root_refusal`); stderr and a non-zero status, so + # a wrapper script cannot mistake a refused run for a generated + # snapshot. print(str(exc), file=sys.stderr) return 1 + except (generator_seam.GeneratorSeamError, + projection_seams.ProjectionSeamError, + corpus_adapter.CorpusRefused) as exc: + # A SEAM'S REFUSAL, reported like every refusal above (plan 034 + # T055): the generator seam's (an input the registered generator does + # not declare, or a projection that cannot be made, such as a checkout + # with neither a pinned nor a resolvable revision), the projection + # seams', and the home corpus's own refusal of the tree it was handed. + # Each is raised before anything is written, so the run leaves no + # snapshot to be mistaken for a result. + print(f"{_command_label(args)} refused: {exc}", file=sys.stderr) + return 1 # --------------------------------------------------------------------------- diff --git a/src/opendox/cli_project.py b/src/opendox/cli_project.py index b64737b..5b3520d 100644 --- a/src/opendox/cli_project.py +++ b/src/opendox/cli_project.py @@ -22,8 +22,9 @@ time and relative to this module's own package, never a module-level `from ideation_dashboard.cli import _report`. A frozen reference would still work and would silently stop honouring the module-level patch sites -the existing tests rely on (`cli_mod._generate_and_write`, -`cli_mod._locate_validator` behind `_validate`) — green, and wrong. The +a test replaces (`cli_mod._generate_and_write`, or `cli_mod._validate_by_kind` +behind `_validate`, which replaced `_locate_validator` at plan 034 T055) — +green, and wrong. The accessor also keeps the two modules importable in either order. """ diff --git a/src/opendox/consumer_reach.py b/src/opendox/consumer_reach.py index b55a33e..a552c2a 100644 --- a/src/opendox/consumer_reach.py +++ b/src/opendox/consumer_reach.py @@ -155,117 +155,6 @@ def __repr__(self) -> str: return f"" -class _LateConsumerCallable: - """One FUNCTION of a consumer module, resolved on first call. - - The module stand-in above cannot serve a `from import ` - site: that form binds the function object itself, and the call sites keep - spelling it as a bare name. This binds a callable that resolves the module, - fetches the attribute and forwards — so the call site is unchanged and the - import is gone. - - Deliberately NOT a `functools.wraps` of the target: wrapping would have to - resolve the consumer at construction time, which is the import-time reach - this class exists to remove. - """ - - __slots__ = ("_module", "_attr") - - def __init__(self, module: _LateConsumerModule, attr: str) -> None: - self._module = module - self._attr = attr - - @property - def name(self) -> str: - return f"{self._module.name}.{self._attr}" - - def __call__(self, *args: Any, **kwargs: Any) -> Any: - return getattr(self._module.resolve(), self._attr)(*args, **kwargs) - - def __repr__(self) -> str: - return f"" - - -class _LateConsumerValue: - """One CONSTANT of a consumer module, resolved on first use. - - THE NARROWEST MEMBER OF THIS FAMILY, and the one that needs an argument for - existing at all. A constant cannot be deferred the way a module or a - function can: a module stand-in defers until an attribute is read and a - callable stand-in defers until it is called, but a name bound to a VALUE is - read by whatever the value is handed to. So this proxy defers to the first - OPERATION on the value — iteration, length, membership, subscript, - comparison, truthiness, `str()` — and forwards it to the resolved object. - - IT IS HERE FOR ONE SITE AND IT SAYS WHICH. `cli.py` re-exports - `openxdox.corpus_root.SCANNED_ROOTS` and reads it at :228 inside a function - body (`for root in SCANNED_ROOTS`), which defers correctly — but :228 is - not a line openxFactory's carve manifest declares for `cli.py`'s row, so - the READ cannot be respelled `corpus_root.SCANNED_ROOTS` and the NAME has - to keep behaving like the tuple it used to be. `SCANNED_ROOTS` is itself - DERIVED at the consumer (`tuple(sorted({*corpus.GOVERNED_ROOTS, - "openspec"}))`, over openxFactory's doc-health corpus), so unlike the - values in `opendox/defaults.py` it cannot simply be owned here: openDox has - no corpus to derive it from. - - NOT A GENERAL VALUE FACADE. `__getattr__` is deliberately NOT forwarded: an - attribute read on a constant is almost always a caller who wanted the - MODULE, and answering it here would turn this into the facade the module - docstring refuses. The operations below are the ones a re-exported - sequence constant is actually subjected to, and a site needing more is a - site that should be respelled instead. - """ - - __slots__ = ("_module", "_attr") - - def __init__(self, module: _LateConsumerModule, attr: str) -> None: - self._module = module - self._attr = attr - - @property - def name(self) -> str: - return f"{self._module.name}.{self._attr}" - - def resolve(self) -> Any: - """The consumer's value, or `ConsumerReachUnavailable` naming the layering.""" - return getattr(self._module.resolve(), self._attr) - - def __iter__(self): - return iter(self.resolve()) - - def __len__(self) -> int: - return len(self.resolve()) - - def __contains__(self, item: Any) -> bool: - return item in self.resolve() - - def __getitem__(self, key: Any) -> Any: - return self.resolve()[key] - - def __eq__(self, other: Any) -> bool: - return self.resolve() == other - - def __ne__(self, other: Any) -> bool: - return self.resolve() != other - - def __hash__(self) -> int: - return hash(self.resolve()) - - def __bool__(self) -> bool: - return bool(self.resolve()) - - def __str__(self) -> str: - return str(self.resolve()) - - def __repr__(self) -> str: - # Deliberately does NOT resolve: `repr` is what a debugger, a pytest - # assertion rewrite and a logging call reach for, and resolving the - # consumer because something formatted a value would make the reach - # fire at a moment no verb chose — the same rule `_LateConsumerModule` - # applies to dunder lookups. - return f"" - - class _LateConsumerColumn: """The BASE-CLASS member of this family: one handler-method column of the consumer, reached on first CALL instead of at class-definition time. @@ -275,16 +164,15 @@ class _LateConsumerColumn: BASES. A base expression is evaluated when the class statement runs, which is when the module loads, so those two lines alone made `import opendox.serve` require openXdox — and a base, unlike a call, cannot - be deferred by any of the stand-ins above: a class needs its bases to exist + be deferred by the module stand-in above: a class needs its bases to exist before its first instance does. So the column is replaced by a base openDox OWNS, carrying one method per name the consumer's column defines. Each forwards `getattr(, name)(self, *args, **kwargs)` — the SAME function object, with the SAME `self`, which is the live request handler — so the - handler behaves exactly as it did when it inherited: `serve.py`'s core - `/snapshot.json` arm still finds `self._serve_snapshot`, and a contributed - binding naming `_handle_gate_action` or `_serve_index` still resolves + handler behaves exactly as it did when it inherited: a contributed binding + naming `_handle_gate_action` or `_serve_index` still resolves against the bound class at wiring time (`route_extension.resolve_handlers`), which is the check that refuses a route that cannot be served BEFORE a socket. @@ -356,16 +244,6 @@ def module(name: str, *, reason: str) -> _LateConsumerModule: return _LateConsumerModule(name, reason=reason) -def function(holder: _LateConsumerModule, attr: str) -> _LateConsumerCallable: - """A late stand-in for one function of a consumer module.""" - return _LateConsumerCallable(holder, attr) - - -def constant(holder: _LateConsumerModule, attr: str) -> _LateConsumerValue: - """A late stand-in for one CONSTANT of a consumer module.""" - return _LateConsumerValue(holder, attr) - - # -------------------------------------------------------------------------- # The reaches this package still makes, each with the reason it still makes it # -------------------------------------------------------------------------- @@ -381,81 +259,43 @@ def constant(holder: _LateConsumerModule, attr: str) -> _LateConsumerValue: reason="the gate-and-commission loop is openXdox's column (design.md § D3) " "and openDox contributes no gate of its own") -#: The snapshot writer and validator — openXdox's projection mechanism. -snapshot = module( - "snapshot", - reason="the snapshot projection mechanism is openXdox's column " - "(design.md § D3)") - -#: The snapshot registry — the projection's per-binding bookkeeping. -snapshot_registry = module( - "snapshot_registry", - reason="the snapshot registry belongs to the projection mechanism, which " - "is openXdox's column (design.md § D3)") - -#: The corpus-root predicate — whether a checkout can be scanned as -#: openxFactory's governed corpus at all. The SHAPE of a corpus is the -#: consumer's business (`SCANNED_ROOTS` is derived from `doc_health.corpus`'s -#: governed roots), and openDox's `cli` asks the question on the way into a -#: generate verb. -corpus_root = module( - "corpus_root", - reason="what counts as a scannable corpus checkout is derived from " - "openxFactory's governed document roots, which openDox does not " - "carry (design.md § D3)") - -#: The snapshot GENERATOR — the projection mechanism's writer half, beside -#: `snapshot` (its validator half) above. -generator = module( - "generator", - reason="the snapshot generator is the projection mechanism, which is " - "openXdox's column (design.md § D3)") - -#: The projection's HTTP column. openDox's `serve` core keeps one reach into it -#: — `hosted_ref_refused`, below — after slice 2b stopped naming the column as -#: a mixin base and stopped re-exporting its names. +#: THE PROJECTION MECHANISM'S FOUR STAND-INS AND THEIR SIX NAMES ARE GONE +#: (plan 034 T055; #1144 5.5 and 4.3 in part; R1Q10 (a), openxFactory#656 +#: comment 5850003126). `snapshot`, `snapshot_registry`, `corpus_root` and +#: `generator` stood here as module stand-ins, with `find_validator`, +#: `corpus_root_refusal`, `generate_snapshot`, `is_rfc3339_datetime`, +#: `hosted_ref_refused` and the one constant stand-in, `scanned_roots`, bound +#: over them. Each was a neutral verb of openDox's reaching the consumer's +#: projection column, so a lone openDox could neither generate nor build a +#: server (plan 034, research R7). They are now read from DECLARED SEAMS, +#: `opendox.projection_seams` (the snapshot registry and source, the +#: corpus-root predicate, the writer and the validator lookup) and +#: `opendox.generator_seam` (the generate operation), each with openDox's own +#: default, which the entry points register where no host has. The consumer +#: contributes its governed mechanism through the same seams (T059). +#: `is_rfc3339_datetime` is openDox's own (`opendox.rfc3339`), and +#: `hosted_ref_refused` is `serve.py`'s own, over the registry seam's +#: `is_publishable_ref`. With no binding left for them, the callable and value +#: stand-ins (`function`, `constant`) went too. + +#: The projection's HTTP column. Named here only to carry +#: `LateProjectionRoutes` below: `serve.py` holds no other reference to it +#: since plan 034 T055. serve_projection = module( "serve_projection", - reason="the snapshot and index routes are openXdox's column " - "(design.md § D3); this core dispatches them through the § 2.4 " - "route extension point instead of inheriting them") - -#: `snapshot.find_validator`, bound as a callable because `workbench.py` and -#: `cli.py` import the FUNCTION rather than the module. -find_validator = function(snapshot, "find_validator") - -#: `corpus_root.corpus_root_refusal` — `cli.py:130` refuses a generate whose -#: `--repo-root` is not a corpus checkout, and the refusal text is the whole -#: message. -corpus_root_refusal = function(corpus_root, "corpus_root_refusal") - -#: `generator.generate_snapshot` and `generator.is_rfc3339_datetime` — -#: `cli.py`'s two generate verbs (:184, :501) and its `--generated-at` shape -#: check (:156). -generate_snapshot = function(generator, "generate_snapshot") -is_rfc3339_datetime = function(generator, "is_rfc3339_datetime") - -#: `serve_projection.hosted_ref_refused` — `serve.py:760` must never NAME a -#: session ref on a hosted response (FR-048), and the predicate that decides it -#: travelled to the projection column with its neighbours. -hosted_ref_refused = function(serve_projection, "hosted_ref_refused") + reason="the snapshot index route is openXdox's column (design.md § D3); " + "this core dispatches it through the § 2.4 route extension point " + "instead of inheriting it") #: `resolve_source_path` STOOD HERE and is `serve.py`'s own definition since #: § 3.4 slice S6 (RULED Q4, openxFactory#656 comment 5642758731). It is the #: single-root entry point to the `/source` pass-through's containment, and that #: pass-through is now a FIXED CORE ARM of the neutral product — so a stand-in #: forwarding into the layer that PINS openDox was the wrong shape for it. The -#: RULE has not moved: `snapshot_registry.resolve_within` is still the consumer's -#: and is still what the entry point calls, through `serve.py`'s `registry_mod` -#: binding below — the same seam `serve_workbench.py` reaches it by at five -#: sites. `notebook_action.py`:52 still imports the name `from .serve`, and now -#: gets a real function rather than a forwarder. - -#: `corpus_root.SCANNED_ROOTS` — the one CONSTANT reach, and the family's -#: narrowest member. `cli.py:228` iterates it to name the roots a rejected -#: checkout is missing; that line is not one the carve manifest declares, so -#: the NAME must keep behaving like the tuple it was. See `_LateConsumerValue`. -scanned_roots = constant(corpus_root, "SCANNED_ROOTS") +#: RULE is the snapshot registry's `resolve_within`, which the entry point +#: calls through `serve.py`'s `registry_mod`, the registry seam's proxy since +#: plan 034 T055. `notebook_action.py`:52 still imports the name `from .serve`, +#: and gets a real function rather than a forwarder. #: The gate console's HTTP column — the door the gate verbs are posted through. #: Named here only to carry `LateGateRoutes` below; `serve.py` holds no other @@ -473,43 +313,30 @@ def constant(holder: _LateConsumerModule, attr: str) -> _LateConsumerValue: ("_handle_gate_action", "_log_gate_failure")) #: `serve_projection.ProjectionRoutes` as a mixin base — `DashboardHandler`'s -#: fourth base until slice 2b. FIVE methods since § 3.4 slice S6: the one the -#: § 2.4 binding names (`_serve_index`), `_serve_snapshot` — which `serve.py`'s -#: own CORE arm calls, because `/snapshot.json`'s handler travelled with its -#: neighbours while its arm stayed core — and the three those two call between -#: them. +#: fourth base until slice 2b. ONE method since plan 034 T055: `_serve_index`, +#: which the column's own § 2.4 binding for `/snapshot-index.json` names, and +#: which T084 hands to the handler-contribution facet with the column. #: -#: IT WAS EIGHT UNTIL S6 (RULED Q4, openxFactory#656 comment 5642758731). -#: `_keyed_source`, `_serve_source` and `_refuse_bare_source` are `serve.py`'s -#: own methods now, because the route they answer is the neutral product's own -#: fixed core arm. `_hosted_entry_refused` stays HERE and is still forwarded: -#: `_serve_snapshot` calls it too, and it is FR-048's hosted-plane confinement, -#: which is the projection column's rule — the route moved, the rule did not. +#: IT WAS EIGHT UNTIL § 3.4 SLICE S6 AND FIVE UNTIL T055. S6 (RULED Q4, +#: openxFactory#656 comment 5642758731) made `_keyed_source`, `_serve_source` +#: and `_refuse_bare_source` `serve.py`'s own, because the route they answer is +#: the neutral product's fixed core arm. T055 did the same for the core +#: `/snapshot.json` arm's four, `_query_key`, `_read_snapshot`, +#: `_serve_snapshot` and `_hosted_entry_refused`: forwarded here, a standalone +#: server refused every `/snapshot.json`. The rules they consult (FR-048's +#: hosted-plane confinement, the registry's per-entry resolution) are the +#: snapshot registry's, reached through its seam. LateProjectionRoutes = route_column( - serve_projection, "ProjectionRoutes", - ("_query_key", "_read_snapshot", "_serve_snapshot", "_hosted_entry_refused", - "_serve_index")) + serve_projection, "ProjectionRoutes", ("_serve_index",)) __all__ = [ "CONSUMER_PACKAGE", "ConsumerReachUnavailable", - "constant", - "corpus_root", - "corpus_root_refusal", - "find_validator", - "function", "gate_console", - "generate_snapshot", - "generator", - "hosted_ref_refused", - "is_rfc3339_datetime", "LateGateRoutes", "LateProjectionRoutes", "module", "route_column", - "scanned_roots", "serve_gate", "serve_projection", - "snapshot", - "snapshot_registry", ] diff --git a/src/opendox/default_generator.py b/src/opendox/default_generator.py index 1c81476..7ca18d4 100644 --- a/src/opendox/default_generator.py +++ b/src/opendox/default_generator.py @@ -40,10 +40,10 @@ six role keys is one (R1Q13 (a)): the line names the document, the value and the six keys, and the snapshot reads that document as a source. -NO VERB REACHES IT YET. The generate verbs still call the consumer's generator, -through `consumer_reach`, until T055 routes them through the seam (T055 comes -after T054, in `cli.py`'s and `serve.py`'s single-writer order). Until then a -library caller reaches it through the seam, `generator_seam.generate()`. +THE VERBS REACH IT THROUGH THE SEAM. Since plan 034's T055 the generate verbs +and the local regenerate generate through `generator_seam.generate()`, so a +process where no host registered a generator of its own generates with this +one. A library caller reaches it the same way. HOW IT IS REGISTERED: by the entry points, and never at import (R1Q3 (a)'s pattern). `cli.build_parser()`, `cli.main()`, `serve.build_server()` and diff --git a/src/opendox/default_projection.py b/src/opendox/default_projection.py new file mode 100644 index 0000000..a7f8149 --- /dev/null +++ b/src/opendox/default_projection.py @@ -0,0 +1,171 @@ +"""openDox's OWN defaults for three of the projection seams: the corpus-root +predicate, the snapshot writer and the validator lookup (plan 034's T055; +R1Q10 (a), `openxFactory#656` comment `5850003126`). + +The entry points register them where no host has +(`projection_seams.register_defaults()`). Each is small, neutral and new code. +A host (openXdox, plan 034's T059) contributes its governed one through the +same seam. + +`CORPUS_ROOT`, THE CORPUS-ROOT PREDICATE. openDox's own corpus is a plain git +repository, read through the home corpus (`local_git_adapter.LocalGitCorpus`, +T022), and that adapter resolves a repository's ROOT and refuses a directory +that is not one, or is only inside one. So the predicate asks the same +question, structurally and without running `git`: is this an existing +directory holding `.git`? A worktree's `.git` is a file, and it counts. A +tree that passes can still be refused by the adapter, which reads git itself, +and the generate verbs report that refusal too. The served image's empty +sentinel directory fails the predicate, so its checkout-bound affordances stay +off, as they do under the governed predicate. openDox's corpus is not scanned +from named roots, so `SCANNED_ROOTS` is empty. And it declares no change +folder with a staged origin, which is what `change_rows` enumerates for +`branch_session`'s proposal custody, so it answers no row. + +`WRITER`, THE SNAPSHOT WRITER. Canonical JSON: keys sorted at every depth, two +spaces of indent, ASCII only, a trailing newline, and no clock. So the same +snapshot is the same bytes. It writes through the interactivity boundary it is +handed, and only there. + +`VALIDATOR`, THE VALIDATOR LOOKUP'S DEFAULT, FOR openDox's OWN KINDS. It is +openDox's own validator, plan 034's T057, which this tree does not carry yet. +Until it does, this stand-in answers every validation `VALIDATOR_UNAVAILABLE`, +naming T057, and never `VALIDATED`: nothing here has checked anything. So a +generate verb warns that its snapshot was not checked, and fails under +`--strict`, and a workbench manifest saved with `validate=True` is refused as +unvalidated, which is what a lone openDox answered before T055 whenever no +validator was reachable. T057's validator replaces it here, under the same +kinds. + +IMPORT WEIGHT. `opendox.generator_seam`, `opendox.projection_seams` and the +standard library. So this module imports with no extra installed and no +sibling present. + +A CREATED FILE: it has no row in openxFactory's +`docs/opendox-carve-manifest.yaml`, because the manifest declares what LEAVES +openxFactory and never what a destination assembles (RULED OQ-C). +""" + +from __future__ import annotations + +import json +from pathlib import Path +from typing import Any + +from opendox import generator_seam, projection_seams + +__all__ = ["CORPUS_ROOT", "CorpusRoot", "OWN_KINDS", "OwnValidatorNotBuilt", + "VALIDATOR", "WRITER", "Writer"] + +#: The workbench manifest's kind, `opendox.workbench.KIND`, restated because +#: `workbench` imports PyYAML and this module must import with nothing extra. +#: `tests/test_projection_seams.py` holds the two spellings together. +WORKBENCH_KIND = "ideation-workbench" + +#: The kinds openDox's own validator answers for, which the entry points +#: register it under: the neutral snapshot every generate verb writes with +#: openDox's own generator, and the workbench manifest `workbench.save()` +#: validates. T057 names its full input set, and registers under it. +OWN_KINDS: tuple[str, ...] = (generator_seam.NEUTRAL_SNAPSHOT_KIND, WORKBENCH_KIND) + + +class CorpusRoot: + """openDox's own corpus-root predicate: a git repository's root.""" + + #: openDox's corpus is not scanned from named roots. + SCANNED_ROOTS: tuple[str, ...] = () + + #: What the refusal says it looked for. + LOOKED_FOR = "the root of a git repository: a directory holding .git" + + @staticmethod + def corpus_scan_defect(repo_root: Path | str) -> str | None: + """Why `repo_root` cannot be read as a corpus checkout, or None when + it can be. Structural only: it answers "could this be read at all", + and an honestly empty repository passes.""" + path = Path(repo_root) + try: + if not path.exists(): + return "the path does not exist" + if not path.is_dir(): + return "the path is not a directory" + if not (path / ".git").exists(): + return "the directory is not the root of a git repository" + except OSError as exc: # an unreadable path is not a corpus checkout + return f"the path could not be read ({exc.strerror or exc})" + return None + + @classmethod + def corpus_root_refusal(cls, repo_root: Path | str, *, + flag: str = "--repo-root", + shape: str = "") -> str | None: + """The operator-facing refusal for a `flag` value that is not a corpus + checkout, or None when it is one. It names the RESOLVED path it + checked and what it looked for, because the mistake it catches is a + path that looks right and resolves somewhere else. `shape` is the + caller's own correct invocation, appended verbatim.""" + defect = cls.corpus_scan_defect(repo_root) + if defect is None: + return None + try: + resolved: Path | str = Path(repo_root).resolve() + except (OSError, RuntimeError): + resolved = repo_root + lines = [ + f"{flag} is not a corpus checkout: {defect}", + f" checked {resolved}", + f" looked for {cls.LOOKED_FOR}", + f" {flag} must name the SERVED CHECKOUT itself: the repository", + " whose documents the snapshot projects. It is not a directory", + " above that repository or inside it, and it is not the same path in", + " another filesystem namespace: a path that resolves inside a", + " container does not resolve on the host, or the reverse. Resolve it", + " where THIS command runs.", + ] + if shape: + lines.append(" a correct invocation has this shape:") + lines.extend(f" {line}" for line in shape.strip().splitlines()) + return "\n".join(lines) + + @staticmethod + def change_rows(checkout_root: Path | str) -> tuple: + """The corpus's change rows, `(change id, status, folder, origin state, + origin)`. openDox's corpus declares no change folder with a staged + origin, so there is none.""" + return () + + +class Writer: + """openDox's own canonical snapshot writer.""" + + @staticmethod + def canonical_json(snapshot: dict[str, Any]) -> str: + """Deterministic JSON: keys sorted at every depth, two spaces of + indent, ASCII only, and a trailing newline.""" + return json.dumps(snapshot, indent=2, sort_keys=True, + ensure_ascii=True) + "\n" + + def write_snapshot(self, snapshot: dict[str, Any], path: Path | str, + boundary) -> Path: + """Render canonically and write through the interactivity boundary, + which writes only under its declared output allowlist.""" + return boundary.write_output(path, self.canonical_json(snapshot)) + + +class OwnValidatorNotBuilt: + """The validator lookup's default until openDox's own validator (T057) is + in this tree. It concludes nothing, and says so.""" + + #: No dependency to install would make it run. + dependency_remedy = None + + def validate(self, path: Path | str, *, strict: bool = False, + search_from: tuple = ()) -> projection_seams.ValidationResult: + return projection_seams.ValidationResult( + False, -1, "", "", None, projection_seams.VALIDATOR_UNAVAILABLE, + "openDox's own validator is plan 034's T057, and this build does " + "not carry it yet, so nothing of openDox's own kinds is checked") + + +CORPUS_ROOT = CorpusRoot() +WRITER = Writer() +VALIDATOR = OwnValidatorNotBuilt() diff --git a/src/opendox/default_registry.py b/src/opendox/default_registry.py new file mode 100644 index 0000000..3b0c1b7 --- /dev/null +++ b/src/opendox/default_registry.py @@ -0,0 +1,590 @@ +"""openDox's OWN snapshot registry and snapshot source: the registry seam's +default (plan 034's T055; R1Q10 (a), `openxFactory#656` comment `5850003126`). + +WHAT IT IS. The registration `projection_seams.registry` holds where no host +has contributed one. It carries what openDox's own code reads off a registry: +the entry, registry and source types, the key and containment rules, and the +refresh bindings. The entry points register it (R1Q3 (a)'s pattern), and a +host contributes its governed registry through the same seam (openXdox's +`snapshot_registry`, T059). + +ONE REGISTRY, KEYED `(repository, ref)`, WITH `ref` DEFAULTING TO `main`. An +entry is one snapshot on disk and the facts a viewer needs to trust it: its +repository and ref, the `source_revision` it projects, its `generated_at`, and +the ROOT its `/source/` pass-through is confined to. A session is an entry at a +session ref whose root is the session's worktree (`branch_session`), and its +owner and base survive a re-registration. + +WHAT IT DOES NOT DO, AND SAYS SO. It reads NO DATA SOURCE and NO INDEX. A +published index is a governed contract (`ideation-dashboard-snapshot-index`), +and openDox has no index kind of its own (plan 034's T053 records that), so +this registry refuses a declared data source and a declared local index rather +than ignoring either, naming the seam a host registers through. The same goes +for aggregates: `compose_view` composes nothing. So standalone, the server +serves the one snapshot it was handed, and a regenerate rewrites it. + +THE LOCAL REGENERATE GOES THROUGH THE SEAMS. `SnapshotSource.refresh` re-runs +the REGISTERED generator (`generator_seam.generate`, looked up on each call) +against the entry's own root, and writes through the REGISTERED writer +(`projection_seams.writer`), inside the interactivity boundary. So the server +and the generate verbs write with the same generator and the same writer. + +THE CONTAINMENT RULE. `resolve_within` is the rule `/source/` has always been +confined by, as openDox states it: no absolute path, no NUL, no escape of the +root (`..`, percent-encoded `..`, a symlink), no dot-directory ever, and a +dot-file only with an extension a document carries. The dot-directory half is +where credentials live (`.git/config` carries a remote's token, T092's defect +10), so nothing below a dot-directory is ever served. + +IMPORT WEIGHT. `opendox.boundary`, `opendox.defaults`, `opendox.generator_seam`, +`opendox.projection_seams` and the standard library. So this module imports +with no extra installed and no sibling present. + +A CREATED FILE: it has no row in openxFactory's +`docs/opendox-carve-manifest.yaml`, because the manifest declares what LEAVES +openxFactory and never what a destination assembles (RULED OQ-C). +""" + +from __future__ import annotations + +import contextlib +import json +import threading +import urllib.parse +from dataclasses import dataclass +from pathlib import Path, PurePosixPath +from typing import Any, Callable + +from opendox import defaults, generator_seam, projection_seams +from opendox.boundary import OutputBoundary + +__all__ = [ + "BINDING_REFETCH", + "BINDING_REGENERATE", + "DEFAULT_INDEX_NAME", + "DEFAULT_PUBLISH_PATH", + "DEFAULT_REF", + "NeutralRegistryRefused", + "ORIGIN_LOCAL", + "PEEK_TTL_SECONDS", + "SERVED_DOTFILE_SUFFIXES", + "SnapshotEntry", + "SnapshotRegistry", + "SnapshotSource", + "data_source_from_options", + "entry_from_snapshot_file", + "github_raw_base_url", + "is_publishable_ref", + "key_id", + "normalize_ref", + "parse_key_id", + "resolve_within", + "snapshot_key", +] + +#: The shared truth every ref-less request means. +DEFAULT_REF = "main" + +#: The two refresh bindings. `refetch` re-reads a declared data source, which +#: this registry never has. `regenerate` re-runs the generator against a real +#: checkout, and it is this registry's only binding. +BINDING_REFETCH = "refetch" +BINDING_REGENERATE = "regenerate" + +#: A snapshot generated on this host from a checkout. The only origin here. +ORIGIN_LOCAL = "local" + +#: The data source's defaults, which `serve.main()` offers as its options' +#: defaults. openDox's own values (`defaults.py`), and an empty publication +#: path, which means the published tree sits at the source's root. +DEFAULT_INDEX_NAME = defaults.DEFAULT_INDEX_NAME +PEEK_TTL_SECONDS = defaults.PEEK_TTL_SECONDS +DEFAULT_PUBLISH_PATH = "" + +#: The dot-file extensions `/source` serves: a dot-file is served only when it +#: is a document by its extension. +SERVED_DOTFILE_SUFFIXES = frozenset({".yaml", ".yml", ".md", ".json"}) + +#: Where every refusal below sends a caller who needs what it refuses. +_REGISTRATION = projection_seams.registry.registration_call + + +class NeutralRegistryRefused(projection_seams.ProjectionSeamError): + """openDox's own registry was asked for something only a host's registry + offers: a data source, a local index, or a refresh it has no binding for. + Refused, never ignored: a server that silently dropped a declared data + source would serve something other than what its operator declared.""" + + +# --------------------------- keys --------------------------- + +def normalize_ref(ref: str | None) -> str: + """`None`, empty and whitespace all mean `main`.""" + if ref is None: + return DEFAULT_REF + text = str(ref).strip() + return text or DEFAULT_REF + + +def snapshot_key(repository: str, ref: str | None = None) -> tuple[str, str]: + """The registry key, `(repository, ref)`, with `ref` defaulting to `main`.""" + return (str(repository), normalize_ref(ref)) + + +def key_id(repository: str, ref: str | None = None) -> str: + """A key's wire form, `repository@ref`.""" + repo, ref_name = snapshot_key(repository, ref) + return f"{repo}@{ref_name}" + + +def parse_key_id(text: str) -> tuple[str, str] | None: + """`key_id`'s inverse, or None where `text` is not a key's form. Parsing + is not admission: the registry still decides whether the pair exists.""" + if not text or "@" not in text: + return None + repo, _, ref = text.rpartition("@") + if not repo: + return None + return snapshot_key(urllib.parse.unquote(repo), urllib.parse.unquote(ref)) + + +def is_publishable_ref(ref: str | None) -> bool: + """Only `main` is shared truth. Every other ref is a session's, which a + hosted plane never names (FR-048).""" + return normalize_ref(ref) == DEFAULT_REF + + +# --------------------------- entries --------------------------- + +@dataclass +class SnapshotEntry: + """One `(repository, ref)` snapshot the registry can serve.""" + + repository: str + ref: str = DEFAULT_REF + snapshot_path: Path | None = None + payload: bytes | None = None + source_root: Path | None = None + source_revision: str | None = None + generated_at: str | None = None + origin: str = ORIGIN_LOCAL + display_name: str | None = None + stale: bool = False + stale_reason: str | None = None + location: str | None = None + unavailable_reason: str | None = None + #: WHOSE session this entry is, `(scope_kind, scope_id)`: set only by the + #: code that opened the session, the one place the answer is known. + session_tile: tuple[str, str] | None = None + #: What the session branched from, `(base_ref, base_revision)`. + session_base: tuple[str, str] | None = None + #: The other recorded spellings of the base's revision. + session_base_aliases: tuple[str, ...] = () + + def __post_init__(self) -> None: + self.ref = normalize_ref(self.ref) + if self.snapshot_path is not None: + self.snapshot_path = Path(self.snapshot_path) + if self.source_root is not None: + self.source_root = Path(self.source_root) + + @property + def key(self) -> tuple[str, str]: + return (self.repository, self.ref) + + @property + def key_id(self) -> str: + return key_id(self.repository, self.ref) + + @property + def available(self) -> bool: + """Whether this entry can serve bytes at all.""" + if self.payload is not None: + return True + return bool(self.snapshot_path and self.snapshot_path.is_file()) + + def read_bytes(self) -> bytes | None: + """The snapshot's bytes: the in-process copy first, then the file.""" + if self.payload is not None: + return self.payload + if self.snapshot_path is None: + return None + try: + return self.snapshot_path.read_bytes() + except OSError: + return None + + def read_json(self) -> dict | None: + raw = self.read_bytes() + if raw is None: + return None + try: + document = json.loads(raw.decode("utf-8")) + except (ValueError, UnicodeDecodeError): + return None + return document if isinstance(document, dict) else None + + def short_revision(self) -> str: + return (self.source_revision or "")[:12] or "unknown" + + def freshness(self) -> dict: + """What a viewer's freshness header reads off the entry.""" + return { + "repository": self.repository, + "ref": self.ref, + "source_revision": self.source_revision, + "source_revision_short": self.short_revision(), + "generated_at": self.generated_at, + "origin": self.origin, + "stale": bool(self.stale), + "stale_reason": self.stale_reason, + "available": self.available, + "unavailable_reason": self.unavailable_reason, + } + + +def entry_from_snapshot_file( + path: Path | str, + *, + repository: str | None = None, + ref: str | None = None, + source_root: Path | str | None = None, + origin: str = ORIGIN_LOCAL, + stale: bool = False, + stale_reason: str | None = None, + display_name: str | None = None, +) -> SnapshotEntry: + """An entry for a snapshot ON DISK. Its repository and its generation + stamps are read OUT OF the snapshot, which is their source, unless the + caller names the repository.""" + path = Path(path) + try: + document = json.loads(path.read_text(encoding="utf-8")) + except (OSError, ValueError): + document = {} + if not isinstance(document, dict): + document = {} + generation = document.get("generation") + generation = generation if isinstance(generation, dict) else {} + return SnapshotEntry( + repository=str(repository or document.get("repository") or "unknown"), + ref=normalize_ref(ref), + snapshot_path=path, + source_root=Path(source_root) if source_root is not None else None, + source_revision=generation.get("source_revision"), + generated_at=generation.get("generated_at"), + origin=origin, + stale=stale, + stale_reason=stale_reason, + display_name=display_name, + ) + + +# --------------------------- containment (pure) --------------------------- + +def resolve_within(root: Path | str, url_tail: str) -> Path | None: + """`url_tail` as an absolute FILE under `root`, or None to refuse it. + + Percent-decoding comes FIRST, so `%2e%2e` and `%2egit` meet the same + checks as their plain spellings. Then it refuses an empty tail, an + absolute one and a NUL; any component but the last that is a dot-directory; + a last component that is a dot-file without a document's extension; any + path that resolves outside `root`, whether by `..` or by a symlink; and + anything that is not a regular file.""" + rel = urllib.parse.unquote(str(url_tail)) + rel = rel.split("?", 1)[0].split("#", 1)[0] + if not rel or rel.startswith("/") or "\x00" in rel: + return None + parts = [part for part in rel.replace("\\", "/").split("/") + if part not in ("", ".")] + if any(part.startswith(".") and part != ".." for part in parts[:-1]): + return None + if parts and parts[-1].startswith(".") and parts[-1] != "..": + if PurePosixPath(parts[-1]).suffix not in SERVED_DOTFILE_SUFFIXES: + return None + try: + base = Path(root).resolve() + resolved = (base / rel).resolve() + except (OSError, RuntimeError, ValueError): + return None + if resolved != base and not resolved.is_relative_to(base): + return None + if not resolved.is_file(): + return None + return resolved + + +# --------------------------- the registry --------------------------- + +class SnapshotRegistry: + """The ONE snapshot registry, keyed `(repository, ref)`. + + Per-process and in memory. `serve.py` answers requests on threads of their + own, so every mutation is serialized under a re-entrant lock, and + `atomically()` holds it across a read-modify-write, as + `branch_session._preserving_active` needs.""" + + def __init__(self) -> None: + self._entries: dict[tuple[str, str], SnapshotEntry] = {} + self._active: tuple[str, str] | None = None + self._lock = threading.RLock() + + @contextlib.contextmanager + def atomically(self): + """Hold the registry across a read-modify-write. Re-entrant.""" + with self._lock: + yield self + + def register(self, entry: SnapshotEntry, *, + active: bool = False) -> SnapshotEntry: + """Register (or replace) `entry`. The first entry registered becomes + active, and so does one registered with `active=True`. + + A replacement that does not know its session's owner or base inherits + them from the entry it replaces: a regenerate rebuilds the entry from + its snapshot, and only the code that OPENED the session knows either.""" + with self._lock: + previous = self._entries.get(entry.key) + if previous is not None and entry.session_tile is None: + entry.session_tile = previous.session_tile + if previous is not None and entry.session_base is None: + entry.session_base = previous.session_base + entry.session_base_aliases = previous.session_base_aliases + self._entries[entry.key] = entry + if active or self._active is None: + self._active = entry.key + return entry + + def drop(self, repository: str, ref: str | None = None) -> None: + with self._lock: + self._entries.pop(snapshot_key(repository, ref), None) + + def get(self, repository: str, ref: str | None = None) -> SnapshotEntry | None: + """A ref-less lookup means `main`.""" + return self._entries.get(snapshot_key(repository, ref)) + + def entries(self) -> list[SnapshotEntry]: + """Every entry, ordered by `(repository, ref)`.""" + with self._lock: + return [self._entries[key] for key in sorted(self._entries)] + + def keys(self) -> list[tuple[str, str]]: + with self._lock: + return sorted(self._entries) + + def __len__(self) -> int: + return len(self._entries) + + @property + def active(self) -> SnapshotEntry | None: + key = self._active + return None if key is None else self._entries.get(key) + + def set_active(self, repository: str, + ref: str | None = None) -> SnapshotEntry | None: + with self._lock: + entry = self.get(repository, ref) + if entry is not None: + self._active = entry.key + return entry + + def resolve(self, repository: str | None, + ref: str | None = None) -> SnapshotEntry | None: + """A named pair, or the ACTIVE entry when no repository is named.""" + if repository is None or repository == "": + return self.active + return self.get(repository, ref) + + def resolve_source(self, repository: str | None, ref: str | None, + tail: str) -> Path | None: + """A `/source/` read through ONE entry, confined to that entry's own + root. An unknown pair, or an entry with no root, serves nothing.""" + entry = self.resolve(repository, ref) + if entry is None or entry.source_root is None: + return None + return resolve_within(entry.source_root, tail) + + +# --------------------------- data sources: none --------------------------- + +def data_source_from_options(*, directory: Path | str | None = None, + url: str | None = None, + token_env: str | None = None, + opener: Callable[..., Any] | None = None): + """None when no data source is declared, which is the only plane this + registry serves. A declared one is REFUSED, never ignored.""" + if directory or url: + raise NeutralRegistryRefused( + "a runtime data source was declared " + f"({'--data-source-dir' if directory else '--data-source-url'}), " + "and openDox's own snapshot registry reads none: a published " + "snapshot tree is located by a snapshot index, which is a governed " + "contract, and openDox has no index kind of its own. Serve the " + "snapshot this checkout generates, or register a host registry " + f"that reads a data source, at process start, with {_REGISTRATION}") + return None + + +def github_raw_base_url(owner_repo: str, *, ref: str = DEFAULT_REF, + path: str = DEFAULT_PUBLISH_PATH) -> str: + """A repository's raw-file base URL, composed from operator flags alone. + openDox's registry reads no data source, so a URL composed here is refused + by `data_source_from_options`, naming the seam.""" + slug = str(owner_repo).strip().strip("/") + if slug.count("/") != 1 or not all(slug.split("/")): + raise ValueError(f"expected OWNER/REPO, got {owner_repo!r}") + tree = str(path or "").strip("/") + tail = f"{slug}/{ref}/" + (f"{tree}/" if tree else "") + return f"https://raw.githubusercontent.com/{tail}" + + +# --------------------------- the source --------------------------- + +class SnapshotSource: + """What a server holds: the registry, the snapshot it was handed, and the + one refresh binding a checkout offers, `regenerate`. + + It takes the keywords a host's source takes, so `serve.build_server` and + `branch_session` build either through the seam with one call. A declared + `data_source` or `local_index` is refused at construction (see the module + docstring). `index_name` and `peek_ttl_seconds` are a data source's, and + are held but never read here. `generator` is a test's seam: a callable + taking the regenerate's call, used instead of the registered generator.""" + + def __init__( + self, + *, + baked_snapshot: Path | str | None = None, + repository: str | None = None, + ref: str | None = None, + checkout_root: Path | str | None = None, + data_source=None, + index_name: str = DEFAULT_INDEX_NAME, + source_roots: dict[str, Path | str] | None = None, + local_index: Path | str | None = None, + generator: Callable[..., dict] | None = None, + project_register: Path | str | None = None, + peek_ttl_seconds: float = PEEK_TTL_SECONDS, + ) -> None: + if data_source is not None: + raise NeutralRegistryRefused( + "a data source was handed to openDox's own snapshot source, " + "which reads none (a published tree is located by a governed " + "snapshot index, and openDox has no index kind). Register a " + f"host registry at process start with {_REGISTRATION}") + if local_index is not None: + raise NeutralRegistryRefused( + f"a local snapshot index ({local_index}) was declared, and " + "openDox's own snapshot source reads none: a snapshot index " + "is a governed contract, and openDox has no index kind of its " + "own. Serve one snapshot, or register a host registry at " + f"process start with {_REGISTRATION}") + self.baked_snapshot = Path(baked_snapshot) if baked_snapshot else None + self.checkout_root = Path(checkout_root) if checkout_root else None + self.data_source = None + self.index_name = index_name + self.local_index = None + self.peek_ttl_seconds = float(peek_ttl_seconds) + # Declared confinement roots, keyed canonically: `repo` means + # `repo@main`, and `repo@ref` names the pair. + self.source_roots = { + key_id(*(parse_key_id(raw) or snapshot_key(raw))): Path(value) + for raw, value in (source_roots or {}).items() + } + self.registry = SnapshotRegistry() + self._refresh_lock = threading.RLock() + self._generator = generator + self.project_register = Path(project_register) if project_register else None + self.errors: list[str] = [] + self.baked_repository = repository + self.baked_ref = normalize_ref(ref) + + def bootstrap(self) -> SnapshotRegistry: + """Register the snapshot this source was handed, where it exists.""" + self._register_baked() + return self.registry + + def _source_root_for(self, repository: str, ref: str) -> Path | None: + """An entry's confinement root: a declared `--source-root` first, the + served checkout for the snapshot this source was handed, and none for + anything else, which then serves no documents.""" + declared = self.source_roots.get(key_id(repository, ref)) + if declared is not None: + return declared + if (repository, ref) == (self.baked_repository, self.baked_ref): + return self.checkout_root + return None + + def _register_baked(self) -> None: + if self.baked_snapshot is None or not self.baked_snapshot.is_file(): + return + entry = entry_from_snapshot_file( + self.baked_snapshot, repository=self.baked_repository, + ref=self.baked_ref) + self.baked_repository = self.baked_repository or entry.repository + entry.source_root = self._source_root_for(entry.repository, entry.ref) + self.registry.register(entry, active=self.registry.active is None) + + def peek_hints(self, *, now: float | None = None) -> dict: + """What a data source advertises. There is none, so nothing.""" + return {} + + def compose_view(self, aggregate_id: str, *, + publishable_only: bool = False) -> dict | None: + """A composed view of several snapshots. openDox's registry composes + none, so an aggregate id is an unknown snapshot.""" + return None + + @property + def refresh_binding(self) -> str | None: + """`regenerate` over a real checkout directory, else no binding.""" + if self.checkout_root is not None and Path(self.checkout_root).is_dir(): + return BINDING_REGENERATE + return None + + def refresh(self, *, repository: str | None = None, + ref: str | None = None) -> dict: + """Run this source's refresh binding for one `(repository, ref)`, + serialized against every other refresh. Raises on refusal, and leaves + the previous snapshot in place.""" + binding = self.refresh_binding + with self._refresh_lock: + if binding == BINDING_REGENERATE: + return self._regenerate(repository=repository, ref=ref) + raise NeutralRegistryRefused( + "no refresh binding is available on this plane: openDox's own " + "source regenerates from a checkout directory, and none was given") + + def _regenerate(self, *, repository: str | None, ref: str | None) -> dict: + """Re-run the REGISTERED generator against the entry's own root and + rewrite that entry's snapshot through the REGISTERED writer, inside + the interactivity boundary. The only write a refresh performs. + + A regenerate keeps `main` active, and never promotes a session's ref: + a session enters the key space and changes nothing the shared + surfaces render (FR-014a).""" + previous = self.registry.active + entry = self.registry.resolve(repository, ref) + if entry is None: + raise ValueError(f"unknown snapshot {repository}@{normalize_ref(ref)}") + if entry.snapshot_path is None: + raise ValueError(f"{entry.key_id}: no local snapshot path to regenerate") + root = entry.source_root or self.checkout_root + if root is None or not Path(root).is_dir(): + raise ValueError(f"{entry.key_id}: no served checkout to regenerate from") + generate = self._generator or generator_seam.generate + snapshot = generate(Path(root), entry.repository, + project_register_source=self.project_register) + target = Path(entry.snapshot_path) + boundary = OutputBoundary(target.parent, [target.name]) + projection_seams.writer.current().write_snapshot(snapshot, target, boundary) + refreshed = entry_from_snapshot_file( + target, repository=entry.repository, ref=entry.ref, + source_root=entry.source_root, origin=ORIGIN_LOCAL) + refreshed.location = entry.location + refreshed.display_name = entry.display_name + already_active = previous is not None and previous.key == entry.key + self.registry.register( + refreshed, active=(already_active or is_publishable_ref(entry.ref))) + return {"binding": BINDING_REGENERATE, "entries": len(self.registry), + **refreshed.freshness()} diff --git a/src/opendox/generator_seam.py b/src/opendox/generator_seam.py index a4d62b5..c7edcd3 100644 --- a/src/opendox/generator_seam.py +++ b/src/opendox/generator_seam.py @@ -14,11 +14,12 @@ implementation"*, *"does not exist yet and is BUILD-arc work"*. This module is that declaration (plan 034's T052). -IT ROUTES NOTHING YET. The generate verbs still call the consumer's generator -through `consumer_reach` until plan 034's T055 routes them through `generate()` -below. T055 comes after T054, which builds openDox's own projection. So from -the day a verb generates through this seam, openDox's own generator can -generate. +WHAT GENERATES THROUGH IT. Since plan 034's T055 the generate verbs +(`cli._generate_and_write`, `cli._gate_snapshot`) and the local regenerate of +openDox's own snapshot source (`default_registry.SnapshotSource.refresh`) call +`generate()` below, looking the generator up on each call. T055 came after +T054, which built openDox's own projection, so the day a verb first generated +through this seam, openDox's own generator could generate. `CorpusAdapter` IS NOT IT, AND STAYS CLOSED AT SIX MEMBERS. The corpus-read interface (`corpus_adapter.py`) declares `resolve`, `list_documents`, `read`, diff --git a/src/opendox/projection_seams.py b/src/opendox/projection_seams.py new file mode 100644 index 0000000..d3d7316 --- /dev/null +++ b/src/opendox/projection_seams.py @@ -0,0 +1,604 @@ +"""THE PROJECTION MECHANISM'S SEAMS: the snapshot registry and source, the +corpus-root predicate, the snapshot writer and the validator lookup. Each is a +host's contribution or openDox's own default, held for late resolution (plan +034's T055; #1144's 5.5, and 4.3 in part). + +WHY THIS FILE EXISTS. openDox's generate verbs and its server reached four +mechanisms of the consumer's projection column through `consumer_reach`: the +snapshot registry and the source built over it (`openxdox.snapshot_registry`), +the corpus-root predicate (`openxdox.corpus_root`), the snapshot writer and its +validator (`openxdox.snapshot`). With openXdox absent, which is the normal state +of a neutral openDox, each of those reaches refused, so a server could not be +BUILT standalone (plan 034, research R7) and a generate verb could not write. +`consumer_reach` names the gap itself: the injection that would retire a reach, +*"openDox naming a protocol and being handed an implementation"*, *"does not +exist yet and is BUILD-arc work"*. This module is that injection for the four. + +R1Q10 (a) (`openxFactory#656` comment `5850003126`, in R-G3's pattern). openDox +grows a small neutral default for each mechanism, and the consumer contributes +its governed one through the same seam. The defaults are new code +(`opendox.default_registry`, `opendox.default_projection`), not openXdox's +modules relocated: option (b), the relocation, was not the ruling. + +THE ENTRY POINTS REGISTER THE DEFAULTS WHERE NO HOST HAS (R1Q3 (a), comment +`5817152735`). `cli.build_parser()`, `cli.main()`, `serve.build_server()` and +`serve.main()` call `register_defaults()`, beside their registrations of the +default profile, the default home corpus and the default generator. So: + +* a process that runs none of them still meets `SeamNotRegistered`, naming the + seam and the call, which is the library caller's case (4.2's discipline: + REFUSAL, NOT A DEFAULT, so nothing here ever answers an empty mechanism that + would read exactly like a working one); +* a host registration made BEFORE the default has been read replaces it; +* AFTER a consumer has read the default, a host's registration is refused as + `SeamAlreadyRegistered`. One process would otherwise hold two registries, two + predicates or two writers, and a registry entry built by one would be + registered in the other. It is the same rule the profile and the generator + keep (R1Q3 (ii); RN-1 (a), comment `5850003126`); +* the same registration again is a no-op, so an idempotent host start is not + punished, and `unregister()` makes a deliberate swap explicit. + +THE FOUR SEAMS. + +* `registry`: the snapshot registry and the source that serves and regenerates + through it. A registration is a MODULE, or any object, carrying + `REGISTRY_CALLABLES` and `REGISTRY_VALUES`: the entry, registry and source + types, the key and containment rules, and the refresh bindings. + openXdox's `snapshot_registry` module carries them as it stands. +* `corpus_root`: whether a path can be scanned as a corpus checkout at all, + the refusal a verb prints when it cannot, the roots a corpus is scanned from, + and the corpus's change rows (`branch_session`'s proposal custody reads + them). A registration carries `CORPUS_ROOT_CALLABLES` and + `CORPUS_ROOT_VALUES`. +* `writer`: the canonical snapshot writer, `write_snapshot(snapshot, path, + boundary)`, which writes through the interactivity boundary. +* `validators`: the validator lookup, ONE REGISTRATION PER KIND. A snapshot is + validated by the validator registered for its own `kind`, which is its + generator's declared contract (`generator_seam.SnapshotGenerator.contract`), + and a workbench manifest by the one registered for `ideation-workbench`. A + host registers a validator for each of its own kinds, and openDox's entry + points register openDox's own for openDox's kinds. So a host that contributes + its governed snapshot validator does not take openDox's own kinds with it. + +RESOLVE PER CALL. A consumer asks `current()` (or reads through `proxy`) at +each use and holds nothing across calls, so every consumer in a process answers +from the one registration that exists at the time. + +ONE LOCK PER SEAM. `serve.py` answers each request on a thread of its own, so +a registration and a read can meet. Each seam's bookkeeping is held under its +own lock, and a registration's own code (probing its names, naming it in a +refusal) never runs under it. + +IMPORT WEIGHT. The standard library only. So this module imports with no extra +installed and no sibling present, and it names no sibling in an import. +`register_defaults()` imports openDox's two default modules when it is CALLED. + +A CREATED FILE: it has no row in openxFactory's +`docs/opendox-carve-manifest.yaml`, because the manifest declares what LEAVES +openxFactory and never what a destination assembles (RULED OQ-C). +""" + +from __future__ import annotations + +import threading +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +__all__ = [ + "CORPUS_ROOT_CALLABLES", + "CORPUS_ROOT_VALUES", + "NOT_CONFORMANT", + "ProjectionSeamError", + "REGISTRY_CALLABLES", + "REGISTRY_VALUES", + "SeamAlreadyRegistered", + "SeamNotRegistered", + "VALIDATED", + "VALIDATOR_CALLABLES", + "VALIDATOR_UNAVAILABLE", + "ValidationResult", + "ValidatorNotRegistered", + "WRITER_CALLABLES", + "corpus_root", + "name_of", + "register_defaults", + "registry", + "validators", + "writer", +] + +#: The ruling every refusal cites for where the defaults come from. +_DEFAULTS_RULING = ( + "R1Q10 (a), openxFactory#656 comment 5850003126, in R1Q3 (a)'s pattern") + +#: The entry points that register the defaults, as every refusal names them. +_ENTRY_POINTS = ("`cli.build_parser()`, `cli.main()`, `serve.build_server()` " + "and `serve.main()`") + +#: How much of a `repr` a refusal quotes before it stops being read. +_NAME_LIMIT = 120 + + +class ProjectionSeamError(RuntimeError): + """A refusal of one of the projection seams. + + One base for every refusal below, so a verb can report them in one clause, + as `cli.main()` and `serve.main()` do.""" + + +class SeamNotRegistered(ProjectionSeamError): + """Nothing is registered at a seam: no host's contribution, and no entry + point's default. Raised instead of falling back to openDox's own default, + which is a registration an ENTRY POINT makes.""" + + +class SeamAlreadyRegistered(ProjectionSeamError): + """A second, different registration was made over a first: over a host's, + or over the entry point's default after a consumer has read it.""" + + +class ValidatorNotRegistered(SeamNotRegistered): + """No validator is registered for a kind. The validator lookup refuses + rather than choosing another kind's validator, because a validator reads a + document by its kind.""" + + +def name_of(value: Any) -> str: + """A registration's most nameable name, for a refusal. Never raises: a + refusal that fails while formatting itself replaces the reader's problem + with a worse one.""" + try: + name = getattr(value, "__name__", None) + if isinstance(name, str) and name: + return name + except Exception: # noqa: BLE001 - naming must never out-raise + pass + try: + kind = type(value) + return f"{kind.__module__}.{kind.__qualname__} instance" + except Exception: # noqa: BLE001 + pass + try: + text = repr(value) + except Exception: # noqa: BLE001 + text = "" + if not text: + return "an unnameable registration" + return text if len(text) <= _NAME_LIMIT else text[:_NAME_LIMIT - 1] + "…" + + +def _probe(registration: Any, callables: tuple[str, ...], + values: tuple[str, ...], call: str, what: str) -> None: + """Refuse a registration that does not carry the seam's names. + + A registration that CANNOT HAND OVER a name, because looking it up raises + (a lazy module whose first use fails, say), lacks that name. Its failure is + chained to the refusal, so the reason stays readable, and it never escapes + the registration as a failure of some other kind (T027's rule for the + status-exemption seam, `doxbench_packet.register_status_exemption`).""" + missing: list[str] = [] + failure: Exception | None = None + for name in (*callables, *values): + try: + member = getattr(registration, name) + except Exception as exc: # noqa: BLE001 - a name it cannot hand over is a name it lacks + failure = failure or exc + missing.append(name) + continue + if name in callables and not callable(member): + missing.append(name) + if registration is None or missing: + wanted = ", ".join(callables) + if values: + wanted += f" (callables) and {', '.join(values)}" + refusal = TypeError( + f"{call} takes {what}: a module, or any object, carrying {wanted}. " + f"{name_of(registration)} lacks {', '.join(missing) or 'them'}. A " + "host with no contribution of its own does not register one: the " + "entry points then register openDox's own default, and a process " + "that runs none of them is refused, naming this call.") + if failure is None: + raise refusal + raise refusal from failure + + +class _SeamProxy: + """A module-shaped stand-in over one seam's registration, resolved on each + attribute read. + + `registry_mod = projection_seams.registry.proxy` lets a module keep its + `registry_mod.X` reads while every one of them resolves the registration + current at the moment it runs. Dunder lookups never resolve: `copy`, + `pickle`, `inspect` and pytest's own assertion rewriting probe for dunders + on arbitrary objects, and a probe must not be what reads a seam. + `consumer_reach._LateConsumerModule` settled the same rule.""" + + __slots__ = ("_seam",) + + def __init__(self, seam: "_Seam") -> None: + self._seam = seam + + def __getattr__(self, attr: str) -> Any: + if attr.startswith("__") and attr.endswith("__"): + raise AttributeError(attr) + return getattr(self._seam.current(), attr) + + def __repr__(self) -> str: + return f"" + + +class _Seam: + """ONE registration: a host's contribution, or the entry point's default. + + `callables` and `values` are the names a registration must carry. + `default` names openDox's own default, for the refusal. The records a + registration keeps are whether it is the entry point's default, and whether + a consumer has read that default since it was registered.""" + + def __init__(self, name: str, *, what: str, callables: tuple[str, ...], + values: tuple[str, ...] = (), default: str, + consequence: str) -> None: + self.name = name + self.what = what + self.callables = callables + self.values = values + self.default = default + self.consequence = consequence + #: The ONE call a host makes, quoted verbatim in every refusal. + self.registration_call = ( + f"opendox.projection_seams.{name}.register()") + self._registered: Any = None + self._is_default = False + self._default_read = False + self._lock = threading.Lock() + #: A module-shaped proxy over whatever is registered (see `_SeamProxy`). + self.proxy = _SeamProxy(self) + + def _begin(self, registration: Any, *, is_default: bool) -> None: + """The one place the registration changes. The caller holds the lock.""" + self._registered = registration + self._is_default = is_default + self._default_read = False + + def register(self, registration: Any) -> Any: + """THE host's registration, made once, at process start. Returns it. + + The SAME registration again is a no-op, and it is not probed a second + time. A different one over a host's is refused. Over the entry point's + default it REPLACES the default until a consumer has read the default, + and it is refused once one has.""" + with self._lock: + if registration is not None and registration is self._registered: + return registration + _probe(registration, self.callables, self.values, + f"projection_seams.{self.name}.register()", f"the host's {self.what}") + with self._lock: + held = self._registered + if held is registration: + return registration + if held is None or (self._is_default and not self._default_read): + self._begin(registration, is_default=False) + return registration + over_a_host = not self._is_default + # Named outside the lock: naming a registration can run its own code. + if over_a_host: + raise SeamAlreadyRegistered( + f"a host's {self.what} is already registered at openDox's " + f"{self.name} seam ({name_of(held)}), and " + f"{name_of(registration)} would replace it. Registration " + "happens ONCE, at process start: one process holding two " + f"would split its consumers between them. Call " + f"opendox.projection_seams.{self.name}.unregister() first if " + "the swap is deliberate.") + raise SeamAlreadyRegistered( + f"openDox's own default {self.what} ({name_of(held)}) is " + f"registered, because an entry point registered it where no host " + f"had, and a consumer has already read it, so " + f"{name_of(registration)} cannot replace it now. A swap would " + "leave one process holding two, as a host's profile after a build " + "would (R1Q3 (ii), openxFactory#656 comment 5817152735; RN-1 (a), " + "comment 5850003126). Register the host's own at process start, " + f"ahead of {_ENTRY_POINTS}. Call " + f"opendox.projection_seams.{self.name}.unregister() first if the " + "swap is deliberate.") + + def register_default(self, registration: Any) -> Any: + """AN ENTRY POINT's registration of openDox's own default (R1Q10 (a)). + + Registers `registration` ONLY where nothing is registered, and leaves + a host's registration, or a default already registered, exactly as it + is. Returns whatever is registered afterwards. It is NOT for hosts. A + default that does not carry the seam's names is refused whether or not + anything is registered, because it is openDox's own defect.""" + with self._lock: + if registration is not None and registration is self._registered: + return registration + _probe(registration, self.callables, self.values, + f"projection_seams.{self.name}.register_default()", + f"openDox's own default {self.what}") + with self._lock: + if self._registered is None: + self._begin(registration, is_default=True) + return self._registered + + def unregister(self) -> None: + """Drop the registration, a host's or the default, and its records. + For test isolation and for a host tearing down.""" + with self._lock: + self._begin(None, is_default=False) + + def is_registered(self) -> bool: + """Is anything registered? Answers without reading or refusing.""" + return self._registered is not None + + def current(self) -> Any: + """The registration, or a refusal naming this seam and its call. + + Reading the entry point's default is what closes its window: from here + a host's registration over it is refused (see `register()`).""" + with self._lock: + registered = self._registered + if registered is not None and self._is_default: + self._default_read = True + if registered is None: + raise SeamNotRegistered( + f"no {self.what} is registered at openDox's {self.name} seam " + f"(opendox.projection_seams.{self.name}), so {self.consequence}. " + f"openDox ships its own, {self.default}, but it is a " + "registration an ENTRY POINT makes and never a fallback here " + f"({_DEFAULTS_RULING}). {_ENTRY_POINTS} register it where no " + "host has. Nothing is registered now, so either nothing in " + "this process has run one of them, or " + f"opendox.projection_seams.{self.name}.unregister() has " + "dropped the registration since. A host that contributes its " + "own registers it at process start with\n\n " + + self.registration_call + "\n\nbefore anything reads it.") + return registered + + +class _KindSeam: + """The validator lookup: ONE registration PER KIND, each a host's or the + entry point's default, with the records `_Seam` keeps, kept per kind.""" + + name = "validators" + what = "validator" + callables = ("validate",) + #: The ONE call a host makes for each of its kinds. + registration_call = ( + "opendox.projection_seams.validators.register(, " + ")") + + def __init__(self) -> None: + #: `kind -> (validator, is_default)`. + self._registered: dict[str, tuple[Any, bool]] = {} + #: The kinds whose entry-point default a consumer has read. + self._default_read: set[str] = set() + self._lock = threading.Lock() + + @staticmethod + def _require_a_kind(kind: Any, call: str) -> str: + if not isinstance(kind, str) or not kind or kind != kind.strip(): + raise ValueError( + f"{call} takes the kind a validator validates, a non-empty name " + f"with no surrounding space, as a document's `kind` carries " + f"it, not {kind!r}") + return kind + + def register(self, kind: str, validator: Any) -> Any: + """A host's validator for ONE of its kinds, registered once, at + process start. Returns it. The rules are `_Seam.register()`'s, kept + per kind.""" + kind = self._require_a_kind(kind, "projection_seams.validators.register()") + with self._lock: + held = self._registered.get(kind) + if held is not None and held[0] is validator: + return validator + _probe(validator, self.callables, (), + "projection_seams.validators.register()", + f"the host's validator for {kind!r}") + with self._lock: + held = self._registered.get(kind) + if held is not None and held[0] is validator: + return validator + if held is None or (held[1] and kind not in self._default_read): + self._registered[kind] = (validator, False) + self._default_read.discard(kind) + return validator + over_a_host = not held[1] + if over_a_host: + raise SeamAlreadyRegistered( + f"a host's validator for {kind!r} is already registered at " + f"openDox's validator lookup ({name_of(held[0])}), and " + f"{name_of(validator)} would replace it. Registration happens " + "ONCE per kind, at process start. Call " + f"opendox.projection_seams.validators.unregister({kind!r}) " + "first if the swap is deliberate.") + raise SeamAlreadyRegistered( + f"openDox's own validator for {kind!r} ({name_of(held[0])}) is " + "registered, because an entry point registered it where no host " + f"had, and a consumer has already read it, so {name_of(validator)} " + "cannot replace it now: a document validated by one validator and " + "then by another under one name is two contracts. Register the " + f"host's own at process start, ahead of {_ENTRY_POINTS}. Call " + f"opendox.projection_seams.validators.unregister({kind!r}) first " + "if the swap is deliberate.") + + def register_default(self, kind: str, validator: Any) -> Any: + """AN ENTRY POINT's registration of openDox's own validator for one of + openDox's kinds, ONLY where nothing is registered for that kind. + Returns whatever is registered for it afterwards.""" + kind = self._require_a_kind( + kind, "projection_seams.validators.register_default()") + with self._lock: + held = self._registered.get(kind) + if held is not None and held[0] is validator: + return validator + _probe(validator, self.callables, (), + "projection_seams.validators.register_default()", + f"openDox's own validator for {kind!r}") + with self._lock: + if kind not in self._registered: + self._registered[kind] = (validator, True) + self._default_read.discard(kind) + return self._registered[kind][0] + + def unregister(self, kind: str | None = None) -> None: + """Drop the registration for `kind`, or every registration.""" + with self._lock: + if kind is None: + self._registered.clear() + self._default_read.clear() + else: + self._registered.pop(kind, None) + self._default_read.discard(kind) + + def is_registered(self, kind: str) -> bool: + """Is a validator registered for `kind`? Answers without reading.""" + return kind in self._registered + + def kinds(self) -> tuple[str, ...]: + """The kinds a validator is registered for, sorted.""" + with self._lock: + return tuple(sorted(self._registered)) + + def for_kind(self, kind: str) -> Any: + """The validator registered for `kind`, or `ValidatorNotRegistered` + naming the kind and the call. It never answers another kind's + validator. Reading a kind's default closes that kind's window.""" + with self._lock: + held = self._registered.get(kind) if isinstance(kind, str) else None + if held is not None and held[1]: + self._default_read.add(kind) + known = tuple(sorted(self._registered)) + if held is None: + raise ValidatorNotRegistered( + f"no validator is registered for kind {kind!r} at openDox's " + "validator lookup (opendox.projection_seams.validators), so " + "nothing of that kind can be validated. A document is " + "validated by the validator registered for its OWN kind, and " + "never by another's. Registered now: " + f"{', '.join(known) or 'none'}. openDox's entry points " + f"register openDox's own validator for openDox's kinds " + f"({_DEFAULTS_RULING}); a host registers its own for each of " + "its kinds at process start with\n\n " + + self.registration_call + "\n") + return held[0] + + +# -------------------------------------------------------------------------- +# the validation result a registered validator answers +# -------------------------------------------------------------------------- + +#: The three outcomes of a validation. "The snapshot is wrong" and "the check +#: could not be performed" are different facts, and the generate verbs give +#: them different consequences (`cli._validate`). +VALIDATED = "validated" +NOT_CONFORMANT = "not-conformant" +VALIDATOR_UNAVAILABLE = "validator-unavailable" + + +@dataclass +class ValidationResult: + """What a registered validator's `validate(path, *, strict, search_from)` + answers. A host's own result type conforms if it carries these attributes; + this one is openDox's. + + `validator` names what ran (a path or a name), or is `None` where no + validator was found at all. `outcome` refines `ok` without displacing it: + left unset it is derived from `ok`. `unavailable_reason` says why nothing + could be concluded, when nothing could.""" + + ok: bool + returncode: int + stdout: str + stderr: str + validator: Path | str | None + outcome: str | None = None + unavailable_reason: str | None = None + + def __post_init__(self) -> None: + if self.outcome is None: + self.outcome = VALIDATED if self.ok else NOT_CONFORMANT + + @property + def available(self) -> bool: + """Did the validator reach a verdict? False means nothing is known + about the document's conformance, not that it is bad.""" + return self.outcome != VALIDATOR_UNAVAILABLE + + def summary(self) -> str: + if self.validator is None: + return f"validator not found ({self.unavailable_reason or 'none registered'})" + tail = (self.stdout or self.stderr).strip().splitlines() + return tail[-1] if tail else f"returncode={self.returncode}" + + +# -------------------------------------------------------------------------- +# the four seams +# -------------------------------------------------------------------------- + +#: What a snapshot registry registration must carry as callables: its entry, +#: registry and source types, the entry-from-a-file reader, the per-entry +#: containment rule, the key parser, the publishable-ref rule the hosted plane +#: confines by (FR-048), and the data-source configuration readers. +REGISTRY_CALLABLES: tuple[str, ...] = ( + "SnapshotEntry", "SnapshotRegistry", "SnapshotSource", + "entry_from_snapshot_file", "resolve_within", "parse_key_id", + "is_publishable_ref", "data_source_from_options", "github_raw_base_url") + +#: ...and as values: the refresh bindings, the default ref, and the data +#: source's defaults `serve.main()` offers as its options' defaults. +REGISTRY_VALUES: tuple[str, ...] = ( + "BINDING_REFETCH", "BINDING_REGENERATE", "DEFAULT_REF", + "DEFAULT_INDEX_NAME", "DEFAULT_PUBLISH_PATH", "PEEK_TTL_SECONDS") + +#: What a corpus-root registration must carry: the predicate, the refusal a +#: verb prints, and the corpus's change rows (callables), and the roots a +#: snapshot is projected from (a value, possibly empty). +CORPUS_ROOT_CALLABLES: tuple[str, ...] = ( + "corpus_scan_defect", "corpus_root_refusal", "change_rows") +CORPUS_ROOT_VALUES: tuple[str, ...] = ("SCANNED_ROOTS",) + +#: What a writer registration must carry. +WRITER_CALLABLES: tuple[str, ...] = ("write_snapshot",) + +#: What a validator registration must carry. +VALIDATOR_CALLABLES: tuple[str, ...] = ("validate",) + +registry = _Seam( + "registry", what="snapshot registry and source", + callables=REGISTRY_CALLABLES, values=REGISTRY_VALUES, + default="opendox.default_registry", + consequence="no snapshot can be registered, served or regenerated") + +corpus_root = _Seam( + "corpus_root", what="corpus-root predicate", + callables=CORPUS_ROOT_CALLABLES, values=CORPUS_ROOT_VALUES, + default="opendox.default_projection.CORPUS_ROOT", + consequence="no path can be accepted as a corpus checkout") + +writer = _Seam( + "writer", what="snapshot writer", callables=WRITER_CALLABLES, + default="opendox.default_projection.WRITER", + consequence="no snapshot can be written") + +validators = _KindSeam() + + +def register_defaults() -> None: + """Register openDox's OWN default at each of the four seams, where no host + has registered one (R1Q10 (a), in R1Q3 (a)'s pattern). + + The entry points call this beside their registrations of the default + profile, home corpus and generator. It registers nothing over a host, and + reads nothing, so a host registration made afterwards, and before any + consumer reads a default, still replaces it. Importing this module + registers nothing.""" + from opendox import default_projection, default_registry + + registry.register_default(default_registry) + corpus_root.register_default(default_projection.CORPUS_ROOT) + writer.register_default(default_projection.WRITER) + for kind in default_projection.OWN_KINDS: + validators.register_default(kind, default_projection.VALIDATOR) diff --git a/src/opendox/rfc3339.py b/src/opendox/rfc3339.py new file mode 100644 index 0000000..72b1baa --- /dev/null +++ b/src/opendox/rfc3339.py @@ -0,0 +1,52 @@ +"""openDox's OWN check of an RFC 3339 date-time: the shape a typed +`--generated-at` must have before a generate verb records it (plan 034's T055). + +WHY IT IS HERE. `cli.py` refuses a malformed `--generated-at` before anything +is generated or written (`GeneratedAtRefused`), and it borrowed the check from +the consumer's generator through `consumer_reach`. The check is not the +generate operation, so the generator seam does not carry it (T052's own +record), and it is not the consumer's either: it is the neutral snapshot +contract's own rule, `generated-at-is-rfc3339` (openDox-spec's +`contracts/schemas/opendox-snapshot.schema.yaml`). So openDox owns it. + +WHAT IT ADMITS. A full date, an explicit time and an explicit offset (`Z` or +`+hh:mm`), in either case of `T` and `Z`, with optional fractional seconds, on +a day the calendar has, and nothing else. That is the neutral rule, which is +narrower than RFC 3339 in two places: the year is never `0000`, and the +seconds never read `60`, which no git commit date can hold and neither Python's +`datetime` nor a browser's `Date` can read. Digits are ASCII only, and the +whole string must match: no surrounding space and no trailing newline, since +the value is recorded verbatim. + +A CREATED FILE: it has no row in openxFactory's +`docs/opendox-carve-manifest.yaml`, because the manifest declares what LEAVES +openxFactory and never what a destination assembles (RULED OQ-C). +""" + +from __future__ import annotations + +import re +from datetime import datetime + +__all__ = ["is_rfc3339_datetime"] + +#: The shape, before the instant. `[0-9]`, never `\d`, which also matches the +#: digits of other scripts. +_SHAPE = re.compile( + r"(?!0000)[0-9]{4}-(?:0[1-9]|1[0-2])-(?:0[1-9]|[12][0-9]|3[01])" + r"[Tt](?:[01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9](?:\.[0-9]+)?" + r"(?:[Zz]|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])") + + +def is_rfc3339_datetime(value: object) -> bool: + """True when `value` is an RFC 3339 date-time as the neutral contract + admits one: the shape above, AND an instant. The shape admits + `2026-02-30T00:00:00Z`, which is no day, so the parse runs too.""" + if not isinstance(value, str) or not _SHAPE.fullmatch(value): + return False + stamp = value[:-1] + "+00:00" if value[-1] in "Zz" else value + try: + datetime.fromisoformat(stamp) + except ValueError: + return False + return True diff --git a/src/opendox/serve.py b/src/opendox/serve.py index a0cd530..0d85aa5 100644 --- a/src/opendox/serve.py +++ b/src/opendox/serve.py @@ -133,19 +133,24 @@ from opendox import doxbench_knowledge # noqa: E402 from opendox import doxbench_packet # noqa: E402 from opendox import doxbench_telemetry # noqa: E402 -# THE PROJECTION REGISTRY, NAMED LATE, AND openDox'S OWN DEFAULTS BESIDE IT -# (BUILD slice 2b). `from openxdox import snapshot_registry as registry_mod` -# stood on this line: an import of the layer that PINS openDox, evaluated when -# this module loads, which `design.md`:243 refuses — *"what must not survive is -# the direction, not the calls."* The stand-in resolves on first attribute -# access, so every `registry_mod.X` read below is unchanged. +# THE PROJECTION REGISTRY, THROUGH ITS SEAM, AND openDox'S OWN DEFAULTS BESIDE +# IT. `from openxdox import snapshot_registry as registry_mod` stood here until +# BUILD slice 2b made it a late `consumer_reach` stand-in, which still refused +# the first read in a lone openDox, so a server could not be built standalone +# (plan 034, research R7). Since plan 034 T055 `registry_mod` is the proxy over +# the snapshot registry SEAM (`projection_seams.registry`): each `registry_mod.X` +# read below resolves the registry registered at that moment, openDox's own +# (`opendox.default_registry`) where no host has contributed one, and every one +# of them is unchanged. # -# EXCEPT `build_server`'s two signature DEFAULTS (:1304, :1308), which no -# stand-in can defer: a default argument is evaluated where the `def` sits, at -# import time. openDox owns those two values (`defaults.py`), and openXdox-code's -# drift guard holds the literals together. +# EXCEPT `build_server`'s two signature DEFAULTS (`index_name` and +# `peek_ttl_seconds`), which no proxy can defer: a default argument is +# evaluated where the `def` sits, at import time. openDox owns those two +# values (`defaults.py`), and openXdox-code's drift guard holds the literals +# together. from opendox import consumer_reach # noqa: E402 from opendox import defaults # noqa: E402 +from opendox import projection_seams # noqa: E402 # § 3.4 slice S5: `build_server()` publishes the VIEW MANIFEST on # `/capabilities`, the one line slice S3 built both ends of and left for the # slice at which a contribution first exists to deliver. @@ -166,7 +171,7 @@ # Both modules are stdlib-only and name no sibling, so this adds no reach. from opendox import default_generator, generator_seam # noqa: E402 -registry_mod = consumer_reach.snapshot_registry # noqa: E402 +registry_mod = projection_seams.registry.proxy # noqa: E402 # THE BY-FUNCTION SPLIT (`split-opendox-two-layer-product` § 2.4, PRs 2 and 3 # of 4). The openDox column's routes live in `serve_workbench.py` (the doxBench # workbench surface) and `serve_project.py` (projects, the notebook tile action, @@ -258,11 +263,14 @@ # this module's name, and now a real definition here rather than a forwarder # into the layer that pins this one. # -# `hosted_ref_refused` KEEPS ITS LATE STAND-IN. It decides at :785 that a hosted -# response must never NAME a session ref, and FR-048 is the PROJECTION column's -# confinement rule, not this core's: the route moved, the hosted-plane rule did -# not. It resolves on first CALL, so it costs no import-time reach. -hosted_ref_refused = consumer_reach.hosted_ref_refused # noqa: E402 +# `hosted_ref_refused` IS DEFINED BELOW SINCE PLAN 034 T055, and the rule it +# applies has not moved. It decides that a hosted response must never NAME a +# session ref (FR-048), and its one dependency is the REGISTRY's own rule for +# "a ref a hosted plane may see", `is_publishable_ref`, which it now reads +# through the registry seam like every other `registry_mod.X`. It had to come +# here with the core `/snapshot.json` arm's handlers (see `_serve_snapshot`), +# which ask it on every response: forwarded to openXdox, a standalone server +# refused every one. from opendox.serve_wire import ( # noqa: E402,F401 AGENT_INVOCATION_REFUSAL, CONTEXT_REDUCED_REASON_MAX_LENGTH, @@ -547,6 +555,24 @@ def _is_loopback(host: str) -> bool: return host in LOOPBACK_HOSTS +# --------------------------- hosted-plane ref confinement (pure) --------------------------- + +def hosted_ref_refused(loopback: bool, ref: str | None) -> bool: + """Whether a request naming `ref` must be REFUSED because this is the + hosted plane (007-workbench-branch-sessions T083, FR-048: "a hosted request + naming a non-`main` ref MUST refuse"). + + The test is the BIND, never the advertised capability: the bind is what + makes a plane hosted. `None` or blank means `main`, so a ref-less request + is untouched, and the LOCAL plane is untouched entirely. "A ref a hosted + plane may see" is the REGISTERED registry's rule, `is_publishable_ref`, + read through the registry seam (plan 034 T055), so there is still one + definition of it per process: the one the registry serves by.""" + if loopback: + return False + return not registry_mod.is_publishable_ref(ref) + + # --------------------------- the human console (FR-019) --------------------------- # # FR-019's THIRD clause — "reject and report any agent or automated invocation" — @@ -613,28 +639,27 @@ def resolve_source_path(checkout_root: Path, url_tail: str) -> Path | None: Percent-decoding happens BEFORE the containment check so `%2e%2e` cannot slip past. - The containment check itself lives in `snapshot_registry.resolve_within` - so the SAME rule applies per registry entry (task 2.2); this stays the + The containment check itself lives in the registry's `resolve_within` so + the SAME rule applies per registry entry (task 2.2); this stays the single-root entry point every existing caller and test uses. ARRIVED HERE AT § 3.4 SLICE S6 (RULED Q4) with the route it confines, from - `openxdox/serve_projection.py`:66. The body is unchanged: the RULE is - `snapshot_registry.resolve_within` and the rule has NOT moved — it is the - projection column's, it is the same rule per registry entry, and openDox - reaches it through the same late `consumer_reach` seam `serve_workbench.py` - already uses at five sites. What moved is the ENTRY POINT, to the module - that now declares the route and to the module `notebook_action.py`:52 - already imported it from. A second copy of the containment rule here would - be the fork `route_extension.py`:89 names; a forwarder into the consumer for - this core's OWN route is the direction `design.md`:243 names. This is - neither.""" + `openxdox/serve_projection.py`:66. The body is unchanged: the RULE is the + registry's `resolve_within`, it is the same rule per registry entry, and + it is reached through the registry SEAM (plan 034 T055) that + `serve_workbench.py` reaches it by too, so a process has one rule: the + registered registry's. What moved is the ENTRY POINT, to the module that + now declares the route and to the module `notebook_action.py`:52 already + imported it from. A second copy of the containment rule here would be the + fork `route_extension.py`:89 names. This is not one.""" return registry_mod.resolve_within(Path(checkout_root), url_tail) def _checkout_real(checkout_root: Path | str) -> bool: """A real corpus checkout, not the served image's empty `/srv/empty` sentinel. - "Real" means SCANNABLE AS A CORPUS (`corpus_root.corpus_scan_defect`) — the same + "Real" means SCANNABLE AS A CORPUS, as the REGISTERED corpus-root predicate + decides (`projection_seams.corpus_root`, plan 034 T055) — the same predicate `cli.py`'s `--repo-root` guard uses, which is the same value under a second spelling (runbook §2). It used to mean merely "an existing, non-empty directory", which is what the sentinel fails; but that let a wrong-but-populated @@ -644,13 +669,12 @@ def _checkout_real(checkout_root: Path | str) -> bool: this makes the code keep the promise. The empty sentinel still fails it, so the hosted image is unchanged. - `corpus_root` is a stdlib-plus-`doc_health.corpus` module for exactly this - reason: this runs on every `build_server`, including the served image's, and - reaching the predicate through `generator` would newly require PyYAML in a - startup path that serves snapshots and scans nothing. Imported lazily, as this - module does for every sibling.""" - from openxdox.corpus_root import corpus_scan_defect - return corpus_scan_defect(checkout_root) is None + This runs on every `build_server`, including the served image's, so the + predicate is structural and imports nothing heavy: openDox's own asks for + a git repository's root, and the governed one for the roots a snapshot is + projected from.""" + return projection_seams.corpus_root.current().corpus_scan_defect( + checkout_root) is None def resolve_actor(checkout_root: Path | str, override: str | None = None) -> str | None: @@ -772,8 +796,13 @@ class DashboardHandler(serve_workbench.WorkbenchRoutes, # `import opendox.serve` require the layer that PINS # openDox. The stand-ins carry the same method names and # forward to the same functions with the same `self` on - # first call, so the core `/snapshot.json` arm and every - # contributed binding behave exactly as before. + # first call, so every contributed binding behaves + # exactly as before. Since plan 034 T055 the core + # `/snapshot.json` arm's handlers are THIS class's own + # (`_serve_snapshot` below), and the projection stand-in + # forwards one method, `_serve_index`, the one its + # contributed `/snapshot-index.json` binding names + # (T084 hands the column to the handler facet). consumer_reach.LateGateRoutes, consumer_reach.LateProjectionRoutes, # Plan 034 T011 (#1144 task 2.2): openxFactory's @@ -1117,6 +1146,88 @@ def do_HEAD(self): # noqa: N802 if not self._route(head_only=True): super().do_HEAD() + # ---- the snapshot: `/snapshot.json`'s core arm (plan 034 T055) ---- + # THIS CLASS'S OWN SINCE T055. The arm (`_route`'s first test) stayed core + # at § 2.4 PR 3, because it tests `path == self.snapshot_route`, a + # per-server keyword a frozen `RouteBinding.pattern` cannot carry, while + # its handlers travelled to openXdox's projection column and were reached + # through `consumer_reach.LateProjectionRoutes`. So a standalone server + # refused every `/snapshot.json`. The four methods below are that route's + # handlers, answering from the REGISTERED snapshot source: the query key, + # the active snapshot's bytes, the route itself, and FR-048's per-entry + # hosted refusal, which `_serve_source` asks too. The rules they consult + # are the registry's, through its seam. + def _query_key(self) -> tuple[str | None, str | None]: + """The optional `?repository=&ref=` of a read route. No repository + means the ACTIVE entry, which is what a query-less request asks for.""" + params = urllib.parse.parse_qs(urllib.parse.urlsplit(self.path).query) + return ((params.get("repository") or [None])[0], + (params.get("ref") or [None])[0]) + + def _read_snapshot(self) -> bytes | None: + """The ACTIVE snapshot's bytes, through the registry when a source is + bound, and from the configured path when none is (a hand-built + handler).""" + entry = self._active_entry() + if entry is not None: + return entry.read_bytes() + try: + return Path(self.snapshot_path).read_bytes() + except OSError: + return None + + def _serve_snapshot(self, head_only: bool) -> None: + """`/snapshot.json`: the active snapshot, or the registered + `(repository, ref)` the query names. An unknown pair is a 404, and the + view the client already has stays as it is. + + On the HOSTED plane a non-`main` ref refuses before anything resolves + (FR-048), and so does a query-less request whose ACTIVE entry is at a + session ref: the refusal follows the entry a request resolved to, not + only the ref it named. An id the registry does not hold may be an + aggregate the source composes, at the default ref only, and composed + from publishable members only off loopback; openDox's own source + composes none.""" + repository, ref = self._query_key() + if hosted_ref_refused(self.loopback, ref): + self._send_json(403, {"ok": False, "error": "session_unavailable", + "message": HOSTED_SESSION_REFUSAL}) + return + if repository and self.source is not None: + entry = self.source.registry.resolve(repository, ref) + if entry is None: + composed = None + if registry_mod.is_publishable_ref(ref): + composed = self.source.compose_view( + repository, publishable_only=not self.loopback) + if composed is not None: + self._serve_bytes(json.dumps(composed).encode("utf-8"), + JSON_CTYPE, head_only) + return + self.send_error(404, "no such snapshot") + return + if self._hosted_entry_refused(entry): + return + self._serve_bytes(entry.read_bytes(), JSON_CTYPE, head_only, + entry=entry) + return + if repository: + self.send_error(404, "no such snapshot") + return + if self._hosted_entry_refused(self._active_entry()): + return + self._serve_bytes(self._read_snapshot(), JSON_CTYPE, head_only) + + def _hosted_entry_refused(self, entry) -> bool: + """Refuse, and answer, when the entry a request RESOLVED to is at a + session ref and this is the hosted plane. Returns whether it answered.""" + if entry is None or not hosted_ref_refused( + self.loopback, getattr(entry, "ref", None)): + return False + self._send_json(403, {"ok": False, "error": "session_unavailable", + "message": HOSTED_SESSION_REFUSAL}) + return True + # ---- source pass-through ---- # ARRIVED HERE AT § 3.4 SLICE S6 (RULED Q4, openxFactory#656 comment # 5642758731) from `openxdox/serve_projection.py`:295-361, byte for byte. @@ -1124,18 +1235,15 @@ def do_HEAD(self): # noqa: N802 # is the optional `@/` split it starts with, and # `_refuse_bare_source` is the bare route's one-line answer. # - # THEY STILL REACH THE CONSUMER, and that is unchanged rather than - # overlooked. `hosted_ref_refused` is FR-048, the projection column's - # hosted-plane confinement; `self.source.registry` and - # `self._hosted_entry_refused` are the snapshot registry's per-entry rules, - # and `_hosted_entry_refused` stays on `consumer_reach.LateProjectionRoutes` - # with `_serve_snapshot`, which also calls it. Q4 rules on ROUTE OWNERSHIP — - # "this is a route-ownership correction at § 4.3 of the packet, not a new - # capability" — so the arm comes here and the rules it consults stay where - # they are, reached through the seam this module already declares. Every one - # of those reaches is guarded by `self.source is not None`, which is None - # only in a hand-built handler, so the SHAPE of the neutral case is already - # here even though the BUILD arc (§ 3.5/3.6) is what makes it reachable. + # THE RULES THEY CONSULT ARE THE REGISTRY'S. `hosted_ref_refused` is + # FR-048's hosted-plane confinement, and `self.source.registry` and + # `self._hosted_entry_refused` apply the snapshot registry's per-entry + # rules. Q4 ruled on ROUTE OWNERSHIP — "this is a route-ownership + # correction at § 4.3 of the packet, not a new capability" — so the arm + # came here and its rules stayed the registry's. Since plan 034 T055 they + # are reached through the registry SEAM, so the route answers in a lone + # openDox, from openDox's own registry, and from a host's where one is + # registered. def _keyed_source(self, tail: str): """Split an optional `@/` prefix off a `/source/` tail. The prefix is honoured ONLY when it names a REGISTERED pair, so a real @@ -1516,11 +1624,12 @@ def _read_source_revision(snapshot_path: Path) -> str | None: def _default_home_factory(root): """`home_corpus`'s shape (`adapter, ref = factory(root)`), over - `WorkingTreeCorpus` at its own bare defaults (`required_fields=()`; phase - 2's T054 sets the neutral fields R1Q13 decides). Its `__init__` takes no - root -- it is root-agnostic, and `resolve(ref)` reads `ref.location` -- - so one `CorpusRef` per call carries the root this factory was given, and - the adapter itself needs none. + `WorkingTreeCorpus` at its own defaults, whose `required_fields` is the + small neutral field set R1Q13 (a) decides (`NEUTRAL_FIELDS`, `title` and + `summary`, since plan 034 T054). Its `__init__` takes no root -- it is + root-agnostic, and `resolve(ref)` reads `ref.location` -- so one + `CorpusRef` per call carries the root this factory was given, and the + adapter itself needs none. READS THE WORKING TREE, uncommitted edits included -- RULING, Brett Heap, 2026-09-27, via the holder: "Working tree (Recommended)". A standalone @@ -1656,6 +1765,12 @@ def build_server( # AND openDox's OWN snapshot generator (5.4, T052; R1Q10 (a), in the same # R1Q3 (a) pattern), registered only where no host has contributed one. generator_seam.register_default(default_generator.GENERATOR) + # AND openDox's OWN snapshot registry and source, corpus-root predicate, + # writer and validators (5.5, T055; the same ruling and pattern), each only + # where no host has registered its own. The snapshot source below is built + # from the registered registry, which is what lets a server be BUILT with + # nothing else installed (plan 034, research R7). + projection_seams.register_defaults() from opendox import doxbench_turns # Imported HERE rather than at module scope, for the reason that is @@ -2121,14 +2236,17 @@ def _refuse_impossible_checkout_root(value: Path | str) -> int: the served image mounts the empty `/srv/empty` sentinel precisely so `_checkout_real` reports false and the write-bearing affordances stay off. So it serves, and says loudly what it will not be able to do — which on a - LOCAL run is the same wrong path, diagnosed.""" - from openxdox.corpus_root import corpus_root_refusal, corpus_scan_defect + LOCAL run is the same wrong path, diagnosed. + + Both answers are the REGISTERED corpus-root predicate's + (`projection_seams.corpus_root`, plan 034 T055).""" + predicate = projection_seams.corpus_root.current() path = Path(value) if not path.is_dir(): - print(corpus_root_refusal(value, flag="--checkout-root", - shape=_SERVE_SHAPE), file=sys.stderr) + print(predicate.corpus_root_refusal(value, flag="--checkout-root", + shape=_SERVE_SHAPE), file=sys.stderr) return 1 - defect = corpus_scan_defect(value) + defect = predicate.corpus_scan_defect(value) if defect is not None: print(f"--checkout-root {path.resolve()} is not a corpus checkout: " f"{defect}", file=sys.stderr) @@ -2156,6 +2274,10 @@ def main(argv: list[str] | None = None) -> int: corpus_adapter.register_default_home(_default_home_factory) # AND openDox's own snapshot generator (5.4, T052), the same way. generator_seam.register_default(default_generator.GENERATOR) + # AND openDox's own projection defaults (5.5, T055), the same way, and + # BEFORE the parser: its option defaults below read the registered + # registry (`registry_mod.DEFAULT_REF` and the data source's defaults). + projection_seams.register_defaults() parser = argparse.ArgumentParser(prog="ideation-dashboard-serve", description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) parser.add_argument("--web-dir", default=str(Path(__file__).resolve().parent / "web"), @@ -2196,7 +2318,8 @@ def main(argv: list[str] | None = None) -> int: help=f"ref of the raw-file source (default: {registry_mod.DEFAULT_REF})") parser.add_argument("--data-source-path", default=registry_mod.DEFAULT_PUBLISH_PATH, help=f"path of the published tree inside the source repository " - f"(default: {registry_mod.DEFAULT_PUBLISH_PATH})") + f"(default: " + f"{registry_mod.DEFAULT_PUBLISH_PATH or 'the source root'})") parser.add_argument("--data-source-index", default=registry_mod.DEFAULT_INDEX_NAME, help=f"index filename inside the data source (default: " f"{registry_mod.DEFAULT_INDEX_NAME})") @@ -2226,21 +2349,30 @@ def main(argv: list[str] | None = None) -> int: rc = _refuse_impossible_checkout_root(args.checkout_root) if rc: return rc - data_source = registry_mod.data_source_from_options( - directory=args.data_source_dir, - url=args.data_source_url or (registry_mod.github_raw_base_url( - args.data_source_github, ref=args.data_source_github_ref, - path=args.data_source_path) if args.data_source_github else None), - token_env=args.data_source_token_env, - ) - serve(args.web_dir, args.snapshot, args.checkout_root, host=args.host, - port=args.port, actor=args.actor, - repository=args.repository, ref=args.ref, - data_source=data_source, index_name=args.data_source_index, - source_roots=_source_roots_from_args(args.source_root), - local_index=args.local_index, - project_register=args.project_register, - peek_ttl_seconds=args.data_source_peek_seconds) + try: + data_source = registry_mod.data_source_from_options( + directory=args.data_source_dir, + url=args.data_source_url or (registry_mod.github_raw_base_url( + args.data_source_github, ref=args.data_source_github_ref, + path=args.data_source_path) if args.data_source_github else None), + token_env=args.data_source_token_env, + ) + serve(args.web_dir, args.snapshot, args.checkout_root, host=args.host, + port=args.port, actor=args.actor, + repository=args.repository, ref=args.ref, + data_source=data_source, index_name=args.data_source_index, + source_roots=_source_roots_from_args(args.source_root), + local_index=args.local_index, + project_register=args.project_register, + peek_ttl_seconds=args.data_source_peek_seconds) + except projection_seams.ProjectionSeamError as exc: + # A PROJECTION SEAM'S REFUSAL, before a socket is bound (plan 034 + # T055): openDox's own registry refuses a declared data source or + # local index rather than ignoring it, since only a host's registry + # reads one. Reported on stderr with a non-zero status, as the + # `--checkout-root` refusal above is. + print(f"serve refused: {exc}", file=sys.stderr) + return 1 return 0 diff --git a/src/opendox/serve_wire.py b/src/opendox/serve_wire.py index 8c2207d..b062a81 100644 --- a/src/opendox/serve_wire.py +++ b/src/opendox/serve_wire.py @@ -64,10 +64,12 @@ the three in `serve.py` would still have forced at least one column to import `serve` while `serve` imported it, which is the cycle this module exists to prevent — this module was the right first home for all three and the wrong -last one for each. `serve.py` takes `hosted_index` and `hosted_ref_refused` -from `serve_projection` and the three strings from this module's re-export, so -`serve.hosted_ref_refused`, `serve.hosted_index` and `serve.JSON_CTYPE` all -resolve exactly as they did for the suites that call them directly. What +last one for each. `serve.py` took `hosted_index` and `hosted_ref_refused` +from `serve_projection` and the three strings from this module's re-export. +Since plan 034 T055 `serve.py` DEFINES `hosted_ref_refused` itself, over the +snapshot registry seam's `is_publishable_ref`, which is the predicate's one +dependency (see above), so `serve.hosted_ref_refused` and `serve.JSON_CTYPE` +still resolve for the suites that call them directly. What remains here is this module's own stated remit: the shared wire vocabulary and pure envelope builders over already-validated inputs, plus fixed refusal prose BOTH columns read. diff --git a/src/opendox/serve_workbench.py b/src/opendox/serve_workbench.py index 0a42c7e..037ed2f 100644 --- a/src/opendox/serve_workbench.py +++ b/src/opendox/serve_workbench.py @@ -46,8 +46,12 @@ from opendox import doxbench_knowledge from opendox import doxbench_packet from opendox import doxbench_threads -from opendox import consumer_reach -registry_mod = consumer_reach.snapshot_registry +# THE SNAPSHOT REGISTRY, THROUGH ITS SEAM (plan 034 T055): each +# `registry_mod.resolve_within` below resolves the registry registered at that +# moment, openDox's own where no host has contributed one, so the workbench +# routes confine by the same rule `/source` does, in a lone openDox too. +from opendox import projection_seams +registry_mod = projection_seams.registry.proxy from opendox.serve_wire import ( DOXBENCH_ABSTRACT_REFUSED_PROSE_BYTES, DOXBENCH_ABSTRACT_REFUSED_SUBJECT_BYTES, @@ -173,7 +177,7 @@ def _workbench_model_port(self): def _indexed_sources(self, projection): """This tile's staged set, as indexable sources. - Read through `snapshot_registry.resolve_within` — the SINGLE + Read through the registry's `resolve_within` — the SINGLE containment authority `/source` already uses — rather than through a second path check of this route's own, because two confinement rules are how one of them drifts. A path that does not resolve, is not a diff --git a/src/opendox/workbench.py b/src/opendox/workbench.py index c3f9743..8740d9b 100644 --- a/src/opendox/workbench.py +++ b/src/opendox/workbench.py @@ -67,12 +67,14 @@ slug, ) from .boundary import OutputBoundary -from . import consumer_reach # The home-corpus seam (4.1, 4.2): stdlib only, and it imports nothing back # from this package, so reading it at import time adds no edge a cycle or a # consumer could hang on. from . import corpus_adapter -find_validator = consumer_reach.find_validator +# The validator lookup (plan 034 T055): stdlib only, like the seam above. A +# manifest is validated by the validator registered for its own kind, +# openDox's own for `ideation-workbench`, and never through the consumer. +from . import projection_seams # -------------------------------------------------------------------------- # contract constants (mirror ideation-workbench.schema.yaml — the READ-ONLY @@ -432,21 +434,45 @@ class ManifestValidation: def summary(self) -> str: if self.validator is None: - return "validator not found (no reachable openxFactory checkout)" + return ("validator not found (no validator registered for " + f"{KIND!r} reached a verdict)") tail = (self.stdout or self.stderr).strip().splitlines() return tail[-1] if tail else f"returncode={self.returncode}" def validate_manifest(path: Path | str, *, validator: Path | None = None, strict: bool = False, search_from: Path | None = None) -> ManifestValidation: - """Validate a written manifest with the pinned validator (single-file mode, - kind auto-detected). Single-file mode does NOT run the committed-manifest - guard (that is a repo scan) — so validating a manifest under a tmp/gitignored - path checks schema + workbench rules cleanly.""" + """Validate a written manifest. + + An EXPLICIT `validator` is a validator script, run on the manifest in + single-file mode (kind auto-detected), as it always was. Single-file mode + does NOT run the committed-manifest guard (that is a repo scan) — so + validating a manifest under a tmp/gitignored path checks schema + workbench + rules cleanly. + + With none given, the manifest is validated by the validator REGISTERED for + its kind, `ideation-workbench` (`projection_seams.validators`, plan 034 + T055), which is openDox's own where no host registered another, and never + by a reach into the consumer. A search starts at `search_from`, else at the + manifest's own directory. No validator registered for the kind, or one that + could not reach a verdict, is `ok=False` with no validator, which is the + answer "validator not found" always gave.""" path = Path(path).resolve() - validator = validator or find_validator(search_from or path.parent) if validator is None: - return ManifestValidation(False, -1, "", "validator not found", None) + try: + registered = projection_seams.validators.for_kind(KIND) + except projection_seams.ValidatorNotRegistered as exc: + return ManifestValidation(False, -1, "", str(exc).split("\n", 1)[0], + None) + result = registered.validate(path, strict=strict, + search_from=(search_from or path.parent,)) + if not result.available: + return ManifestValidation( + False, result.returncode, result.stdout, + result.stderr or str(result.unavailable_reason or ""), None) + return ManifestValidation(bool(result.ok), result.returncode, + result.stdout, result.stderr, + result.validator) cmd = [sys.executable, str(validator), str(path)] if strict: cmd.append("--strict") diff --git a/tests/test_authoring_seam.py b/tests/test_authoring_seam.py index 440ef88..4af72c8 100644 --- a/tests/test_authoring_seam.py +++ b/tests/test_authoring_seam.py @@ -357,15 +357,17 @@ def test_an_entry_point_registers_the_local_git_corpus_when_no_host_has() -> Non def test_build_server_registers_the_default_where_no_host_has() -> None: """`serve.build_server()`'s half of 4.1a, run from its own source lines. - `serve.build_server()` cannot be called directly today, unlike - `cli.build_parser()`: T011 only made `opendox.serve` IMPORTABLE, and - `build_server()`'s own body still reaches `openxdox.snapshot_registry`/ - `openxdox.corpus_root` (unrelated, later tasks; plan 034 research R7), so - calling it whole fails for a reason that has nothing to do with this - registration (Copilot review of openDox-code#45, "Missing test coverage - for server default registration": the entry-point falsifier above - exercises only `cli.build_parser()`, so a regression removing `serve - .build_server()`'s own registration line would leave this suite green). + `serve.build_server()` could not be called directly when this case was + written, unlike `cli.build_parser()`: T011 only made `opendox.serve` + IMPORTABLE, and `build_server()`'s own body reached + `openxdox.snapshot_registry`/`openxdox.corpus_root` until plan 034 T055 + routed both through declared seams (research R7), so calling it whole + failed for a reason that had nothing to do with this registration (Copilot + review of openDox-code#45, "Missing test coverage for server default + registration": the entry-point falsifier above exercises only + `cli.build_parser()`, so a regression removing `serve.build_server()`'s + own registration line would leave this suite green). It stays lifted, + because that isolates the one line under test. Lifted by AST instead, mirroring `test_profile_registration.py`'s own technique for the identical problem with `ROUTE_EXTENSIONS`: the module- @@ -530,10 +532,10 @@ def test_the_default_can_actually_classify_a_proposal() -> None: (`_stage_as_a_repository_if_git_is_available`). This proves the fix through the REGISTERED default, not a mock of it. - Deliberately NOT asserting the exact `required_fields` tuple: T054 - (phase 2) sets the neutral fields; today's bare default is `()`, and - hard-coding that would make this test wrong the day T054 lands rather - than testing what it actually claims to -- that the call SUCCEEDS. + Deliberately NOT asserting the exact `required_fields` tuple (T054 made + the default `NEUTRAL_FIELDS`, where it was `()`): hard-coding either would + test the field set rather than what this case claims -- that the call + SUCCEEDS. SKIPS WHERE `git` IS NOT ON PATH, like the falsifier above (Copilot review of openDox-code#45, "Skip Git-dependent integration test when diff --git a/tests/test_consumer_reach.py b/tests/test_consumer_reach.py index 4155ade..0248a22 100644 --- a/tests/test_consumer_reach.py +++ b/tests/test_consumer_reach.py @@ -142,6 +142,17 @@ def _import_in_subprocess(module: str, *, consumer_blocked: bool, # moves with it. They move here IN THE SAME ACT, as the record test asks. "opendox.cli", "opendox.serve", + # Plan 034 T055 (#1144 5.5 and 4.3 in part; R1Q10 (a)). The projection + # mechanism's seams and openDox's own defaults behind them, which retired + # the `snapshot`, `snapshot_registry`, `corpus_root` and `generator` + # stand-ins and the six names bound over them. A seam whose whole job is to + # let a HOST hand openDox its governed registry, predicate, writer and + # validators is where a reach into `openxdox` would look reasonable, and + # none of the four makes one. + "opendox.projection_seams", + "opendox.default_registry", + "opendox.default_projection", + "opendox.rfc3339", ) #: Modules that STILL require the consumer at import time, with the reason. They @@ -574,8 +585,8 @@ def test_resolution_forwards_to_the_real_module_and_caches(tmp_path: Path) -> No assert reach.ANSWER == 42 assert reach.resolve() is module_object assert reach.resolve() is module_object, "the resolved module is cached" - assert consumer_reach.function(reach, "verb")(3) == 6, ( - "a late callable forwards arguments and the return value") + assert reach.verb(3) == 6, ( + "a resolved module's function is the module's own, called through") finally: sys.modules.pop("openxdox.pretend", None) if parent_was_created: @@ -597,128 +608,13 @@ def test_a_dunder_lookup_does_not_resolve_the_consumer() -> None: assert "unresolved" in repr(reach) -@pytest.fixture() -def pretend_corpus_root(): - """A stand-in for `openxdox.corpus_root` carrying a `SCANNED_ROOTS` tuple. - - Installed in `sys.modules` rather than imported, so the test holds on a - machine with openXdox-code present and in CI where it is absent — and the - fake PARENT is removed again only if this fixture created it, because - leaving an empty `openxdox` package behind would make every later test in - the process see an importable-but-empty consumer. - """ - module_object = type(sys)("openxdox.corpus_root") - module_object.SCANNED_ROOTS = ("contracts", "docs", "openspec") - sys.modules["openxdox.corpus_root"] = module_object - parent_was_created = "openxdox" not in sys.modules - if parent_was_created: - sys.modules["openxdox"] = type(sys)("openxdox") - try: - yield module_object - finally: - sys.modules.pop("openxdox.corpus_root", None) - if parent_was_created: - sys.modules.pop("openxdox", None) - - -def test_constructing_a_constant_resolves_nothing() -> None: - """The laziness claim, made for the VALUE member of the family. - - `scanned_roots` is built at `consumer_reach` import time, in a package that - must import with no consumer present. If construction resolved, the module - that exists to remove import-time reaches would itself be one. - """ - from opendox import consumer_reach - - absent = consumer_reach.constant( - consumer_reach.module("no_such_column", reason="a test's own"), "ROOTS") - assert "no_such_column.ROOTS" in repr(absent) - assert " None: - """Every operation `SCANNED_ROOTS` is actually subjected to, forwarded. - - `cli.py`:228 iterates it; the rest are what a re-exported sequence constant - meets from a caller who believes it is still the tuple it was. The point of - the assertions is that the stand-in is INDISTINGUISHABLE from the tuple at - these operations — anything less and the name could not have been left in - place on a line the carve manifest does not declare. - """ - from opendox import consumer_reach - - roots = consumer_reach.constant( - consumer_reach.module("corpus_root", reason="a test's own"), - "SCANNED_ROOTS") - real = pretend_corpus_root.SCANNED_ROOTS - - assert list(roots) == list(real), "iteration — cli.py:228's `for root in ...`" - assert len(roots) == 3 - assert "docs" in roots - assert "no-such-root" not in roots - assert roots[0] == "contracts" - assert roots[-1] == "openspec" - assert list(roots[1:]) == ["docs", "openspec"], "slicing is indexing too" - assert roots == real, "equality against the real tuple" - assert not (roots != real) - assert roots != ("something", "else") - assert bool(roots) is True - assert str(roots) == str(real) - assert hash(roots) == hash(real), ( - "a constant re-exported into a set or a dict key must hash as its value") - assert roots.resolve() is real - - -def test_a_constant_does_not_forward_attribute_reads(pretend_corpus_root) -> None: - """`__getattr__` is deliberately absent — the refusal is part of the design. - - An attribute read on a constant is almost always a caller who wanted the - MODULE, and answering it would turn the value stand-in into the general - facade this module refuses to be. It must fail as an `AttributeError`, the - way the tuple it stands for would. - """ - from opendox import consumer_reach - - roots = consumer_reach.constant( - consumer_reach.module("corpus_root", reason="a test's own"), - "SCANNED_ROOTS") - with pytest.raises(AttributeError): - roots.corpus_root_refusal - - -def test_an_unavailable_consumer_refuses_at_the_operation_not_at_the_binding() -> None: - """The error path, and WHERE it fires: at use, naming the layering. - - With no consumer installed the binding is still constructed — that is the - whole point — so the refusal has to arrive at the first operation, and it - has to name which way the pin runs rather than reading as a missing-module - accident. - """ - from opendox import consumer_reach - - absent = consumer_reach.constant( - consumer_reach.module("no_such_column", reason="a test's own"), "ROOTS") - - for operation in (lambda: list(absent), - lambda: len(absent), - lambda: "x" in absent, - lambda: absent[0], - lambda: absent == ("x",), - lambda: absent != ("x",), - lambda: hash(absent), - lambda: bool(absent), - lambda: str(absent), - lambda: absent.resolve()): - with pytest.raises(consumer_reach.ConsumerReachUnavailable) as caught: - operation() - message = str(caught.value) - assert "openxdox.no_such_column" in message - assert "RULED OQ-2" in message - assert isinstance(caught.value.__cause__, ModuleNotFoundError) +# THE VALUE AND CALLABLE STAND-INS ARE RETIRED (plan 034 T055). `constant` +# stood for ONE site, `cli.py`'s `SCANNED_ROOTS`, and `function` for six +# callables (`find_validator`, `corpus_root_refusal`, `generate_snapshot`, +# `is_rfc3339_datetime`, `hosted_ref_refused` among them); every one of them is +# read from a declared seam now, or is openDox's own, so the four cases that +# held `_LateConsumerValue` to the tuple it stood for went with it. +# `tests/test_projection_seams.py` holds the seams that replaced them. @pytest.fixture() @@ -866,10 +762,10 @@ def test_a_column_standing_in_for_nothing_is_refused() -> None: def test_the_two_live_columns_name_the_methods_serve_dispatches() -> None: """The real bindings, held against the names `serve.py` and § 2.4 rely on. - `_serve_snapshot` is the one to watch: `/snapshot.json`'s HANDLER travelled - to the projection column while its dispatch ARM stayed core, so `serve.py` - itself calls `self._serve_snapshot`. Dropping it from the list would leave - every import green and `/snapshot.json` broken. + `_serve_index` is the projection column's ONE forwarded method since plan + 034 T055: the § 2.4 binding for `/snapshot-index.json` names it. The core + `/snapshot.json` arm's handlers, which travelled to the column at § 2.4 + PR 3 while their dispatch ARM stayed core, are `serve.py`'s own again. """ from opendox import consumer_reach @@ -883,9 +779,10 @@ def test_the_two_live_columns_name_the_methods_serve_dispatches() -> None: consumer_reach.LateProjectionRoutes.LATE_COLUMN assert (proj_module, proj_class) == ("openxdox.serve_projection", "ProjectionRoutes") - for required in ("_serve_snapshot", "_serve_index"): - assert required in proj_methods, ( - f"{required} is dispatched by name and must be on the column") + assert proj_methods == ("_serve_index",), ( + "the projection column forwards the ONE method its contributed " + "`/snapshot-index.json` binding names. The core `/snapshot.json` " + "arm's handlers are serve.py's own since plan 034 T055") # § 3.4 SLICE S6, RULED Q4 (openxFactory#656 comment 5642758731): the # `/source` pair is `serve.py`'s own FIXED CORE ARM now, so the three # methods that answer it must NOT be forwarded into the consumer. Asserted @@ -898,6 +795,13 @@ def test_the_two_live_columns_name_the_methods_serve_dispatches() -> None: f"{departed} answers /source, which RULED Q4 makes openDox's own " "core arm; it must be defined in serve.py, not forwarded to " "openxdox.serve_projection") + # PLAN 034 T055: the same for the core `/snapshot.json` arm. Forwarded, + # its four handlers refused every `/snapshot.json` of a standalone server. + for departed in ("_query_key", "_read_snapshot", "_serve_snapshot", + "_hosted_entry_refused"): + assert departed not in proj_methods, ( + f"{departed} answers /snapshot.json, the neutral product's own core " + "arm; since plan 034 T055 it is defined in serve.py") def test_the_prefix_is_refused_rather_than_doubled() -> None: @@ -926,29 +830,23 @@ def test_the_prefix_is_refused_rather_than_doubled() -> None: # docstring gives), and it was the one module the table left out; # * `cli.py`'s other four aliases, the two late callables BUILD slice 2b # bound for the generate verbs and the `--generated-at` check, and the - # one late constant. Calling a late callable, or iterating the late - # constant, at import time resolves the consumer as surely as reading - # `gate_mod` does. + # one late constant. # None of the five is read at import time, so the tree was already right. - "branch_session.py": ("gate_console",), - "cli.py": ("gate_mod", "snapshot_mod", "corpus_root_refusal", - "generate_snapshot", "is_rfc3339_datetime", "SCANNED_ROOTS"), - "serve_workbench.py": ("registry_mod",), - "workbench.py": ("find_validator",), - # Slice 2b step 4. `consumer_reach` and `defaults` are deliberately NOT - # listed: both ARE read at import time and must be. Constructing a stand-in - # resolves nothing (`test_importing_the_seam_resolves_nothing`), and the two - # late COLUMNS are mixin bases, which a class statement needs before its - # first instance exists; `defaults` is openDox's own module, which is the - # point of it. What must not be read at import time is a name BOUND to a - # stand-in, and these are serve.py's two. # - # § 3.4 SLICE S6, RULED Q4: `resolve_source_path` LEFT this tuple because it - # stopped being a stand-in — it is a real `def` in `serve.py` now, reaching - # `registry_mod.resolve_within` from inside a function body like every other - # deferred use. Nothing is unguarded by the removal: `registry_mod` is still - # listed, and that is the name the containment call actually reads. - "serve.py": ("registry_mod", "hosted_ref_refused"), + # NARROWED BY PLAN 034 T055 to the gate column alone. `cli.py`'s + # `snapshot_mod`, `corpus_root_refusal`, `generate_snapshot`, + # `is_rfc3339_datetime` and `SCANNED_ROOTS`, `serve.py`'s `registry_mod` + # and `hosted_ref_refused`, `serve_workbench.py`'s `registry_mod` and + # `workbench.py`'s `find_validator` bind no stand-in any more: the + # projection mechanism is read from declared seams + # (`opendox.projection_seams`, `opendox.generator_seam`), and the two + # `registry_mod`s are the registry seam's proxy, which + # `tests/test_projection_seams.py` holds to the same import-time rule. + # `serve.py` still names the two late COLUMNS as mixin bases, which a class + # statement needs before its first instance exists, so it binds no name + # here either. + "branch_session.py": ("gate_console",), + "cli.py": ("gate_mod",), } diff --git a/tests/test_doxbench_entrypoint.py b/tests/test_doxbench_entrypoint.py index 0eac480..bda1149 100644 --- a/tests/test_doxbench_entrypoint.py +++ b/tests/test_doxbench_entrypoint.py @@ -24,44 +24,34 @@ That is the same discipline `tests/hermeticity.py` now applies to the `omp` binary itself (§2.3): two independent layers, neither one's substitute. -THE ENTRYPOINT RUNS FOR REAL, AND SIX OF ITS REACHES ARE STOOD IN (plan 034 -T035). `cmd_generate_and_open` is what these cases exist to drive, and they now -drive it in a lone checkout. Before this, all four fixture cases errored in -fixture setup at the first reach across the carve, `openxdox.corpus_root` -(`ConsumerReachUnavailable`). The reaches are the ones plan 034 names as its -phase-1 limit (plan.md, "Phase-1 limit"; research R7), and phase 2 gives -openDox its own snapshot and generator. Until then each one is stood in here, -and nothing else is: - - * three in `cli`, the `consumer_reach` names `generate-and-open` calls: - `corpus_root_refusal`, `generate_snapshot` and - `snapshot_mod.write_snapshot`. A fourth, `SCANNED_ROOTS`, is read only to - warn about a snapshot with no document, and the stand-in snapshot carries - one, so it is never reached; - * three in `serve`, T011's `standalone` stand-ins - (`tests/test_route_handler_contribution.py`, section 6): the snapshot - source, through `build_server`'s own `snapshot_source=` seam; - `_checkout_real`; and `registry_mod`'s two `BINDING_*` constants. - -`_checkout_real` ANSWERS TRUE HERE, where T011's answers false. The model port -is gated on the `session` verdict (`serve_workbench._workbench_model_port`): -loopback, a real checkout, and a resolved human actor. The carve's `base-repo` -fixture, which these cases used to serve, was a real corpus checkout, so the -stand-in gives the answer that fixture gave. The predicate itself is -openXdox's (`corpus_root.corpus_scan_defect`), and a lone checkout cannot ask -it. +THE ENTRYPOINT RUNS FOR REAL, AND NOTHING OF IT IS STOOD IN (plan 034 T055). +`cmd_generate_and_open` is what these cases exist to drive, and they drive it +in a lone checkout. Plan 034 T035 first made them run here, by standing in for +six reaches across the carve that phase 1 could not route (plan.md, +"Phase-1 limit"; research R7): the corpus-root predicate, the generator and +the writer in `cli`, and the snapshot source, `_checkout_real` and the +registry's refresh bindings in `serve`. T055 routes every one of them through +a declared seam where the entry points register openDox's own default, so the +fixture below runs openDox's own generator over a real git checkout, writes +through openDox's own writer, and builds the server over openDox's own +snapshot registry and source. + +`_checkout_real` IS TRUE HERE because the checkout is real: a git repository, +which is what openDox's own corpus-root predicate asks for. The model port is +gated on the `session` verdict (`serve_workbench._workbench_model_port`): +loopback, a real checkout, and a resolved human actor. What these cases test is openDox's own: the port the entrypoint declares (`doxbench_install.declared_model_port_factory`), the one adapter it resolves -to, and the session root it is handed. None of them reads the snapshot. The -checkout is an empty scratch directory rather than the carve's `base-repo` -fixture, which stayed in openxFactory and does not exist at this leg. +to, and the session root it is handed. None of them reads the snapshot. """ from __future__ import annotations -import json -import types +import os +import shutil +import subprocess +from pathlib import Path import pytest @@ -81,38 +71,36 @@ def _handler_class(httpd): # -------------------------------------------------------------------------- -# the phase-1 stand-ins (see the module docstring), and nothing else +# a real git checkout to generate from and serve # -------------------------------------------------------------------------- -class _StandInSource: - """`build_server`'s snapshot source as T011's `standalone` fixture injects - it: nothing registered, nothing baked, so no session is re-derived.""" +#: T050's plain-documents fixture: documents across the six stations. +PLAIN_DOCUMENTS = Path(__file__).resolve().parent / "fixtures" / "plain-documents" - refresh_binding = None - baked_repository = None - class registry: - active = None +def _git(root: Path, *args: str) -> None: + """`git` in `root`, with no inherited `GIT_*` variable and no user or + system configuration, as the fixture's own identity at a fixed date.""" + env = {k: v for k, v in os.environ.items() if not k.startswith("GIT_")} + env.update({ + "GIT_AUTHOR_NAME": "fixture", "GIT_AUTHOR_EMAIL": "fixture@example.invalid", + "GIT_COMMITTER_NAME": "fixture", + "GIT_COMMITTER_EMAIL": "fixture@example.invalid", + "GIT_AUTHOR_DATE": "2026-09-27T12:00:00+00:00", + "GIT_COMMITTER_DATE": "2026-09-27T12:00:00+00:00", + "GIT_CONFIG_GLOBAL": os.devnull, "GIT_CONFIG_SYSTEM": os.devnull, + }) + subprocess.run(["git", "-C", str(root), *args], check=True, + capture_output=True, env=env) - def bootstrap(self): - pass - -def _stand_in_generate_snapshot(repo_root, repository, *, source_revision=None, - **_ignored): - """`generate_snapshot`'s stand-in: exactly the fields `generate-and-open` - reads back when it reports the run. ONE document, so `_report` has no empty - projection to warn about and never reaches `SCANNED_ROOTS`.""" - return {"repository": repository, - "generation": {"source_revision": source_revision}, - "documents": [{"id": "stand-in.md"}]} - - -def _stand_in_write_snapshot(snapshot, output, boundary): - """`snapshot.write_snapshot`'s stand-in: the file `build_server` is handed, - which it opens only to read `generation.source_revision` back.""" - output.write_text(json.dumps(snapshot), encoding="utf-8") - return output +def _checkout(tmp_path: Path) -> Path: + root = tmp_path / "checkout" + shutil.copytree(PLAIN_DOCUMENTS, root) + _git(root, "-c", "init.defaultBranch=main", "init", "-q") + _git(root, "add", "-A") + _git(root, "commit", "-qm", "fixture") + return root @pytest.fixture() @@ -134,34 +122,19 @@ def _refuse_spawn(argv, environment, cwd): monkeypatch.setattr(br, "_spawn_child", _refuse_spawn) - monkeypatch.setattr(cli_mod, "corpus_root_refusal", - lambda root, shape=None: None) - monkeypatch.setattr(cli_mod, "generate_snapshot", _stand_in_generate_snapshot) - monkeypatch.setattr(cli_mod, "snapshot_mod", types.SimpleNamespace( - write_snapshot=_stand_in_write_snapshot)) - monkeypatch.setattr(serve_mod, "_checkout_real", lambda root: True) - monkeypatch.setattr(serve_mod, "registry_mod", types.SimpleNamespace( - BINDING_REGENERATE="regenerate", BINDING_REFETCH="refetch")) - built = [] real_build_server = serve_mod.build_server def _capture(*args, **kwargs): - # A TRIPWIRE, not only an injection: the day the entrypoint declares a - # snapshot source of its own (phase 2), this stand-in must go, and this - # line says so instead of silently standing in over it. - assert "snapshot_source" not in kwargs, ( - "the entrypoint now declares its own snapshot source; drop the " - "phase-1 stand-in from this fixture") - httpd = real_build_server(*args, snapshot_source=_StandInSource(), - **kwargs) + # A CAPTURE, and nothing else: the server is the entrypoint's own, + # built over the snapshot source the registered registry makes. + httpd = real_build_server(*args, **kwargs) built.append(httpd) return httpd monkeypatch.setattr(serve_mod, "build_server", _capture) - checkout = tmp_path / "checkout" - checkout.mkdir() + checkout = _checkout(tmp_path) session_root = tmp_path / "model-sessions" args = cli_mod.build_parser().parse_args([ "generate-and-open", diff --git a/tests/test_generator_seam.py b/tests/test_generator_seam.py index b96921b..392edd6 100644 --- a/tests/test_generator_seam.py +++ b/tests/test_generator_seam.py @@ -40,9 +40,11 @@ deadlocking the seam. What the default generates is T054's neutral projection, and `tests/test_neutral_projection.py` holds it. 7. EACH ENTRY POINT REGISTERS IT. `cli.build_parser()` and `cli.main()` run for - real. `serve.build_server()` and `serve.main()` still cannot run in a lone - checkout (research R7), so their registration is executed from their own - source lines, as `tests/test_authoring_seam.py` does for the home corpus. + real. `serve.build_server()`'s and `serve.main()`'s registration is executed + from their own source lines, as `tests/test_authoring_seam.py` does for the + home corpus, because that isolates the one statement under test. Since plan + 034 T055 both also run whole in a lone checkout + (`tests/test_projection_seams.py`). 8. `CorpusAdapter` STAYS CLOSED AT SIX MEMBERS. `--noconftest` SAFE. The autouse fixture below saves and restores the three @@ -919,10 +921,11 @@ def test_a_host_registered_first_is_kept_by_the_cli_entry_points() -> None: @pytest.mark.parametrize("function", ("build_server", "main")) def test_the_server_entry_points_register_openDoxs_own_generator(function: str) -> None: - """`serve.build_server()` and `serve.main()` cannot run in a lone checkout - until phase 2 routes their snapshot source (research R7). So the import - and the registration are lifted out of `serve.py` and executed: the tree's - own statements, not a paraphrase of them.""" + """The import and the registration, lifted out of `serve.py` and executed: + the tree's own statements, not a paraphrase of them. They were lifted + because `serve.build_server()` and `serve.main()` could not run in a lone + checkout until plan 034 T055 routed their snapshot source (research R7); + they stay lifted because that isolates the one statement under test.""" module_body, body = _module_and_function(SERVE, function) lifted = [stmt for stmt in module_body if _binds_the_seam(stmt)] + \ [stmt for stmt in body if _is_the_registration(stmt)] diff --git a/tests/test_neutral_projection.py b/tests/test_neutral_projection.py index fb74cc7..e1812f9 100644 --- a/tests/test_neutral_projection.py +++ b/tests/test_neutral_projection.py @@ -11,8 +11,9 @@ source. Those are the first three cases below, in that order. F5.3 itself runs -`python -m opendox.cli generate`, whose verb reaches the consumer's generator -until T055 routes it, so T056 and T063 quote it. +`python -m opendox.cli generate`, whose verb generates through the generator +seam since T055, with openDox's own generator where no host registered one, +and T056 and T063 quote it. THE SCHEMA. `tests/fixtures/opendox-snapshot.schema.yaml` is openDox-spec's neutral snapshot contract, copied byte for byte from openDox-spec#16 at diff --git a/tests/test_profile_registration.py b/tests/test_profile_registration.py index 1b16d33..1866283 100644 --- a/tests/test_profile_registration.py +++ b/tests/test_profile_registration.py @@ -32,14 +32,14 @@ own rewriting all depend on it. 7. THE SERVED COMPOSITION POINT IS EXECUTED, NOT DESCRIBED (RULED ASK-6 -> 1, `5635150678`). `serve.build_server()` reads `ROUTE_EXTENSIONS` through this - proxy. `opendox.serve` imports in a lone checkout since plan 034's T011, but a - SERVER still cannot be BUILT in one until phase 2: `build_server()` reaches - `openxdox.snapshot_registry` for its snapshot source and - `openxdox.corpus_root` in `_checkout_real` (plan 034, research R7). So the - statements that make up the composition point are lifted OUT of - `build_server`'s body BY AST and executed against a stand-in - `route_extension` seam. That runs the real source lines — an assertion about - the tree, not a paraphrase of it. + proxy. `opendox.serve` imports in a lone checkout since plan 034's T011, and + a SERVER can be BUILT in one since T055, which routed `build_server()`'s + snapshot source and `_checkout_real` through declared seams (plan 034, + research R7, measured the limit). The statements that make up the + composition point are lifted OUT of `build_server`'s body BY AST and + executed against a stand-in `route_extension` seam, which isolates them. + That runs the real source lines — an assertion about the tree, not a + paraphrase of it. 8. THE ENTRY POINTS REGISTER openDox's OWN DEFAULT (R1Q3 (a), with (i) and (ii), `5817152735`; RN-1 (a), `5850003126`). `cli.build_parser()`, `serve.build_server()` and both `main()`s register `opendox.default_profile` @@ -512,10 +512,11 @@ class _Empty: # own files BY AST and executed against stand-ins for the two § 2.4 seams. # # Plan 034's T011 made both modules import, and section 10 below builds the -# parser for real. A server still cannot be BUILT in a lone checkout until -# phase 2 (research R7), so its composition point stays lifted. The parser's -# lifted cases stay beside it: they hold the READ itself, apart from the entry -# point's registration of the default that now precedes it. +# parser for real. A server can be BUILT in a lone checkout since T055 +# (research R7 measured the limit it lifted), and its composition point stays +# lifted because that holds the READ itself, as the parser's lifted cases +# beside it do, apart from the entry point's registration of the default that +# now precedes it. # # What runs is the tree's own statements — the module-level or function-level # binding of the proxy, and the statement that reads a facet off it — so a @@ -979,10 +980,11 @@ def test_serve_main_registers_the_default_first_and_a_host_may_still_replace_it( register its own, and registering builds nothing. So a host that registers afterwards still replaces the default. - `serve.main()` cannot yet be run in a lone checkout, even to `--help`: its - own option defaults read `openxdox.snapshot_registry` (research R7; T055 - routes that reach in phase 2). So its registration is executed from its own - source, as the server's is. + Its registration is executed from its own source, as the server's is, + which isolates the two statements this case is about. `serve.main()` could + not be run in a lone checkout at all until plan 034 T055, even to + `--help`, because its option defaults read `openxdox.snapshot_registry` + (research R7); they read the registry seam now. """ _module_body_, main_body = _module_body(SERVE, "main") first = main_body[:2] diff --git a/tests/test_projection_seams.py b/tests/test_projection_seams.py new file mode 100644 index 0000000..838d43c --- /dev/null +++ b/tests/test_projection_seams.py @@ -0,0 +1,1233 @@ +"""The projection mechanism's seams, openDox's own defaults behind them, and +the verbs routed through them: plan 034's T055. + +T055 realizes #1144's 5.5 and 4.3 in part. It gives the snapshot registry and +source, the corpus-root predicate, the snapshot writer and the validator lookup +a declared seam each (`opendox.projection_seams`) with an openDox default each +(`opendox.default_registry`, `opendox.default_projection`), which the entry +points register where no host has (R1Q10 (a), in R1Q3 (a)'s pattern). It +routes `cli.py`'s two generations and the registry's regenerate through +`generator_seam.generate()`, chooses a validator by the snapshot's `kind`, and +retires the `consumer_reach` stand-ins those reaches used. Its falsifier is +F4.1's scan, down by these reaches; the PR body quotes that scan. This file +holds the behaviour behind it: + +1. NOTHING REGISTERED REFUSES, naming the seam and its call, at each seam. +2. THE REGISTRATION RULES: names are probed, one registration, a host replaces + the default only until a consumer has read it, the same one again is a + no-op, and the validator lookup keeps all of that per kind. +3. THE ENTRY POINTS REGISTER THE DEFAULTS, and keep a host registered first. +4. STANDALONE, with no sibling importable, `generate` writes and a server + builds and answers `/snapshot.json`, `/capabilities` and `/source/`. +5. openDox's own registry and source: containment, keys, the baked entry, the + refusals of what only a host's registry offers, and the regenerate through + the registered generator and writer. +6. openDox's own corpus-root predicate, writer and validator stand-in. +7. The generate verbs: the report, the refusals, and validation by kind. +8. `branch_session`'s routed sites and `workbench.validate_manifest`. +9. No proxy over a seam is read at import time. +10. openDox's own RFC 3339 check. + +Every case starts with nothing registered at the four seams or the generator +seam, and puts back every registry it found. + +A CREATED FILE: no carve-manifest row (RULED OQ-C). +""" + +from __future__ import annotations + +import argparse +import ast +import http.client +import json +import os +import shutil +import subprocess +import sys +import textwrap +import threading +import types +from pathlib import Path + +import pytest + +from opendox import cli +from opendox import corpus_adapter +from opendox import default_generator +from opendox import default_projection +from opendox import default_registry +from opendox import defaults +from opendox import domain_profile +from opendox import generator_seam as gs +from opendox import projection_seams as ps +from opendox import rfc3339 +from opendox import serve +from opendox import workbench +from opendox import branch_session as bs +from opendox import consumer_reach +from opendox.boundary import BoundaryViolation, OutputBoundary + +ROOT = Path(__file__).resolve().parent.parent +PACKAGE = ROOT / "src" / "opendox" +PLAIN_DOCUMENTS = ROOT / "tests" / "fixtures" / "plain-documents" +ANCHOR_DATE = "2026-09-27T12:00:00+00:00" +NEUTRAL = gs.NEUTRAL_SNAPSHOT_KIND +SIBLINGS = ("openxdox", "ideation_dashboard", "doc_health", + "corpus_adapter_openxfactory") + +SINGLE_SEAMS = {"registry": ps.registry, "corpus_root": ps.corpus_root, + "writer": ps.writer} + + +# --------------------------------------------------------------------------- +# isolation, stand-ins and fixtures +# --------------------------------------------------------------------------- + +@pytest.fixture(autouse=True) +def _isolated_registries(): + """Nothing registered at the four seams or the generator seam, the entry + points' default home corpus registered, and every registry PUT BACK whole, + records included, so an entry point's default is never handed back as a + host's.""" + single = {name: (seam._registered, seam._is_default, seam._default_read) + for name, seam in SINGLE_SEAMS.items()} + kinds = (dict(ps.validators._registered), set(ps.validators._default_read)) + generator = (gs._registered, gs._is_default, gs._generated_from_default, + gs._default_generations_under_way, gs._registration_serial) + profile = (domain_profile._registered, domain_profile._is_default, + domain_profile._built_from_default) + home = corpus_adapter._home_factory + for seam in SINGLE_SEAMS.values(): + seam.unregister() + ps.validators.unregister() + gs.unregister() + corpus_adapter.register_home(cli._default_home_factory) + yield + for name, seam in SINGLE_SEAMS.items(): + seam._registered, seam._is_default, seam._default_read = single[name] + ps.validators._registered, ps.validators._default_read = kinds + (gs._registered, gs._is_default, gs._generated_from_default, + gs._default_generations_under_way, gs._registration_serial) = generator + (domain_profile._registered, domain_profile._is_default, + domain_profile._built_from_default) = profile + corpus_adapter._home_factory = home + + +def _stub(seam) -> types.SimpleNamespace: + """A registration carrying exactly the seam's names.""" + members = {name: (lambda *args, **kwargs: None) for name in seam.callables} + members.update({name: f"<{name}>" for name in seam.values}) + return types.SimpleNamespace(**members) + + +class _Validator: + """A stand-in validator that records its calls and answers `result`.""" + + def __init__(self, result=None, remedy=None): + self.calls = [] + self.result = result or ps.ValidationResult( + True, 0, "stand-in: 0 error(s)", "", "stand-in") + self.dependency_remedy = remedy + + def validate(self, path, *, strict=False, search_from=()): + self.calls.append({"path": Path(path), "strict": strict, + "search_from": tuple(search_from)}) + return self.result + + +def _git(root: Path, *args: str) -> str: + """`git` in `root` as the fixture's own identity at a fixed date, with no + inherited `GIT_*` variable and no user or system configuration.""" + env = {k: v for k, v in os.environ.items() if not k.startswith("GIT_")} + env.update({ + "GIT_AUTHOR_NAME": "fixture", "GIT_AUTHOR_EMAIL": "fixture@example.invalid", + "GIT_COMMITTER_NAME": "fixture", + "GIT_COMMITTER_EMAIL": "fixture@example.invalid", + "GIT_AUTHOR_DATE": ANCHOR_DATE, "GIT_COMMITTER_DATE": ANCHOR_DATE, + "GIT_CONFIG_GLOBAL": os.devnull, "GIT_CONFIG_SYSTEM": os.devnull, + }) + return subprocess.run(["git", "-C", str(root), *args], check=True, + capture_output=True, env=env).stdout.decode() + + +def _repository(tmp_path: Path, *, copy: Path | None = PLAIN_DOCUMENTS, + files: dict[str, str] | None = None, + name: str = "repository") -> Path: + """A fresh plain git repository, committed.""" + root = tmp_path / name + if copy is not None: + shutil.copytree(copy, root) + else: + root.mkdir(parents=True) + for relative, body in (files or {}).items(): + path = root / relative + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(body, encoding="utf-8") + _git(root, "-c", "init.defaultBranch=main", "init", "-q") + _git(root, "add", "-A") + _git(root, "commit", "-q", "--allow-empty", "-m", "fixture") + return root + + +def _generate(repo: Path, out: Path, *extra: str) -> int: + return cli.main(["generate", "--repo-root", str(repo), + "--repository", "garden", "--output", str(out), *extra]) + + +def _get(httpd, path: str) -> tuple[int, dict, bytes]: + worker = threading.Thread(target=httpd.serve_forever, daemon=True) + worker.start() + try: + host, port = httpd.server_address[:2] + connection = http.client.HTTPConnection(host, port, timeout=10) + connection.request("GET", path) + response = connection.getresponse() + return response.status, dict(response.getheaders()), response.read() + finally: + httpd.shutdown() + httpd.server_close() + worker.join(timeout=10) + + +def _served(tmp_path: Path, **kwargs): + """A real server over a generated snapshot of a real git checkout.""" + repo = _repository(tmp_path) + out = tmp_path / "run" / "snapshot.json" + assert _generate(repo, out, "--no-validate") == 0 + (tmp_path / "web").mkdir(exist_ok=True) + return repo, out, serve.build_server(tmp_path / "web", out, repo, port=0, + **kwargs) + + +# --------------------------------------------------------------------------- +# 1 — nothing registered refuses, naming the seam and its call (4.2) +# --------------------------------------------------------------------------- + +@pytest.mark.parametrize("name", sorted(SINGLE_SEAMS)) +def test_each_seam_refuses_when_nothing_is_registered(name) -> None: + seam = SINGLE_SEAMS[name] + with pytest.raises(ps.SeamNotRegistered) as caught: + seam.current() + message = str(caught.value) + for expected in (seam.registration_call, f"opendox.projection_seams.{name}", + seam.default, "ENTRY POINT", "cli.build_parser()", + "cli.main()", "serve.build_server()", "serve.main()", + "R1Q10 (a)", "5850003126", "unregister()"): + assert expected in message, f"the refusal no longer says {expected!r}" + + +def test_the_validator_lookup_refuses_a_kind_with_no_validator() -> None: + """Never another kind's validator: a document is read by its own kind.""" + other = _Validator() + ps.validators.register("some-kind", other) + with pytest.raises(ps.ValidatorNotRegistered) as caught: + ps.validators.for_kind("another-kind") + message = str(caught.value) + assert "'another-kind'" in message and "some-kind" in message + assert ps.validators.registration_call in message + assert isinstance(caught.value, ps.SeamNotRegistered) + + +def test_a_proxy_resolves_on_each_read_and_never_on_a_dunder() -> None: + proxy = ps.registry.proxy + assert "registry" in repr(proxy), "repr must not resolve the seam" + with pytest.raises(AttributeError): + proxy.__wrapped__ + first, second = _stub(ps.registry), _stub(ps.registry) + first.DEFAULT_REF, second.DEFAULT_REF = "first", "second" + ps.registry.register(first) + assert proxy.DEFAULT_REF == "first" + ps.registry.unregister() + ps.registry.register(second) + assert proxy.DEFAULT_REF == "second", "the proxy holds nothing across reads" + ps.registry.unregister() + with pytest.raises(ps.SeamNotRegistered): + proxy.DEFAULT_REF + + +# --------------------------------------------------------------------------- +# 2 — the registration rules +# --------------------------------------------------------------------------- + +@pytest.mark.parametrize("name", sorted(SINGLE_SEAMS)) +def test_a_registration_missing_a_name_is_refused_naming_it(name) -> None: + seam = SINGLE_SEAMS[name] + for missing in (*seam.callables, *seam.values): + registration = _stub(seam) + delattr(registration, missing) + with pytest.raises(TypeError) as caught: + seam.register(registration) + assert f"lacks {missing}." in str(caught.value) + assert not seam.is_registered() + + +def test_a_callable_name_that_is_not_callable_is_refused() -> None: + registration = _stub(ps.writer) + registration.write_snapshot = "not callable" + with pytest.raises(TypeError, match="lacks write_snapshot"): + ps.writer.register(registration) + + +def test_a_registration_that_cannot_hand_a_name_over_is_refused_with_the_cause() -> None: + class _Lazy: + def __getattr__(self, name): + raise RuntimeError(f"cannot load {name}") + + with pytest.raises(TypeError) as caught: + ps.corpus_root.register(_Lazy()) + assert isinstance(caught.value.__cause__, RuntimeError) + assert not ps.corpus_root.is_registered() + + +@pytest.mark.parametrize("name", sorted(SINGLE_SEAMS)) +def test_one_registration_and_a_deliberate_swap(name) -> None: + seam = SINGLE_SEAMS[name] + host, other = _stub(seam), _stub(seam) + assert seam.register(host) is host + assert seam.register(host) is host, "the same registration again is a no-op" + with pytest.raises(ps.SeamAlreadyRegistered) as caught: + seam.register(other) + assert f"opendox.projection_seams.{name}.unregister()" in str(caught.value) + assert seam.current() is host + seam.unregister() + assert seam.register(other) is other + + +@pytest.mark.parametrize("name", sorted(SINGLE_SEAMS)) +def test_a_host_replaces_the_default_only_until_it_is_read(name) -> None: + seam = SINGLE_SEAMS[name] + default, host, late = _stub(seam), _stub(seam), _stub(seam) + assert seam.register_default(default) is default + assert seam.register_default(_stub(seam)) is default, ( + "register_default never displaces what is registered") + assert seam.register(host) is host, "nothing read the default, so it yields" + seam.unregister() + seam.register_default(default) + assert seam.current() is default + with pytest.raises(ps.SeamAlreadyRegistered) as caught: + seam.register(late) + message = str(caught.value) + for expected in ("openDox's own default", "already read", "R1Q3 (ii)", + "5817152735", "RN-1 (a)", "unregister()"): + assert expected in message, f"the refusal no longer says {expected!r}" + assert seam.current() is default + assert seam.register_default(_stub(seam)) is default + + +@pytest.mark.parametrize("name", sorted(SINGLE_SEAMS)) +def test_register_default_never_displaces_a_host(name) -> None: + seam = SINGLE_SEAMS[name] + host = _stub(seam) + seam.register(host) + assert seam.register_default(_stub(seam)) is host + assert seam.current() is host + + +def test_a_default_missing_a_name_is_refused_even_over_a_registration() -> None: + ps.writer.register(_stub(ps.writer)) + with pytest.raises(TypeError, match="lacks write_snapshot"): + ps.writer.register_default(types.SimpleNamespace()) + + +def test_the_validator_lookup_keeps_the_rules_per_kind() -> None: + default, host, late = _Validator(), _Validator(), _Validator() + ps.validators.register_default(NEUTRAL, default) + assert ps.validators.register_default(NEUTRAL, _Validator()) is default + assert ps.validators.register(NEUTRAL, host) is host, ( + "nothing read the kind's default, so it yields") + with pytest.raises(ps.SeamAlreadyRegistered): + ps.validators.register(NEUTRAL, late) + ps.validators.unregister(NEUTRAL) + ps.validators.register_default(NEUTRAL, default) + ps.validators.register("host-kind", host) + assert ps.validators.for_kind(NEUTRAL) is default + with pytest.raises(ps.SeamAlreadyRegistered, match="already read"): + ps.validators.register(NEUTRAL, late) + assert ps.validators.register("host-kind", host) is host + assert ps.validators.kinds() == ("host-kind", NEUTRAL) + for bad in ("", " padded ", None, 3): + with pytest.raises(ValueError): + ps.validators.register(bad, host) + with pytest.raises(TypeError, match="lacks validate"): + ps.validators.register("x-kind", types.SimpleNamespace()) + ps.validators.unregister() + assert ps.validators.kinds() == () + + +def test_register_defaults_registers_openDoxs_own_at_each_seam_and_reads_nothing() -> None: + ps.register_defaults() + assert ps.registry._registered is default_registry + assert ps.corpus_root._registered is default_projection.CORPUS_ROOT + assert ps.writer._registered is default_projection.WRITER + for kind in default_projection.OWN_KINDS: + assert ps.validators._registered[kind] == (default_projection.VALIDATOR, True) + for seam in SINGLE_SEAMS.values(): + host = _stub(seam) + assert seam.register(host) is host, ( + "registering the defaults read nothing, so a host still replaces them") + + +def test_openDoxs_own_kinds_are_the_neutral_snapshot_and_the_workbench_manifest() -> None: + assert default_projection.OWN_KINDS == (NEUTRAL, "ideation-workbench") + assert default_projection.WORKBENCH_KIND == workbench.KIND + + +def test_the_default_registry_carries_openDoxs_own_values() -> None: + assert default_registry.DEFAULT_INDEX_NAME == defaults.DEFAULT_INDEX_NAME + assert default_registry.PEEK_TTL_SECONDS == defaults.PEEK_TTL_SECONDS + assert default_registry.DEFAULT_REF == "main" + assert (default_registry.BINDING_REFETCH, + default_registry.BINDING_REGENERATE) == ("refetch", "regenerate") + + +def test_importing_the_seams_and_defaults_registers_and_resolves_nothing() -> None: + script = _blocking_siblings() + textwrap.dedent(""" + from opendox import projection_seams as ps + import opendox.default_registry, opendox.default_projection, opendox.rfc3339 + assert not ps.registry.is_registered() + assert not ps.corpus_root.is_registered() + assert not ps.writer.is_registered() + assert ps.validators.kinds() == () + print("clean") + """) + done = subprocess.run([sys.executable, "-c", script], capture_output=True, + text=True, cwd=ROOT, timeout=120) + assert done.returncode == 0 and "clean" in done.stdout, done.stderr + + +# --------------------------------------------------------------------------- +# 3 — the entry points register the defaults, and keep a host's +# --------------------------------------------------------------------------- + +def _assert_defaults_registered() -> None: + assert ps.registry._registered is default_registry + assert ps.corpus_root._registered is default_projection.CORPUS_ROOT + assert ps.writer._registered is default_projection.WRITER + assert set(default_projection.OWN_KINDS) <= set(ps.validators.kinds()) + + +def test_the_cli_entry_points_register_the_projection_defaults() -> None: + cli.build_parser() + _assert_defaults_registered() + for seam in SINGLE_SEAMS.values(): + seam.unregister() + ps.validators.unregister() + with pytest.raises(SystemExit) as exited: + cli.main(["--help"]) + assert exited.value.code == 0 + _assert_defaults_registered() + + +def test_serve_main_runs_its_help_standalone_over_the_registered_registry(capsys) -> None: + """`serve.main()` could not run in a lone checkout even to `--help` until + T055 (research R7): its option defaults read the consumer's registry.""" + with pytest.raises(SystemExit) as exited: + serve.main(["--help"]) + assert exited.value.code == 0 + _assert_defaults_registered() + out = " ".join(capsys.readouterr().out.split()) + assert "(default: main" in out + assert "(default: the source root)" in out + + +def test_a_host_registered_first_is_kept_by_every_entry_point() -> None: + hosts = {name: _stub(seam) for name, seam in SINGLE_SEAMS.items()} + hosts["registry"] = types.SimpleNamespace(**{ + name: getattr(default_registry, name) + for name in (*ps.REGISTRY_CALLABLES, *ps.REGISTRY_VALUES)}) + for name, seam in SINGLE_SEAMS.items(): + seam.register(hosts[name]) + validator = _Validator() + ps.validators.register(NEUTRAL, validator) + cli.build_parser() + with pytest.raises(SystemExit): + cli.main(["--help"]) + with pytest.raises(SystemExit): + serve.main(["--help"]) + for name, seam in SINGLE_SEAMS.items(): + assert seam.current() is hosts[name], name + assert ps.validators.for_kind(NEUTRAL) is validator + + +def test_build_server_builds_standalone_over_openDoxs_own_registry(tmp_path) -> None: + """Research R7's probe, which refused at the snapshot source, now builds.""" + repo, out, httpd = _served(tmp_path) + try: + bound = httpd.RequestHandlerClass.func + assert isinstance(bound.source, default_registry.SnapshotSource) + assert bound.source.registry.active.repository == "garden" + assert Path(bound.source.registry.active.source_root).resolve() == repo.resolve() + assert bound.capabilities["refresh"]["binding"] == "regenerate" + assert serve._checkout_real(repo) is True + finally: + httpd.server_close() + _assert_defaults_registered() + + +# --------------------------------------------------------------------------- +# 4 — standalone: no sibling importable +# --------------------------------------------------------------------------- + +_STANDALONE = textwrap.dedent(""" + import http.client, json, sys, threading + from pathlib import Path + from opendox import cli, serve + tmp = Path(sys.argv[1]) + out = tmp / "out" / "snapshot.json" + rc = cli.main(["generate", "--repo-root", str(tmp / "repo"), + "--repository", "garden", "--output", str(out)]) + (tmp / "web").mkdir() + httpd = serve.build_server(tmp / "web", out, tmp / "repo", port=0) + worker = threading.Thread(target=httpd.serve_forever, daemon=True) + worker.start() + host, port = httpd.server_address[:2] + answers = {} + for path in ("/snapshot.json", "/capabilities", + "/source/notes-toolshed-inventory.md", "/source/.git/config"): + connection = http.client.HTTPConnection(host, port, timeout=10) + connection.request("GET", path) + response = connection.getresponse() + answers[path] = [response.status, response.read().decode("utf-8", "replace")] + httpd.shutdown(); httpd.server_close() + loaded = sorted(name for name, module in sys.modules.items() + if module is not None and name.split(".")[0] in SIBLINGS) + print(json.dumps({"rc": rc, "answers": answers, "loaded": loaded, + "kind": json.loads(out.read_text())["kind"]})) + """) + + +def _blocking_siblings() -> str: + """A preamble that makes every sibling unimportable, as a lone openDox + checkout is: a `None` in `sys.modules` fails the import at once.""" + return ("import sys\n" + "".join( + f"sys.modules[{name!r}] = None\n" for name in SIBLINGS) + + f"SIBLINGS = {set(SIBLINGS)!r}\n") + + +def test_generate_and_serve_run_with_no_sibling_importable(tmp_path) -> None: + """T055's claim end to end, in a process where `openxdox`, + `ideation_dashboard`, `doc_health` and `corpus_adapter_openxfactory` cannot + be imported: `generate` writes the neutral snapshot, and a server builds + and answers the core routes from openDox's own registry.""" + _repository(tmp_path, name="repo") + script = _blocking_siblings() + _STANDALONE + done = subprocess.run([sys.executable, "-c", script, str(tmp_path)], + capture_output=True, text=True, cwd=ROOT, timeout=300) + assert done.returncode == 0, done.stderr + result = json.loads(done.stdout.strip().splitlines()[-1]) + assert result["rc"] == 0, done.stderr + assert result["kind"] == NEUTRAL + assert result["loaded"] == [] + answers = result["answers"] + assert answers["/snapshot.json"][0] == 200 + assert json.loads(answers["/snapshot.json"][1])["kind"] == NEUTRAL + assert answers["/capabilities"][0] == 200 + assert answers["/source/notes-toolshed-inventory.md"][0] == 200 + assert answers["/source/.git/config"][0] == 404 + + +# --------------------------------------------------------------------------- +# 5 — openDox's own registry and source +# --------------------------------------------------------------------------- + +@pytest.fixture() +def tree(tmp_path) -> Path: + root = tmp_path / "tree" + (root / "docs").mkdir(parents=True) + (root / "docs" / "a.md").write_text("# a\n", encoding="utf-8") + (root / ".git").mkdir() + (root / ".git" / "config").write_text("[remote]\n", encoding="utf-8") + (root / ".env").write_text("TOKEN=x\n", encoding="utf-8") + (root / "changes").mkdir() + (root / "changes" / ".openspec.yaml").write_text("x: 1\n", encoding="utf-8") + outside = tmp_path / "outside.md" + outside.write_text("secret\n", encoding="utf-8") + (root / "escape.md").symlink_to(outside) + return root + + +@pytest.mark.parametrize("tail,served", [ + ("docs/a.md", True), + ("docs%2Fa.md", True), + ("changes/.openspec.yaml", True), + ("../outside.md", False), + ("%2e%2e/outside.md", False), + ("/etc/passwd", False), + ("docs/a.md\x00", False), + ("", False), + (".git/config", False), + ("%2egit/config", False), + (".env", False), + ("docs", False), + ("escape.md", False), + ("docs/missing.md", False), +]) +def test_resolve_within_is_the_containment_source_has_always_had(tree, tail, served) -> None: + resolved = default_registry.resolve_within(tree, tail) + if served: + assert resolved is not None and resolved.is_file() + assert resolved.is_relative_to(tree.resolve()) + else: + assert resolved is None + + +def test_the_keys_default_to_main_and_parse_back() -> None: + reg = default_registry + assert reg.snapshot_key("r") == ("r", "main") + assert reg.snapshot_key("r", " ") == ("r", "main") + assert reg.key_id("r", "draft/t") == "r@draft/t" + assert reg.parse_key_id("r@draft%2Ft") == ("r", "draft/t") + assert reg.parse_key_id("no-at-sign") is None + assert reg.parse_key_id("@main") is None + assert reg.is_publishable_ref(None) and reg.is_publishable_ref("main") + assert not reg.is_publishable_ref("draft/t") + + +def test_the_source_registers_the_snapshot_it_was_handed(tmp_path) -> None: + snapshot = tmp_path / "snapshot.json" + snapshot.write_text(json.dumps({ + "repository": "garden", + "generation": {"source_revision": "abc123", "generated_at": ANCHOR_DATE}, + }), encoding="utf-8") + checkout = tmp_path / "checkout" + checkout.mkdir() + source = default_registry.SnapshotSource( + baked_snapshot=snapshot, checkout_root=checkout, + source_roots={"other@main": tmp_path}) + registry = source.bootstrap() + entry = registry.active + assert (entry.repository, entry.ref) == ("garden", "main") + assert entry.source_root == checkout + assert (entry.source_revision, entry.generated_at) == ("abc123", ANCHOR_DATE) + assert source.baked_repository == "garden" + assert source.refresh_binding == "regenerate" + assert source.compose_view("garden") is None + assert source.peek_hints() == {} + assert source._source_root_for("other", "main") == tmp_path + assert source._source_root_for("stranger", "main") is None + assert default_registry.SnapshotSource( + baked_snapshot=snapshot, checkout_root=tmp_path / "gone").refresh_binding is None + + +def test_the_source_refuses_what_only_a_hosts_registry_reads(tmp_path) -> None: + for kwargs in ({"data_source": object()}, {"local_index": tmp_path / "index.json"}): + with pytest.raises(default_registry.NeutralRegistryRefused) as caught: + default_registry.SnapshotSource(**kwargs) + assert ps.registry.registration_call in str(caught.value) + assert isinstance(caught.value, ps.ProjectionSeamError) + assert default_registry.data_source_from_options() is None + for kwargs in ({"directory": tmp_path}, {"url": "https://example.invalid/x"}): + with pytest.raises(default_registry.NeutralRegistryRefused): + default_registry.data_source_from_options(**kwargs) + assert default_registry.github_raw_base_url("o/r") == \ + "https://raw.githubusercontent.com/o/r/main/" + assert default_registry.github_raw_base_url("o/r", ref="x", path="a/b") == \ + "https://raw.githubusercontent.com/o/r/x/a/b/" + with pytest.raises(ValueError): + default_registry.github_raw_base_url("not-a-slug") + + +def test_serve_main_refuses_a_data_source_openDoxs_registry_cannot_read( + tmp_path, capsys, monkeypatch) -> None: + reached = [] + monkeypatch.setattr(serve, "serve", lambda *a, **k: reached.append(a)) + repo = _repository(tmp_path) + rc = serve.main(["--snapshot", str(tmp_path / "s.json"), "--checkout-root", + str(repo), "--data-source-dir", str(tmp_path)]) + assert rc == 1 and not reached + err = capsys.readouterr().err + assert "serve refused:" in err and "--data-source-dir" in err + + +def test_the_registry_keeps_a_sessions_owner_and_base_and_confines_each_entry(tmp_path) -> None: + reg = default_registry + registry = reg.SnapshotRegistry() + main = registry.register(reg.SnapshotEntry("garden", source_root=tmp_path)) + session = reg.SnapshotEntry("garden", "draft/t", source_root=tmp_path / "wt", + session_tile=("staged-topic", "t"), + session_base=("main", "abc"), + session_base_aliases=("def",)) + registry.register(session) + assert registry.active is main, "the first entry registered is active" + rebuilt = registry.register(reg.SnapshotEntry("garden", "draft/t")) + assert rebuilt.session_tile == ("staged-topic", "t") + assert rebuilt.session_base == ("main", "abc") + assert rebuilt.session_base_aliases == ("def",) + assert registry.resolve(None) is main + assert registry.resolve("garden", "draft/t") is rebuilt + assert registry.keys() == [("garden", "draft/t"), ("garden", "main")] + (tmp_path / "doc.md").write_text("x", encoding="utf-8") + assert registry.resolve_source("garden", None, "doc.md") == (tmp_path / "doc.md").resolve() + assert registry.resolve_source("garden", "draft/t", "doc.md") is None, ( + "an entry with no root serves nothing, never another entry's checkout") + assert registry.resolve_source("stranger", None, "doc.md") is None + with registry.atomically() as held: + assert held is registry + assert registry.set_active("garden", "draft/t") is rebuilt + registry.drop("garden", "draft/t") + assert registry.get("garden", "draft/t") is None and len(registry) == 1 + + +def test_the_regenerate_runs_the_registered_generator_and_writer(tmp_path) -> None: + calls, writes = [], [] + + def operation(repo_root, repository, *, source_revision=None, generated_at=None): + calls.append((repo_root, repository, source_revision)) + return {"schema_version": 1, "kind": "host-snapshot", "repository": repository, + "generation": {"source_revision": "fresh"}} + + gs.register(gs.SnapshotGenerator(contract="host-snapshot", generate=operation)) + + class _Writer: + def write_snapshot(self, snapshot, path, boundary): + writes.append(snapshot) + return boundary.write_output(path, json.dumps(snapshot)) + + ps.writer.register(_Writer()) + snapshot = tmp_path / "snapshot.json" + snapshot.write_text(json.dumps({"repository": "garden", + "generation": {"source_revision": "old"}}), + encoding="utf-8") + source = default_registry.SnapshotSource(baked_snapshot=snapshot, + checkout_root=tmp_path) + source.bootstrap() + result = source.refresh(repository="garden") + assert calls == [(tmp_path, "garden", None)], ( + "the regenerate generates through the registered generator, and an " + "unset project register asks nothing of it") + assert writes and writes[0]["generation"]["source_revision"] == "fresh" + assert result["binding"] == "regenerate" and result["source_revision"] == "fresh" + assert source.registry.active.source_revision == "fresh" + + +def test_the_regenerate_refuses_an_input_the_generator_does_not_declare(tmp_path) -> None: + gs.register_default(default_generator.GENERATOR) + snapshot = tmp_path / "snapshot.json" + snapshot.write_text("{}", encoding="utf-8") + source = default_registry.SnapshotSource( + baked_snapshot=snapshot, checkout_root=tmp_path, + project_register=tmp_path / "register.yaml") + source.bootstrap() + with pytest.raises(gs.GeneratorInputRefused): + source.refresh() + assert snapshot.read_text(encoding="utf-8") == "{}", "nothing was written" + + +def test_an_injected_generator_is_used_and_a_session_ref_is_never_promoted(tmp_path) -> None: + ps.register_defaults() + used = [] + + def generator(root, repository, *, project_register_source=None): + used.append(repository) + return {"schema_version": 1, "kind": NEUTRAL, "repository": repository, + "generation": {"source_revision": "s1"}} + + source = default_registry.SnapshotSource(checkout_root=tmp_path, generator=generator) + registry = source.registry + main = registry.register(default_registry.SnapshotEntry( + "garden", snapshot_path=tmp_path / "main.json", source_root=tmp_path)) + registry.register(default_registry.SnapshotEntry( + "garden", "draft/t", snapshot_path=tmp_path / "session.json", + source_root=tmp_path)) + source.refresh(repository="garden", ref="draft/t") + assert used == ["garden"] + assert registry.active.key == main.key, "FR-014a: main stays active" + assert json.loads((tmp_path / "session.json").read_text())["generation"] == \ + {"source_revision": "s1"} + + +# --------------------------------------------------------------------------- +# 5b — the core `/snapshot.json` arm, answered from the registered source +# --------------------------------------------------------------------------- + +def test_the_snapshot_arm_serves_the_active_and_the_named_entry(tmp_path) -> None: + _repo, out, httpd = _served(tmp_path) + status, headers, body = _get(httpd, "/snapshot.json") + assert status == 200 and json.loads(body)["kind"] == NEUTRAL + assert headers["X-Snapshot-Repository"] == "garden" + _repo2, _out2, httpd = _served(tmp_path / "second") + status, _headers, body = _get(httpd, "/snapshot.json?repository=garden&ref=main") + assert status == 200 and json.loads(body)["repository"] == "garden" + _repo3, _out3, httpd = _served(tmp_path / "third") + status, _headers, _body = _get(httpd, "/snapshot.json?repository=nobody") + assert status == 404, "an unknown pair, and no aggregate composes" + + +def test_the_hosted_plane_refuses_a_session_ref_named_or_active(tmp_path) -> None: + _repo, _out, httpd = _served(tmp_path) + bound = httpd.RequestHandlerClass.func + bound.loopback = False + status, _headers, body = _get(httpd, "/snapshot.json?repository=garden&ref=draft/t") + assert status == 403 and json.loads(body)["error"] == "session_unavailable" + _repo, _out, httpd = _served(tmp_path / "again") + bound = httpd.RequestHandlerClass.func + bound.loopback = False + registry = bound.source.registry + registry.register(default_registry.SnapshotEntry( + "garden", "draft/t", snapshot_path=bound.snapshot_path), active=True) + status, _headers, body = _get(httpd, "/snapshot.json") + assert status == 403, "the refusal follows the entry the request resolved to" + + +def test_hosted_ref_refused_asks_the_registered_registry() -> None: + ps.register_defaults() + assert serve.hosted_ref_refused(True, "draft/t") is False + assert serve.hosted_ref_refused(False, None) is False + assert serve.hosted_ref_refused(False, "main") is False + assert serve.hosted_ref_refused(False, "draft/t") is True + ps.registry.unregister() + host = types.SimpleNamespace(**{ + name: getattr(default_registry, name) + for name in (*ps.REGISTRY_CALLABLES, *ps.REGISTRY_VALUES)}) + host.is_publishable_ref = lambda ref: ref in (None, "main", "release") + ps.registry.register(host) + assert serve.hosted_ref_refused(False, "release") is False, ( + "the rule is the registered registry's, not a second spelling of it") + + +# --------------------------------------------------------------------------- +# 6 — openDox's own corpus-root predicate, writer and validator stand-in +# --------------------------------------------------------------------------- + +def test_the_corpus_root_predicate_asks_for_a_git_repositorys_root(tmp_path) -> None: + predicate = default_projection.CORPUS_ROOT + assert predicate.corpus_scan_defect(tmp_path / "gone") == "the path does not exist" + (tmp_path / "file").write_text("x", encoding="utf-8") + assert predicate.corpus_scan_defect(tmp_path / "file") == "the path is not a directory" + (tmp_path / "plain").mkdir() + assert "not the root of a git repository" in predicate.corpus_scan_defect(tmp_path / "plain") + repo = _repository(tmp_path, copy=None, files={"a.md": "# a\n"}) + assert predicate.corpus_scan_defect(repo) is None + assert predicate.corpus_scan_defect(repo / "a.md") == "the path is not a directory" + worktree = tmp_path / "worktree" + worktree.mkdir() + (worktree / ".git").write_text("gitdir: elsewhere\n", encoding="utf-8") + assert predicate.corpus_scan_defect(worktree) is None, "a worktree's .git is a file" + assert predicate.SCANNED_ROOTS == () + assert predicate.change_rows(repo) == () + + +def test_the_corpus_root_refusal_names_the_path_what_it_wanted_and_a_shape(tmp_path) -> None: + predicate = default_projection.CORPUS_ROOT + refusal = predicate.corpus_root_refusal(tmp_path / "gone", flag="--checkout-root", + shape="a\n b") + assert refusal.startswith("--checkout-root is not a corpus checkout: the path does not exist") + assert str((tmp_path / "gone").resolve()) in refusal + assert predicate.LOOKED_FOR in refusal + for expected in ("SERVED CHECKOUT", "namespace", "a correct invocation has this shape:", + " a", " b"): + assert expected in refusal + assert predicate.corpus_root_refusal(_repository(tmp_path)) is None + + +def test_the_writer_is_canonical_and_writes_only_through_the_boundary(tmp_path) -> None: + writer = default_projection.WRITER + snapshot = {"b": 1, "a": {"d": "é", "c": [2, 1]}} + boundary = OutputBoundary(tmp_path, ["snapshot.json"]) + written = writer.write_snapshot(snapshot, tmp_path / "snapshot.json", boundary) + text = written.read_text(encoding="utf-8") + assert text == writer.canonical_json(snapshot) + assert text == ('{\n "a": {\n "c": [\n 2,\n 1\n ],\n ' + '"d": "\\u00e9"\n },\n "b": 1\n}\n') + with pytest.raises(BoundaryViolation): + writer.write_snapshot(snapshot, tmp_path / "elsewhere.json", boundary) + + +def test_the_validator_stand_in_concludes_nothing_and_names_T057(tmp_path) -> None: + result = default_projection.VALIDATOR.validate(tmp_path / "x.json") + assert result.available is False and result.ok is False + assert result.validator is None and "T057" in result.unavailable_reason + assert default_projection.VALIDATOR.dependency_remedy is None + + +def test_a_validation_results_outcome_follows_ok_unless_given() -> None: + assert ps.ValidationResult(True, 0, "", "", "v").outcome == ps.VALIDATED + assert ps.ValidationResult(False, 1, "", "", "v").outcome == ps.NOT_CONFORMANT + unavailable = ps.ValidationResult(False, -1, "", "", None, ps.VALIDATOR_UNAVAILABLE, "why") + assert not unavailable.available and "why" in unavailable.summary() + assert ps.ValidationResult(True, 0, "a\nlast line\n", "", "v").summary() == "last line" + + +# --------------------------------------------------------------------------- +# 7 — the generate verbs through the seams +# --------------------------------------------------------------------------- + +def test_generate_writes_the_neutral_snapshot_and_names_its_kind(tmp_path, capsys) -> None: + repo = _repository(tmp_path) + out = tmp_path / "out" / "snapshot.json" + assert _generate(repo, out, "--no-validate") == 0 + snapshot = json.loads(out.read_text(encoding="utf-8")) + assert snapshot["kind"] == NEUTRAL and snapshot["documents"] + assert out.read_text(encoding="utf-8") == default_projection.WRITER.canonical_json(snapshot) + captured = capsys.readouterr() + assert " repository=garden kind=opendox-snapshot" in captured.out + assert "project=" not in captured.out, "the neutral contract carries no project" + + +def test_the_report_keeps_every_other_kinds_line(capsys, tmp_path) -> None: + ps.register_defaults() + cli._report({"kind": "ideation-dashboard-snapshot", "repository": "r", + "generation": {"source_revision": "x"}, "documents": [{}]}, + tmp_path / "s.json", tmp_path) + assert " repository=r project= project_group=" in capsys.readouterr().out + + +def test_an_input_the_generator_does_not_declare_is_refused_before_a_write(tmp_path, capsys) -> None: + repo = _repository(tmp_path) + out = tmp_path / "out" / "snapshot.json" + assert _generate(repo, out, "--project-register", str(tmp_path / "r.yaml")) == 1 + err = capsys.readouterr().err + assert "generate refused:" in err and "project_register_source" in err + assert not out.exists() + + +def test_a_root_openDoxs_predicate_refuses_is_refused_with_its_message(tmp_path, capsys) -> None: + (tmp_path / "plain").mkdir() + assert _generate(tmp_path / "plain", tmp_path / "out.json") == 1 + err = capsys.readouterr().err + assert "--repo-root is not a corpus checkout" in err + assert default_projection.CorpusRoot.LOOKED_FOR in err + assert "python3 src/opendox/cli.py generate-and-open" in err + assert not (tmp_path / "out.json").exists() + + +def test_a_malformed_generated_at_is_refused_by_openDoxs_own_rule(tmp_path, capsys) -> None: + repo = _repository(tmp_path) + assert _generate(repo, tmp_path / "out.json", "--generated-at", "2026-02-30T00:00:00Z") == 1 + assert "--generated-at is not an RFC 3339 date-time" in capsys.readouterr().err + assert not (tmp_path / "out.json").exists() + + +def test_an_empty_projection_warns_in_the_predicates_own_terms(tmp_path, capsys) -> None: + repo = _repository(tmp_path, copy=None) + assert _generate(repo, tmp_path / "out.json", "--no-validate") == 0 + err = capsys.readouterr().err + assert "ZERO documents" in err + assert "accepted as a corpus checkout, but nothing in it was read as a document" in err + assert "because it holds" not in err + + +def test_the_empty_projection_warning_names_a_predicates_roots(tmp_path, capsys) -> None: + predicate = _stub(ps.corpus_root) + predicate.SCANNED_ROOTS = ("ideation", "openspec") + ps.corpus_root.register(predicate) + (tmp_path / "ideation").mkdir() + cli._warn_on_empty_projection({"documents": 0}, tmp_path) + assert "because it holds ideation/," in capsys.readouterr().err + + +def test_the_gate_snapshot_generates_through_the_seam(tmp_path) -> None: + calls = [] + + def operation(repo_root, repository, *, source_revision=None, generated_at=None, + project_register_source=None, possibles_source=None): + calls.append((repository, source_revision, project_register_source)) + return {"schema_version": 1, "kind": "host-snapshot"} + + gs.register(gs.SnapshotGenerator( + contract="host-snapshot", generate=operation, + inputs=("project_register_source", "possibles_source"))) + args = argparse.Namespace(repo_root=str(tmp_path), repository="garden", + source_revision="abc", project_register=None, + possibles=None) + root, snapshot = cli._gate_snapshot(args) + assert root == tmp_path.resolve() and snapshot["kind"] == "host-snapshot" + assert calls == [("garden", "abc", None)] + + +def test_a_cli_session_registry_is_the_registered_registrys(tmp_path) -> None: + class _HostRegistry(default_registry.SnapshotRegistry): + pass + + host = types.SimpleNamespace(**{ + name: getattr(default_registry, name) + for name in (*ps.REGISTRY_CALLABLES, *ps.REGISTRY_VALUES)}) + host.SnapshotRegistry = _HostRegistry + ps.registry.register(host) + registry = cli._session_registry(_repository(tmp_path), "garden") + assert type(registry) is _HostRegistry + + +def _validate_args(repo: Path, *extra: str) -> argparse.Namespace: + return argparse.Namespace(repo_root=str(repo), no_validate=False, + strict="--strict" in extra) + + +def _written(tmp_path: Path, kind: str | None = NEUTRAL) -> Path: + path = tmp_path / "out" / "snapshot.json" + path.parent.mkdir(parents=True, exist_ok=True) + document = {"schema_version": 1, "repository": "garden"} + if kind is not None: + document["kind"] = kind + path.write_text(json.dumps(document), encoding="utf-8") + return path + + +def test_validation_is_by_the_written_snapshots_kind(tmp_path, capsys) -> None: + ps.register_defaults() + host_validator = _Validator() + ps.validators.register("host-snapshot", host_validator) + written = _written(tmp_path, "host-snapshot") + assert cli._validate(written, _validate_args(tmp_path)) == 0 + assert host_validator.calls == [{"path": written, "strict": False, + "search_from": (written.parent, tmp_path.resolve())}] + assert ps.validators.for_kind(NEUTRAL) is default_projection.VALIDATOR, ( + "the host's kind took nothing from openDox's own") + assert "validation: stand-in: 0 error(s)" in capsys.readouterr().out + + +def test_openDoxs_own_kind_meets_the_stand_in_and_strict_makes_it_fatal(tmp_path, capsys) -> None: + ps.register_defaults() + written = _written(tmp_path) + assert cli._validate(written, _validate_args(tmp_path)) == 0 + err = capsys.readouterr().err + assert "validation SKIPPED" in err and "'opendox-snapshot'" in err and "T057" in err + assert str(written.parent) in err and str(tmp_path.resolve()) in err + assert "the ENVIRONMENT, not the snapshot" in err + assert cli._validate(written, _validate_args(tmp_path, "--strict")) == 1 + assert "--strict was given" in capsys.readouterr().err + + +def test_a_kind_with_no_validator_is_unavailable_not_another_kinds(tmp_path, capsys) -> None: + ps.register_defaults() + assert cli._validate(_written(tmp_path, "stranger"), _validate_args(tmp_path)) == 0 + err = capsys.readouterr().err + assert "no validator is registered for kind 'stranger'" in err + + +def test_a_rejection_fails_and_blames_the_snapshot(tmp_path, capsys) -> None: + rejecting = _Validator(ps.ValidationResult( + False, 1, "ERROR rule-x: broken\n1 error(s)", "", "stand-in")) + ps.validators.register(NEUTRAL, rejecting) + assert cli._validate(_written(tmp_path), _validate_args(tmp_path)) == 1 + err = capsys.readouterr().err + assert "REJECTED" in err and "rule-x" in err and "pip install" not in err + + +def test_a_validator_that_could_not_run_warns_with_its_own_remedy(tmp_path, capsys) -> None: + stuck = _Validator(ps.ValidationResult( + False, 2, "", "ERROR a library is missing", "stand-in", + ps.VALIDATOR_UNAVAILABLE, "it exited 2"), remedy="pip install something") + ps.validators.register(NEUTRAL, stuck) + assert cli._validate(_written(tmp_path), _validate_args(tmp_path)) == 0 + err = capsys.readouterr().err + for expected in ("could not run: it exited 2", "ERROR a library is missing", + "-m pip install something", "the ENVIRONMENT, not the snapshot"): + assert expected in err + assert cli._validate(_written(tmp_path), _validate_args(tmp_path, "--strict")) == 1 + + +def test_a_snapshot_that_declares_no_kind_fails(tmp_path, capsys) -> None: + ps.register_defaults() + assert cli._validate(_written(tmp_path, None), _validate_args(tmp_path)) == 1 + assert "declares no kind" in capsys.readouterr().err + + +# --------------------------------------------------------------------------- +# 8 — `branch_session`'s routed sites, and the workbench manifest +# --------------------------------------------------------------------------- + +def _host_registry_with_entry(entry_type) -> types.SimpleNamespace: + host = types.SimpleNamespace(**{ + name: getattr(default_registry, name) + for name in (*ps.REGISTRY_CALLABLES, *ps.REGISTRY_VALUES)}) + host.SnapshotEntry = entry_type + return host + + +def test_a_session_entry_is_the_registered_registrys_entry(tmp_path) -> None: + class _HostEntry(default_registry.SnapshotEntry): + pass + + ps.registry.register(_host_registry_with_entry(_HostEntry)) + entry = bs.session_entry("garden", "draft/t", tmp_path) + assert type(entry) is _HostEntry + assert (entry.repository, entry.ref, entry.source_root) == ("garden", "draft/t", tmp_path) + + +def test_a_registered_session_reads_its_snapshot_through_the_registry(tmp_path) -> None: + ps.register_defaults() + checkout = tmp_path / "garden" + checkout.mkdir() + branch = "draft/t" + snapshot = bs.session_snapshot_path(checkout, branch) + snapshot.parent.mkdir(parents=True, exist_ok=True) + snapshot.write_text(json.dumps({"repository": "garden", + "generation": {"source_revision": "s"}}), + encoding="utf-8") + registry = default_registry.SnapshotRegistry() + entry = bs.register_session_entry(registry, repository="garden", branch=branch, + worktree=tmp_path / "wt", checkout_root=checkout, + regenerate=False) + assert isinstance(entry, default_registry.SnapshotEntry) + assert entry.source_revision == "s" and registry.get("garden", branch) is entry + + +def test_a_session_snapshot_and_the_main_view_regenerate_through_the_seams(tmp_path) -> None: + ps.register_defaults() + gs.register_default(default_generator.GENERATOR) + repo = _repository(tmp_path) + registry = default_registry.SnapshotRegistry() + main = registry.register(default_registry.SnapshotEntry( + "garden", snapshot_path=tmp_path / "main.json", source_root=repo)) + registry.register(default_registry.SnapshotEntry( + "garden", "draft/t", snapshot_path=tmp_path / "session.json", + source_root=repo)) + result = bs.refresh_session_snapshot(registry, repository="garden", + branch="draft/t", worktree=repo) + assert result["binding"] == "regenerate" + assert json.loads((tmp_path / "session.json").read_text())["kind"] == NEUTRAL + assert registry.active.key == main.key, "FR-014a: main stays active" + view = bs.refresh_main_view(registry, repository="garden", checkout_root=repo) + assert view["binding"] == "regenerate" + assert json.loads((tmp_path / "main.json").read_text())["kind"] == NEUTRAL + + +def test_the_change_rows_are_the_registered_predicates(tmp_path) -> None: + ps.register_defaults() + assert bs._change_rows(tmp_path) == () + ps.corpus_root.unregister() + predicate = _stub(ps.corpus_root) + predicate.change_rows = lambda root: [("c-1", "active", Path(root), "absent", None)] + ps.corpus_root.register(predicate) + assert bs._change_rows(tmp_path) == (("c-1", "active", tmp_path, "absent", None),) + + +def test_a_manifest_is_validated_by_the_validator_for_its_kind(tmp_path) -> None: + manifest = tmp_path / "set.yaml" + manifest.write_text("kind: ideation-workbench\n", encoding="utf-8") + ruling = _Validator() + ps.validators.register(workbench.KIND, ruling) + result = workbench.validate_manifest(manifest) + assert result.ok and result.validator == "stand-in" + assert ruling.calls == [{"path": manifest.resolve(), "strict": False, + "search_from": (manifest.resolve().parent,)}] + ps.validators.unregister() + ps.register_defaults() + unchecked = workbench.validate_manifest(manifest, search_from=tmp_path) + assert not unchecked.ok and unchecked.validator is None + assert "T057" in unchecked.stderr and "validator not found" in unchecked.summary() + ps.validators.unregister() + refused = workbench.validate_manifest(manifest) + assert not refused.ok and "ideation-workbench" in refused.stderr + + +def test_an_explicit_manifest_validator_script_still_runs(tmp_path) -> None: + script = tmp_path / "validator.py" + script.write_text("import sys\nprint('explicit: ok', sys.argv[1:])\n", encoding="utf-8") + manifest = tmp_path / "set.yaml" + manifest.write_text("kind: ideation-workbench\n", encoding="utf-8") + result = workbench.validate_manifest(manifest, validator=script, strict=True) + assert result.ok and result.validator == script + assert "--strict" in result.stdout and str(manifest.resolve()) in result.stdout + + +# --------------------------------------------------------------------------- +# 9 — no proxy over a seam is read at import time +# --------------------------------------------------------------------------- + +def _proxy_bindings(tree: ast.Module) -> set[str]: + """Module-level names bound to `projection_seams..proxy`.""" + bound = set() + for node in tree.body: + if isinstance(node, ast.Assign) and isinstance(node.value, ast.Attribute) \ + and node.value.attr == "proxy" \ + and isinstance(node.value.value, ast.Attribute) \ + and isinstance(node.value.value.value, ast.Name) \ + and node.value.value.value.id == "projection_seams": + bound |= {t.id for t in node.targets if isinstance(t, ast.Name)} + return bound + + +def _import_time_reads(tree: ast.Module, names: set[str]) -> list[int]: + """Lines where `names` are read when the module is IMPORTED: module-level + statements, class bodies, and a function's defaults, annotations and + decorators. Only a function's body defers.""" + hits = [] + + def reads(node): + for inner in ast.walk(node): + if isinstance(inner, ast.Name) and inner.id in names \ + and isinstance(inner.ctx, ast.Load): + hits.append(inner.lineno) + + def walk(body): + for node in body: + if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)): + args = node.args + for part in [*node.decorator_list, *args.defaults, + *(d for d in args.kw_defaults if d), + *(a.annotation for a in [*args.posonlyargs, *args.args, + *args.kwonlyargs, args.vararg, + args.kwarg] + if a is not None and a.annotation is not None), + *([node.returns] if node.returns else [])]: + reads(part) + continue + if isinstance(node, ast.ClassDef): + for part in [*node.bases, *node.keywords, *node.decorator_list]: + reads(part) + walk(node.body) + continue + nested = [] + for _field, value in ast.iter_fields(node): + for item in value if isinstance(value, list) else [value]: + if isinstance(item, ast.stmt): + nested.append(item) + elif isinstance(item, ast.AST): + reads(item) + walk(nested) + + walk(tree.body) + return sorted(hits) + + +def test_no_proxy_over_a_seam_is_read_at_import_time() -> None: + """A proxy read at import time would resolve the seam before any entry + point registered anything, so the import would refuse.""" + found = {} + for path in sorted(PACKAGE.rglob("*.py")): + tree = ast.parse(path.read_text(encoding="utf-8")) + names = _proxy_bindings(tree) + if names: + found[path.relative_to(PACKAGE).as_posix()] = ( + sorted(names), _import_time_reads(tree, names)) + assert found == {"serve.py": (["registry_mod"], []), + "serve_workbench.py": (["registry_mod"], [])}, found + + +def test_the_retired_stand_ins_are_gone_from_consumer_reach() -> None: + for name in ("snapshot", "snapshot_registry", "corpus_root", "generator", + "find_validator", "corpus_root_refusal", "generate_snapshot", + "is_rfc3339_datetime", "hosted_ref_refused", "scanned_roots", + "function", "constant"): + assert not hasattr(consumer_reach, name), name + assert name not in consumer_reach.__all__, name + assert consumer_reach.LateProjectionRoutes.LATE_COLUMN[2] == ("_serve_index",) + + +# --------------------------------------------------------------------------- +# 10 — openDox's own RFC 3339 check +# --------------------------------------------------------------------------- + +@pytest.mark.parametrize("value,admitted", [ + ("2026-09-04T01:23:45Z", True), + ("2026-09-04t01:23:45.500z", True), + ("2026-09-04T01:23:45+05:30", True), + ("2024-02-29T00:00:00-00:00", True), + ("2026-02-30T00:00:00Z", False), + ("2025-02-29T00:00:00Z", False), + ("2026-09-04T01:23:45+24:00", False), + ("2026-09-04T01:23:60Z", False), + ("0000-01-01T00:00:00Z", False), + ("2026-09-04T01:23:45", False), + ("2026-09-04", False), + ("2026-09-04 01:23:45Z", False), + ("2026-09-04T01:23:45Z\n", False), + (" 2026-09-04T01:23:45Z", False), + ("\u0662\u0660\u0662\u0666-09-04T01:23:45Z", False), + (None, False), + (20260904, False), +]) +def test_the_rfc3339_check_is_the_neutral_contracts_rule(value, admitted) -> None: + assert rfc3339.is_rfc3339_datetime(value) is admitted diff --git a/tests/test_reach_sweep.py b/tests/test_reach_sweep.py index cfaec86..ad993a9 100644 --- a/tests/test_reach_sweep.py +++ b/tests/test_reach_sweep.py @@ -13,10 +13,11 @@ closed, each by its own phase-1 task: `authoring.py:318` (T021), `serve.py:713` (T012), `workbench.py:746` (T025), `workbench.py:1407-1409` (T026), and `serve_wire.py:1369` and `doxbench_packet.py:177` (T027). -* Nineteen deferred reaches remain, every one of them into `openxdox` and - inside a function body. The release map routes them in phases 2 and 3 - (T055, T084), and openXdox-code's `OPENDOX_BACK_IMPORTS` ratchet counts them - module by module. +* Eleven deferred reaches remain, every one of them into `openxdox` and + inside a function body. Phase 2's T055 routed eight of the nineteen (the + snapshot registry and source, the corpus-root predicate and the change rows, + through declared seams); phase 3's T084 routes the rest, and openXdox-code's + `OPENDOX_BACK_IMPORTS` ratchet counts them module by module. THIS FILE HOLDS THE FIRST TWO FACTS, AND DOES NOT PIN THE THIRD'S COUNT. A deferred reach into `openxdox` is still lawful in release 1. Each one that a @@ -108,7 +109,7 @@ class body, an `if`/`try`/`with` at module level, and a `def`'s decorators, CONTEXT_NAMES, UNREADABLE, importer_escapes, importing_calls, names_a_forbidden_package, names_imported_by, rebinds, string_literals) -#: The consumer. Its deferred reaches are phase 2's and 3's to route. +#: The consumer. Its deferred reaches left are phase 3's to route (T084). CONSUMER = "openxdox" #: openxFactory's packages, spelled as F2.1 and F4.1 spell them. openDox can @@ -283,7 +284,7 @@ def test_no_code_under_src_spells_an_openxfactory_package(): def test_every_reach_the_sweep_finds_is_deferred_into_the_consumer(): - """The positive form of the two above: what is left is phase 2's and 3's. + """The positive form of the two above: what is left is phase 3's (T084). Every reach names `openxdox` and sits in a function body. Their count is openXdox-code's ratchet's, and it is not pinned here.""" diff --git a/tests/test_route_handler_contribution.py b/tests/test_route_handler_contribution.py index aa0e8ec..98b1cc8 100644 --- a/tests/test_route_handler_contribution.py +++ b/tests/test_route_handler_contribution.py @@ -1028,20 +1028,21 @@ def test_a_name_the_bound_class_sets_is_refused_before_build_servers_side_effect # 6. THE CASE THROUGH `build_server` (T011). `opendox.serve` imports now. # --------------------------------------------------------------------------- # -# THE PHASE-1 LIMIT, as measured (plan 034 plan.md, "Phase-1 limit"; research -# R7). A standalone `build_server` still meets three reaches that phase 2 -# routes, and each case below stands in for exactly those three, no more: -# * the snapshot source, `serve.py`'s `registry_mod.SnapshotSource(...)`, -# through `build_server`'s own `snapshot_source=` seam; -# * `_checkout_real`, whose body imports `openxdox.corpus_root`; -# * `compute_capabilities`' two reads of `registry_mod.BINDING_*`, the late -# `openxdox.snapshot_registry` constants. -# Everything else `build_server` does runs for real, the composition first -# among it. +# THE PHASE-1 LIMIT IS LIFTED (plan 034 T055; research R7 measured it). A +# standalone `build_server` met three reaches into openXdox: the snapshot +# source (`registry_mod.SnapshotSource(...)`), `_checkout_real` (which +# imported `openxdox.corpus_root`), and `compute_capabilities`' two reads of +# `registry_mod.BINDING_*`. Phase 1's cases stood in for all three. Since T055 +# each is read from a declared seam where `build_server` registers openDox's +# own default, so the predicate and the capabilities below are the real ones. +# The snapshot source stays injected, through `build_server`'s own +# `snapshot_source=` seam, for one reason: the refusal-order case reads whether +# its `bootstrap()` ran. Everything else `build_server` does runs for real, the +# composition first among it. class _StandInSource: - """The snapshot source phase 1 injects: nothing registered, nothing baked. - It records whether `bootstrap()` ran, so a refusal can be shown to come + """An injected snapshot source: nothing registered, nothing baked. It + records whether `bootstrap()` ran, so a refusal can be shown to come BEFORE the expensive work does.""" refresh_binding = None @@ -1092,19 +1093,17 @@ class _HostProfile: @pytest.fixture() -def standalone(monkeypatch, tmp_path): - """`build_server` with a stand-in host profile, the three phase-1 stand-ins - above, and the registry put back afterwards. The root conftest registers - its own empty profile at process start, and it must find it again.""" - import types - +def standalone(tmp_path): + """`build_server` with a stand-in host profile, the injected source above, + and the registry put back afterwards. The root conftest registers its own + empty profile at process start, and it must find it again. The corpus-root + predicate and the registry's bindings are the registered defaults, and + `repo` is not a git repository, so the checkout is not real, as phase 1's + stand-in answered.""" from opendox import domain_profile, serve previous = domain_profile.current() if domain_profile.is_registered() else None domain_profile.unregister() - monkeypatch.setattr(serve, "_checkout_real", lambda root: False) - monkeypatch.setattr(serve, "registry_mod", types.SimpleNamespace( - BINDING_REGENERATE="regenerate", BINDING_REFETCH="refetch")) (tmp_path / "web").mkdir() (tmp_path / "repo").mkdir() (tmp_path / "snapshot.json").write_text("{}", encoding="utf-8") diff --git a/tests/test_source_core_arm.py b/tests/test_source_core_arm.py index 9f36c54..62bebef 100644 --- a/tests/test_source_core_arm.py +++ b/tests/test_source_core_arm.py @@ -68,10 +68,11 @@ which is what makes them unshadowable: a contributed binding for `/source/` can no longer take the route back by arriving first. 5. THE CONTAINMENT AUTHORITY IS SINGLE AND UNMOVED. `resolve_source_path` is a - real `def` here now, and its body is one call to - `snapshot_registry.resolve_within` — the SAME rule, still the projection - column's, reached through the late seam `serve_workbench.py` already uses at - five sites. A second copy of the check beside it is the fork + real `def` here now, and its body is one call to the registry's + `resolve_within` — the SAME rule, reached through the registry seam + `serve_workbench.py` reaches it by too (a late `consumer_reach` stand-in + until plan 034 T055, the registry seam's proxy since). A second copy of the + check beside it is the fork `route_extension.py`:89 names; that is what this asserts against, and it is also what carries openXdox-code's `tests/test_source_dot_directories.py` (T092 defect 10: `/source/.git/config` answering 200 with a remote's @@ -85,9 +86,12 @@ original move was careful not to introduce), and `send_error(404, "unreadable source")` for an OSError. Plus the two content types, and the absence of any write arm naming the route. -7. FR-048 IS STILL ASKED. The hosted-plane confinement is the PROJECTION - column's rule, not this core's: the route moved and the rule did not, so both - `hosted_ref_refused` and `_hosted_entry_refused` must still be consulted. +7. FR-048 IS STILL ASKED. The hosted-plane confinement is the snapshot + registry's rule (`is_publishable_ref`): the route moved and the rule did + not, so both `hosted_ref_refused` and `_hosted_entry_refused` must still be + consulted. Since plan 034 T055 both are `serve.py`'s own, with the core + `/snapshot.json` arm's handlers, and the rule is read through the registry + seam. 8. THE FRONT END DID NOT CHANGE. § 5 row S6 says `views/viewer.js` and `views/wheel.js`:97 "become clean" — and they do it WITHOUT AN EDIT, because the route moved to them rather than them to it. Asserted as a positive: the @@ -203,18 +207,27 @@ def test_the_consumer_column_no_longer_forwards_them(name): "is how a move looks complete and is not") -def test_the_column_keeps_the_rules_the_ruling_did_not_move(): - """`_serve_index` and `_serve_snapshot` stay the projection column's, and so - does `_hosted_entry_refused` — FR-048's per-entry hosted refusal, which - `_serve_snapshot` calls too. Q4 ruled on route OWNERSHIP, not on the rules a - route consults.""" +SNAPSHOT_METHODS = ("_query_key", "_read_snapshot", "_serve_snapshot", + "_hosted_entry_refused") + + +def test_the_column_keeps_only_its_own_contributed_route(): + """`_serve_index` stays the projection column's: its contributed + `/snapshot-index.json` binding names it. Since plan 034 T055 it is the ONE + method forwarded. The core `/snapshot.json` arm's four handlers, which S6 + left on the column, are this handler's own, because forwarded they refused + every `/snapshot.json` of a standalone server.""" from opendox import consumer_reach _module, _cls, methods = consumer_reach.LateProjectionRoutes.LATE_COLUMN - for kept in ("_serve_index", "_serve_snapshot", "_hosted_entry_refused"): - assert kept in methods, ( - f"{kept} is still openXdox's and must still be forwarded; S6 moved " - "the /source pair and nothing else") + assert methods == ("_serve_index",), methods + + +@pytest.mark.parametrize("name", SNAPSHOT_METHODS) +def test_the_snapshot_arms_handlers_are_defined_on_this_handler(name): + assert _method(name) is not None, ( + f"DashboardHandler does not define {name}. Since plan 034 T055 the " + "core /snapshot.json arm's handlers are the neutral product's own") # --------------------------------------------------------------------------- @@ -275,10 +288,10 @@ def test_the_containment_rule_is_the_one_authority_and_is_not_re_implemented(): statements = [line for line in body.splitlines() if line.strip() and not line.strip().startswith(("#", '"', "'"))] assert "registry_mod.resolve_within(" in body, ( - "the containment rule is snapshot_registry.resolve_within and it did " - "NOT move: it is the projection column's, it is the same rule applied " - "per registry entry (task 2.2), and openDox reaches it through the " - "late consumer_reach seam serve_workbench.py already uses") + "the containment rule is the snapshot registry's resolve_within and it " + "did NOT move: it is the same rule applied per registry entry (task " + "2.2), and openDox reaches it through the registry seam " + "serve_workbench.py reaches it by too") assert len([s for s in statements if s.startswith(" return ")]) == 1, ( "resolve_source_path is a delegation and must stay one. A second copy " "of the containment rule here is the fork route_extension.py:89 names " @@ -368,12 +381,21 @@ def test_the_arm_still_asks_the_hosted_plane_refusal(): assert "HOSTED_SESSION_REFUSAL" in body -def test_hosted_ref_refused_is_still_reached_through_the_late_seam(): - """The route moved; FR-048 did not. `hosted_ref_refused` stays bound to the - consumer stand-in, and `tests/test_consumer_reach.py` holds it to being read - only from inside a function body.""" - node = _module_assign("hosted_ref_refused") - assert node is not None and "consumer_reach.hosted_ref_refused" in _source_of(node) +def test_hosted_ref_refused_asks_the_registered_registrys_rule(): + """The route moved; FR-048 did not. Since plan 034 T055 `hosted_ref_refused` + is `serve.py`'s own `def`, and "a ref a hosted plane may see" is still the + snapshot registry's rule, `is_publishable_ref`, read through the registry + seam. So there is one definition of it per process, and a standalone + server can answer at all.""" + assert _module_assign("hosted_ref_refused") is None, ( + "hosted_ref_refused is bound to a stand-in again; it is serve.py's own") + node = _module_function("hosted_ref_refused") + assert node is not None + body = _source_of(node) + assert "if loopback:" in body and "return False" in body, ( + "the LOCAL plane is never confined") + assert "registry_mod.is_publishable_ref(ref)" in body, ( + "the rule is the registered registry's, not a second spelling of it") # --------------------------------------------------------------------------- From ea49c424294ac1b9ac19c685489d608864d7bbe1 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Mon, 28 Sep 2026 18:22:37 +0000 Subject: [PATCH 14/38] T055: the canonical path meets the hidden-name rule; a token variable is a declaration Taken from Copilot's review of c27eac35: - r4125556296. resolve_within read the dot-directory and dot-file rule only from the URL's spelling. A symlink inside the root that led to a dot-directory, or to a dot-file with no document extension, got past it: link -> .git served /source/link/config, which is .git/config, and a symlink named alias.md served .env. The same rule now also runs on the canonical path, relative to the root, once symlinks are resolved. A symlink to a document is still served. Five cases are added, and dropping the new check makes them fail. openXdox's governed resolve_within has the same gap, and it is flagged to the holder. - r4125556360. data_source_from_options ignored token_env, so --data-source-token-env on its own was dropped in silence. Any declared option is now refused by name, an explicitly empty one included (is not None). The CLI's refusal names the flag. - r4125556420. An extra "by" is gone from the sentence in test_source_core_arm.py, and from the same sentence in resolve_source_path's docstring and in one assertion message. Whole suite: 2689 passed, 11 skipped. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/default_registry.py | 52 ++++++++++++++++++++++++++------- src/opendox/serve.py | 4 +-- tests/test_projection_seams.py | 38 +++++++++++++++++++----- tests/test_source_core_arm.py | 8 ++--- 4 files changed, 78 insertions(+), 24 deletions(-) diff --git a/src/opendox/default_registry.py b/src/opendox/default_registry.py index 3b0c1b7..f3cb719 100644 --- a/src/opendox/default_registry.py +++ b/src/opendox/default_registry.py @@ -34,7 +34,9 @@ root (`..`, percent-encoded `..`, a symlink), no dot-directory ever, and a dot-file only with an extension a document carries. The dot-directory half is where credentials live (`.git/config` carries a remote's token, T092's defect -10), so nothing below a dot-directory is ever served. +10), so nothing below a dot-directory is ever served. openDox's statement is +stricter in one place: the hidden-name half reads the CANONICAL path too, so a +symlink inside the root cannot lead to what the URL could not name. IMPORT WEIGHT. `opendox.boundary`, `opendox.defaults`, `opendox.generator_seam`, `opendox.projection_seams` and the standard library. So this module imports @@ -282,6 +284,18 @@ def entry_from_snapshot_file( # --------------------------- containment (pure) --------------------------- +def _names_something_hidden(parts: list[str] | tuple[str, ...]) -> bool: + """Whether a relative path's components name what `/source` never serves: + a dot-directory anywhere but the last component, or a last component that + is a dot-file without a document's extension. `..` is not a name, and the + escape check decides it.""" + if any(part.startswith(".") and part != ".." for part in parts[:-1]): + return True + last = parts[-1] if parts else "" + return (last.startswith(".") and last != ".." + and PurePosixPath(last).suffix not in SERVED_DOTFILE_SUFFIXES) + + def resolve_within(root: Path | str, url_tail: str) -> Path | None: """`url_tail` as an absolute FILE under `root`, or None to refuse it. @@ -290,18 +304,22 @@ def resolve_within(root: Path | str, url_tail: str) -> Path | None: absolute one and a NUL; any component but the last that is a dot-directory; a last component that is a dot-file without a document's extension; any path that resolves outside `root`, whether by `..` or by a symlink; and - anything that is not a regular file.""" + anything that is not a regular file. + + THE HIDDEN-NAME RULE IS APPLIED TWICE: to the path as the URL spells it, + and to the CANONICAL path, relative to `root`, once symlinks are resolved. + The second is what refuses a symlink inside the root that leads to what the + first refuses by name. `link -> .git` would otherwise serve + `/source/link/config`, which is `.git/config`, and `notes.md -> .env` would + serve `.env`: both resolve inside the root, to a regular file.""" rel = urllib.parse.unquote(str(url_tail)) rel = rel.split("?", 1)[0].split("#", 1)[0] if not rel or rel.startswith("/") or "\x00" in rel: return None parts = [part for part in rel.replace("\\", "/").split("/") if part not in ("", ".")] - if any(part.startswith(".") and part != ".." for part in parts[:-1]): + if _names_something_hidden(parts): return None - if parts and parts[-1].startswith(".") and parts[-1] != "..": - if PurePosixPath(parts[-1]).suffix not in SERVED_DOTFILE_SUFFIXES: - return None try: base = Path(root).resolve() resolved = (base / rel).resolve() @@ -309,6 +327,8 @@ def resolve_within(root: Path | str, url_tail: str) -> Path | None: return None if resolved != base and not resolved.is_relative_to(base): return None + if _names_something_hidden(resolved.relative_to(base).parts): + return None if not resolved.is_file(): return None return resolved @@ -412,12 +432,22 @@ def data_source_from_options(*, directory: Path | str | None = None, token_env: str | None = None, opener: Callable[..., Any] | None = None): """None when no data source is declared, which is the only plane this - registry serves. A declared one is REFUSED, never ignored.""" - if directory or url: + registry serves. A declared one is REFUSED, never ignored, and so is any + one of its options given alone: a token variable named with no source is + still a declaration this registry would otherwise drop in silence. A value + counts as declared when it is given at all, so an explicitly empty one is + refused too. `opener` is how a source's URL is fetched, a caller's + injection rather than an operator's declaration, and with no URL there is + nothing for it to fetch.""" + declared = [flag for flag, value in ( + ("--data-source-dir", directory), + ("--data-source-url (or --data-source-github, which composes one)", url), + ("--data-source-token-env", token_env), + ) if value is not None] + if declared: raise NeutralRegistryRefused( - "a runtime data source was declared " - f"({'--data-source-dir' if directory else '--data-source-url'}), " - "and openDox's own snapshot registry reads none: a published " + f"a runtime data source was declared ({', '.join(declared)}), and " + "openDox's own snapshot registry reads none: a published " "snapshot tree is located by a snapshot index, which is a governed " "contract, and openDox has no index kind of its own. Serve the " "snapshot this checkout generates, or register a host registry " diff --git a/src/opendox/serve.py b/src/opendox/serve.py index 0d85aa5..9060fb0 100644 --- a/src/opendox/serve.py +++ b/src/opendox/serve.py @@ -646,8 +646,8 @@ def resolve_source_path(checkout_root: Path, url_tail: str) -> Path | None: ARRIVED HERE AT § 3.4 SLICE S6 (RULED Q4) with the route it confines, from `openxdox/serve_projection.py`:66. The body is unchanged: the RULE is the registry's `resolve_within`, it is the same rule per registry entry, and - it is reached through the registry SEAM (plan 034 T055) that - `serve_workbench.py` reaches it by too, so a process has one rule: the + it is reached through the registry SEAM (plan 034 T055), as + `serve_workbench.py` reaches it too, so a process has one rule: the registered registry's. What moved is the ENTRY POINT, to the module that now declares the route and to the module `notebook_action.py`:52 already imported it from. A second copy of the containment rule here would be the diff --git a/tests/test_projection_seams.py b/tests/test_projection_seams.py index 838d43c..14321c8 100644 --- a/tests/test_projection_seams.py +++ b/tests/test_projection_seams.py @@ -543,6 +543,13 @@ def tree(tmp_path) -> Path: outside = tmp_path / "outside.md" outside.write_text("secret\n", encoding="utf-8") (root / "escape.md").symlink_to(outside) + # Symlinks that stay INSIDE the root: two lead to what no URL may name + # (a dot-directory, an undocumented dot-file), and two to what it may. + (root / "link").symlink_to(".git", target_is_directory=True) + (root / "docs" / "hidden").symlink_to("../.git", target_is_directory=True) + (root / "alias.md").symlink_to(".env") + (root / "same.md").symlink_to("docs/a.md") + (root / "spec.yaml").symlink_to("changes/.openspec.yaml") return root @@ -561,8 +568,17 @@ def tree(tmp_path) -> Path: ("docs", False), ("escape.md", False), ("docs/missing.md", False), + ("link/config", False), + ("docs/hidden/config", False), + ("alias.md", False), + ("same.md", True), + ("spec.yaml", True), ]) def test_resolve_within_is_the_containment_source_has_always_had(tree, tail, served) -> None: + """The rule `/source` has always been confined by, and one place where + openDox's statement of it is stricter: a symlink inside the root that + leads into a dot-directory or at an undocumented dot-file is refused as + its target would be, so `link -> .git` cannot serve `.git/config`.""" resolved = default_registry.resolve_within(tree, tail) if served: assert resolved is not None and resolved.is_file() @@ -616,9 +632,15 @@ def test_the_source_refuses_what_only_a_hosts_registry_reads(tmp_path) -> None: assert ps.registry.registration_call in str(caught.value) assert isinstance(caught.value, ps.ProjectionSeamError) assert default_registry.data_source_from_options() is None - for kwargs in ({"directory": tmp_path}, {"url": "https://example.invalid/x"}): - with pytest.raises(default_registry.NeutralRegistryRefused): + for kwargs, flag in (({"directory": tmp_path}, "--data-source-dir"), + ({"directory": ""}, "--data-source-dir"), + ({"url": "https://example.invalid/x"}, "--data-source-url"), + ({"token_env": "SECRET_ENV"}, "--data-source-token-env"), + ({"token_env": ""}, "--data-source-token-env")): + with pytest.raises(default_registry.NeutralRegistryRefused) as caught: default_registry.data_source_from_options(**kwargs) + assert flag in str(caught.value), ( + "a declared option is refused by name, never dropped in silence") assert default_registry.github_raw_base_url("o/r") == \ "https://raw.githubusercontent.com/o/r/main/" assert default_registry.github_raw_base_url("o/r", ref="x", path="a/b") == \ @@ -632,11 +654,13 @@ def test_serve_main_refuses_a_data_source_openDoxs_registry_cannot_read( reached = [] monkeypatch.setattr(serve, "serve", lambda *a, **k: reached.append(a)) repo = _repository(tmp_path) - rc = serve.main(["--snapshot", str(tmp_path / "s.json"), "--checkout-root", - str(repo), "--data-source-dir", str(tmp_path)]) - assert rc == 1 and not reached - err = capsys.readouterr().err - assert "serve refused:" in err and "--data-source-dir" in err + for flag, value in (("--data-source-dir", str(tmp_path)), + ("--data-source-token-env", "SECRET_ENV")): + rc = serve.main(["--snapshot", str(tmp_path / "s.json"), "--checkout-root", + str(repo), flag, value]) + assert rc == 1 and not reached, flag + err = capsys.readouterr().err + assert "serve refused:" in err and flag in err def test_the_registry_keeps_a_sessions_owner_and_base_and_confines_each_entry(tmp_path) -> None: diff --git a/tests/test_source_core_arm.py b/tests/test_source_core_arm.py index 62bebef..5d51f48 100644 --- a/tests/test_source_core_arm.py +++ b/tests/test_source_core_arm.py @@ -69,8 +69,8 @@ can no longer take the route back by arriving first. 5. THE CONTAINMENT AUTHORITY IS SINGLE AND UNMOVED. `resolve_source_path` is a real `def` here now, and its body is one call to the registry's - `resolve_within` — the SAME rule, reached through the registry seam - `serve_workbench.py` reaches it by too (a late `consumer_reach` stand-in + `resolve_within` — the SAME rule, reached through the registry seam, as + `serve_workbench.py` reaches it too (a late `consumer_reach` stand-in until plan 034 T055, the registry seam's proxy since). A second copy of the check beside it is the fork `route_extension.py`:89 names; that is what this asserts against, and it is @@ -290,8 +290,8 @@ def test_the_containment_rule_is_the_one_authority_and_is_not_re_implemented(): assert "registry_mod.resolve_within(" in body, ( "the containment rule is the snapshot registry's resolve_within and it " "did NOT move: it is the same rule applied per registry entry (task " - "2.2), and openDox reaches it through the registry seam " - "serve_workbench.py reaches it by too") + "2.2), and openDox reaches it through the registry seam, as " + "serve_workbench.py does") assert len([s for s in statements if s.startswith(" return ")]) == 1, ( "resolve_source_path is a delegation and must stay one. A second copy " "of the containment rule here is the fork route_extension.py:89 names " From bfb9c47929696e23d92d5ee14481852b7c92c7a6 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Mon, 28 Sep 2026 18:34:29 +0000 Subject: [PATCH 15/38] T055: serve.main hands each data-source option over as given; dropping the active entry clears its key Taken from Copilot's review of ea49c424: - r4125666251. serve.main computed `url=args.data_source_url or (...)`, which turned an explicitly empty --data-source-url into None, and `if args.data_source_github` did the same to an empty GitHub slug. So the registry never saw either. Each option is now tested against None. A GitHub source is composed into the URL only when no URL was given. A slug that cannot be composed, such as an empty one, is refused as "serve refused: --data-source-github: ..." rather than raising ValueError. openDox's own registry then refuses every one by name, and new cases cover the empty URL, a GitHub slug and an empty slug. - The same review's overview (no thread) notes that SnapshotRegistry.drop left the active key pointing at a removed entry. In that state no later register() became active, and a ref-less request met a key with nothing behind it. Dropping the active entry now clears the key. A new case covers it, and removing the clear makes it fail. openXdox's governed registry drops the same way, and that is flagged to the holder. Whole suite: 2690 passed, 11 skipped. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/default_registry.py | 8 +++++++- src/opendox/serve.py | 21 +++++++++++++++++--- tests/test_projection_seams.py | 34 ++++++++++++++++++++++++++++----- 3 files changed, 54 insertions(+), 9 deletions(-) diff --git a/src/opendox/default_registry.py b/src/opendox/default_registry.py index f3cb719..8ef2c8a 100644 --- a/src/opendox/default_registry.py +++ b/src/opendox/default_registry.py @@ -376,8 +376,14 @@ def register(self, entry: SnapshotEntry, *, return entry def drop(self, repository: str, ref: str | None = None) -> None: + """Remove an entry. Dropping the ACTIVE entry clears the active key, + so no ref-less request meets a key with nothing behind it, and the + next entry registered becomes active, as the first one did.""" with self._lock: - self._entries.pop(snapshot_key(repository, ref), None) + key = snapshot_key(repository, ref) + self._entries.pop(key, None) + if self._active == key: + self._active = None def get(self, repository: str, ref: str | None = None) -> SnapshotEntry | None: """A ref-less lookup means `main`.""" diff --git a/src/opendox/serve.py b/src/opendox/serve.py index 9060fb0..5f194c5 100644 --- a/src/opendox/serve.py +++ b/src/opendox/serve.py @@ -2350,11 +2350,26 @@ def main(argv: list[str] | None = None) -> int: if rc: return rc try: + # EVERY DATA-SOURCE OPTION IS HANDED OVER AS GIVEN (plan 034 T055). + # `None` means "not given", and anything else, an explicitly empty + # value included, is a declaration the REGISTERED registry decides + # about: openDox's own refuses each one by name, and a host's reads + # it. So each option is tested against None, never for truth. A + # GitHub source is composed into the URL only where no URL was given, + # and a slug that cannot be composed is refused like any other. + url = args.data_source_url + if url is None and args.data_source_github is not None: + try: + url = registry_mod.github_raw_base_url( + args.data_source_github, ref=args.data_source_github_ref, + path=args.data_source_path) + except ValueError as exc: + print(f"serve refused: --data-source-github: {exc}", + file=sys.stderr) + return 1 data_source = registry_mod.data_source_from_options( directory=args.data_source_dir, - url=args.data_source_url or (registry_mod.github_raw_base_url( - args.data_source_github, ref=args.data_source_github_ref, - path=args.data_source_path) if args.data_source_github else None), + url=url, token_env=args.data_source_token_env, ) serve(args.web_dir, args.snapshot, args.checkout_root, host=args.host, diff --git a/tests/test_projection_seams.py b/tests/test_projection_seams.py index 14321c8..9504dca 100644 --- a/tests/test_projection_seams.py +++ b/tests/test_projection_seams.py @@ -654,13 +654,37 @@ def test_serve_main_refuses_a_data_source_openDoxs_registry_cannot_read( reached = [] monkeypatch.setattr(serve, "serve", lambda *a, **k: reached.append(a)) repo = _repository(tmp_path) - for flag, value in (("--data-source-dir", str(tmp_path)), - ("--data-source-token-env", "SECRET_ENV")): + for flags, named in ( + (["--data-source-dir", str(tmp_path)], "--data-source-dir"), + (["--data-source-token-env", "SECRET_ENV"], "--data-source-token-env"), + # An explicitly EMPTY option is a declaration too: `serve.main()` + # hands it over as given, and never drops it for being falsy. + (["--data-source-url", ""], "--data-source-url"), + (["--data-source-github", "o/r"], + "--data-source-github, which composes one"), + (["--data-source-github", ""], + "serve refused: --data-source-github: expected OWNER/REPO")): rc = serve.main(["--snapshot", str(tmp_path / "s.json"), "--checkout-root", - str(repo), flag, value]) - assert rc == 1 and not reached, flag + str(repo), *flags]) + assert rc == 1 and not reached, flags err = capsys.readouterr().err - assert "serve refused:" in err and flag in err + assert "serve refused:" in err and named in err, (flags, err) + + +def test_dropping_the_active_entry_clears_the_active_key(tmp_path) -> None: + """No ref-less request meets a key with nothing behind it, and the next + entry registered becomes active, as the first one did.""" + reg = default_registry + registry = reg.SnapshotRegistry() + first = registry.register(reg.SnapshotEntry("garden", source_root=tmp_path)) + assert registry.active is first + registry.drop("garden") + assert registry.active is None and registry.resolve(None) is None + second = registry.register(reg.SnapshotEntry("orchard", source_root=tmp_path)) + assert registry.active is second + registry.register(reg.SnapshotEntry("orchard", "draft/t")) + registry.drop("orchard", "draft/t") + assert registry.active is second, "dropping another entry leaves the active one" def test_the_registry_keeps_a_sessions_owner_and_base_and_confines_each_entry(tmp_path) -> None: From 244d7b7f9e8b0a3f95ce41ff1b7ca72f5ec3d921 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Mon, 28 Sep 2026 18:46:53 +0000 Subject: [PATCH 16/38] T055: the unavailable-validator warning says which of its two cases happened Taken from the overview of Copilot's review of bfb9c479. That review had no findings and no threads. _warn_validator_not_found described a filesystem search, "no validator for kind X was reachable from ... or from ", in both of its cases. When nothing is registered for the kind, the lookup is the process's own registry, and no path could make a validator reachable, so the old text pointed at a remedy that does not exist. The warning now separates the two cases: - Nothing is registered for the kind. The warning names no path, and gives the lookup's refusal whole, on one line. That refusal ends with the call that registers a validator. It used to be cut at its first line, which ended in "at process start with". workbench.validate_manifest made the same cut, and it is corrected there too. - The registered validator reached no verdict. The warning says it was offered both roots to search from, the OUTPUT path first and --repo-root second (T092 defect 8), and then gives the validator's own reason. Two assertions pin the cases apart, and both mutations fail them. Whole suite: 2690 passed, 11 skipped. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/cli.py | 40 +++++++++++++++++++++------------- src/opendox/workbench.py | 2 +- tests/test_projection_seams.py | 6 +++++ 3 files changed, 32 insertions(+), 16 deletions(-) diff --git a/src/opendox/cli.py b/src/opendox/cli.py index 116ac96..09a76e4 100644 --- a/src/opendox/cli.py +++ b/src/opendox/cli.py @@ -341,34 +341,43 @@ def _validate_by_kind(written: Path, kind: str, *, strict: bool, directory first and the SERVED CHECKOUT second (T092 acceptance sweep, defect 8, which is why the order is kept). No validator registered for the kind is VALIDATOR UNAVAILABLE, sub-case "nothing to run", and the lookup's - own refusal is the reason given.""" + own refusal is the reason given, whole, on one line: it ends with the call + that registers a validator, which is the remedy.""" try: validator = projection_seams.validators.for_kind(kind) except projection_seams.ValidatorNotRegistered as exc: return None, projection_seams.ValidationResult( False, -1, "", "", None, projection_seams.VALIDATOR_UNAVAILABLE, - str(exc).split("\n", 1)[0]) + " ".join(str(exc).split())) return validator, validator.validate(written, strict=strict, search_from=search_from) def _warn_validator_not_found(written: Path, repo_root: Path, kind: str, - result) -> None: + result, *, registered: bool) -> None: """VALIDATOR UNAVAILABLE, sub-case "nothing to run". - NOT routine. Both roots were offered to the search, so the message names - BOTH and blames neither on its own, and then gives the lookup's own reason: - reaching here means no validator for this snapshot's kind could be reached - from the OUTPUT path OR from the served checkout, and the snapshot went - unvalidated however good the corpus was. The old one-liner ("no reachable - openxFactory checkout") read as routine while quietly meaning - "unvalidated", and pointed at a checkout that was present and fine — which - is exactly where it sent the T092 pass.""" + NOT routine: the snapshot went unvalidated however good the corpus was. + The old one-liner ("no reachable openxFactory checkout") read as routine + while quietly meaning "unvalidated", and pointed at a checkout that was + present and fine, which is exactly where it sent the T092 pass. So this + says which of the two things happened, and then gives the reason: + + * NOTHING IS REGISTERED FOR THE KIND (`registered` false). The validator + lookup is the process's own registry, so no path can make a validator + reachable. Moving the output or the checkout would change nothing, and + the message names neither. The lookup's refusal says what registers one. + * THE VALIDATOR REGISTERED FOR THE KIND REACHED NO VERDICT. It was offered + both roots to search from, the OUTPUT path's directory first and the + served checkout second (T092 acceptance sweep, defect 8), so the message + names BOTH and blames neither on its own. Its own reason follows.""" print(" validation SKIPPED — this snapshot was NOT checked against the " "pinned schema", file=sys.stderr) - print(f" no validator for kind {kind!r} was reachable from " - f"{written.parent} (the OUTPUT path, searched first) or from " - f"{repo_root} (--repo-root, the fallback)", file=sys.stderr) + if registered: + print(f" the validator registered for kind {kind!r} reached no " + f"verdict. It was offered {written.parent} (the OUTPUT path) " + f"first, then {repo_root} (--repo-root), to search from", + file=sys.stderr) if result.unavailable_reason: print(f" {result.unavailable_reason}", file=sys.stderr) @@ -441,7 +450,8 @@ def _validate(written: Path, args: argparse.Namespace, *, written, kind, strict=args.strict, search_from=(written.parent, repo_root)) if not result.available: if result.validator is None: - _warn_validator_not_found(written, repo_root, kind, result) + _warn_validator_not_found(written, repo_root, kind, result, + registered=validator is not None) else: _warn_validator_could_not_run(result, validator) if args.strict: diff --git a/src/opendox/workbench.py b/src/opendox/workbench.py index 8740d9b..216c296 100644 --- a/src/opendox/workbench.py +++ b/src/opendox/workbench.py @@ -462,7 +462,7 @@ def validate_manifest(path: Path | str, *, validator: Path | None = None, try: registered = projection_seams.validators.for_kind(KIND) except projection_seams.ValidatorNotRegistered as exc: - return ManifestValidation(False, -1, "", str(exc).split("\n", 1)[0], + return ManifestValidation(False, -1, "", " ".join(str(exc).split()), None) result = registered.validate(path, strict=strict, search_from=(search_from or path.parent,)) diff --git a/tests/test_projection_seams.py b/tests/test_projection_seams.py index 9504dca..bab16dc 100644 --- a/tests/test_projection_seams.py +++ b/tests/test_projection_seams.py @@ -1030,6 +1030,7 @@ def test_openDoxs_own_kind_meets_the_stand_in_and_strict_makes_it_fatal(tmp_path assert cli._validate(written, _validate_args(tmp_path)) == 0 err = capsys.readouterr().err assert "validation SKIPPED" in err and "'opendox-snapshot'" in err and "T057" in err + assert "the validator registered for kind 'opendox-snapshot' reached no verdict" in err assert str(written.parent) in err and str(tmp_path.resolve()) in err assert "the ENVIRONMENT, not the snapshot" in err assert cli._validate(written, _validate_args(tmp_path, "--strict")) == 1 @@ -1041,6 +1042,11 @@ def test_a_kind_with_no_validator_is_unavailable_not_another_kinds(tmp_path, cap assert cli._validate(_written(tmp_path, "stranger"), _validate_args(tmp_path)) == 0 err = capsys.readouterr().err assert "no validator is registered for kind 'stranger'" in err + assert ps.validators.registration_call in err, ( + "the lookup's refusal is given whole: it ends with the call that registers one") + assert "to search from" not in err and str(tmp_path / "out") not in err, ( + "the lookup is the process's registry: no path could make a validator " + "reachable, so the warning names none") def test_a_rejection_fails_and_blames_the_snapshot(tmp_path, capsys) -> None: From 19c2678295d8c0c8273830e90b2e6e2f5bebbe7a Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Mon, 28 Sep 2026 19:00:02 +0000 Subject: [PATCH 17/38] T055: the writer refuses what JSON cannot carry; one request serves the entry its refusal checked Taken from Copilot's review of 244d7b7f: - r4125900060. The default writer's json.dumps ran with allow_nan=True, so a registered generator's NaN or infinity was written as NaN or Infinity. No JSON reader parses those. The writer now refuses them with allow_nan=False. It also refuses a value of no JSON type and a structure that contains itself. The refusal is SnapshotNotWritable, a ProjectionSeamError, raised before anything is written. A generate verb reports it as "generate refused: ...". Five writer cases and a CLI case are new. - r4125900164. On the query-less path, _serve_snapshot read the active entry once for the hosted refusal, and _read_snapshot() read it again for the body. So a refresh that made a session active between the two could pass the refusal with main and serve the session's bytes, which FR-048 forbids. The handlers came from openXdox unchanged, and the race came with them. The active entry is now resolved once, and that entry is handed to the refusal, the body (_read_snapshot(entry)) and the headers. A new case makes a session active right after the refusal check passes. The response is still main's bytes and ref, and the double read fails it. Whole suite: 2697 passed, 11 skipped. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/default_projection.py | 32 ++++++++++++--- src/opendox/serve.py | 23 +++++++---- tests/test_projection_seams.py | 65 +++++++++++++++++++++++++++++++ 3 files changed, 108 insertions(+), 12 deletions(-) diff --git a/src/opendox/default_projection.py b/src/opendox/default_projection.py index a7f8149..a06c5ac 100644 --- a/src/opendox/default_projection.py +++ b/src/opendox/default_projection.py @@ -24,7 +24,9 @@ `WRITER`, THE SNAPSHOT WRITER. Canonical JSON: keys sorted at every depth, two spaces of indent, ASCII only, a trailing newline, and no clock. So the same snapshot is the same bytes. It writes through the interactivity boundary it is -handed, and only there. +handed, and only there. A snapshot holding what JSON cannot carry (NaN, an +infinity, a value of no JSON type) is refused, `SnapshotNotWritable`, before +anything is written. `VALIDATOR`, THE VALIDATOR LOOKUP'S DEFAULT, FOR openDox's OWN KINDS. It is openDox's own validator, plan 034's T057, which this tree does not carry yet. @@ -54,7 +56,7 @@ from opendox import generator_seam, projection_seams __all__ = ["CORPUS_ROOT", "CorpusRoot", "OWN_KINDS", "OwnValidatorNotBuilt", - "VALIDATOR", "WRITER", "Writer"] + "SnapshotNotWritable", "VALIDATOR", "WRITER", "Writer"] #: The workbench manifest's kind, `opendox.workbench.KIND`, restated because #: `workbench` imports PyYAML and this module must import with nothing extra. @@ -134,15 +136,35 @@ def change_rows(checkout_root: Path | str) -> tuple: return () +class SnapshotNotWritable(projection_seams.ProjectionSeamError): + """The snapshot holds a value JSON cannot carry, so the writer refuses it + and writes nothing. A generate verb reports it as a refusal.""" + + class Writer: """openDox's own canonical snapshot writer.""" @staticmethod def canonical_json(snapshot: dict[str, Any]) -> str: """Deterministic JSON: keys sorted at every depth, two spaces of - indent, ASCII only, and a trailing newline.""" - return json.dumps(snapshot, indent=2, sort_keys=True, - ensure_ascii=True) + "\n" + indent, ASCII only, and a trailing newline. + + NOTHING JSON CANNOT CARRY. NaN and the infinities are refused + (`allow_nan=False`) rather than written as the `NaN` and `Infinity` + that Python's `json` would otherwise emit, which no JSON reader parses: + not the server's, not a browser's, not a validator's. So is a value of + no JSON type, and a structure that contains itself. A registered + generator can answer any of them, and the seam checks only a + snapshot's kind and version.""" + try: + return json.dumps(snapshot, indent=2, sort_keys=True, + ensure_ascii=True, allow_nan=False) + "\n" + except (TypeError, ValueError) as exc: + raise SnapshotNotWritable( + f"the snapshot holds a value JSON cannot carry ({exc}). NaN, " + "the infinities and values of no JSON type have no JSON " + "spelling, and a file carrying one would be one no JSON reader " + "parses, so nothing was written") from exc def write_snapshot(self, snapshot: dict[str, Any], path: Path | str, boundary) -> Path: diff --git a/src/opendox/serve.py b/src/opendox/serve.py index 5f194c5..f61c8f1 100644 --- a/src/opendox/serve.py +++ b/src/opendox/serve.py @@ -1164,11 +1164,16 @@ def _query_key(self) -> tuple[str | None, str | None]: return ((params.get("repository") or [None])[0], (params.get("ref") or [None])[0]) - def _read_snapshot(self) -> bytes | None: - """The ACTIVE snapshot's bytes, through the registry when a source is - bound, and from the configured path when none is (a hand-built - handler).""" - entry = self._active_entry() + def _read_snapshot(self, entry=None) -> bytes | None: + """The bytes of `entry`, the active entry a request resolved to, and + of the configured path where it resolved none (a hand-built handler, or + a source with nothing registered). + + It never resolves the active entry itself. `_serve_snapshot` resolves + it ONCE and hands the same entry to the refusal, this read and the + headers, so a refresh that changes the active entry mid-request cannot + pass the hosted refusal with one entry and serve another's bytes + (FR-048).""" if entry is not None: return entry.read_bytes() try: @@ -1214,9 +1219,13 @@ def _serve_snapshot(self, head_only: bool) -> None: if repository: self.send_error(404, "no such snapshot") return - if self._hosted_entry_refused(self._active_entry()): + # ONE RESOLUTION of the active entry, for the refusal, the body and + # the headers alike (see `_read_snapshot`). + entry = self._active_entry() + if self._hosted_entry_refused(entry): return - self._serve_bytes(self._read_snapshot(), JSON_CTYPE, head_only) + self._serve_bytes(self._read_snapshot(entry), JSON_CTYPE, head_only, + entry=entry) def _hosted_entry_refused(self, entry) -> bool: """Refuse, and answer, when the entry a request RESOLVED to is at a diff --git a/tests/test_projection_seams.py b/tests/test_projection_seams.py index bab16dc..6db1b3c 100644 --- a/tests/test_projection_seams.py +++ b/tests/test_projection_seams.py @@ -817,6 +817,44 @@ def test_the_hosted_plane_refuses_a_session_ref_named_or_active(tmp_path) -> Non assert status == 403, "the refusal follows the entry the request resolved to" +def test_one_request_serves_the_entry_its_refusal_checked(tmp_path, monkeypatch) -> None: + """FR-048 against a concurrent refresh. The active entry is resolved ONCE + for the refusal, the body and the headers: a refresh that makes a session + active right after the refusal passed `main` cannot put the session's bytes + in that response.""" + _repo, _out, httpd = _served(tmp_path) + bound = httpd.RequestHandlerClass.func + bound.loopback = False + main = bound.source.registry.active + session_path = tmp_path / "session.json" + session_path.write_text('{"kind": "a-session-snapshot"}', encoding="utf-8") + session = default_registry.SnapshotEntry("garden", "draft/t", + snapshot_path=session_path) + landed = [] + + class _Registry(default_registry.SnapshotRegistry): + @property + def active(self): + return session if landed else main + + registry = _Registry() + registry.register(main) + registry.register(session) + bound.source.registry = registry + checked = bound._hosted_entry_refused + + def refusal_then_a_refresh_lands(self, entry): + answered = checked(self, entry) + landed.append(True) + return answered + + monkeypatch.setattr(bound, "_hosted_entry_refused", refusal_then_a_refresh_lands) + status, headers, body = _get(httpd, "/snapshot.json") + assert status == 200 and landed, "the refusal ran, and a refresh landed after it" + assert json.loads(body)["kind"] == NEUTRAL, "the body is the entry the refusal checked" + assert headers["X-Snapshot-Ref"] == "main" + + def test_hosted_ref_refused_asks_the_registered_registry() -> None: ps.register_defaults() assert serve.hosted_ref_refused(True, "draft/t") is False @@ -881,6 +919,33 @@ def test_the_writer_is_canonical_and_writes_only_through_the_boundary(tmp_path) writer.write_snapshot(snapshot, tmp_path / "elsewhere.json", boundary) +@pytest.mark.parametrize("value", [float("nan"), float("inf"), float("-inf"), + Path("not-json"), {"a": {1, 2}}]) +def test_the_writer_refuses_what_json_cannot_carry(tmp_path, value) -> None: + """Never the `NaN`/`Infinity` that `json` would otherwise write, which no + JSON reader parses; and nothing is written.""" + boundary = OutputBoundary(tmp_path, ["snapshot.json"]) + with pytest.raises(default_projection.SnapshotNotWritable) as caught: + default_projection.WRITER.write_snapshot( + {"kind": NEUTRAL, "value": value}, tmp_path / "snapshot.json", boundary) + assert isinstance(caught.value, ps.ProjectionSeamError) + assert "JSON cannot carry" in str(caught.value) + assert not (tmp_path / "snapshot.json").exists() + + +def test_generate_refuses_a_snapshot_json_cannot_carry(tmp_path, capsys) -> None: + def operation(repo_root, repository, *, source_revision=None, generated_at=None): + return {"schema_version": 1, "kind": "host-snapshot", "repository": repository, + "generation": {"source_revision": "r"}, "score": float("nan")} + + gs.register(gs.SnapshotGenerator(contract="host-snapshot", generate=operation)) + out = tmp_path / "out" / "snapshot.json" + assert _generate(_repository(tmp_path), out, "--no-validate") == 1 + err = capsys.readouterr().err + assert "generate refused:" in err and "JSON cannot carry" in err + assert not out.exists() + + def test_the_validator_stand_in_concludes_nothing_and_names_T057(tmp_path) -> None: result = default_projection.VALIDATOR.validate(tmp_path / "x.json") assert result.available is False and result.ok is False From 8c09d74cb84fe5e988ae149b60952b546a6ac1e5 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Mon, 28 Sep 2026 19:13:14 +0000 Subject: [PATCH 18/38] T055: the source arm resolves its entry once, and asks for the path by that entry's pair Taken from Copilot's review of 19c26782 (r4126022918). _serve_source resolved the entry twice. It called registry.resolve_source(repository, ref, rest) for the path, and then registry.resolve(repository, ref) for the hosted refusal and the headers. In the unkeyed form both read the ACTIVE entry. A refresh that switched the active entry from a session to main between the two reads could take the path from the session's worktree, pass the refusal with main, and serve the session's file on a hosted plane. That breaks FR-048, the same race the snapshot arm had. The arm now resolves the entry once, checks the refusal against it, and asks resolve_source for that entry's own (repository, ref), never for "the active entry" again. The path is still confined through the registry's resolve_source, so the single-entry-point rule that test_source_core_arm holds is kept. That test now also counts one resolve() in the arm. A new case holds a session active until the first path lookup, then main. The response is 403, and the session's bytes never appear in it. With the double read restored, the session's file is served and the case fails. Whole suite: 2698 passed, 11 skipped. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/serve.py | 15 +++++++++++--- tests/test_projection_seams.py | 37 ++++++++++++++++++++++++++++++++++ tests/test_source_core_arm.py | 7 ++++++- 3 files changed, 55 insertions(+), 4 deletions(-) diff --git a/src/opendox/serve.py b/src/opendox/serve.py index f61c8f1..eb9ae71 100644 --- a/src/opendox/serve.py +++ b/src/opendox/serve.py @@ -1292,12 +1292,21 @@ def _serve_source(self, tail: str, head_only: bool) -> None: return entry = None if self.source is not None: - target = self.source.registry.resolve_source(repository, ref, rest) + # ONE RESOLUTION of the entry, for the refusal, the confined path + # and the headers alike. The UNKEYED form resolves to the ACTIVE + # entry, which the query never named, the same ref-less hole + # `_serve_snapshot` closes. Resolved twice, once for the path and + # once for the refusal, a refresh that changed the active entry + # between the two could serve one entry's file under another's + # refusal and headers (FR-048). So the entry is resolved once, and + # the path is asked for by THAT entry's own pair, which is the pair + # the refusal checked, never "the active entry" a second time. entry = self.source.registry.resolve(repository, ref) - # the UNKEYED form resolves to the ACTIVE entry, which the query never - # named — the same ref-less hole `_serve_snapshot` closes if self._hosted_entry_refused(entry): return + target = (None if entry is None else + self.source.registry.resolve_source( + entry.repository, entry.ref, rest)) else: target = resolve_source_path(Path(self.checkout_root), rest) if target is None: diff --git a/tests/test_projection_seams.py b/tests/test_projection_seams.py index 6db1b3c..549197c 100644 --- a/tests/test_projection_seams.py +++ b/tests/test_projection_seams.py @@ -855,6 +855,43 @@ def refusal_then_a_refresh_lands(self, entry): assert headers["X-Snapshot-Ref"] == "main" +def test_the_source_arm_confines_to_the_entry_its_refusal_checked(tmp_path) -> None: + """FR-048 on `/source/`, against a concurrent refresh. The unkeyed form + resolves the ACTIVE entry once, and asks for the path by that entry's own + pair. Here the active entry is a session until the first path lookup and + `main` after it. Read twice, the path came from the session's worktree and + the refusal passed `main`, so a hosted plane served the session's file.""" + _repo, _out, httpd = _served(tmp_path) + bound = httpd.RequestHandlerClass.func + bound.loopback = False + main = bound.source.registry.active + worktree = tmp_path / "worktree" + worktree.mkdir() + (worktree / "secret.md").write_text("SESSION-ONLY BYTES\n", encoding="utf-8") + session = default_registry.SnapshotEntry("garden", "draft/t", source_root=worktree) + + class _Registry(default_registry.SnapshotRegistry): + looked_up = False + + @property + def active(self): + return main if self.looked_up else session + + def resolve_source(self, repository, ref, tail): + try: + return super().resolve_source(repository, ref, tail) + finally: + self.looked_up = True + + registry = _Registry() + registry.register(main) + registry.register(session) + bound.source.registry = registry + status, _headers, body = _get(httpd, "/source/secret.md") + assert b"SESSION-ONLY" not in body, "a hosted plane served a session's file" + assert status == 403 and json.loads(body)["error"] == "session_unavailable" + + def test_hosted_ref_refused_asks_the_registered_registry() -> None: ps.register_defaults() assert serve.hosted_ref_refused(True, "draft/t") is False diff --git a/tests/test_source_core_arm.py b/tests/test_source_core_arm.py index 5d51f48..486d6aa 100644 --- a/tests/test_source_core_arm.py +++ b/tests/test_source_core_arm.py @@ -305,11 +305,16 @@ def test_the_arm_resolves_through_that_entry_point_and_the_registry_only(): body = _source_of(_method("_serve_source")) assert "resolve_source_path(Path(self.checkout_root), rest)" in body, ( "the no-registry path resolves through the single-root entry point") - assert "self.source.registry.resolve_source(repository, ref, rest)" in body, ( + assert "self.source.registry.resolve_source(" in body and \ + "entry.repository, entry.ref, rest)" in body, ( "the registry path resolves through the entry's OWN root (per-entry " "confinement, task 2.2) — not through the served checkout with a key " "stripped off, which is the silent-wrong-data failure the seam exists " "to prevent") + assert body.count("self.source.registry.resolve(") == 1, ( + "ONE resolution of the entry, whose own pair the path is then asked " + "for (plan 034 T055, FR-048): resolved a second time, the active " + "entry could change between the refusal and the path") assert "resolve_within(" not in body, ( "the arm must not call the containment rule directly: one entry point, " "so a change to the rule cannot reach the route by one path and miss " From f4ef63d37e0fc6de7f57631fc02c5e2c3d03f884 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Mon, 28 Sep 2026 19:25:51 +0000 Subject: [PATCH 19/38] T055: the default writer replaces the snapshot atomically Taken from Copilot's review of 8c09d74c (r4126138808). serve.py answers /snapshot.json on threads of its own while a refresh rewrites the snapshot it serves. The default writer used OutputBoundary.write_output, which truncates the file and then writes it, so a request could read a truncated snapshot. The writer now sends the bytes to a temporary sibling and moves them over the target with one os.replace. A reader sees the whole old snapshot or the whole new one. The boundary still decides the destination. permit_output, the same check write_output makes (root, allowlist, refusal and ledger), runs first, so a refused target leaves nothing behind. The sibling is created exclusively beside the permitted target, with an ordinary write's mode. Its name is a dot-file with no document extension, so /source never serves it, and it is removed if the write or the move fails. Three new cases. A reader holding the old file still reads the whole old snapshot, and the new bytes arrive by rename. A failed move leaves the old snapshot and no sibling. The sibling's name is refused by /source. Restoring the in-place write, or dropping the cleanup, makes a case fail. Whole suite: 2701 passed, 11 skipped. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/default_projection.py | 42 ++++++++++++++++++++++++++++--- tests/test_projection_seams.py | 39 ++++++++++++++++++++++++++++ 2 files changed, 77 insertions(+), 4 deletions(-) diff --git a/src/opendox/default_projection.py b/src/opendox/default_projection.py index a06c5ac..18f31d9 100644 --- a/src/opendox/default_projection.py +++ b/src/opendox/default_projection.py @@ -26,7 +26,8 @@ snapshot is the same bytes. It writes through the interactivity boundary it is handed, and only there. A snapshot holding what JSON cannot carry (NaN, an infinity, a value of no JSON type) is refused, `SnapshotNotWritable`, before -anything is written. +anything is written. The write is ATOMIC: a temporary sibling, then one +`os.replace`, so a request never reads a half-written snapshot. `VALIDATOR`, THE VALIDATOR LOOKUP'S DEFAULT, FOR openDox's OWN KINDS. It is openDox's own validator, plan 034's T057, which this tree does not carry yet. @@ -49,7 +50,10 @@ from __future__ import annotations +import contextlib import json +import os +import uuid from pathlib import Path from typing import Any @@ -168,9 +172,39 @@ def canonical_json(snapshot: dict[str, Any]) -> str: def write_snapshot(self, snapshot: dict[str, Any], path: Path | str, boundary) -> Path: - """Render canonically and write through the interactivity boundary, - which writes only under its declared output allowlist.""" - return boundary.write_output(path, self.canonical_json(snapshot)) + """Render canonically and write where the interactivity boundary + permits: its root, under its declared output allowlist. + + ATOMICALLY. `serve.py` answers `/snapshot.json` on threads of its own + while a refresh rewrites the very snapshot it serves, and a write in + place (truncate, then write) let a request read a truncated file. So + the bytes go to a temporary sibling first, and one `os.replace` moves + them over the target: a reader sees the whole old snapshot or the + whole new one, never part of either. + + THE BOUNDARY STILL DECIDES THE DESTINATION. `permit_output` is the + check `write_output` makes, root and allowlist, with its refusal and + its ledger, and it runs first, so a refused target leaves nothing + behind. The sibling is created exclusively beside the permitted + target, with the mode an ordinary write would give it. Its name is a + dot-file with no document extension, so `/source` never serves it, + and it is removed if the write or the move fails.""" + data = self.canonical_json(snapshot).encode("ascii") + target = boundary.permit_output(path) + target.parent.mkdir(parents=True, exist_ok=True) + temporary = target.with_name(f".{target.name}.{uuid.uuid4().hex}.tmp") + descriptor = os.open(temporary, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o666) + try: + with os.fdopen(descriptor, "wb") as stream: + stream.write(data) + stream.flush() + os.fsync(stream.fileno()) + os.replace(temporary, target) + except BaseException: + with contextlib.suppress(OSError): + os.unlink(temporary) + raise + return target class OwnValidatorNotBuilt: diff --git a/tests/test_projection_seams.py b/tests/test_projection_seams.py index 549197c..ce5cfa8 100644 --- a/tests/test_projection_seams.py +++ b/tests/test_projection_seams.py @@ -550,6 +550,8 @@ def tree(tmp_path) -> Path: (root / "alias.md").symlink_to(".env") (root / "same.md").symlink_to("docs/a.md") (root / "spec.yaml").symlink_to("changes/.openspec.yaml") + # The writer's temporary sibling, as a crash mid-write would leave it. + (root / "docs" / ".snapshot.json.0123abcd.tmp").write_text("{", encoding="utf-8") return root @@ -573,6 +575,7 @@ def tree(tmp_path) -> Path: ("alias.md", False), ("same.md", True), ("spec.yaml", True), + ("docs/.snapshot.json.0123abcd.tmp", False), ]) def test_resolve_within_is_the_containment_source_has_always_had(tree, tail, served) -> None: """The rule `/source` has always been confined by, and one place where @@ -954,6 +957,42 @@ def test_the_writer_is_canonical_and_writes_only_through_the_boundary(tmp_path) '"d": "\\u00e9"\n },\n "b": 1\n}\n') with pytest.raises(BoundaryViolation): writer.write_snapshot(snapshot, tmp_path / "elsewhere.json", boundary) + assert sorted(p.name for p in tmp_path.iterdir()) == ["snapshot.json"], ( + "the boundary refused the destination before anything was created") + + +def test_the_writer_replaces_the_snapshot_atomically(tmp_path) -> None: + """A request mid-read while a refresh rewrites the snapshot sees the + whole old snapshot, never a truncated one: the new bytes arrive by one + rename over the target, not by a rewrite in place.""" + writer = default_projection.WRITER + target = tmp_path / "snapshot.json" + boundary = OutputBoundary(tmp_path, ["snapshot.json"]) + writer.write_snapshot({"kind": NEUTRAL, "n": 1}, target, boundary) + before = os.stat(target).st_ino + with open(target, "rb") as reader: + writer.write_snapshot({"kind": NEUTRAL, "n": 2}, target, boundary) + assert json.loads(reader.read()) == {"kind": NEUTRAL, "n": 1} + assert json.loads(target.read_text(encoding="utf-8")) == {"kind": NEUTRAL, "n": 2} + assert os.stat(target).st_ino != before, "the bytes arrived by a rename" + assert sorted(p.name for p in tmp_path.iterdir()) == ["snapshot.json"], ( + "no temporary sibling is left behind") + + +def test_a_failed_move_leaves_the_old_snapshot_and_no_sibling(tmp_path, monkeypatch) -> None: + writer = default_projection.WRITER + target = tmp_path / "snapshot.json" + boundary = OutputBoundary(tmp_path, ["snapshot.json"]) + writer.write_snapshot({"kind": NEUTRAL, "n": 1}, target, boundary) + + def the_move_fails(source, destination): + raise OSError("the move failed") + + monkeypatch.setattr(default_projection.os, "replace", the_move_fails) + with pytest.raises(OSError, match="the move failed"): + writer.write_snapshot({"kind": NEUTRAL, "n": 2}, target, boundary) + assert json.loads(target.read_text(encoding="utf-8")) == {"kind": NEUTRAL, "n": 1} + assert sorted(p.name for p in tmp_path.iterdir()) == ["snapshot.json"] @pytest.mark.parametrize("value", [float("nan"), float("inf"), float("-inf"), From 521de95c2ed8f2c7e12714379fc9c7e79bc61d52 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Tue, 29 Sep 2026 16:19:00 +0000 Subject: [PATCH 20/38] T050: the docstring says the required check collects this suite Since T036 (#52), `validate` runs the whole suite (`python -m pytest -q` over the configured testpaths) instead of an explicit list of files. So the paragraph that said this suite was "not yet wired" into that list, and ran by node id, has been false since T036 landed. It now says that the required check collects this suite like every other. The text is #57's correction of the same paragraph, byte for byte. Copilot raised it on #57 (r4117299033). It is taken here because the paragraph is already false at main, so it must not land with #53. When #57 takes this head, that hunk merges clean. The opening paragraph ("T052 and T054 ... have not landed") stays. It is true until T054 lands, and #57 corrects it. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/test_plain_documents_fixture.py | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/tests/test_plain_documents_fixture.py b/tests/test_plain_documents_fixture.py index 6a0f23d..02a9258 100644 --- a/tests/test_plain_documents_fixture.py +++ b/tests/test_plain_documents_fixture.py @@ -35,13 +35,10 @@ will require regardless of station — `title` and `summary` — per T050's task line and the answer's own example. -NOT YET WIRED INTO `.github/workflows/validate.yml`'s explicit pytest list: -the phase-2 draft-ahead scope keeps this PR out of that file (conftest.py, -pyproject.toml, validate.yml and README.md are the phase-1 chain's), so this -suite runs by node id today, exactly as other narrowed-out suites in this -tree have (`tests/test_display_facet_leaves.py`'s own S7-residue history). -It joins the enumerated list whichever later task next touches it — most -likely F5.3/T056, or T049's own close. +COLLECTED BY THE REQUIRED CHECK. Since T036, `validate` runs the whole suite +(`python -m pytest -q` over the configured testpaths) instead of an explicit +list of files, so this suite is collected like every other and needs no entry +anywhere. A CREATED file: no row in openxFactory's `docs/opendox-carve-manifest.yaml` (RULED OQ-C: the manifest declares what LEAVES openxFactory, never what a From 7028f6136560ab1617bbd37c815806cb48f63e85 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Tue, 29 Sep 2026 16:27:11 +0000 Subject: [PATCH 21/38] T050: the unrelated source shares no topic with the pair, not just no phrase Copilot at 521de95c (r4135803935) is right. The case showed only that the source outside the rain-barrel pair lacks the literal phrase. A later fixture edit could give the sources another topic in common and still pass, which defeats the guarantee the case exists for. The case now compares the two things a topic that sources share can be derived from when nobody declares one (R1Q13 (a) with (c)): the words of each document's name, and the other documents each one names. - Each rain-barrel note's name carries the pair's topic. - The source outside the pair shares no name word with either note. Only the article and conjunction in FUNCTION_WORDS are dropped. - No document on either side names one on the other, by title or by file name. It needs no projection, because T054 is not in this tree. T054's projection test proves the grouping itself over this fixture. The new case goes red on three plants, and the old one passes all three: - a shared name word ("garden" in both titles), which is Copilot's scenario; - the unrelated note naming a pair member by its file name; - a pair member naming the unrelated note by its title. The real fixture passes, with 20 cases, so the triple does not move. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/test_plain_documents_fixture.py | 89 ++++++++++++++++++++++++--- 1 file changed, 79 insertions(+), 10 deletions(-) diff --git a/tests/test_plain_documents_fixture.py b/tests/test_plain_documents_fixture.py index 02a9258..e615caf 100644 --- a/tests/test_plain_documents_fixture.py +++ b/tests/test_plain_documents_fixture.py @@ -199,17 +199,86 @@ def test_at_least_two_sources_share_a_topic() -> None: ) +#: The two words this fixture's titles use that carry no topic under any +#: reading of a name: an article and a conjunction. They are dropped before +#: two names are compared, and nothing else is. +FUNCTION_WORDS = frozenset({"the", "and"}) + +_LETTER_RUN = re.compile(r"[^\W\d_]+") + + +def _tokens(text: str) -> list[str]: + """The runs of letters in `text`, case-folded, in order.""" + return [run.casefold() for run in _LETTER_RUN.findall(text)] + + +def _name_words(header: dict[str, str]) -> set[str]: + """The words of a document's name: its `title:` (every document here + declares one, `test_required_fields_present`), as runs of three or more + letters, without `FUNCTION_WORDS`.""" + return {word for word in _tokens(header.get("title", "")) + if len(word) >= 3} - FUNCTION_WORDS + + +def _names(text: str, path: Path, header: dict[str, str]) -> bool: + """Whether `text` names the document at `path`: whether it holds that + document's title, or its file name, as a run of whole words.""" + words = _tokens(text) + for name in (header.get("title", ""), path.stem): + run = _tokens(name) + if run and any(words[i:i + len(run)] == run + for i in range(len(words) - len(run) + 1)): + return True + return False + + def test_at_least_one_source_shares_no_topic() -> None: - """Keeps the assertion above honest: a fixture where every source shared - one word would not exercise topic-based selection at all.""" - singleton = [ - path for path, (header, body) in _documents().items() - if "stage" not in header - and SHARED_SOURCE_TOPIC not in " ".join( - (header.get("title", ""), header.get("summary", ""), body) - ).lower() - ] - assert singleton, "every source shares the same topic phrase" + """At least one source shares no topic with the rain-barrel pair. + Without it, a fixture where every source shared one topic would not + exercise topic-based grouping at all. + + NO TOPIC, not just not the phrase. The phrase check alone would pass a + fixture edit that gave the sources some other topic in common, so this + compares the two things a topic that sources share can be derived from + when nobody declares one (R1Q13 (a) with (c)): + - the words of each document's name; + - the other documents each one names. + + So each rain-barrel note's name carries the pair's topic. A source + outside the pair shares no name word with either note, and it and the + notes never name each other, in either direction. Words elsewhere in + the body are not compared. That the projection then forms exactly this + group is proved over this fixture by T054's projection test. + """ + sources = {path: parts for path, parts in _documents().items() + if "stage" not in parts[0]} + pair = [path for path, (header, body) in sources.items() + if SHARED_SOURCE_TOPIC in " ".join( + (header.get("title", ""), header.get("summary", ""), body) + ).lower()] + others = [path for path in sources if path not in pair] + assert others, "every source shares the same topic phrase" + + topic = set(SHARED_SOURCE_TOPIC.split()) + for member in pair: + assert topic <= _name_words(sources[member][0]), ( + f"{member.name}'s name does not carry {SHARED_SOURCE_TOPIC!r}") + + for other in others: + other_header = sources[other][0] + for member in pair: + member_header = sources[member][0] + shared = _name_words(other_header) & _name_words(member_header) + assert not shared, ( + f"{other.name} and {member.name} share the name word(s) " + f"{sorted(shared)}, so a topic rule could group them") + for writer, named, named_header in ( + (other, member, member_header), + (member, other, other_header)): + assert not _names(writer.read_text(encoding="utf-8"), + named, named_header), ( + f"{writer.name} names {named.name}, so its topics " + "could reach across the pair") if __name__ == "__main__": From 4ba410ff91200d288b78043fb694c3c372e1df4b Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Tue, 29 Sep 2026 16:33:21 +0000 Subject: [PATCH 22/38] T050: compare each source's whole derived topic set; pin the fixture's shape Copilot at 7028f613 (r4135883800) is right. Comparing name words and direct naming misses a topic two sources share through a third document: if both name the same other document, both inherit its name words. The case now derives each source's whole topic set: the words of its name, plus the name words of every other fixture document it names, whatever that document's station. Then it compares whole sets. That catches a common name word, one source naming the other, and both naming the same third document. Only "the" and "and" are dropped, so the comparison is stricter than a topic rule with a longer stop list, never looser. The review's overview also asked that the intended source declaration and count be enforced. The module docstring states both, so the cases now hold them: - no document declares `stage: source`, because a source here is a document that declares nothing (R1Q13 (a) with (c)); - there are three sources, two of which share "rain barrel". Six plants each turn the cases red: - a shared name word; - the unrelated note naming a pair member; - a pair member naming the unrelated note; - both naming grouping-compost-corner.md (the review's scenario); - `stage: source` on the unrelated note; - a fourth source. 7028f613's cases pass the third-document plant and the fourth source. Still 20 cases, all passing, so the triple does not move. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/test_plain_documents_fixture.py | 93 ++++++++++++++++++--------- 1 file changed, 61 insertions(+), 32 deletions(-) diff --git a/tests/test_plain_documents_fixture.py b/tests/test_plain_documents_fixture.py index e615caf..e00446e 100644 --- a/tests/test_plain_documents_fixture.py +++ b/tests/test_plain_documents_fixture.py @@ -158,7 +158,12 @@ def test_required_fields_present(path: Path) -> None: def test_spread_across_the_six_stations() -> None: """One document per explicit station, and every undeclared document is - a source (R1Q13 (a) with (c)).""" + a source (R1Q13 (a) with (c)). + + No document declares `stage: source`. This fixture's sources are the + documents that declare nothing, which is the case R1Q13 (a) with (c) + names ("a document that declares nothing is a source"), so the fixture + exercises that reading and not a declared one.""" by_role: dict[str, list[Path]] = {role: [] for role in STAGE_ROLES} for path, (header, _) in _documents().items(): stage = header.get("stage") @@ -168,6 +173,10 @@ def test_spread_across_the_six_stations() -> None: assert stage in STAGE_ROLES, ( f"{path.name} declares stage {stage!r}, not one of {STAGE_ROLES}" ) + assert stage != "source", ( + f"{path.name} declares `stage: source`; a source in this fixture " + "declares nothing, so that it exercises that reading" + ) by_role[stage].append(path) empty = [role for role, paths in by_role.items() if not paths] @@ -201,7 +210,9 @@ def test_at_least_two_sources_share_a_topic() -> None: #: The two words this fixture's titles use that carry no topic under any #: reading of a name: an article and a conjunction. They are dropped before -#: two names are compared, and nothing else is. +#: two names are compared, and nothing else is. So the comparison is +#: stricter than a topic rule that drops more words: it may call a word a +#: shared topic where such a rule would not, but it never misses one. FUNCTION_WORDS = frozenset({"the", "and"}) _LETTER_RUN = re.compile(r"[^\W\d_]+") @@ -232,53 +243,71 @@ def _names(text: str, path: Path, header: dict[str, str]) -> bool: return False +def _derived_topics(path: Path, + documents: dict[Path, tuple[dict[str, str], str]]) -> set[str]: + """The topics a document that declares none is read to carry: the words + of its own name, and the name words of every other document in the + fixture that it names, whatever that document's station.""" + header, _ = documents[path] + topics = set(_name_words(header)) + text = path.read_text(encoding="utf-8") + for named, (named_header, _) in documents.items(): + if named != path and _names(text, named, named_header): + topics |= _name_words(named_header) + return topics + + def test_at_least_one_source_shares_no_topic() -> None: """At least one source shares no topic with the rain-barrel pair. Without it, a fixture where every source shared one topic would not exercise topic-based grouping at all. NO TOPIC, not just not the phrase. The phrase check alone would pass a - fixture edit that gave the sources some other topic in common, so this - compares the two things a topic that sources share can be derived from - when nobody declares one (R1Q13 (a) with (c)): - - the words of each document's name; - - the other documents each one names. - - So each rain-barrel note's name carries the pair's topic. A source - outside the pair shares no name word with either note, and it and the - notes never name each other, in either direction. Words elsewhere in - the body are not compared. That the projection then forms exactly this - group is proved over this fixture by T054's projection test. + fixture edit that gave the sources some other topic in common. So this + compares each source's whole derived topic set: what a topic that + sources share can come from when nobody declares one (R1Q13 (a) with + (c)). That set is: + - the words of the source's name; + - the name words of every other document it names. + Comparing whole sets catches every way two sources come to share a + topic: a common name word, one naming the other, or both naming the same + third document. + + THE SHAPE is the one the module docstring states: three sources, two of + which share "rain barrel". Each note in the pair carries the pair's + topic. The third source's topics meet neither note's. Words elsewhere in + a body are not topics and are not compared. That the projection then + forms exactly this group is proved over this fixture by T054's + projection test. """ - sources = {path: parts for path, parts in _documents().items() - if "stage" not in parts[0]} - pair = [path for path, (header, body) in sources.items() + documents = _documents() + sources = [path for path, (header, _) in documents.items() + if "stage" not in header] + pair = [path for path in sources if SHARED_SOURCE_TOPIC in " ".join( - (header.get("title", ""), header.get("summary", ""), body) + (documents[path][0].get("title", ""), + documents[path][0].get("summary", ""), + documents[path][1]) ).lower()] others = [path for path in sources if path not in pair] - assert others, "every source shares the same topic phrase" + assert (len(pair), len(others)) == (2, 1), ( + f"want two sources sharing {SHARED_SOURCE_TOPIC!r} and one outside " + f"them; the pair is {[p.name for p in pair]} and the rest " + f"{[p.name for p in others]}" + ) + derived = {path: _derived_topics(path, documents) for path in sources} topic = set(SHARED_SOURCE_TOPIC.split()) for member in pair: - assert topic <= _name_words(sources[member][0]), ( - f"{member.name}'s name does not carry {SHARED_SOURCE_TOPIC!r}") - + assert topic <= derived[member], ( + f"{member.name} does not carry {SHARED_SOURCE_TOPIC!r}: " + f"{sorted(derived[member])}") for other in others: - other_header = sources[other][0] for member in pair: - member_header = sources[member][0] - shared = _name_words(other_header) & _name_words(member_header) + shared = derived[other] & derived[member] assert not shared, ( - f"{other.name} and {member.name} share the name word(s) " + f"{other.name} and {member.name} share the topic(s) " f"{sorted(shared)}, so a topic rule could group them") - for writer, named, named_header in ( - (other, member, member_header), - (member, other, other_header)): - assert not _names(writer.read_text(encoding="utf-8"), - named, named_header), ( - f"{writer.name} names {named.name}, so its topics " - "could reach across the pair") if __name__ == "__main__": From 8e7da4a23c66590a2cba1d786b959af2ad39b28f Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Tue, 29 Sep 2026 17:01:42 +0000 Subject: [PATCH 23/38] T054: openDox's own scaffold carries the small neutral field set The holder's decision of 2026-09-28: render_scaffold()'s output read as missing title and summary under NEUTRAL_FIELDS. The H1 comes first, so leading_header() stops at the blank line after it, and its field is the capitalized `Summary:`. So standalone agent_capture refused a document `create` had written, and the projection lost the supplied summary (Copilot r4126022820, reproduced on #59). The scaffold now LEADS with the neutral fields the registered corpus obliges: - SCAFFOLD_LEAD_FIELDS = ("title", "summary"), filled from the create's own input. The title drops the family suffix, which stays on the H1. - render_scaffold(lead_fields=...) writes them first, as `name: value` lines, then a blank line, then the governed block unchanged. It refuses a field it has no value for. With lead_fields=() (the default) its output is byte-identical to before. - create_scaffold() passes scaffold_lead_fields(): the fields of that tuple that required_header_fields() answers. Why it follows the corpus rather than always leading: openxFactory's gated create (tests/ideation-dashboard/test_gate_routes.py) and openXdox's authoring suite (tests/test_authoring_agent.py) pin the governed layout with the H1 on the first line. openxFactory's adapter obliges only the governed block's own fields, which it reads over a 15-line window. So there the answer is (), and the created bytes do not change. It asks the corpus the way agent_capture does, so with no home registered create_scaffold now refuses, naming the seam (4.2). Every create path runs after an entry point registers one. The owed half of openDox-code#53's review is taken here too: T054's falsifier case now also asserts that notes-toolshed-inventory.md lands in no group. The pre-change case passes a projection that keeps its stopwords; this one does not. Tests, in tests/test_neutral_projection.py (25 -> 29 cases): - the scaffold under openDox's default leads with both fields, missing_required_headers() is empty, agent_capture accepts the text, and the projection keeps its title and summary; - under a corpus that obliges neither, the bytes equal render_scaffold()'s default, H1 first; - the lead block is SCAFFOLD_LEAD_FIELDS' order, drops the suffix, refuses an unknown field, and covers NEUTRAL_FIELDS; - with nothing registered, create_scaffold refuses and writes nothing. Evidence: - mutations 8/8 killed; - the whole suite as CI runs it: selected=2598 passed=2587 skipped=11; - F4.1's scan: 19 = 19. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/authoring.py | 91 ++++++++++++++++-- src/opendox/runtime/local_git_adapter.py | 8 +- tests/test_neutral_projection.py | 115 +++++++++++++++++++++++ 3 files changed, 204 insertions(+), 10 deletions(-) diff --git a/src/opendox/authoring.py b/src/opendox/authoring.py index f4a151e..ca0303b 100644 --- a/src/opendox/authoring.py +++ b/src/opendox/authoring.py @@ -9,7 +9,10 @@ family's ` — Brainstorm` suffix, `Status:`/`Kind:`/`Summary:`/`Topics:`/ `Repository context:`/`Captured:` pre-filled, plus a `## Possible feats` seed section per the openxFactory `ideation/README.md` "Ideation Header - Format") into the CHOSEN ideation area and writes it through + Format"). Where the registered corpus obliges openDox's small neutral + field set, the skeleton leads with a `title:`/`summary:` block above + the H1 (`scaffold_lead_fields`, plan 034 T054). It renders that skeleton + into the CHOSEN ideation area and writes it through `boundary.OutputBoundary.create_document` — create-only, so scaffolding over an existing path refuses as a SOURCE_EDIT (never silently overwritten). `edit_target`/`edit_command` back "select-to-edit": they @@ -105,6 +108,21 @@ # directory" while only the TRAVERSAL half was checked (PR #49 wave-2 critic). IDEATION_PREFIX = "ideation/" +#: THE FIELDS A SCAFFOLD CAN LEAD WITH (plan 034 T054; the holder's decision of +#: 2026-09-28): openDox's small neutral field set, `local_git_adapter +#: .NEUTRAL_FIELDS` (RULED R1Q13 (a)), each filled from the create's own input. +#: openDox's own default adapter reads a document's fields from its LEADING +#: `name: value` block (`local_git_adapter.leading_header`), which ends at the +#: first blank line. The governed block below the H1 carries neither name: its +#: `Summary:` is a different field, and it has no title field at all. So a +#: scaffold that must satisfy that adapter leads with them. +#: +#: They are written only where the registered corpus obliges them +#: (`scaffold_lead_fields`). Where it obliges neither, the governed layout is +#: unchanged: openxFactory's gated create and openXdox's authoring suite pin it +#: with the H1 on the first line. +SCAFFOLD_LEAD_FIELDS: tuple[str, ...] = ("title", "summary") + def _utcnow() -> str: return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") @@ -118,7 +136,7 @@ def render_scaffold( *, title: str, summary: str, topics: Sequence[str], repository_context: str, kind: str = DEFAULT_KIND, status: str = DEFAULT_STATUS, possible_feats: Sequence[str] = (), source: str | None = None, - now: str | None = None, + now: str | None = None, lead_fields: Sequence[str] = (), ) -> str: """Render a header-compliant skeleton for a NEW ideation document: the H1 with the family's ` — Brainstorm` suffix (so `grep '— Brainstorm$'` finds @@ -132,15 +150,40 @@ def render_scaffold( the keyword-lens recipe at a named `source_revision`). It is additive: the six required fields keep their declared order and their positions, so every existing header check still passes and a scaffold with no source is - byte-identical to what this function rendered before.""" + byte-identical to what this function rendered before. + + `lead_fields` (plan 034 T054) names the fields of `SCAFFOLD_LEAD_FIELDS` + that the scaffold LEADS with. + - Each is written first, as a `name: value` line, in that tuple's order, + and a blank line then separates the block from the H1. + - `title` is the title as given, without the family suffix. `summary` is + the summary as given. + - That block is where openDox's own default adapter reads its small + neutral field set, so a created document satisfies it. + `create_scaffold` passes `scaffold_lead_fields()`. + - A name outside `SCAFFOLD_LEAD_FIELDS` is refused (`ValueError`), since + the scaffold has no value for it. + - With none, which is the default, the scaffold is byte-identical to what + this function rendered before, H1 first.""" + unknown = [name for name in lead_fields if name not in SCAFFOLD_LEAD_FIELDS] + if unknown: + raise ValueError( + f"a scaffold can lead only with {SCAFFOLD_LEAD_FIELDS}, and it has " + f"no value for {unknown}") now = now or _utcnow() - heading = title if title.endswith(BRAINSTORM_SUFFIX) else f"{title}{BRAINSTORM_SUFFIX}" + suffixed = title.endswith(BRAINSTORM_SUFFIX) + heading = title if suffixed else f"{title}{BRAINSTORM_SUFFIX}" + lead_values = {"title": title[:-len(BRAINSTORM_SUFFIX)] if suffixed else title, + "summary": summary} + lead = "".join(f"{name}: {lead_values[name]}\n" + for name in SCAFFOLD_LEAD_FIELDS if name in lead_fields) topics_line = ", ".join(t.strip() for t in topics if t and t.strip()) feats = [f for f in possible_feats if f and f.strip()] or [DEFAULT_POSSIBLE_FEAT] feat_lines = "\n".join(f"- {f}" for f in feats) source_line = f"Source: {source.strip()}\n" if source and source.strip() else "" return ( - f"# {heading}\n\n" + (f"{lead}\n" if lead else "") + + f"# {heading}\n\n" f"Status: {status}\n" f"Kind: {kind}\n" f"Summary: {summary}\n" @@ -248,12 +291,25 @@ def create_scaffold( `staged` from an `ideation/staging/` area; the ruling REVERSED that and the `status_for_area` helper it needed is gone.) `source` records the optional provenance citation. Both are additive: omitting them reproduces the - previous behaviour exactly.""" + previous behaviour exactly. + + THE SCAFFOLD LEADS WITH WHAT THE REGISTERED CORPUS OBLIGES (plan 034 T054; + the holder's decision of 2026-09-28). The fields come from + `scaffold_lead_fields()`, and they are the small neutral field set, + `title` and `summary`, wherever the corpus obliges them, as openDox's own + default adapter does. Before this, openDox's own scaffold read there as + missing both: its H1 comes first, and its `Summary:` is capitalized. So + standalone `agent_capture` refused a document `create` had written, and + the neutral projection lost its summary. Under a corpus that obliges + neither, openxFactory's among them, the scaffold is byte-identical to what + this function wrote before. Asking the corpus means a bare process, with + no home registered, refuses here as `agent_capture` does.""" rel = scaffold_relpath(area, title) text = render_scaffold(title=title, summary=summary, topics=topics, repository_context=repository_context, kind=kind, status=status or DEFAULT_STATUS, - possible_feats=possible_feats, source=source, now=now) + possible_feats=possible_feats, source=source, now=now, + lead_fields=scaffold_lead_fields()) return boundary.create_document(rel, text) @@ -446,6 +502,27 @@ def required_header_fields() -> tuple[str, ...]: return _classify_proposal("").required_fields +def scaffold_lead_fields() -> tuple[str, ...]: + """The fields a scaffold for THIS corpus leads with: those of + `SCAFFOLD_LEAD_FIELDS` that the registered corpus obliges + (`required_header_fields()`), in `SCAFFOLD_LEAD_FIELDS`' order. + + - Under openDox's own default adapter, `WorkingTreeCorpus`, that is both. + So a document `create` writes carries its small neutral field set where + that adapter reads it. `agent_capture` then accepts it, and the neutral + projection copies its title and summary (plan 034 T054). + - Under an adapter that obliges neither, the answer is `()`, and the + scaffold keeps the governed layout, H1 first. openxFactory's adapter is + one such: its fields are the governed block's own `Status:`, `Kind:`, + `Summary:` and the rest. + + It asks the corpus the way `agent_capture` does. So with nothing + registered it refuses as that does (4.2), rather than guessing a layout. + """ + obliged = set(required_header_fields()) + return tuple(name for name in SCAFFOLD_LEAD_FIELDS if name in obliged) + + def missing_required_headers(text: str) -> list[str]: """Required ideation headers absent — OR value-empty — in `text`'s header window. diff --git a/src/opendox/runtime/local_git_adapter.py b/src/opendox/runtime/local_git_adapter.py index de37dd3..a2ad764 100644 --- a/src/opendox/runtime/local_git_adapter.py +++ b/src/opendox/runtime/local_git_adapter.py @@ -167,9 +167,11 @@ #: standalone default adapter, `WorkingTreeCorpus`, obliges of a document it #: recognizes, so `authoring.required_header_fields()` answers it. They are #: the `title` and `summary` of T053's neutral snapshot, which openDox's own -#: projection copies from the same header (`leading_header`). A document -#: without them is still listed and read, as a source: a missing field is -#: reported, never a reason to refuse or drop the document. +#: projection copies from the same header (`leading_header`). openDox's own +#: `create` writes them there wherever the corpus obliges them +#: (`authoring.scaffold_lead_fields`). A document without them is still +#: listed and read, as a source: a missing field is reported, never a reason +#: to refuse or drop the document. NEUTRAL_FIELDS: tuple[str, ...] = ("title", "summary") #: The corpus's ONE verdict of its own, and it is a fact about git rather than diff --git a/tests/test_neutral_projection.py b/tests/test_neutral_projection.py index fb74cc7..16bf854 100644 --- a/tests/test_neutral_projection.py +++ b/tests/test_neutral_projection.py @@ -40,6 +40,10 @@ 5. THE FIELD SET: openDox's default adapter obliges `title` and `summary` (R1Q13 (a)), so `authoring.required_header_fields()` answers them through the entry point, and a document without them is still read. + (5a) THE SCAFFOLD. Following the holder's decision, what openDox's own + `create` writes carries that field set where the adapter reads it, so + `agent_capture` accepts it and the projection keeps its title and summary. + Where the corpus obliges neither field, the governed layout is unchanged. 6. THE VALUES: `SNAPSHOT_VALUES`' defaults, in Python and in `display.js`, are the neutral schema's values, so openDox's views match openDox's snapshot. And the wheel's grouping tile counts a group's edges where the group carries @@ -79,6 +83,7 @@ ) from opendox import generator_seam as gs from opendox import neutral_projection as projection +from opendox.boundary import HUMAN, OutputBoundary from opendox.runtime import local_git_adapter as lga ROOT = Path(__file__).resolve().parents[1] @@ -567,6 +572,15 @@ def test_the_projection_over_the_plain_documents_fixture_is_a_neutral_snapshot( assert derived, snapshot["clusters"] assert any(set(c["topics"]) >= {"rain", "barrel"} for c in derived), derived + # AND THE SOURCE OUTSIDE THAT PAIR LANDS IN NO GROUP. T050's own fixture + # test pins what the topics are derived from, the names and what each + # document names (openDox-code#53, Copilot r4135803935 and r4135883800). + # This pins the projection's own reading of them. + grouped = {e["document"] for c in snapshot["clusters"] + for e in c["document_edges"]} + assert "notes-toolshed-inventory.md" in sources + assert "notes-toolshed-inventory.md" not in grouped, snapshot["clusters"] + # --------------------------------------------------------------------------- # the falsifier, part 2: the topic rule over repository (b), no front matter @@ -868,6 +882,107 @@ def test_the_header_reader_keeps_an_empty_value_and_stops_at_a_blank_line() -> N "a": "2", "b": ""} +# --------------------------------------------------------------------------- +# 5a — openDox's own scaffold carries the small neutral field set +# --------------------------------------------------------------------------- + +#: One create's input, with its date pinned, so two scaffolds compare as bytes. +_CREATE: dict[str, Any] = { + "title": "Shed roof", "summary": "Where the water gets in after a storm.", + "topics": ["shed"], "repository_context": "garden", + "now": "2026-09-29T12:00:00Z"} + + +def test_openDoxs_own_scaffold_carries_the_small_neutral_field_set( + tmp_path: Path) -> None: + """The holder's decision of 2026-09-28 on this PR: what `create` writes + must satisfy `NEUTRAL_FIELDS` under openDox's own default adapter. + + It read there as missing both fields, because its H1 comes first and its + `Summary:` is capitalized. So standalone `agent_capture` refused what + `create` had written, and the projection lost the supplied summary + (Copilot r4126022820, reproduced on openDox-code#59). The scaffold now + leads with `title:` and `summary:`, and its governed block follows it + unchanged.""" + root = _repository(tmp_path, files={"notes-first.md": "title: F\nsummary: S\n"}) + assert authoring.scaffold_lead_fields() == lga.NEUTRAL_FIELDS + written = authoring.create_scaffold(OutputBoundary(root, actor=HUMAN), + **_CREATE) + text = written.read_text(encoding="utf-8") + + assert lga.leading_header(text) == {"title": "Shed roof", + "summary": _CREATE["summary"]} + assert authoring.missing_required_headers(text) == [] + assert text == (f"title: Shed roof\nsummary: {_CREATE['summary']}\n\n" + + authoring.render_scaffold(**_CREATE)) + + # AN AGENT'S CAPTURE OF THE SAME TEXT is not refused as header-incomplete. + captured = authoring.agent_capture( + authoring.agent_boundary(root), + path="ideation/brainstorm/shed-roof-again.md", text=text) + assert captured.read_text(encoding="utf-8") == text + + # AND THE PROJECTION KEEPS the title and summary the create was given. + documents = _by_path(_generate(root)) + for path in (written, captured): + document = documents[path.relative_to(root).as_posix()] + assert (document["title"], document["summary"]) == ( + "Shed roof", _CREATE["summary"]) + + +def test_a_scaffold_keeps_the_governed_layout_where_the_corpus_obliges_neither( + tmp_path: Path) -> None: + """Under a corpus that obliges neither neutral field, the scaffold is + byte-identical to `render_scaffold`'s default, H1 first. openxFactory's + adapter is one such, since its fields are the governed block's own. + openxFactory's gated create (`tests/ideation-dashboard/test_gate_routes.py`) + and openXdox's authoring suite (`tests/test_authoring_agent.py`) pin that + layout.""" + governed = ("Status", "Kind", "Summary", "Topics", "Repository context", + "Captured") + corpus_adapter.register_home(lambda root: ( + lga.WorkingTreeCorpus(required_fields=governed), + corpus_adapter.CorpusRef(name="home", location=str(root)))) + assert authoring.required_header_fields() == governed + assert authoring.scaffold_lead_fields() == () + root = _repository(tmp_path, files={"notes-first.md": "title: F\nsummary: S\n"}) + written = authoring.create_scaffold(OutputBoundary(root, actor=HUMAN), + **_CREATE) + text = written.read_text(encoding="utf-8") + assert text == authoring.render_scaffold(**_CREATE) + assert text.startswith("# Shed roof — Brainstorm\n") + + +def test_the_lead_block_is_the_neutral_fields_in_order_then_a_blank_line() -> None: + """The lead is `SCAFFOLD_LEAD_FIELDS`' order whatever order it is asked + in. The neutral title is the title without the family suffix, which + stays on the H1. A field the scaffold has no value for is refused. And + every field openDox's own default obliges is one a scaffold can lead + with, so a created document always satisfies it.""" + suffixed = {**_CREATE, "title": "Shed roof — Brainstorm"} + plain = authoring.render_scaffold(**suffixed) + assert plain.startswith("# Shed roof — Brainstorm\n") + assert authoring.render_scaffold(**suffixed, lead_fields=("summary", "title")) == ( + f"title: Shed roof\nsummary: {_CREATE['summary']}\n\n" + plain) + assert authoring.render_scaffold(**suffixed, lead_fields=("summary",)) == ( + f"summary: {_CREATE['summary']}\n\n" + plain) + with pytest.raises(ValueError, match=r"no value for \['author'\]"): + authoring.render_scaffold(**suffixed, lead_fields=("title", "author")) + assert set(lga.NEUTRAL_FIELDS) <= set(authoring.SCAFFOLD_LEAD_FIELDS) + + +def test_a_scaffold_with_no_corpus_registered_refuses_and_writes_nothing( + tmp_path: Path) -> None: + """`create_scaffold` asks the corpus which fields to lead with, as + `agent_capture` asks it which to require. So with no home registered it + refuses as that does (4.2), naming the seam, and it writes nothing.""" + corpus_adapter._home_factory = corpus_adapter._UNSET + with pytest.raises(corpus_adapter.CorpusRefused) as refused: + authoring.create_scaffold(OutputBoundary(tmp_path, actor=HUMAN), **_CREATE) + assert refused.value.refusal.kind == corpus_adapter.ADAPTER_NOT_REGISTERED + assert list(tmp_path.rglob("*")) == [] + + # --------------------------------------------------------------------------- # 6 — the product's views match the product's snapshot # --------------------------------------------------------------------------- From d7aa9d8cdb32c511646129f6b4d9a06a196dd30d Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Tue, 29 Sep 2026 17:25:13 +0000 Subject: [PATCH 24/38] T055: an empty --project-register or --possibles is refused, fail closed The holder's decision of 2026-09-28. The generate verbs tested each source option for truth, which predates T055, so an explicitly empty value was DROPPED: it was not passed, and the run went on as if the option had never been given. Testing it against None alone would make it Path("").resolve(), the current directory. The decision is neither: refuse. - cli.SourceOptionRefused, raised by one helper, _source_option(args, attr, flag). It answers None for an option not given, refuses an empty one, and resolves anything else. Both call sites use it: _generate_and_write (generate, generate-and-open) and _gate_snapshot (the gate verbs). - The guard runs beside the other two, before any generation or write. In generate-and-open it runs early too, so a refused run mints no run directory. main() reports it as " refused: ..." with exit 1. - serve.main refuses an empty --project-register the same way, before a socket is bound. SnapshotSource tests the value for truth, and would otherwise drop it. #58 (T057) is still not merged here. The validator stand-in stays until T058 wires opendox.validator. Tests, in tests/test_projection_seams.py (117 -> 124 cases): - both flags, through `generate`: refused before the generator is called, and nothing is written. The generator declares both inputs, so the refusal is the option's own; - both options, through _gate_snapshot; - a given option is resolved, and an unset one is passed as None; - generate-and-open: no run directory is minted; - serve: an empty --project-register is refused, and a given one is handed over as given. Mutations: 8 of 9 killed. The ninth removes the explicit guard in _generate_and_write, and it is equivalent: the argument expression refuses before generate() is called. The guard is kept beside the other two, so the order does not rest on argument evaluation. The whole suite as CI runs it: selected=2723 passed=2712 skipped=11. That is +11: #57's 4 scaffold cases, merged in, and these 7. F4.1's scan: 11 = 11. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/cli.py | 54 ++++++++++++++++++-- src/opendox/serve.py | 11 ++++ tests/test_projection_seams.py | 91 ++++++++++++++++++++++++++++++++++ 3 files changed, 151 insertions(+), 5 deletions(-) diff --git a/src/opendox/cli.py b/src/opendox/cli.py index 09a76e4..e7b7bad 100644 --- a/src/opendox/cli.py +++ b/src/opendox/cli.py @@ -201,6 +201,40 @@ class GeneratedAtRefused(Exception): saying why.""" +class SourceOptionRefused(Exception): + """`--project-register` or `--possibles` was given an EMPTY path. + + REFUSED, fail closed, on the holder's decision of 2026-09-28 (plan 034 + T055). This used to be decided by testing the value for truth, which + DROPPED an empty one: the option was silently not passed, and the run went + on as if it had never been given. Resolving it instead, as + `Path("").resolve()`, would name the CURRENT DIRECTORY as the register to + read. Neither is what a caller who typed the option asked for, so an empty + value ends the run, before anything is generated or written. An option + that is not given at all is `None` and is still simply not passed.""" + + +def _source_option(args: argparse.Namespace, attr: str, flag: str) -> Path | None: + """The file a `--project-register`/`--possibles` option names, resolved; + `None` when the option was not given; `SourceOptionRefused` when it was + given an empty path (see that class).""" + value = getattr(args, attr, None) + if value is None: + return None + if value == "": + raise SourceOptionRefused( + f"{flag} was given an empty path. It is refused: dropping it would " + f"ignore the option without a word, and resolving it would read " + f"the current directory. Name the file to read, or leave {flag} out") + return Path(value).resolve() + + +def _refuse_empty_source_options(args: argparse.Namespace) -> None: + """Raise `SourceOptionRefused` if either source option is an empty path.""" + _source_option(args, "project_register", "--project-register") + _source_option(args, "possibles", "--possibles") + + def _refuse_malformed_generated_at(args: argparse.Namespace) -> None: """Raise `GeneratedAtRefused` unless `--generated-at`, when given, is an RFC 3339 date-time (`opendox.rfc3339.is_rfc3339_datetime`, the neutral @@ -238,19 +272,22 @@ def _generate_and_write(args: argparse.Namespace, output: Path) -> tuple[dict, P (`generator_seam.generate`, which looks it up on each call), and it is written by the registered writer. An option given as `None` is not passed, so an unset `--project-register` or `--possibles` asks nothing of a - generator that declares no such input. A given one that the registered + generator that declares no such input. An EMPTY one is refused as + `SourceOptionRefused`, never dropped and never read as the current + directory (the holder, 2026-09-28). A given one that the registered generator does not declare is refused as `GeneratorInputRefused`, before anything is generated or written, and `main` reports it.""" _refuse_non_corpus_repo_root(args) _refuse_malformed_generated_at(args) + _refuse_empty_source_options(args) repo_root = Path(args.repo_root).resolve() snapshot = generator_seam.generate( repo_root, args.repository, source_revision=args.source_revision, generated_at=args.generated_at, - project_register_source=Path(args.project_register).resolve() if args.project_register else None, - possibles_source=Path(args.possibles).resolve() if args.possibles else None, + project_register_source=_source_option(args, "project_register", "--project-register"), + possibles_source=_source_option(args, "possibles", "--possibles"), ) boundary = OutputBoundary(output.parent, [output.name]) written = projection_seams.writer.current().write_snapshot( @@ -480,6 +517,7 @@ def cmd_generate_and_open(args: argparse.Namespace, *, opener=webbrowser.open) - # (it is the one no caller can skip); these are the same checks, earlier. _refuse_non_corpus_repo_root(args) _refuse_malformed_generated_at(args) + _refuse_empty_source_options(args) run_dir = Path(args.run_dir).resolve() if args.run_dir else Path( tempfile.mkdtemp(prefix="ideation-dashboard-")) run_dir.mkdir(parents=True, exist_ok=True) @@ -613,8 +651,8 @@ def _gate_snapshot(args: argparse.Namespace) -> tuple[Path, dict]: repo_root = Path(args.repo_root).resolve() snapshot = generator_seam.generate( repo_root, args.repository, source_revision=args.source_revision, - project_register_source=Path(args.project_register).resolve() if args.project_register else None, - possibles_source=Path(args.possibles).resolve() if args.possibles else None) + project_register_source=_source_option(args, "project_register", "--project-register"), + possibles_source=_source_option(args, "possibles", "--possibles")) return repo_root, snapshot @@ -1146,6 +1184,12 @@ def main(argv: list[str] | None = None, *, # stderr, rather than degrading to a stamp that is quietly absent. print(str(exc), file=sys.stderr) return 1 + except SourceOptionRefused as exc: + # An EMPTY `--project-register`/`--possibles`, refused before + # anything is generated or written (the holder, 2026-09-28): the + # option was typed, so it is neither dropped nor read as `.`. + print(f"{_command_label(args)} refused: {exc}", file=sys.stderr) + return 1 except RepoRootRefused as exc: # The refusal is the whole message (the registered corpus-root # predicate's `corpus_root_refusal`); stderr and a non-zero status, so diff --git a/src/opendox/serve.py b/src/opendox/serve.py index eb9ae71..3100291 100644 --- a/src/opendox/serve.py +++ b/src/opendox/serve.py @@ -2364,6 +2364,17 @@ def main(argv: list[str] | None = None) -> int: help="project-register instance passed to the generator on a " "local regenerate (grouping resolution)") args = parser.parse_args(argv) + if args.project_register == "": + # AN EMPTY `--project-register` IS REFUSED, fail closed (the holder, + # 2026-09-28; the generate verbs refuse it the same way, as + # `cli.SourceOptionRefused`). The snapshot source tests the value for + # truth, so an empty one would be dropped without a word, and + # `Path("")` would name the current directory. Neither is what a + # caller who typed the option asked for. + print("serve refused: --project-register was given an empty path. " + "Name the file to read, or leave --project-register out", + file=sys.stderr) + return 1 rc = _refuse_impossible_checkout_root(args.checkout_root) if rc: return rc diff --git a/tests/test_projection_seams.py b/tests/test_projection_seams.py index ce5cfa8..6bdfde5 100644 --- a/tests/test_projection_seams.py +++ b/tests/test_projection_seams.py @@ -674,6 +674,22 @@ def test_serve_main_refuses_a_data_source_openDoxs_registry_cannot_read( assert "serve refused:" in err and named in err, (flags, err) +def test_serve_main_refuses_an_empty_project_register(tmp_path, capsys, monkeypatch) -> None: + """The holder's decision of 2026-09-28, at `serve`'s own option: an EMPTY + `--project-register` is refused before a socket is bound, and never + dropped, as `SnapshotSource` would drop it by testing it for truth. A + given one is still handed over as given.""" + reached: list = [] + monkeypatch.setattr(serve, "serve", lambda *a, **k: reached.append(k)) + repo = _repository(tmp_path) + base = ["--snapshot", str(tmp_path / "s.json"), "--checkout-root", str(repo)] + assert serve.main([*base, "--project-register", ""]) == 1 and not reached + assert ("serve refused: --project-register was given an empty path" + in capsys.readouterr().err) + assert serve.main([*base, "--project-register", "register.yaml"]) == 0 + assert reached and reached[0]["project_register"] == "register.yaml" + + def test_dropping_the_active_entry_clears_the_active_key(tmp_path) -> None: """No ref-less request meets a key with nothing behind it, and the next entry registered becomes active, as the first one did.""" @@ -1070,6 +1086,81 @@ def test_an_input_the_generator_does_not_declare_is_refused_before_a_write(tmp_p assert not out.exists() +def _declaring_generator(calls: list) -> None: + """Register a host generator that DECLARES both source inputs, so that a + refusal of an empty option is the option's own, not the seam's + `GeneratorInputRefused` for an input the generator does not take.""" + def operation(repo_root, repository, *, source_revision=None, generated_at=None, + project_register_source=None, possibles_source=None): + calls.append((project_register_source, possibles_source)) + return {"schema_version": 1, "kind": "host-snapshot", + "repository": repository, "generation": {"source_revision": "s"}, + "documents": []} + + gs.register(gs.SnapshotGenerator( + contract="host-snapshot", generate=operation, + inputs=("project_register_source", "possibles_source"))) + + +@pytest.mark.parametrize("flag", ["--project-register", "--possibles"]) +def test_an_empty_source_option_is_refused_before_anything_is_generated( + flag, tmp_path, capsys) -> None: + """The holder's decision of 2026-09-28: an EMPTY `--project-register` or + `--possibles` is refused, fail closed. It is not dropped, as testing the + value for truth used to drop it. And it is not `Path("").resolve()`, which + is the current directory. The generator declares the input, so nothing + else refuses it.""" + calls: list = [] + _declaring_generator(calls) + repo = _repository(tmp_path) + out = tmp_path / "out" / "snapshot.json" + assert _generate(repo, out, flag, "") == 1 + err = capsys.readouterr().err + assert f"generate refused: {flag} was given an empty path" in err, err + assert calls == [] and not out.exists() + + +@pytest.mark.parametrize("attr, flag", [("project_register", "--project-register"), + ("possibles", "--possibles")]) +def test_the_gate_snapshot_refuses_an_empty_source_option(attr, flag, tmp_path) -> None: + calls: list = [] + _declaring_generator(calls) + args = argparse.Namespace(repo_root=str(tmp_path), repository="garden", + source_revision="abc", project_register=None, + possibles=None) + setattr(args, attr, "") + with pytest.raises(cli.SourceOptionRefused, match=f"{flag} was given an empty path"): + cli._gate_snapshot(args) + assert calls == [] + + +def test_a_given_source_option_is_resolved_and_an_unset_one_is_not_passed( + tmp_path, monkeypatch) -> None: + calls: list = [] + _declaring_generator(calls) + monkeypatch.chdir(tmp_path) + args = argparse.Namespace(repo_root=str(tmp_path), repository="garden", + source_revision="abc", project_register="register.yaml", + possibles=None) + cli._gate_snapshot(args) + assert calls == [(tmp_path.resolve() / "register.yaml", None)] + + +def test_generate_and_open_refuses_an_empty_source_option_before_its_run_dir( + tmp_path, capsys) -> None: + calls: list = [] + _declaring_generator(calls) + repo = _repository(tmp_path) + run_dir = tmp_path / "run" + rc = cli.main(["generate-and-open", "--repo-root", str(repo), "--repository", + "garden", "--run-dir", str(run_dir), "--no-open", "--no-serve", + "--possibles", ""]) + assert rc == 1 + assert ("generate-and-open refused: --possibles was given an empty path" + in capsys.readouterr().err) + assert calls == [] and not run_dir.exists() + + def test_a_root_openDoxs_predicate_refuses_is_refused_with_its_message(tmp_path, capsys) -> None: (tmp_path / "plain").mkdir() assert _generate(tmp_path / "plain", tmp_path / "out.json") == 1 From 687d37bf5fa150e6508e6505cf3b3945741c63b5 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Tue, 29 Sep 2026 17:38:32 +0000 Subject: [PATCH 25/38] T055: a refresh keeps the snapshot's permissions Copilot at d7aa9d8c (r4136439187) is right. The atomic writer (f4ef63d3) creates its temporary sibling with a new file's ordinary mode, and os.replace carries the sibling's mode over the target. So a snapshot kept restricted (0600, say) was widened by every refresh, to 0644 under a usual umask. Where the target exists, its permission bits now go onto the sibling (os.fchmod) before a byte is written. A new target keeps the ordinary mode, as before. If the bits cannot be copied, the write fails: - the stream closes the descriptor, and the existing handler removes the sibling; - the old snapshot stands, with its mode; - nothing is silently widened. The copy runs inside the stream's `with`, so a failure there cannot leak the descriptor. Tests, in tests/test_projection_seams.py (124 -> 128 cases): - test_a_refresh_keeps_the_snapshots_permissions, over 0600, 0640 and 0444: a new snapshot has an ordinary file's mode, and a refresh keeps the mode it was given; - test_a_mode_that_cannot_be_kept_fails_the_write_and_widens_nothing: fchmod raises, the write fails, and the old snapshot and its 0600 stay, with no sibling left. All four fail against f4ef63d3's writer. The whole suite: selected=2727 passed=2716 skipped=11, which is +4. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/default_projection.py | 21 +++++++++++++--- tests/test_projection_seams.py | 40 +++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+), 3 deletions(-) diff --git a/src/opendox/default_projection.py b/src/opendox/default_projection.py index 18f31d9..f687437 100644 --- a/src/opendox/default_projection.py +++ b/src/opendox/default_projection.py @@ -53,6 +53,7 @@ import contextlib import json import os +import stat import uuid from pathlib import Path from typing import Any @@ -186,9 +187,11 @@ def write_snapshot(self, snapshot: dict[str, Any], path: Path | str, check `write_output` makes, root and allowlist, with its refusal and its ledger, and it runs first, so a refused target leaves nothing behind. The sibling is created exclusively beside the permitted - target, with the mode an ordinary write would give it. Its name is a - dot-file with no document extension, so `/source` never serves it, - and it is removed if the write or the move fails.""" + target, with the mode an ordinary write would give it, or with the + target's own permission bits where the target exists, so a refresh + never widens a restricted snapshot. Its name is a dot-file with no + document extension, so `/source` never serves it, and it is removed if + the write or the move fails.""" data = self.canonical_json(snapshot).encode("ascii") target = boundary.permit_output(path) target.parent.mkdir(parents=True, exist_ok=True) @@ -196,6 +199,18 @@ def write_snapshot(self, snapshot: dict[str, Any], path: Path | str, descriptor = os.open(temporary, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o666) try: with os.fdopen(descriptor, "wb") as stream: + # THE SNAPSHOT KEEPS ITS PERMISSIONS (Copilot at + # openDox-code#59 d7aa9d8c, r4136439187). `os.replace` carries + # the SIBLING's mode over the target, and the sibling is + # created with a new file's ordinary mode, so a snapshot kept + # restricted (0600, say) was widened by every refresh. Where + # the target exists, its permission bits go onto the sibling + # before a byte is written. A new target keeps the ordinary + # mode. If they cannot be copied, the write fails: the stream + # closes the descriptor, the sibling is removed below, and the + # snapshot is never silently widened. + with contextlib.suppress(FileNotFoundError): + os.fchmod(stream.fileno(), stat.S_IMODE(os.stat(target).st_mode)) stream.write(data) stream.flush() os.fsync(stream.fileno()) diff --git a/tests/test_projection_seams.py b/tests/test_projection_seams.py index 6bdfde5..bdea1f1 100644 --- a/tests/test_projection_seams.py +++ b/tests/test_projection_seams.py @@ -42,6 +42,7 @@ import json import os import shutil +import stat import subprocess import sys import textwrap @@ -1011,6 +1012,45 @@ def the_move_fails(source, destination): assert sorted(p.name for p in tmp_path.iterdir()) == ["snapshot.json"] +@pytest.mark.parametrize("mode", [0o600, 0o640, 0o444]) +def test_a_refresh_keeps_the_snapshots_permissions(tmp_path, mode) -> None: + """Copilot at d7aa9d8c (r4136439187). `os.replace` carries the sibling's + mode over the target, and the sibling had a new file's ordinary mode, so + a snapshot kept restricted was widened by every refresh. Its permission + bits now carry over. A new snapshot gets an ordinary new file's mode.""" + writer = default_projection.WRITER + target = tmp_path / "snapshot.json" + boundary = OutputBoundary(tmp_path, ["snapshot.json"]) + writer.write_snapshot({"kind": NEUTRAL, "n": 1}, target, boundary) + ordinary = tmp_path / "ordinary" + ordinary.write_bytes(b"") + assert (stat.S_IMODE(os.stat(target).st_mode) + == stat.S_IMODE(os.stat(ordinary).st_mode)), "a new snapshot is ordinary" + os.chmod(target, mode) + writer.write_snapshot({"kind": NEUTRAL, "n": 2}, target, boundary) + assert stat.S_IMODE(os.stat(target).st_mode) == mode + assert json.loads(target.read_text(encoding="utf-8")) == {"kind": NEUTRAL, "n": 2} + + +def test_a_mode_that_cannot_be_kept_fails_the_write_and_widens_nothing( + tmp_path, monkeypatch) -> None: + writer = default_projection.WRITER + target = tmp_path / "snapshot.json" + boundary = OutputBoundary(tmp_path, ["snapshot.json"]) + writer.write_snapshot({"kind": NEUTRAL, "n": 1}, target, boundary) + os.chmod(target, 0o600) + + def the_mode_cannot_be_set(descriptor, mode): + raise OSError("fchmod is not permitted here") + + monkeypatch.setattr(default_projection.os, "fchmod", the_mode_cannot_be_set) + with pytest.raises(OSError, match="fchmod is not permitted here"): + writer.write_snapshot({"kind": NEUTRAL, "n": 2}, target, boundary) + assert json.loads(target.read_text(encoding="utf-8")) == {"kind": NEUTRAL, "n": 1} + assert stat.S_IMODE(os.stat(target).st_mode) == 0o600 + assert sorted(p.name for p in tmp_path.iterdir()) == ["snapshot.json"] + + @pytest.mark.parametrize("value", [float("nan"), float("inf"), float("-inf"), Path("not-json"), {"a": {1, 2}}]) def test_the_writer_refuses_what_json_cannot_carry(tmp_path, value) -> None: From 96f18c45978c88359deee714f67b4c34088e53ae Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Tue, 29 Sep 2026 18:12:39 +0000 Subject: [PATCH 26/38] T055: the source arm's two lookups are made under the registry's lock Copilot at 687d37bf (r4136585695) is right. The arm resolves its entry once and then asks resolve_source for the path by that entry's pair, and resolve_source looks the pair up again. A refresh that re-registered the same key between the two lookups put another entry, with another root, behind the path. The arm then served the replacement's file under the first entry's refusal and headers. Both lookups now run under the registry's own lock, registry.atomically(). That is the lock register and drop take, in openDox's registry and in openXdox's alike. So no refresh can land between the two lookups. - The path still comes through resolve_source. That is the one entry point to the containment rule, which tests/test_source_core_arm.py records: the arm never calls resolve_within itself. - Nothing is sent while the lock is held, so a slow client cannot hold a refresh up. The refusal is decided on the entry in hand after the lock is released, and a refused entry's path is never read. Tests: - tests/test_projection_seams.py (128 -> 130 cases): test_a_refresh_cannot_land_between_the_source_arms_two_lookups, keyed and unkeyed. A refresh on another thread re-registers garden@main with another root right after the first lookup. The body must be the first entry's file, and the refresh lands once the read is done. - Both cases fail against 687d37bf's arm. - Both also fail with the hold removed and the new order kept, so it is the lock that closes the race. - tests/test_source_core_arm.py: the single-entry-point test also pins both lookups inside the hold. It fails with the hold removed. Copilot's other thread at 687d37bf (r4136585637, default_registry.py:561) does not reproduce. _register_baked sets baked_repository (line 560) before it asks _source_root_for (line 561), and has done since c27eac35. Two existing tests pin that order, and both fail when the two lines are swapped: - test_the_source_registers_the_snapshot_it_was_handed: None == checkout; - test_generate_and_serve_run_with_no_sibling_importable: 404 == 200. The whole suite: selected=2729 passed=2718 skipped=11, which is +2. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/serve.py | 21 ++++++++++--- tests/test_projection_seams.py | 56 ++++++++++++++++++++++++++++++++++ tests/test_source_core_arm.py | 7 +++++ 3 files changed, 80 insertions(+), 4 deletions(-) diff --git a/src/opendox/serve.py b/src/opendox/serve.py index 3100291..8ffb61c 100644 --- a/src/opendox/serve.py +++ b/src/opendox/serve.py @@ -1301,12 +1301,25 @@ def _serve_source(self, tail: str, head_only: bool) -> None: # refusal and headers (FR-048). So the entry is resolved once, and # the path is asked for by THAT entry's own pair, which is the pair # the refusal checked, never "the active entry" a second time. - entry = self.source.registry.resolve(repository, ref) + # + # AND BOTH LOOKUPS ARE MADE UNDER THE REGISTRY'S OWN LOCK (Copilot + # at openDox-code#59 687d37bf, r4136585695). `resolve_source` + # looks that pair up again, so a refresh that re-registered the + # key in between put another entry, with another root, behind the + # path. `atomically()` holds the lock that `register` and `drop` + # take, in openDox's registry and in a host's, so no refresh lands + # between the two. The path still comes through `resolve_source`, + # the one entry point to the containment rule. Nothing is sent + # while the lock is held, so a slow client cannot hold a refresh + # up: the refusal is decided on the entry in hand afterwards, and + # a refused entry's path is never read. + with self.source.registry.atomically(): + entry = self.source.registry.resolve(repository, ref) + target = (None if entry is None else + self.source.registry.resolve_source( + entry.repository, entry.ref, rest)) if self._hosted_entry_refused(entry): return - target = (None if entry is None else - self.source.registry.resolve_source( - entry.repository, entry.ref, rest)) else: target = resolve_source_path(Path(self.checkout_root), rest) if target is None: diff --git a/tests/test_projection_seams.py b/tests/test_projection_seams.py index bdea1f1..d9948cb 100644 --- a/tests/test_projection_seams.py +++ b/tests/test_projection_seams.py @@ -38,6 +38,7 @@ import argparse import ast +import dataclasses import http.client import json import os @@ -912,6 +913,61 @@ def resolve_source(self, repository, ref, tail): assert status == 403 and json.loads(body)["error"] == "session_unavailable" +@pytest.mark.parametrize("query", ["", "?repository=garden&ref=main"], + ids=["unkeyed", "keyed"]) +def test_a_refresh_cannot_land_between_the_source_arms_two_lookups( + tmp_path, query) -> None: + """FR-048 on `/source/`, against a refresh that REPLACES the key. The + arm resolves the entry once and asks for the path by that entry's pair, + and `resolve_source` looks the pair up again. Here a refresh on another + thread re-registers `garden@main`, with another root, right after the + first lookup. Unheld, it landed between the two, and the path came from + the replacement's root under the first entry's refusal and headers + (Copilot at openDox-code#59 687d37bf, r4136585695). Made under the + registry's own lock, the two lookups see one entry, and the refresh + lands after the read.""" + repo, _out, httpd = _served(tmp_path) + bound = httpd.RequestHandlerClass.func + first = bound.source.registry.active + elsewhere = tmp_path / "elsewhere" + elsewhere.mkdir() + (elsewhere / "notes-toolshed-inventory.md").write_text( + "THE REPLACEMENT'S BYTES\n", encoding="utf-8") + replacement = dataclasses.replace(first, source_root=elsewhere, + source_revision="replaced") + attempting = threading.Event() + + def refresh() -> None: + attempting.set() + registry.register(replacement) + + refresher = threading.Thread(target=refresh, daemon=True) + + class _Registry(default_registry.SnapshotRegistry): + def resolve(self, repository, ref=None): + entry = super().resolve(repository, ref) + if entry is first and refresher.ident is None: + refresher.start() + attempting.wait(timeout=10) + # Time to land, where nothing holds the registry. A held + # lock keeps it waiting, and this wait simply runs out. + refresher.join(timeout=0.5) + return entry + + registry = _Registry() + registry.register(first) + bound.source.registry = registry + status, _headers, body = _get( + httpd, f"/source/notes-toolshed-inventory.md{query}") + refresher.join(timeout=10) + assert registry.get(*first.key) is replacement, "the refresh landed after" + assert b"THE REPLACEMENT'S BYTES" not in body, ( + "a refresh landed between the arm's two lookups, and the path came " + "from the replacement's root") + assert status == 200 + assert body == (repo / "notes-toolshed-inventory.md").read_bytes() + + def test_hosted_ref_refused_asks_the_registered_registry() -> None: ps.register_defaults() assert serve.hosted_ref_refused(True, "draft/t") is False diff --git a/tests/test_source_core_arm.py b/tests/test_source_core_arm.py index 486d6aa..f19e8fe 100644 --- a/tests/test_source_core_arm.py +++ b/tests/test_source_core_arm.py @@ -315,6 +315,13 @@ def test_the_arm_resolves_through_that_entry_point_and_the_registry_only(): "ONE resolution of the entry, whose own pair the path is then asked " "for (plan 034 T055, FR-048): resolved a second time, the active " "entry could change between the refusal and the path") + held = body.split("with self.source.registry.atomically():", 1) + assert len(held) == 2 and "self.source.registry.resolve(" in held[1] \ + and "self.source.registry.resolve_source(" in held[1], ( + "both lookups under the registry's own lock (openDox-code#59, " + "r4136585695): resolve_source looks the pair up again, and a " + "refresh that re-registered the key in between put another root " + "behind the path") assert "resolve_within(" not in body, ( "the arm must not call the containment rule directly: one entry point, " "so a change to the rule cannot reach the route by one path and miss " From e3ef506a8036c1360d88ff75cab50f8b8e5d5fca Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Tue, 29 Sep 2026 18:35:13 +0000 Subject: [PATCH 27/38] T055: the arm confines its entry in hand, reads take the lock, the report reads any shape Copilot at 96f18c45 raised three threads, and all three are taken. r4136863569 (serve.py): the seam does not declare atomically(), and 96f18c45's arm called it on whatever registry is registered. So a registry that passes the seam's probe could fail every /source request. - The arm now asks nothing of the registry beyond the one resolution. It confines the path to the root of the entry it resolved, through resolve_source_path. That is its own single entry point, and the one the no-registry path already uses. It applies the registry's declared per-entry containment rule, resolve_within, as serve_workbench.py does for the roots it holds. - There is no second lookup, so a refresh that re-registers the key cannot put another root behind the path (r4136585695), and no lock is needed for that. - tests/test_source_core_arm.py now pins that shape: one resolve, the entry's own root through resolve_source_path, and no resolve_source or atomically in the arm. It fails against 687d37bf's arm and 96f18c45's. r4136863481 (default_registry.py): get, active and len() read without the lock that register, drop and atomically() take. A reader that does not wait for a held read-modify-write can answer from the middle of it. - Now all three read under the lock, as entries() and keys() already did. - openXdox's registry has the same gap. It is noted for T059 in the PR body. r4136863311 (cli.py): a registered generator owes the seam only its kind and an integer schema_version. _report indexed repository and generation as if every contract carried them, so such a snapshot raised KeyError after it was written. - The report now shows a field that is lacking, or in another shape, as . - _stats counts only a list or a mapping. - The verb then goes on to the validator for the snapshot's kind. Tests, in tests/test_projection_seams.py (130 -> 136 cases): - test_a_refresh_that_replaces_the_key_cannot_move_the_source_arms_root (renamed from 96f18c45's race test, keyed and unkeyed). The replacement now provably lands before the path is confined, and the body is still the first entry's file. Red against 687d37bf's arm. - test_a_read_waits_for_a_held_read_modify_write, over active, resolve, get and len. Each case fails with its own lock removed. - test_generate_reports_a_snapshot_whatever_else_its_contract_carries, over a minimal snapshot and one with other shapes. Both fail against 96f18c45's report, with KeyError. The whole suite: selected=2735 passed=2724 skipped=11, which is +6. F4.1's scan is unchanged, at 12. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/cli.py | 28 ++++++-- src/opendox/default_registry.py | 23 +++++-- src/opendox/serve.py | 37 +++++----- tests/test_projection_seams.py | 117 +++++++++++++++++++++++++++----- tests/test_source_core_arm.py | 30 ++++---- 5 files changed, 173 insertions(+), 62 deletions(-) diff --git a/src/opendox/cli.py b/src/opendox/cli.py index e7b7bad..cbfb034 100644 --- a/src/opendox/cli.py +++ b/src/opendox/cli.py @@ -300,20 +300,30 @@ def _report(snapshot: dict, written: Path, repo_root: Path) -> None: The neutral snapshot (`opendox-snapshot`) has no project and no project group, so its line names its kind instead, rather than reporting a grouping - its contract does not carry. Every other kind's line is unchanged.""" + its contract does not carry. Every other kind's line is unchanged. + + A REGISTERED GENERATOR OWES THE SEAM ONLY ITS DECLARED `kind` AND AN INTEGER + `schema_version` (`generator_seam`), so nothing else is indexed here as if + every contract carried it (Copilot at openDox-code#59 96f18c45, + r4136863311). A field the snapshot lacks, or carries in another shape, is + reported ``. The snapshot is already written, and whether its own + contract required the field is its validator's to say, which runs next.""" stats = _stats(snapshot) + generation = snapshot.get("generation") + generation = generation if isinstance(generation, dict) else {} + repository = snapshot.get("repository", "") print(f"wrote {written}") if snapshot.get("kind") == generator_seam.NEUTRAL_SNAPSHOT_KIND: - print(f" repository={snapshot['repository']} kind={snapshot['kind']}") + print(f" repository={repository} kind={snapshot['kind']}") else: - print(f" repository={snapshot['repository']} " + print(f" repository={repository} " f"project={snapshot.get('project', '')} " f"project_group={snapshot.get('project_group', '')}") - print(f" source_revision={snapshot['generation']['source_revision']}") + print(f" source_revision={generation.get('source_revision', '')}") # Printed even when absent: a missing freshness stamp used to be invisible # (the schema makes it optional, so nothing downstream complains), and a run # that meant to pin one needs to see whether it landed. - print(f" generated_at={snapshot['generation'].get('generated_at', '')}") + print(f" generated_at={generation.get('generated_at', '')}") print(f" documents={stats['documents']} clusters={stats['clusters']} " f"possibles={stats['possibles']} staged_topics={stats['staged_topics']} " f"changes={stats['changes']} keywords={stats['keyword_index']}") @@ -933,7 +943,13 @@ def _pull_request_port(repo_root: Path): def _stats(snapshot: dict) -> dict[str, int]: - return {k: len(snapshot.get(k, [])) for k in ( + """How many of each collection the snapshot carries. One it lacks, or + carries as something other than a list or a mapping, counts 0: a + registered generator's contract need carry none of them (see `_report`).""" + def count(value: object) -> int: + return len(value) if isinstance(value, (list, dict)) else 0 + + return {k: count(snapshot.get(k)) for k in ( "documents", "clusters", "possibles", "staged_topics", "changes", "keyword_index")} diff --git a/src/opendox/default_registry.py b/src/opendox/default_registry.py index 8ef2c8a..80bc49b 100644 --- a/src/opendox/default_registry.py +++ b/src/opendox/default_registry.py @@ -342,7 +342,15 @@ class SnapshotRegistry: Per-process and in memory. `serve.py` answers requests on threads of their own, so every mutation is serialized under a re-entrant lock, and `atomically()` holds it across a read-modify-write, as - `branch_session._preserving_active` needs.""" + `branch_session._preserving_active` needs. + + EVERY READ TAKES THE LOCK TOO (Copilot at openDox-code#59 96f18c45, + r4136863481). A read-modify-write is atomic only to a reader that waits + for it. Read unheld, `active` and `get` could answer from the middle of a + block `atomically()` holds: a key promoted and not yet put back, or an + entry registered and not yet dropped. So `get`, `active` and `len()` read + under the lock, as `entries()` and `keys()` already did. Each answers the + registry as it stood before a held block or after it, never half-way.""" def __init__(self) -> None: self._entries: dict[tuple[str, str], SnapshotEntry] = {} @@ -387,7 +395,9 @@ def drop(self, repository: str, ref: str | None = None) -> None: def get(self, repository: str, ref: str | None = None) -> SnapshotEntry | None: """A ref-less lookup means `main`.""" - return self._entries.get(snapshot_key(repository, ref)) + key = snapshot_key(repository, ref) + with self._lock: + return self._entries.get(key) def entries(self) -> list[SnapshotEntry]: """Every entry, ordered by `(repository, ref)`.""" @@ -399,12 +409,15 @@ def keys(self) -> list[tuple[str, str]]: return sorted(self._entries) def __len__(self) -> int: - return len(self._entries) + with self._lock: + return len(self._entries) @property def active(self) -> SnapshotEntry | None: - key = self._active - return None if key is None else self._entries.get(key) + """The active entry, its key and its entry read as one.""" + with self._lock: + key = self._active + return None if key is None else self._entries.get(key) def set_active(self, repository: str, ref: str | None = None) -> SnapshotEntry | None: diff --git a/src/opendox/serve.py b/src/opendox/serve.py index 8ffb61c..5ab4700 100644 --- a/src/opendox/serve.py +++ b/src/opendox/serve.py @@ -1298,28 +1298,27 @@ def _serve_source(self, tail: str, head_only: bool) -> None: # `_serve_snapshot` closes. Resolved twice, once for the path and # once for the refusal, a refresh that changed the active entry # between the two could serve one entry's file under another's - # refusal and headers (FR-048). So the entry is resolved once, and - # the path is asked for by THAT entry's own pair, which is the pair - # the refusal checked, never "the active entry" a second time. + # refusal and headers (FR-048). So the entry is resolved once. # - # AND BOTH LOOKUPS ARE MADE UNDER THE REGISTRY'S OWN LOCK (Copilot - # at openDox-code#59 687d37bf, r4136585695). `resolve_source` - # looks that pair up again, so a refresh that re-registered the - # key in between put another entry, with another root, behind the - # path. `atomically()` holds the lock that `register` and `drop` - # take, in openDox's registry and in a host's, so no refresh lands - # between the two. The path still comes through `resolve_source`, - # the one entry point to the containment rule. Nothing is sent - # while the lock is held, so a slow client cannot hold a refresh - # up: the refusal is decided on the entry in hand afterwards, and - # a refused entry's path is never read. - with self.source.registry.atomically(): - entry = self.source.registry.resolve(repository, ref) - target = (None if entry is None else - self.source.registry.resolve_source( - entry.repository, entry.ref, rest)) + # AND THE PATH IS CONFINED TO THAT ENTRY'S OWN ROOT, with no second + # lookup (Copilot at openDox-code#59: r4136585695 at 687d37bf and + # r4136863569 at 96f18c45). Asked for by the entry's pair, the + # registry looked the pair up again, so a refresh that + # re-registered the key in between put another root behind the + # path. Holding the registry's lock across both lookups would ask + # a contributed registry for a method the seam does not declare. + # So the entry in hand is confined through this arm's one entry + # point, `resolve_source_path`, as the no-registry path below is. + # It applies the registry's declared per-entry containment rule + # (`projection_seams.REGISTRY_CALLABLES`), as `serve_workbench.py` + # does for the roots it holds. An entry with no root serves + # nothing. + entry = self.source.registry.resolve(repository, ref) if self._hosted_entry_refused(entry): return + root = None if entry is None else entry.source_root + target = (None if root is None else + resolve_source_path(Path(root), rest)) else: target = resolve_source_path(Path(self.checkout_root), rest) if target is None: diff --git a/tests/test_projection_seams.py b/tests/test_projection_seams.py index d9948cb..24e8e96 100644 --- a/tests/test_projection_seams.py +++ b/tests/test_projection_seams.py @@ -737,6 +737,55 @@ def test_the_registry_keeps_a_sessions_owner_and_base_and_confines_each_entry(tm assert registry.get("garden", "draft/t") is None and len(registry) == 1 +@pytest.mark.parametrize("read,after", [ + ("active", "main"), + ("resolve", "main"), + ("get", None), + ("len", 1), +]) +def test_a_read_waits_for_a_held_read_modify_write(read, after, tmp_path) -> None: + """Every read takes the registry's lock, so none answers from the middle + of a block `atomically()` holds (Copilot at openDox-code#59 96f18c45, + r4136863481). Here the block registers a session, promotes it, and then + puts `main` back and drops the session, as a read-modify-write may. A + read on another thread starts while the session is promoted. Unheld, it + saw the session, or two entries. Held, it waits, and it answers the + registry as the block left it.""" + reg = default_registry + registry = reg.SnapshotRegistry() + main = registry.register(reg.SnapshotEntry("garden", source_root=tmp_path)) + session = reg.SnapshotEntry("garden", "draft/t", source_root=tmp_path) + reads = { + "active": lambda: registry.active, + "resolve": lambda: registry.resolve(None), + "get": lambda: registry.get("garden", "draft/t"), + "len": lambda: len(registry), + } + promoted, reading = threading.Event(), threading.Event() + answered: list = [] + + def reader() -> None: + promoted.wait(timeout=10) + reading.set() + answered.append(reads[read]()) + + worker = threading.Thread(target=reader, daemon=True) + worker.start() + with registry.atomically(): + registry.register(session, active=True) + promoted.set() + reading.wait(timeout=10) + # Time to answer, where nothing holds the registry. A read that + # waits for the lock is still waiting when this runs out. + worker.join(timeout=0.5) + registry.set_active("garden") + registry.drop("garden", "draft/t") + worker.join(timeout=10) + expected = {"main": main, None: None, 1: 1}[after] + assert answered == [expected], ( + f"{read} answered from the middle of a held block: {answered!r}") + + def test_the_regenerate_runs_the_registered_generator_and_writer(tmp_path) -> None: calls, writes = [], [] @@ -915,17 +964,16 @@ def resolve_source(self, repository, ref, tail): @pytest.mark.parametrize("query", ["", "?repository=garden&ref=main"], ids=["unkeyed", "keyed"]) -def test_a_refresh_cannot_land_between_the_source_arms_two_lookups( +def test_a_refresh_that_replaces_the_key_cannot_move_the_source_arms_root( tmp_path, query) -> None: - """FR-048 on `/source/`, against a refresh that REPLACES the key. The - arm resolves the entry once and asks for the path by that entry's pair, - and `resolve_source` looks the pair up again. Here a refresh on another - thread re-registers `garden@main`, with another root, right after the - first lookup. Unheld, it landed between the two, and the path came from - the replacement's root under the first entry's refusal and headers - (Copilot at openDox-code#59 687d37bf, r4136585695). Made under the - registry's own lock, the two lookups see one entry, and the refresh - lands after the read.""" + """FR-048 on `/source/`, against a refresh that REPLACES the key. Here a + refresh on another thread re-registers `garden@main`, with another root, + right after the arm's one resolution. When the path was asked for by the + entry's pair, `resolve_source` looked the pair up again, and the path + came from the replacement's root under the first entry's refusal and + headers (Copilot at openDox-code#59 687d37bf, r4136585695). Confined to + the root of the entry in hand, it comes from the entry the refusal + checked, and the refresh lands as it would.""" repo, _out, httpd = _served(tmp_path) bound = httpd.RequestHandlerClass.func first = bound.source.registry.active @@ -942,6 +990,7 @@ def refresh() -> None: registry.register(replacement) refresher = threading.Thread(target=refresh, daemon=True) + landed_before_the_path: list[bool] = [] class _Registry(default_registry.SnapshotRegistry): def resolve(self, repository, ref=None): @@ -949,9 +998,9 @@ def resolve(self, repository, ref=None): if entry is first and refresher.ident is None: refresher.start() attempting.wait(timeout=10) - # Time to land, where nothing holds the registry. A held - # lock keeps it waiting, and this wait simply runs out. - refresher.join(timeout=0.5) + refresher.join(timeout=10) + landed_before_the_path.append( + self.get(*first.key) is replacement) return entry registry = _Registry() @@ -959,11 +1008,11 @@ def resolve(self, repository, ref=None): bound.source.registry = registry status, _headers, body = _get( httpd, f"/source/notes-toolshed-inventory.md{query}") - refresher.join(timeout=10) - assert registry.get(*first.key) is replacement, "the refresh landed after" + assert landed_before_the_path == [True], ( + "the refresh landed after the arm's one resolution, and before its " + "path was confined") assert b"THE REPLACEMENT'S BYTES" not in body, ( - "a refresh landed between the arm's two lookups, and the path came " - "from the replacement's root") + "the path was looked up again, and came from the replacement's root") assert status == 200 assert body == (repo / "notes-toolshed-inventory.md").read_bytes() @@ -1134,6 +1183,40 @@ def operation(repo_root, repository, *, source_revision=None, generated_at=None) assert not out.exists() +@pytest.mark.parametrize("extra", [ + {}, + {"generation": "r1", "documents": 7, "keyword_index": "k"}, +], ids=["only-what-the-seam-requires", "other-shapes"]) +def test_generate_reports_a_snapshot_whatever_else_its_contract_carries( + extra, tmp_path, capsys) -> None: + """A registered generator owes the seam only its declared `kind` and an + integer `schema_version`. Its snapshot is written and reported, with + what it lacks shown as ``, and the verb goes on to the validator + for its kind. The report indexed `repository` and `generation` as if + every contract carried them, so such a snapshot failed with a `KeyError` + after it was written (Copilot at openDox-code#59 96f18c45, + r4136863311).""" + snapshot = {"schema_version": 1, "kind": "host-snapshot", **extra} + + def operation(repo_root, repository, *, source_revision=None, generated_at=None): + return dict(snapshot) + + gs.register(gs.SnapshotGenerator(contract="host-snapshot", generate=operation)) + validator = _Validator() + ps.validators.register("host-snapshot", validator) + out = tmp_path / "out" / "snapshot.json" + assert _generate(_repository(tmp_path), out) == 0 + assert json.loads(out.read_text(encoding="utf-8")) == snapshot + lines = capsys.readouterr().out.splitlines() + for line in (" repository= project= project_group=", + " source_revision=", " generated_at=", + " documents=0 clusters=0 possibles=0 staged_topics=0 " + "changes=0 keywords=0"): + assert line in lines, lines + assert [call["path"].resolve() for call in validator.calls] == [out.resolve()], ( + "the verb goes on to the validator registered for the snapshot's kind") + + def test_the_validator_stand_in_concludes_nothing_and_names_T057(tmp_path) -> None: result = default_projection.VALIDATOR.validate(tmp_path / "x.json") assert result.available is False and result.ok is False diff --git a/tests/test_source_core_arm.py b/tests/test_source_core_arm.py index f19e8fe..745f9d5 100644 --- a/tests/test_source_core_arm.py +++ b/tests/test_source_core_arm.py @@ -305,23 +305,23 @@ def test_the_arm_resolves_through_that_entry_point_and_the_registry_only(): body = _source_of(_method("_serve_source")) assert "resolve_source_path(Path(self.checkout_root), rest)" in body, ( "the no-registry path resolves through the single-root entry point") - assert "self.source.registry.resolve_source(" in body and \ - "entry.repository, entry.ref, rest)" in body, ( + assert "root = None if entry is None else entry.source_root" in body and \ + "resolve_source_path(Path(root), rest)" in body, ( "the registry path resolves through the entry's OWN root (per-entry " - "confinement, task 2.2) — not through the served checkout with a key " - "stripped off, which is the silent-wrong-data failure the seam exists " - "to prevent") + "confinement, task 2.2), by the same entry point — not through the " + "served checkout with a key stripped off, which is the " + "silent-wrong-data failure the seam exists to prevent") assert body.count("self.source.registry.resolve(") == 1, ( - "ONE resolution of the entry, whose own pair the path is then asked " - "for (plan 034 T055, FR-048): resolved a second time, the active " - "entry could change between the refusal and the path") - held = body.split("with self.source.registry.atomically():", 1) - assert len(held) == 2 and "self.source.registry.resolve(" in held[1] \ - and "self.source.registry.resolve_source(" in held[1], ( - "both lookups under the registry's own lock (openDox-code#59, " - "r4136585695): resolve_source looks the pair up again, and a " - "refresh that re-registered the key in between put another root " - "behind the path") + "ONE resolution of the entry, whose own root the path is then " + "confined to (plan 034 T055, FR-048): resolved a second time, the " + "active entry could change between the refusal and the path") + assert "resolve_source(" not in body and "atomically(" not in body, ( + "and the path is never looked up again (openDox-code#59, " + "r4136585695): asked for by the entry's pair, a refresh that " + "re-registered the key in between put another root behind the path. " + "Nor does the arm hold the registry's lock for it, which asks a " + "contributed registry for a method the seam does not declare " + "(r4136863569)") assert "resolve_within(" not in body, ( "the arm must not call the containment rule directly: one entry point, " "so a change to the rule cannot reach the route by one path and miss " From 03e06ccd56cf88fd4a787eac4e87fc8ee537e9f8 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Tue, 29 Sep 2026 23:42:04 +0000 Subject: [PATCH 28/38] T054: a scaffold's leading title and summary are one header line each Copilot at 1a603677 (r4139383472) is right. The lead block interpolated title and summary as given, so summary="ok\nstage: completion" wrote a line that leading_header() reads as the document's own stage:. A create could then land a document in a station the caller never asked for. render_scaffold() now refuses a lead value that carries a control character (Unicode Cc: \n, \r, \t, \x85 and the rest) or a line or paragraph separator (Zl, Zp), which covers every break str.splitlines ends a line at. It raises ValueError before anything is rendered, so create_scaffold writes nothing. With no lead block (a corpus that obliges neither field) nothing new is refused, and the governed layout is unchanged. The regression case is test_a_lead_value_that_is_not_one_header_line_is_refused, over title and summary and ten breakers (20 cases). All 20 fail against 1a603677's authoring.py. The whole suite: selected=2618 passed=2607 skipped=11, which is +20. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/authoring.py | 23 +++++++++++++++++++++++ tests/test_neutral_projection.py | 26 ++++++++++++++++++++++++++ 2 files changed, 49 insertions(+) diff --git a/src/opendox/authoring.py b/src/opendox/authoring.py index ca0303b..33d7517 100644 --- a/src/opendox/authoring.py +++ b/src/opendox/authoring.py @@ -55,6 +55,7 @@ import shlex import tempfile +import unicodedata from datetime import datetime, timezone from pathlib import Path from typing import Sequence @@ -163,6 +164,12 @@ def render_scaffold( `create_scaffold` passes `scaffold_lead_fields()`. - A name outside `SCAFFOLD_LEAD_FIELDS` is refused (`ValueError`), since the scaffold has no value for it. + - A lead value that is not ONE header line is refused (`ValueError`) + before anything is rendered: a line break, any other control + character, or a line or paragraph separator. Written into the block, `summary="ok\nstage: + completion"` would be read as the document's own `stage:` line, so a + create could land a document in a station the caller never asked for + (Copilot at openDox-code#57 1a603677, r4139383472). - With none, which is the default, the scaffold is byte-identical to what this function rendered before, H1 first.""" unknown = [name for name in lead_fields if name not in SCAFFOLD_LEAD_FIELDS] @@ -175,6 +182,12 @@ def render_scaffold( heading = title if suffixed else f"{title}{BRAINSTORM_SUFFIX}" lead_values = {"title": title[:-len(BRAINSTORM_SUFFIX)] if suffixed else title, "summary": summary} + for name in SCAFFOLD_LEAD_FIELDS: + if name in lead_fields and not _one_header_line(lead_values[name]): + raise ValueError( + f"a scaffold's leading {name}: must be one header line, and " + f"{lead_values[name]!r} carries a line break or another " + "control character") lead = "".join(f"{name}: {lead_values[name]}\n" for name in SCAFFOLD_LEAD_FIELDS if name in lead_fields) topics_line = ", ".join(t.strip() for t in topics if t and t.strip()) @@ -198,6 +211,16 @@ def render_scaffold( ) +def _one_header_line(value: str) -> bool: + """Whether `value` can stand as one `name: value` header line: no control + character (Unicode `Cc`, which holds `\n`, `\r`, `\t`, `\x85` and the + other ASCII breaks) and no line or paragraph separator (`Zl`, `Zp`). That + is every character `str.splitlines` ends a line at, and every other + control a header reader could stumble on.""" + return not any(unicodedata.category(char) in ("Cc", "Zl", "Zp") + for char in value) + + def normalized_area(area: str) -> str: """`area` with the one trailing slash `scaffold_relpath` assumes.""" return area if area.endswith("/") else area + "/" diff --git a/tests/test_neutral_projection.py b/tests/test_neutral_projection.py index 16bf854..ff68e31 100644 --- a/tests/test_neutral_projection.py +++ b/tests/test_neutral_projection.py @@ -971,6 +971,32 @@ def test_the_lead_block_is_the_neutral_fields_in_order_then_a_blank_line() -> No assert set(lga.NEUTRAL_FIELDS) <= set(authoring.SCAFFOLD_LEAD_FIELDS) +@pytest.mark.parametrize("field", ["title", "summary"]) +@pytest.mark.parametrize("breaker", ["\n", "\r", "\r\n", "\x0b", "\x1e", "\x85", + "
", "
", "\t", "\x00"]) +def test_a_lead_value_that_is_not_one_header_line_is_refused( + field: str, breaker: str, tmp_path: Path) -> None: + """A leading `title:` or `summary:` is ONE header line. A value that + breaks it would put a line of the caller's choosing into the block the + default adapter reads: `summary="ok\\nstage: completion"` read as the + document's own `stage:` (Copilot at openDox-code#57 1a603677, + r4139383472). So it is refused before anything is rendered or written. + A value with no lead block to break keeps the governed layout as before, + and an ordinary value with inner spaces and dashes still leads.""" + value = f"ok{breaker}stage: completion" + asked = {**_CREATE, field: value} + with pytest.raises(ValueError, match=f"leading {field}: must be one header line"): + authoring.render_scaffold(**asked, lead_fields=("title", "summary")) + assert authoring.render_scaffold(**asked).startswith("# "), ( + "with no lead block, nothing new is refused") + with pytest.raises(ValueError): + authoring.create_scaffold(OutputBoundary(tmp_path, actor=HUMAN), **asked) + assert not any(tmp_path.rglob("*.md")), "a refused create writes nothing" + fine = {**_CREATE, field: "A shed roof — pitched, not flat"} + assert authoring.render_scaffold(**fine, lead_fields=(field,)).startswith( + f"{field}: A shed roof — pitched, not flat\n") + + def test_a_scaffold_with_no_corpus_registered_refuses_and_writes_nothing( tmp_path: Path) -> None: """`create_scaffold` asks the corpus which fields to lead with, as From 65c943ffd81cfd4c33324cf81137a35ed4f56870 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Tue, 29 Sep 2026 23:59:32 +0000 Subject: [PATCH 29/38] T056: the standalone generate path, end to end, and the served URL is flushed Plan 034 T056 (#1144 5.1, in part). python -m opendox.cli generate and generate-and-open --no-open run on T050's fixture in a process of their own, with neither sibling importable, and the server STARTS and answers. Research R7 measured that start as refused. tests/test_standalone_generate_path.py runs each verb as a real child process. A sitecustomize on PYTHONPATH refuses openxdox, ideation_dashboard, doc_health and corpus_adapter_openxfactory at the first finder, and logs every refused name. Each case asserts the log is empty. - F5.3 as #1144 writes it: generate over a fresh copy of the fixture gives a non-empty neutral snapshot with none of the declared vocabulary. - The verb's half of the stage: edge case. An out-of-six value is reported, naming the document, the value and the six keys, and it is read as a source. The unedited fixture is the control. - generate-and-open without --no-serve, and python -m opendox.serve, each start. Each answers /index.html, /snapshot.json, /capabilities and /source/, refuses /source/.git/config, and exits 0 on SIGINT with its port closed. The fix the start needed: cli.cmd_generate_and_open and serve.serve printed the URL and then blocked in serve_forever() without flushing. On a pipe, stdout is block-buffered, so a wrapper never saw the line while the server ran: zero lines in 20 s at #59 e3ef506a. Both now flush before they serve. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/cli.py | 10 +- src/opendox/serve.py | 7 +- tests/test_standalone_generate_path.py | 447 +++++++++++++++++++++++++ 3 files changed, 461 insertions(+), 3 deletions(-) create mode 100644 tests/test_standalone_generate_path.py diff --git a/src/opendox/cli.py b/src/opendox/cli.py index cbfb034..7d11a97 100644 --- a/src/opendox/cli.py +++ b/src/opendox/cli.py @@ -598,7 +598,13 @@ def cmd_generate_and_open(args: argparse.Namespace, *, opener=webbrowser.open) - url = serve_mod.server_url(httpd, "/index.html") print(f" serving {url}") print(f" snapshot {serve_mod.server_url(httpd, '/snapshot.json')}") - print(url) # the URL is ALWAYS printed on its own line + # The URL is ALWAYS printed on its own line, AND FLUSHED (plan 034 T056). + # Where standard output is a pipe or a file, Python buffers it by block, + # and the process is about to block in `serve_forever()`. So without the + # flush, a wrapper reading this line never sees it while the server runs, + # and it cannot learn an ephemeral port or tell that the server started. + # Measured at openDox-code#59 e3ef506a: zero lines in 20 s on a pipe. + print(url, flush=True) if not args.no_open: try: @@ -610,7 +616,7 @@ def cmd_generate_and_open(args: argparse.Namespace, *, opener=webbrowser.open) - httpd.server_close() return 0 - print(" serving until interrupted (Ctrl-C to stop)") + print(" serving until interrupted (Ctrl-C to stop)", flush=True) try: httpd.serve_forever() except KeyboardInterrupt: diff --git a/src/opendox/serve.py b/src/opendox/serve.py index 5ab4700..c642091 100644 --- a/src/opendox/serve.py +++ b/src/opendox/serve.py @@ -2216,7 +2216,12 @@ def serve( checkout_root=checkout_root)) httpd = build_server(web_dir, snapshot_path, checkout_root, host=host, port=port, quiet=quiet, actor=actor, **build_kwargs) - print(f"serving ideation dashboard at {server_url(httpd, '/index.html')}") + # FLUSHED before the process blocks (plan 034 T056): where standard + # output is a pipe or a file it is block-buffered, so an unflushed line + # never reaches a wrapper while the server runs, and the wrapper cannot + # learn an ephemeral port or tell that the server started. + print(f"serving ideation dashboard at {server_url(httpd, '/index.html')}", + flush=True) try: httpd.serve_forever() except KeyboardInterrupt: diff --git a/tests/test_standalone_generate_path.py b/tests/test_standalone_generate_path.py new file mode 100644 index 0000000..c55e88e --- /dev/null +++ b/tests/test_standalone_generate_path.py @@ -0,0 +1,447 @@ +"""The standalone generate path, end to end, through the verbs themselves: +plan 034's T056 (#1144's 5.1, in part). + +T054 tests the neutral projection in process, and T055 routes the verbs +through the seams. What neither shows is the path a user runs: +`python -m opendox.cli generate` and `generate-and-open`, in a process of their +own, with neither sibling importable, until the server STARTS and answers. That +is what research R7 measured as refused, and this file holds the lifted limit: + +1. F5.3, #1144's falsifier for Group 5, run the way it is written: + `python -m opendox.cli generate` over a fresh repository copied from T050's + `tests/fixtures/plain-documents`. The snapshot is not empty, it is the + neutral kind, and none of openxFactory's declared vocabulary is in a string + value of it. +2. The verb's half of spec.md's `stage:` edge case. Over a copy of T050's + fixture in which one document declares a `stage:` value outside the six + role keys, the verb reports it, naming the document, the value and the six + keys, and the snapshot it writes reads that document as a source. T054 + tests the projection's half in process. +3. `python -m opendox.cli generate-and-open --no-open` STARTS a server, which + answers `/index.html`, `/snapshot.json`, `/capabilities` and `/source/`, + refuses `/source/.git/config`, and stops on an interrupt with status 0. +4. `python -m opendox.serve`, the server's own entry point, starts and answers + the same way. + +HOW "NEITHER SIBLING IS IMPORTABLE" IS MADE TRUE. Each run is a real child +process, `python -m ...`, whose interpreter loads a `sitecustomize` from a +directory put first on `PYTHONPATH`. It installs a meta-path finder that +refuses `openxdox`, `ideation_dashboard`, `doc_health` and +`corpus_adapter_openxfactory`, whatever is installed, and it records every +refused name in a log. Each case asserts that the log is EMPTY. A refused +import that some `except ImportError` swallowed would otherwise pass as a +degraded run, so the case holds both halves: the siblings cannot be imported, +and nothing on the path tries to. + +THE CHILD'S STANDARD OUTPUT IS A PIPE, with Python's default buffering, as +any wrapper that reads the URL sees it. `PYTHONUNBUFFERED` is taken out of the +child's environment on purpose. A server that printed its URL and then +blocked in `serve_forever()` without flushing never delivered that line on a +pipe, so a caller could neither learn an ephemeral port nor tell that the +server had started (measured at openDox-code#59 `e3ef506a`: zero lines in 20 +seconds). T056 flushes it in both entry points, and cases 3 and 4 fail +without that. + +NOT HERE: F10.1's run through a plain install, with the console script and no +`--local`, arrives in phase 3 (T070, and T077 as batch H amends it). + +A CREATED FILE: no carve-manifest row (RULED OQ-C). +""" + +from __future__ import annotations + +import http.client +import json +import os +import queue +import re +import shutil +import signal +import socket +import subprocess +import sys +import threading +import time +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent +PLAIN_DOCUMENTS = ROOT / "tests" / "fixtures" / "plain-documents" +NEUTRAL = "opendox-snapshot" +SIBLINGS = ("openxdox", "ideation_dashboard", "doc_health", + "corpus_adapter_openxfactory") + +#: The six station role keys, in the order the neutral contract lists them. +ROLE_KEYS = ("source", "grouping", "candidate", "selection", "submission", + "completion") + +#: F5.3's declared vocabulary, verbatim: the eight lifecycle `Status:` words +#: and the change/spec/delta nouns, as #1144's Group 5 falsifier spells them. +F53_WORDS = ["brainstorm", "staged", "draft", "ratified", "standard", + "superseded", "retired", "record", "openspec", "proposal.md", + "tasks.md", "design.md", "added requirements", + "modified requirements"] + +#: How long a child may take to say where it serves. Generation over the +#: fixture takes about a second; the margin is for a loaded CI runner. +START_DEADLINE_SECONDS = 120.0 + +#: How long a child may take to stop once interrupted. +STOP_DEADLINE_SECONDS = 30.0 + +#: The `sitecustomize` every child loads. It refuses the siblings at the +#: FIRST finder, drops any that a `.pth` file imported before it ran, and +#: logs each refused name to the file `OPENDOX_T056_REFUSED` names. +_BLOCKER = f'''\ +import os +import sys + +_SIBLINGS = {SIBLINGS!r} + +for _name in list(sys.modules): + if _name.split(".")[0] in _SIBLINGS: + del sys.modules[_name] + + +class _RefuseTheSiblings: + """Neither sibling is importable in this process (plan 034 T056).""" + + def find_spec(self, name, path=None, target=None): + if name.split(".")[0] not in _SIBLINGS: + return None + log = os.environ.get("OPENDOX_T056_REFUSED") + if log: + with open(log, "a", encoding="utf-8") as stream: + stream.write(name + "\\n") + raise ModuleNotFoundError( + f"No module named {{name!r}} (refused: a lone openDox has no " + f"sibling)", name=name) + + +sys.meta_path.insert(0, _RefuseTheSiblings()) +''' + + +# --------------------------------------------------------------------------- +# helpers +# --------------------------------------------------------------------------- + +def _git(root: Path, *args: str) -> None: + """`git` in `root` as F5.3's fixture identity, with no inherited `GIT_*` + variable and no user or system configuration.""" + env = {k: v for k, v in os.environ.items() if not k.startswith("GIT_")} + env.update({ + "GIT_AUTHOR_NAME": "fixture", "GIT_AUTHOR_EMAIL": "fixture@example.invalid", + "GIT_COMMITTER_NAME": "fixture", + "GIT_COMMITTER_EMAIL": "fixture@example.invalid", + "GIT_CONFIG_GLOBAL": os.devnull, "GIT_CONFIG_SYSTEM": os.devnull, + }) + subprocess.run(["git", "-C", str(root), *args], check=True, + capture_output=True, env=env) + + +def _fresh_repository(tmp_path: Path, *, edits: dict[str, str] | None = None) -> Path: + """F5.3's preamble: T050's fixture copied into a FRESH repository, with + `edits` (a document's new text, by name) applied before the commit.""" + root = tmp_path / "plain-documents" + shutil.copytree(PLAIN_DOCUMENTS, root) + for name, text in (edits or {}).items(): + (root / name).write_text(text, encoding="utf-8") + _git(root, "-c", "init.defaultBranch=main", "init", "-q") + _git(root, "add", "-A") + _git(root, "commit", "-qm", "fixture") + return root + + +class _Child: + """One `python -m ...` child with the siblings refused, its + standard output a buffered pipe.""" + + def __init__(self, tmp_path: Path, module: str, *args: str) -> None: + blocker = tmp_path / "sibling-blocker" + blocker.mkdir(exist_ok=True) + (blocker / "sitecustomize.py").write_text(_BLOCKER, encoding="utf-8") + self.refused_log = tmp_path / "refused-imports.log" + env = dict(os.environ) + env.pop("PYTHONUNBUFFERED", None) + env["PYTHONPATH"] = os.pathsep.join( + [str(blocker), *filter(None, [env.get("PYTHONPATH")])]) + env["OPENDOX_T056_REFUSED"] = str(self.refused_log) + self.argv = [sys.executable, "-m", module, *args] + verb = args[0] if args and not args[0].startswith("-") else "" + self.label = f"python -m {module} {verb}".strip() + self.process = subprocess.Popen( + self.argv, cwd=ROOT, env=env, stdout=subprocess.PIPE, + stderr=subprocess.PIPE, text=True, encoding="utf-8", + errors="replace") + self.lines: queue.Queue[str | None] = queue.Queue() + self.stdout: list[str] = [] + self.stderr: list[str] = [] + self._pumps = [ + threading.Thread(target=self._pump, args=(self.process.stdout, True), + daemon=True), + threading.Thread(target=self._pump, args=(self.process.stderr, False), + daemon=True), + ] + for pump in self._pumps: + pump.start() + + def _pump(self, stream, is_stdout: bool) -> None: + for line in stream: + (self.stdout if is_stdout else self.stderr).append(line) + if is_stdout: + self.lines.put(line) + if is_stdout: + self.lines.put(None) + + def wait_for_line(self, pattern: re.Pattern[str]) -> re.Match[str]: + """The first standard-output line matching `pattern`, read while the + child runs. Fails, naming what the child said, if it exits first or + the deadline passes.""" + deadline = time.monotonic() + START_DEADLINE_SECONDS + while time.monotonic() < deadline: + try: + line = self.lines.get(timeout=0.25) + except queue.Empty: + continue + if line is None: + break + match = pattern.search(line) + if match: + return match + raise AssertionError( + f"{self.label} never printed a line matching " + f"{pattern.pattern!r} on its (buffered) standard output while it " + f"ran: exit status {self.process.poll()}, standard output " + f"{''.join(self.stdout)!r}, standard error {self.stderr_text()[-2000:]!r}") + + def interrupt(self) -> int: + """Send SIGINT, as Ctrl-C would, and answer the exit status.""" + self.process.send_signal(signal.SIGINT) + try: + return self.process.wait(timeout=STOP_DEADLINE_SECONDS) + finally: + for pump in self._pumps: + pump.join(timeout=STOP_DEADLINE_SECONDS) + + def kill(self) -> None: + if self.process.poll() is None: + self.process.kill() + self.process.wait(timeout=STOP_DEADLINE_SECONDS) + + def stderr_text(self) -> str: + return "".join(self.stderr) + + def refused(self) -> list[str]: + if not self.refused_log.exists(): + return [] + return self.refused_log.read_text(encoding="utf-8").split() + + +def _run(tmp_path: Path, module: str, *args: str) -> tuple[_Child, int]: + """A child run to completion.""" + child = _Child(tmp_path, module, *args) + try: + status = child.process.wait(timeout=300) + finally: + child.kill() + for pump in child._pumps: + pump.join(timeout=STOP_DEADLINE_SECONDS) + return child, status + + +def _get(base: tuple[str, int], path: str) -> tuple[int, str, bytes]: + connection = http.client.HTTPConnection(*base, timeout=30) + try: + connection.request("GET", path) + response = connection.getresponse() + return (response.status, response.getheader("Content-Type") or "", + response.read()) + finally: + connection.close() + + +def _string_values(value): + """Every string VALUE in a JSON document: keys are the product's own + structure, as F5.3 reads it.""" + if isinstance(value, dict): + for item in value.values(): + yield from _string_values(item) + elif isinstance(value, list): + for item in value: + yield from _string_values(item) + elif isinstance(value, str): + yield value + + +def _leaks(snapshot: dict) -> list[str]: + """F5.3's assertion, verbatim in substance: the declared words found as + whole words in a lower-cased string value.""" + pattern = re.compile(r"\b(" + "|".join(re.escape(w) for w in F53_WORDS) + r")\b") + return sorted({m.group(1) for text in _string_values(snapshot) + for m in pattern.finditer(text.lower())}) + + +def _assert_the_server_answers(base: tuple[str, int], written: Path, + repo: Path) -> None: + """The core routes a standalone server answers, from openDox's own + registry and source.""" + status, kind, body = _get(base, "/index.html") + assert status == 200 and " None: + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as probe: + probe.settimeout(5) + assert probe.connect_ex(base) != 0, f"{base} still accepts connections" + + +_URL = re.compile(r"^(http://([0-9.]+):([0-9]+))/index\.html$") +_SERVE_URL = re.compile(r"^serving ideation dashboard at " + r"(http://([0-9.]+):([0-9]+))/index\.html$") + + +# --------------------------------------------------------------------------- +# 1 — F5.3, through the module +# --------------------------------------------------------------------------- + +def test_F5_3_generate_through_the_module_writes_the_neutral_snapshot(tmp_path) -> None: + """#1144's F5.3, as written: `python -m opendox.cli generate` over a fresh + copy of T050's fixture, with neither sibling importable.""" + repo = _fresh_repository(tmp_path) + out = tmp_path / "snap.json" + child, status = _run(tmp_path, "opendox.cli", "generate", + "--repo-root", str(repo), "--repository", "fixture", + "--output", str(out)) + assert status == 0, child.stderr_text() + assert child.refused() == [], child.refused() + snapshot = json.loads(out.read_text(encoding="utf-8")) + assert snapshot["documents"], "snapshot is empty" + assert snapshot["kind"] == NEUTRAL + assert snapshot["repository"] == "fixture" + assert _leaks(snapshot) == [], ( + f"openxFactory's vocabulary leaked into the neutral projection: " + f"{_leaks(snapshot)}") + assert f"wrote {out}" in "".join(child.stdout) + assert "notice:" not in child.stderr_text() + + +# --------------------------------------------------------------------------- +# 2 — the verb's half of the `stage:` edge case +# --------------------------------------------------------------------------- + +def test_the_verb_reports_a_stage_outside_the_six_and_reads_it_as_a_source( + tmp_path) -> None: + """spec.md's edge case, through `python -m opendox.cli generate`: the value + is not a declaration, the verb names the document, the value and the six + keys, and the snapshot reads the document as a source.""" + document = "candidate-toolshed-rebuild.md" + original = (PLAIN_DOCUMENTS / document).read_text(encoding="utf-8") + assert original.startswith("stage: candidate\n"), original[:40] + repo = _fresh_repository( + tmp_path, edits={document: original.replace( + "stage: candidate\n", "stage: someday\n", 1)}) + out = tmp_path / "snap.json" + child, status = _run(tmp_path, "opendox.cli", "generate", + "--repo-root", str(repo), "--repository", "fixture", + "--output", str(out)) + assert status == 0, child.stderr_text() + assert child.refused() == [], child.refused() + notices = [line for line in child.stderr_text().splitlines() + if line.startswith("notice:")] + assert len(notices) == 1, child.stderr_text() + notice = notices[0] + assert document in notice + assert "'someday'" in notice + for key in ROLE_KEYS: + assert re.search(rf"\b{key}\b", notice), (key, notice) + assert "read as a source" in notice + snapshot = json.loads(out.read_text(encoding="utf-8")) + [entry] = [d for d in snapshot["documents"] if d["path"] == document] + assert entry["stage"] == "source" + assert "someday" not in set(_string_values(snapshot)) + assert {d["stage"] for d in snapshot["documents"]} <= set(ROLE_KEYS) + + +def test_the_unedited_fixture_declares_that_document_a_candidate(tmp_path) -> None: + """The control for the case above: the same document, as T050 ships it, + is a declared candidate and draws no notice. So the reading above is the + out-of-six value's doing.""" + repo = _fresh_repository(tmp_path) + out = tmp_path / "snap.json" + child, status = _run(tmp_path, "opendox.cli", "generate", + "--repo-root", str(repo), "--repository", "fixture", + "--output", str(out)) + assert status == 0, child.stderr_text() + snapshot = json.loads(out.read_text(encoding="utf-8")) + [entry] = [d for d in snapshot["documents"] + if d["path"] == "candidate-toolshed-rebuild.md"] + assert entry["stage"] == "candidate" + assert "notice:" not in child.stderr_text() + + +# --------------------------------------------------------------------------- +# 3 — `generate-and-open` STARTS the server (research R7's limit, lifted) +# --------------------------------------------------------------------------- + +def test_generate_and_open_starts_a_server_that_answers_with_no_sibling(tmp_path) -> None: + """`python -m opendox.cli generate-and-open --no-open`, with no + `--no-serve`: the server starts, says where on a buffered pipe, answers + the core routes, and stops on an interrupt with status 0.""" + repo = _fresh_repository(tmp_path) + run_dir = tmp_path / "run" + child = _Child(tmp_path, "opendox.cli", "generate-and-open", + "--repo-root", str(repo), "--repository", "fixture", + "--no-open", "--port", "0", "--run-dir", str(run_dir)) + try: + match = child.wait_for_line(_URL) + base = (match.group(2), int(match.group(3))) + assert child.process.poll() is None, "the server exited after printing its URL" + _assert_the_server_answers(base, run_dir / "snapshot.json", repo) + assert child.interrupt() == 0, child.stderr_text() + finally: + child.kill() + _assert_the_port_is_closed(base) + assert child.refused() == [], child.refused() + assert "serving until interrupted" in "".join(child.stdout) + + +# --------------------------------------------------------------------------- +# 4 — the server's own entry point starts the same way +# --------------------------------------------------------------------------- + +def test_serve_main_starts_a_server_that_answers_with_no_sibling(tmp_path) -> None: + """`python -m opendox.serve` over a snapshot `generate` wrote: it starts, + announces its URL on a buffered pipe, answers, and stops on an interrupt.""" + repo = _fresh_repository(tmp_path) + out = tmp_path / "out" / "snapshot.json" + generated, status = _run(tmp_path, "opendox.cli", "generate", + "--repo-root", str(repo), "--repository", "fixture", + "--output", str(out), "--no-validate") + assert status == 0, generated.stderr_text() + child = _Child(tmp_path, "opendox.serve", "--snapshot", str(out), + "--checkout-root", str(repo), "--port", "0") + try: + match = child.wait_for_line(_SERVE_URL) + base = (match.group(2), int(match.group(3))) + _assert_the_server_answers(base, out, repo) + assert child.interrupt() == 0, child.stderr_text() + finally: + child.kill() + _assert_the_port_is_closed(base) + assert child.refused() == [], child.refused() From 1597511d8f6659d339faf386331ce47eaf432a06 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Wed, 30 Sep 2026 00:10:14 +0000 Subject: [PATCH 30/38] T056: the lone-openDox child process becomes a test helper, for T058's F7.2 tests/standalone_child.py holds what tests/test_standalone_generate_path.py built for itself: python -m as a child process with the four siblings refused at its first finder (and every refused name logged), its standard output a buffered pipe read on threads, and #1144's fresh-repository preamble. T058's F7.2 case runs the generate verbs the same way, so the harness is written once. The five cases are unchanged, and all ten mutations are still killed. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/standalone_child.py | 224 +++++++++++++++++++++++++ tests/test_standalone_generate_path.py | 192 ++------------------- 2 files changed, 238 insertions(+), 178 deletions(-) create mode 100644 tests/standalone_child.py diff --git a/tests/standalone_child.py b/tests/standalone_child.py new file mode 100644 index 0000000..0398d1e --- /dev/null +++ b/tests/standalone_child.py @@ -0,0 +1,224 @@ +"""A lone openDox, as a CHILD PROCESS: `python -m ...` with neither +sibling importable (plan 034's T056, and T058's F7.2 case after it). + +A test that means "the verb, the way a user runs it" runs the verb in a process +of its own, through `python -m`, as #1144's falsifiers are written. This module +builds that child and reads it: + +* NEITHER SIBLING IS IMPORTABLE. The child's interpreter loads a + `sitecustomize` from a directory put first on `PYTHONPATH`. It installs a + meta-path finder that refuses `openxdox`, `ideation_dashboard`, `doc_health` + and `corpus_adapter_openxfactory`, whatever is installed, and drops any of + them that a `.pth` file imported before it ran. It records every refused name + in a log (`Child.refused()`). A case asserts that the log is EMPTY: a refused + import that some `except ImportError` swallowed would otherwise pass as a + degraded run. +* ITS STANDARD OUTPUT IS A PIPE WITH PYTHON'S DEFAULT BUFFERING, as any + wrapper that reads it sees it. `PYTHONUNBUFFERED` is taken out of the + child's environment on purpose, so a line the child prints and does not + flush before it blocks is a line a case never reads. +* It is read on threads while it runs (`Child.wait_for_line`), so a server that + never exits can still be asked where it serves, and then interrupted + (`Child.interrupt`, SIGINT, as Ctrl-C sends). + +`fresh_repository()` is #1144's preamble: a fixture copied into a FRESH git +repository and committed as the fixture's own identity. + +A helper module, not a test module: it holds no case. A CREATED FILE, with no +carve-manifest row (RULED OQ-C). +""" + +from __future__ import annotations + +import os +import queue +import re +import shutil +import signal +import subprocess +import sys +import threading +import time +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent + +#: The four packages a neutral openDox must run without (#1144's F2.1). +SIBLINGS = ("openxdox", "ideation_dashboard", "doc_health", + "corpus_adapter_openxfactory") + +#: How long a child may take to print what a case waits for. Generation over a +#: fixture takes about a second; the margin is for a loaded CI runner. +START_DEADLINE_SECONDS = 120.0 + +#: How long a child may take to stop once interrupted, or to finish a run. +STOP_DEADLINE_SECONDS = 30.0 +RUN_DEADLINE_SECONDS = 300.0 + +#: The environment variable naming the child's refused-import log. +REFUSED_LOG_ENV = "OPENDOX_STANDALONE_CHILD_REFUSED" + +#: The `sitecustomize` every child loads. +_BLOCKER = f'''\ +import os +import sys + +_SIBLINGS = {SIBLINGS!r} + +for _name in list(sys.modules): + if _name.split(".")[0] in _SIBLINGS: + del sys.modules[_name] + + +class _RefuseTheSiblings: + """Neither sibling is importable in this process (plan 034 T056).""" + + def find_spec(self, name, path=None, target=None): + if name.split(".")[0] not in _SIBLINGS: + return None + log = os.environ.get({REFUSED_LOG_ENV!r}) + if log: + with open(log, "a", encoding="utf-8") as stream: + stream.write(name + "\\n") + raise ModuleNotFoundError( + f"No module named {{name!r}} (refused: a lone openDox has no " + f"sibling)", name=name) + + +sys.meta_path.insert(0, _RefuseTheSiblings()) +''' + + +def git(root: Path, *args: str) -> None: + """`git` in `root` as #1144's fixture identity, with no inherited `GIT_*` + variable and no user or system configuration.""" + env = {k: v for k, v in os.environ.items() if not k.startswith("GIT_")} + env.update({ + "GIT_AUTHOR_NAME": "fixture", "GIT_AUTHOR_EMAIL": "fixture@example.invalid", + "GIT_COMMITTER_NAME": "fixture", + "GIT_COMMITTER_EMAIL": "fixture@example.invalid", + "GIT_CONFIG_GLOBAL": os.devnull, "GIT_CONFIG_SYSTEM": os.devnull, + }) + subprocess.run(["git", "-C", str(root), *args], check=True, + capture_output=True, env=env) + + +def fresh_repository(fixture: Path, parent: Path, *, + edits: dict[str, str] | None = None) -> Path: + """#1144's preamble: `fixture` copied into a FRESH repository under + `parent`, named as the fixture is, with `edits` (a document's new text, by + name) applied before the one commit.""" + root = parent / fixture.name + shutil.copytree(fixture, root) + for name, text in (edits or {}).items(): + (root / name).write_text(text, encoding="utf-8") + git(root, "-c", "init.defaultBranch=main", "init", "-q") + git(root, "add", "-A") + git(root, "commit", "-qm", "fixture") + return root + + +class Child: + """One `python -m ...` child, with the siblings refused and its + standard output a buffered pipe. `workdir` holds the blocker and the log.""" + + def __init__(self, workdir: Path, module: str, *args: str) -> None: + blocker = workdir / "sibling-blocker" + blocker.mkdir(parents=True, exist_ok=True) + (blocker / "sitecustomize.py").write_text(_BLOCKER, encoding="utf-8") + self.refused_log = workdir / "refused-imports.log" + env = dict(os.environ) + env.pop("PYTHONUNBUFFERED", None) + env["PYTHONPATH"] = os.pathsep.join( + [str(blocker), *filter(None, [env.get("PYTHONPATH")])]) + env[REFUSED_LOG_ENV] = str(self.refused_log) + self.argv = [sys.executable, "-m", module, *args] + verb = args[0] if args and not args[0].startswith("-") else "" + self.label = f"python -m {module} {verb}".strip() + self.process = subprocess.Popen( + self.argv, cwd=ROOT, env=env, stdout=subprocess.PIPE, + stderr=subprocess.PIPE, text=True, encoding="utf-8", + errors="replace") + self.lines: queue.Queue[str | None] = queue.Queue() + self.stdout: list[str] = [] + self.stderr: list[str] = [] + self._pumps = [ + threading.Thread(target=self._pump, args=(self.process.stdout, True), + daemon=True), + threading.Thread(target=self._pump, args=(self.process.stderr, False), + daemon=True), + ] + for pump in self._pumps: + pump.start() + + def _pump(self, stream, is_stdout: bool) -> None: + for line in stream: + (self.stdout if is_stdout else self.stderr).append(line) + if is_stdout: + self.lines.put(line) + if is_stdout: + self.lines.put(None) + + def wait_for_line(self, pattern: re.Pattern[str]) -> re.Match[str]: + """The first standard-output line matching `pattern`, read while the + child runs. Fails, naming what the child said, if it exits first or + the deadline passes.""" + deadline = time.monotonic() + START_DEADLINE_SECONDS + while time.monotonic() < deadline: + try: + line = self.lines.get(timeout=0.25) + except queue.Empty: + continue + if line is None: + break + match = pattern.search(line) + if match: + return match + raise AssertionError( + f"{self.label} never printed a line matching " + f"{pattern.pattern!r} on its (buffered) standard output while it " + f"ran: exit status {self.process.poll()}, standard output " + f"{''.join(self.stdout)!r}, standard error {self.stderr_text()[-2000:]!r}") + + def wait(self) -> int: + """Run the child to completion and answer its exit status.""" + try: + return self.process.wait(timeout=RUN_DEADLINE_SECONDS) + finally: + self.kill() + self._join() + + def interrupt(self) -> int: + """Send SIGINT, as Ctrl-C would, and answer the exit status.""" + self.process.send_signal(signal.SIGINT) + try: + return self.process.wait(timeout=STOP_DEADLINE_SECONDS) + finally: + self._join() + + def kill(self) -> None: + if self.process.poll() is None: + self.process.kill() + self.process.wait(timeout=STOP_DEADLINE_SECONDS) + + def _join(self) -> None: + for pump in self._pumps: + pump.join(timeout=STOP_DEADLINE_SECONDS) + + def stdout_text(self) -> str: + return "".join(self.stdout) + + def stderr_text(self) -> str: + return "".join(self.stderr) + + def refused(self) -> list[str]: + """Every sibling name the child tried to import, in order.""" + if not self.refused_log.exists(): + return [] + return self.refused_log.read_text(encoding="utf-8").split() + + +def run_module(workdir: Path, module: str, *args: str) -> tuple[Child, int]: + """A child run to completion: `(child, exit status)`.""" + child = Child(workdir, module, *args) + return child, child.wait() diff --git a/tests/test_standalone_generate_path.py b/tests/test_standalone_generate_path.py index c55e88e..eb47fcb 100644 --- a/tests/test_standalone_generate_path.py +++ b/tests/test_standalone_generate_path.py @@ -24,18 +24,16 @@ the same way. HOW "NEITHER SIBLING IS IMPORTABLE" IS MADE TRUE. Each run is a real child -process, `python -m ...`, whose interpreter loads a `sitecustomize` from a -directory put first on `PYTHONPATH`. It installs a meta-path finder that -refuses `openxdox`, `ideation_dashboard`, `doc_health` and -`corpus_adapter_openxfactory`, whatever is installed, and it records every -refused name in a log. Each case asserts that the log is EMPTY. A refused +process, `python -m ...`, built by `tests/standalone_child.py`. The child's +interpreter refuses `openxdox`, `ideation_dashboard`, `doc_health` and +`corpus_adapter_openxfactory` at its first finder, whatever is installed, and +logs every refused name. Each case asserts that the log is EMPTY. A refused import that some `except ImportError` swallowed would otherwise pass as a degraded run, so the case holds both halves: the siblings cannot be imported, and nothing on the path tries to. THE CHILD'S STANDARD OUTPUT IS A PIPE, with Python's default buffering, as -any wrapper that reads the URL sees it. `PYTHONUNBUFFERED` is taken out of the -child's environment on purpose. A server that printed its URL and then +any wrapper that reads the URL sees it. A server that printed its URL and then blocked in `serve_forever()` without flushing never delivered that line on a pipe, so a caller could neither learn an ephemeral port nor tell that the server had started (measured at openDox-code#59 `e3ef506a`: zero lines in 20 @@ -52,23 +50,15 @@ import http.client import json -import os -import queue import re -import shutil -import signal import socket -import subprocess -import sys -import threading -import time from pathlib import Path +from standalone_child import Child, fresh_repository, run_module + ROOT = Path(__file__).resolve().parent.parent PLAIN_DOCUMENTS = ROOT / "tests" / "fixtures" / "plain-documents" NEUTRAL = "opendox-snapshot" -SIBLINGS = ("openxdox", "ideation_dashboard", "doc_health", - "corpus_adapter_openxfactory") #: The six station role keys, in the order the neutral contract lists them. ROLE_KEYS = ("source", "grouping", "candidate", "selection", "submission", @@ -81,172 +71,18 @@ "tasks.md", "design.md", "added requirements", "modified requirements"] -#: How long a child may take to say where it serves. Generation over the -#: fixture takes about a second; the margin is for a loaded CI runner. -START_DEADLINE_SECONDS = 120.0 - -#: How long a child may take to stop once interrupted. -STOP_DEADLINE_SECONDS = 30.0 - -#: The `sitecustomize` every child loads. It refuses the siblings at the -#: FIRST finder, drops any that a `.pth` file imported before it ran, and -#: logs each refused name to the file `OPENDOX_T056_REFUSED` names. -_BLOCKER = f'''\ -import os -import sys - -_SIBLINGS = {SIBLINGS!r} - -for _name in list(sys.modules): - if _name.split(".")[0] in _SIBLINGS: - del sys.modules[_name] - - -class _RefuseTheSiblings: - """Neither sibling is importable in this process (plan 034 T056).""" - - def find_spec(self, name, path=None, target=None): - if name.split(".")[0] not in _SIBLINGS: - return None - log = os.environ.get("OPENDOX_T056_REFUSED") - if log: - with open(log, "a", encoding="utf-8") as stream: - stream.write(name + "\\n") - raise ModuleNotFoundError( - f"No module named {{name!r}} (refused: a lone openDox has no " - f"sibling)", name=name) - - -sys.meta_path.insert(0, _RefuseTheSiblings()) -''' - # --------------------------------------------------------------------------- # helpers # --------------------------------------------------------------------------- -def _git(root: Path, *args: str) -> None: - """`git` in `root` as F5.3's fixture identity, with no inherited `GIT_*` - variable and no user or system configuration.""" - env = {k: v for k, v in os.environ.items() if not k.startswith("GIT_")} - env.update({ - "GIT_AUTHOR_NAME": "fixture", "GIT_AUTHOR_EMAIL": "fixture@example.invalid", - "GIT_COMMITTER_NAME": "fixture", - "GIT_COMMITTER_EMAIL": "fixture@example.invalid", - "GIT_CONFIG_GLOBAL": os.devnull, "GIT_CONFIG_SYSTEM": os.devnull, - }) - subprocess.run(["git", "-C", str(root), *args], check=True, - capture_output=True, env=env) +def _fresh_repository(tmp_path: Path, *, edits: dict[str, str] | None = None) -> Path: + """F5.3's preamble over T050's fixture (`standalone_child.fresh_repository`).""" + return fresh_repository(PLAIN_DOCUMENTS, tmp_path, edits=edits) -def _fresh_repository(tmp_path: Path, *, edits: dict[str, str] | None = None) -> Path: - """F5.3's preamble: T050's fixture copied into a FRESH repository, with - `edits` (a document's new text, by name) applied before the commit.""" - root = tmp_path / "plain-documents" - shutil.copytree(PLAIN_DOCUMENTS, root) - for name, text in (edits or {}).items(): - (root / name).write_text(text, encoding="utf-8") - _git(root, "-c", "init.defaultBranch=main", "init", "-q") - _git(root, "add", "-A") - _git(root, "commit", "-qm", "fixture") - return root - - -class _Child: - """One `python -m ...` child with the siblings refused, its - standard output a buffered pipe.""" - - def __init__(self, tmp_path: Path, module: str, *args: str) -> None: - blocker = tmp_path / "sibling-blocker" - blocker.mkdir(exist_ok=True) - (blocker / "sitecustomize.py").write_text(_BLOCKER, encoding="utf-8") - self.refused_log = tmp_path / "refused-imports.log" - env = dict(os.environ) - env.pop("PYTHONUNBUFFERED", None) - env["PYTHONPATH"] = os.pathsep.join( - [str(blocker), *filter(None, [env.get("PYTHONPATH")])]) - env["OPENDOX_T056_REFUSED"] = str(self.refused_log) - self.argv = [sys.executable, "-m", module, *args] - verb = args[0] if args and not args[0].startswith("-") else "" - self.label = f"python -m {module} {verb}".strip() - self.process = subprocess.Popen( - self.argv, cwd=ROOT, env=env, stdout=subprocess.PIPE, - stderr=subprocess.PIPE, text=True, encoding="utf-8", - errors="replace") - self.lines: queue.Queue[str | None] = queue.Queue() - self.stdout: list[str] = [] - self.stderr: list[str] = [] - self._pumps = [ - threading.Thread(target=self._pump, args=(self.process.stdout, True), - daemon=True), - threading.Thread(target=self._pump, args=(self.process.stderr, False), - daemon=True), - ] - for pump in self._pumps: - pump.start() - - def _pump(self, stream, is_stdout: bool) -> None: - for line in stream: - (self.stdout if is_stdout else self.stderr).append(line) - if is_stdout: - self.lines.put(line) - if is_stdout: - self.lines.put(None) - - def wait_for_line(self, pattern: re.Pattern[str]) -> re.Match[str]: - """The first standard-output line matching `pattern`, read while the - child runs. Fails, naming what the child said, if it exits first or - the deadline passes.""" - deadline = time.monotonic() + START_DEADLINE_SECONDS - while time.monotonic() < deadline: - try: - line = self.lines.get(timeout=0.25) - except queue.Empty: - continue - if line is None: - break - match = pattern.search(line) - if match: - return match - raise AssertionError( - f"{self.label} never printed a line matching " - f"{pattern.pattern!r} on its (buffered) standard output while it " - f"ran: exit status {self.process.poll()}, standard output " - f"{''.join(self.stdout)!r}, standard error {self.stderr_text()[-2000:]!r}") - - def interrupt(self) -> int: - """Send SIGINT, as Ctrl-C would, and answer the exit status.""" - self.process.send_signal(signal.SIGINT) - try: - return self.process.wait(timeout=STOP_DEADLINE_SECONDS) - finally: - for pump in self._pumps: - pump.join(timeout=STOP_DEADLINE_SECONDS) - - def kill(self) -> None: - if self.process.poll() is None: - self.process.kill() - self.process.wait(timeout=STOP_DEADLINE_SECONDS) - - def stderr_text(self) -> str: - return "".join(self.stderr) - - def refused(self) -> list[str]: - if not self.refused_log.exists(): - return [] - return self.refused_log.read_text(encoding="utf-8").split() - - -def _run(tmp_path: Path, module: str, *args: str) -> tuple[_Child, int]: - """A child run to completion.""" - child = _Child(tmp_path, module, *args) - try: - status = child.process.wait(timeout=300) - finally: - child.kill() - for pump in child._pumps: - pump.join(timeout=STOP_DEADLINE_SECONDS) - return child, status +def _run(tmp_path: Path, module: str, *args: str) -> tuple[Child, int]: + return run_module(tmp_path, module, *args) def _get(base: tuple[str, int], path: str) -> tuple[int, str, bytes]: @@ -405,7 +241,7 @@ def test_generate_and_open_starts_a_server_that_answers_with_no_sibling(tmp_path the core routes, and stops on an interrupt with status 0.""" repo = _fresh_repository(tmp_path) run_dir = tmp_path / "run" - child = _Child(tmp_path, "opendox.cli", "generate-and-open", + child = Child(tmp_path, "opendox.cli", "generate-and-open", "--repo-root", str(repo), "--repository", "fixture", "--no-open", "--port", "0", "--run-dir", str(run_dir)) try: @@ -434,7 +270,7 @@ def test_serve_main_starts_a_server_that_answers_with_no_sibling(tmp_path) -> No "--repo-root", str(repo), "--repository", "fixture", "--output", str(out), "--no-validate") assert status == 0, generated.stderr_text() - child = _Child(tmp_path, "opendox.serve", "--snapshot", str(out), + child = Child(tmp_path, "opendox.serve", "--snapshot", str(out), "--checkout-root", str(repo), "--port", "0") try: match = child.wait_for_line(_SERVE_URL) From 50b0d42b7c2ea89e73511dd65bc39fd9254ffa32 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Wed, 30 Sep 2026 00:11:29 +0000 Subject: [PATCH 31/38] T054: one emptiness rule in both vocabularies; the commit date uses the corpus's git Copilot at 03e06ccd raised two threads, and both are taken. r4139523226 (local_git_adapter.py): the suffix vocabulary counts a field given with nothing after its colon as missing, but the kind_field path counted it present by its key alone. _classify_by_header now uses the same test, `not header.get(field)`, so a required field means the same whichever vocabulary a corpus is built with. r4139523258 (default_generator.py): _commit_date ran a bare `git` from PATH, while the home corpus may run another. LocalGitCorpus now exposes its `executable`, and generate() reads the commit date with the home corpus's own git where it is a LocalGitCorpus, and with `git` otherwise. Tests, in tests/test_neutral_projection.py: - test_an_empty_field_is_missing_whichever_vocabulary_is_in_force runs both vocabularies over one repository. It fails against 03e06ccd's adapter. - test_the_commit_date_is_read_by_the_home_corpus_s_own_git registers a home corpus whose git is a logging wrapper. It fails against 03e06ccd's generator. The whole suite: selected=2620 passed=2609 skipped=11, which is +2. F4.1's scan is unchanged, at 20. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/default_generator.py | 17 ++++++--- src/opendox/runtime/local_git_adapter.py | 12 ++++++- tests/test_neutral_projection.py | 44 ++++++++++++++++++++++++ 3 files changed, 67 insertions(+), 6 deletions(-) diff --git a/src/opendox/default_generator.py b/src/opendox/default_generator.py index 1c81476..d99cb69 100644 --- a/src/opendox/default_generator.py +++ b/src/opendox/default_generator.py @@ -70,7 +70,7 @@ from typing import Any from opendox import corpus_adapter, generator_seam, neutral_projection -from opendox.runtime.local_git_adapter import GitCommandFailed, GitRunner +from opendox.runtime.local_git_adapter import GitCommandFailed, GitRunner, LocalGitCorpus __all__ = ["GENERATOR", "generate"] @@ -86,13 +86,17 @@ r"(?:Z|[+-][0-9]{2}:[0-9]{2})") -def _commit_date(location: str, revision: str) -> str | None: +def _commit_date(location: str, revision: str, *, + executable: str = "git") -> str | None: """The committer date of `revision` in the checkout at `location`, or - None where `git` cannot answer one.""" + None where `git` cannot answer one. `executable` is the git to run: the + home corpus's own where it is a `LocalGitCorpus`, so the date is read by + the git that resolved the revision (Copilot at openDox-code#57 03e06ccd, + r4139523258).""" if not _OBJECT_ID.fullmatch(revision): return None try: - stamp = GitRunner(Path(location)).out( + stamp = GitRunner(Path(location), executable).out( "show", "-s", "--format=%cI", f"{revision}^{{commit}}", "--") except (GitCommandFailed, OSError): return None @@ -109,7 +113,10 @@ def generate(repo_root: Path, repository: str, *, corpus = adapter.resolve(ref) anchor = source_revision if source_revision is not None else corpus.revision if generated_at is None and anchor is not None: - generated_at = _commit_date(corpus.location, anchor) + generated_at = _commit_date( + corpus.location, anchor, + executable=(adapter.executable if isinstance(adapter, LocalGitCorpus) + else "git")) projection = neutral_projection.project( adapter, corpus, repository, source_revision=anchor, generated_at=generated_at) diff --git a/src/opendox/runtime/local_git_adapter.py b/src/opendox/runtime/local_git_adapter.py index a2ad764..8383640 100644 --- a/src/opendox/runtime/local_git_adapter.py +++ b/src/opendox/runtime/local_git_adapter.py @@ -1615,6 +1615,13 @@ def __init__(self, *, executable: str = "git", self._kind_field = kind_field self._required_fields = tuple(required_fields) + @property + def executable(self) -> str: + """The `git` this corpus runs, for a caller that reads the same + checkout with its own `GitRunner` (`default_generator`'s commit date), + so both run one git.""" + return self._executable + # -- resolve ---------------------------------------------------------- def resolve(self, ref: CorpusRef) -> ResolvedCorpus: @@ -1999,10 +2006,13 @@ def _classify_by_header(self, corpus: ResolvedCorpus, unclassifiable=( f"{document.key!r} carries no {self._kind_field!r} header; " "it is still listed and still readable")) + # THE SAME EMPTINESS RULE AS THE SUFFIX PATH (Copilot at + # openDox-code#57 03e06ccd, r4139523226): a field given with nothing + # after its colon is no field, whichever vocabulary is in force. return Classification( id=document, kind=kind, required_fields=self._required_fields, missing_fields=tuple(field for field in self._required_fields - if field not in header)) + if not header.get(field))) def _header_of(self, corpus: ResolvedCorpus, document: DocumentId) -> dict[str, str]: diff --git a/tests/test_neutral_projection.py b/tests/test_neutral_projection.py index ff68e31..e4073e7 100644 --- a/tests/test_neutral_projection.py +++ b/tests/test_neutral_projection.py @@ -868,6 +868,50 @@ def test_the_default_adapter_obliges_the_small_neutral_field_set(tmp_path: Path) assert _by_path(_generate(root))["none.md"]["stage"] == "source" +def test_an_empty_field_is_missing_whichever_vocabulary_is_in_force( + tmp_path: Path) -> None: + """The suffix vocabulary and the `kind_field` one read a field given with + nothing after its colon the same way: as no field. The `kind_field` path + counted it present by its key alone (Copilot at openDox-code#57 + 03e06ccd, r4139523226).""" + root = _repository(tmp_path, files={ + "both.md": "kind: note\ntitle: T\nsummary: S\n", + "empty.md": "kind: note\ntitle:\nsummary: S\n", + "none.md": "kind: note\n"}) + ref = corpus_adapter.CorpusRef(name="home", location=str(root)) + by_suffix = lga.LocalGitCorpus(required_fields=lga.NEUTRAL_FIELDS) + by_header = lga.LocalGitCorpus(kind_field="kind", + required_fields=lga.NEUTRAL_FIELDS) + for adapter in (by_suffix, by_header): + corpus = adapter.resolve(ref) + missing = {d.key: adapter.classify(corpus, d).missing_fields + for d in adapter.list_documents(corpus)} + assert missing == {"both.md": (), "empty.md": ("title",), + "none.md": ("title", "summary")}, adapter._kind_field + + +def test_the_commit_date_is_read_by_the_home_corpus_s_own_git(tmp_path: Path) -> None: + """`generated_at` is read with the git the home corpus runs, not a bare + `git` from PATH, so the date and the revision it stamps come from one + git (Copilot at openDox-code#57 03e06ccd, r4139523258).""" + from opendox import default_generator + + root = _repository(tmp_path, files={"a.md": "title: A\nsummary: S\n"}) + real = shutil.which("git") + log = tmp_path / "calls.log" + wrapper = tmp_path / "logging-git" + wrapper.write_text(f'#!/bin/sh\necho "$@" >> "{log}"\nexec "{real}" "$@"\n', + encoding="utf-8") + wrapper.chmod(0o755) + corpus_adapter.register_home(lambda location: ( + lga.WorkingTreeCorpus(executable=str(wrapper)), + corpus_adapter.CorpusRef(name="home", location=str(location)))) + snapshot = default_generator.generate(root, "garden") + assert snapshot["generation"].get("generated_at"), "a date was read" + assert "--format=%cI" in log.read_text(encoding="utf-8"), ( + "the commit date was read by the home corpus's own git") + + def test_required_header_fields_answers_the_neutral_set_through_the_entry_point() -> None: from opendox import cli From d7954fc604dc5fd46ef7fc0c995220a59a3a5a23 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Wed, 30 Sep 2026 00:22:56 +0000 Subject: [PATCH 32/38] T054: the projection reads keys as paths only where the adapter declares so Copilot at 50b0d42b (r4139607560) is right. DocumentId.key is opaque to the CorpusAdapter interface: an implementation may make it an object id or a row key. The neutral projection read every key as a repository path and every document's leading block as its header, so a valid non-git home adapter would have its keys written into the snapshot as paths. The projection now names the requirement. An adapter declares `document_keys_are_paths = True` (neutral_projection.PATH_KEYS) for its keys to be read as repository paths and its documents' leading `Name: value` block as their header. LocalGitCorpus declares it, so WorkingTreeCorpus and the conformance corpus do too. project() refuses any other adapter with ProjectionRefused, naming the declaration, before it reads a document. The same review's r4139607585 does not reproduce: a path reaches the invalid-stage notice only after _unwritable() has passed it, and that refuses any control character (U+0000 to U+001F, U+007F to U+009F). Such a path is named by its repr in the "left out" notice and never reaches the stage check. The regression case is test_an_adapter_whose_keys_are_not_declared_paths_is_refused. It fails against 50b0d42b's projection. The whole suite: selected=2621 passed=2610 skipped=11, which is +1. F4.1's scan is unchanged. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/neutral_projection.py | 18 +++++++++++++ src/opendox/runtime/local_git_adapter.py | 7 +++++ tests/test_neutral_projection.py | 33 ++++++++++++++++++++++++ 3 files changed, 58 insertions(+) diff --git a/src/opendox/neutral_projection.py b/src/opendox/neutral_projection.py index d0addd8..1d9e1ed 100644 --- a/src/opendox/neutral_projection.py +++ b/src/opendox/neutral_projection.py @@ -94,6 +94,7 @@ __all__ = [ "GENERATOR_VERSION", "Notice", + "PATH_KEYS", "Projection", "ProjectionRefused", "SCHEMA_VERSION", @@ -162,6 +163,13 @@ _CONTROL = re.compile("[\u0000-\u001f\u007f-\u009f]") +#: What an adapter declares, as a class or instance attribute set to True, +#: for this projection to read its `DocumentId.key` as a repository path and +#: its documents' leading `Name: value` block as their header. `LocalGitCorpus` +#: declares it. An adapter that does not is refused, not guessed at. +PATH_KEYS = "document_keys_are_paths" + + class ProjectionRefused(generator_seam.GeneratorSeamError): """The projection cannot write a snapshot the neutral contract admits. @@ -420,6 +428,16 @@ def project(adapter: CorpusAdapter, corpus: ResolvedCorpus, repository: str, the projection refuses, since the contract requires an anchor. `generated_at`, where given, is recorded verbatim, and it is otherwise absent. `repository` is recorded as given.""" + if getattr(adapter, PATH_KEYS, False) is not True: + raise ProjectionRefused( + f"the neutral snapshot records each document by its repository " + f"path, and {type(adapter).__name__} does not declare " + f"`{PATH_KEYS} = True`. A `DocumentId.key` is opaque to the " + "CorpusAdapter interface: it may be an object id or a row key, and " + "read as a path it would fabricate one. Register a home corpus " + "whose keys are repository paths and whose documents open with a " + "`Name: value` header, and have it declare so (Copilot at " + "openDox-code#57 50b0d42b, r4139607560).") revision = source_revision if source_revision is not None else corpus.revision if revision is None: raise ProjectionRefused( diff --git a/src/opendox/runtime/local_git_adapter.py b/src/opendox/runtime/local_git_adapter.py index 8383640..54472ef 100644 --- a/src/opendox/runtime/local_git_adapter.py +++ b/src/opendox/runtime/local_git_adapter.py @@ -1606,6 +1606,13 @@ class LocalGitCorpus: #: (Copilot review of openDox-code#26). A parameter that cannot change an #: outcome is removed rather than wired up, because wiring it up would #: reintroduce the assumption `_served_ref` exists to refuse. + #: THIS ADAPTER'S KEYS ARE REPOSITORY PATHS, and its documents open with a + #: leading `Name: value` header. `DocumentId.key` is opaque to the + #: `CorpusAdapter` interface, so a consumer that needs a path, openDox's + #: neutral projection, reads keys as paths only where the adapter says so + #: (`neutral_projection.PATH_KEYS`). + document_keys_are_paths = True + def __init__(self, *, executable: str = "git", write_path: str | None = WRITE_PATH, kind_field: str | None = None, diff --git a/tests/test_neutral_projection.py b/tests/test_neutral_projection.py index e4073e7..84ce709 100644 --- a/tests/test_neutral_projection.py +++ b/tests/test_neutral_projection.py @@ -835,6 +835,39 @@ def test_the_anchors_come_from_the_source_revision_and_the_bytes_repeat( assert "generated_at" not in unknown +def test_an_adapter_whose_keys_are_not_declared_paths_is_refused(tmp_path: Path) -> None: + """`DocumentId.key` is opaque to the `CorpusAdapter` interface, so the + projection reads keys as repository paths only for an adapter that + declares `PATH_KEYS`. One that does not is refused, naming the + declaration, rather than having an object id or a row key written as a + path (Copilot at openDox-code#57 50b0d42b, r4139607560).""" + root = _repository(tmp_path, files={"a.md": "title: A\nsummary: S\n"}) + assert projection.PATH_KEYS == "document_keys_are_paths" + assert lga.LocalGitCorpus.document_keys_are_paths is True + + class _Opaque: + """Delegates everything to a path adapter, and declares nothing.""" + def __init__(self, inner): + self._inner = inner + + def __getattr__(self, name): + if name == projection.PATH_KEYS: + raise AttributeError(name) + return getattr(self._inner, name) + + inner = lga.WorkingTreeCorpus() + corpus = inner.resolve(corpus_adapter.CorpusRef(name="home", location=str(root))) + with pytest.raises(projection.ProjectionRefused) as caught: + projection.project(_Opaque(inner), corpus, "garden") + assert "document_keys_are_paths = True" in str(caught.value) + assert isinstance(caught.value, gs.GeneratorSeamError) + declared = _Opaque(inner) + declared.document_keys_are_paths = True + paths = [d["path"] for d in + projection.project(declared, corpus, "garden").snapshot["documents"]] + assert paths == ["a.md"] + + def test_a_repository_with_no_commit_needs_a_pinned_revision(tmp_path: Path) -> None: root = _repository(tmp_path, files={"a.md": "# A\n"}, commit=False) with pytest.raises(projection.ProjectionRefused) as caught: From 42a08a31d84090ba077facb71837d0950940defd Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Wed, 30 Sep 2026 00:35:10 +0000 Subject: [PATCH 33/38] T056: an interrupted child that will not stop is killed before its pipes are joined Copilot at openDox-code#66 1597511d (r4139607689). Child.interrupt() waited STOP_DEADLINE_SECONDS for the child, then, in its finally, joined both pipe readers for up to another deadline each. They block until the pipes close, and nothing had killed the child yet, so a server that ignored Ctrl-C held the caller for three deadlines, on exactly the path the helper exists to report. interrupt() now kills the child before joining, as wait() already did, and the TimeoutExpired is still raised. test_a_child_that_ignores_the_interrupt_is_killed_at_the_deadline runs a module that ignores SIGINT, with the deadline patched to 1 s. It fails without the fix ("the child outlived its deadline") and passes with it. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/standalone_child.py | 10 ++++++- tests/test_standalone_generate_path.py | 39 ++++++++++++++++++++++++++ 2 files changed, 48 insertions(+), 1 deletion(-) diff --git a/tests/standalone_child.py b/tests/standalone_child.py index 0398d1e..5362f2a 100644 --- a/tests/standalone_child.py +++ b/tests/standalone_child.py @@ -189,11 +189,19 @@ def wait(self) -> int: self._join() def interrupt(self) -> int: - """Send SIGINT, as Ctrl-C would, and answer the exit status.""" + """Send SIGINT, as Ctrl-C would, and answer the exit status. + + A child that does not stop within `STOP_DEADLINE_SECONDS` is KILLED + before its pipes are joined, and the timeout is raised (Copilot at + openDox-code#66 1597511d, r4139607689). Its readers block until the + pipes close, so joining first would hold the caller for two more + deadlines on exactly the path, an interrupt that is ignored, this + helper exists to report.""" self.process.send_signal(signal.SIGINT) try: return self.process.wait(timeout=STOP_DEADLINE_SECONDS) finally: + self.kill() self._join() def kill(self) -> None: diff --git a/tests/test_standalone_generate_path.py b/tests/test_standalone_generate_path.py index eb47fcb..64ee976 100644 --- a/tests/test_standalone_generate_path.py +++ b/tests/test_standalone_generate_path.py @@ -22,6 +22,9 @@ refuses `/source/.git/config`, and stops on an interrupt with status 0. 4. `python -m opendox.serve`, the server's own entry point, starts and answers the same way. +5. The harness itself: a child that ignores the interrupt is killed at the + deadline, and the timeout is raised, so a server that will not stop is + reported rather than waited out. HOW "NEITHER SIBLING IS IMPORTABLE" IS MADE TRUE. Each run is a real child process, `python -m ...`, built by `tests/standalone_child.py`. The child's @@ -52,8 +55,14 @@ import json import re import socket +import subprocess +import textwrap +import time from pathlib import Path +import pytest + +import standalone_child from standalone_child import Child, fresh_repository, run_module ROOT = Path(__file__).resolve().parent.parent @@ -281,3 +290,33 @@ def test_serve_main_starts_a_server_that_answers_with_no_sibling(tmp_path) -> No child.kill() _assert_the_port_is_closed(base) assert child.refused() == [], child.refused() + + +# --------------------------------------------------------------------------- +# 5 — the harness itself: an ignored interrupt is reported, not waited out +# --------------------------------------------------------------------------- + +def test_a_child_that_ignores_the_interrupt_is_killed_at_the_deadline( + tmp_path, monkeypatch) -> None: + """The regression path cases 3 and 4 guard: a server that does not stop + on Ctrl-C. `Child.interrupt()` raises at its deadline with the child + already killed, so the caller is not held while its pipes drain.""" + monkeypatch.setattr(standalone_child, "STOP_DEADLINE_SECONDS", 1.0) + blocker = tmp_path / "sibling-blocker" + blocker.mkdir() + (blocker / "t056_ignores_sigint.py").write_text(textwrap.dedent(""" + import signal, time + signal.signal(signal.SIGINT, signal.SIG_IGN) + print("ready", flush=True) + time.sleep(600) + """), encoding="utf-8") + child = Child(tmp_path, "t056_ignores_sigint") + try: + child.wait_for_line(re.compile(r"^ready$")) + started = time.monotonic() + with pytest.raises(subprocess.TimeoutExpired): + child.interrupt() + assert child.process.poll() is not None, "the child outlived its deadline" + assert time.monotonic() - started < 10 + finally: + child.kill() From e939c31ff8e5ed4adb08ac07c31f34ab5e0462f8 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Wed, 30 Sep 2026 01:05:51 +0000 Subject: [PATCH 34/38] T056: the undeclared stage: value is looked for inside every string, not only as one Copilot at openDox-code#66 5a26532e (r4139809410). The stage: case asserted that no string value EQUALS "someday". A snapshot carrying the value inside a larger string, such as "stage: someday", would still have passed, though the case says the undeclared value appears nowhere in the snapshot. It now fails on any string value that contains it, and names that string. Checked with a mutant that sets the document's summary to "stage: someday": it passes the old equality check and fails the new one. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/test_standalone_generate_path.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/test_standalone_generate_path.py b/tests/test_standalone_generate_path.py index 64ee976..58fd505 100644 --- a/tests/test_standalone_generate_path.py +++ b/tests/test_standalone_generate_path.py @@ -219,7 +219,9 @@ def test_the_verb_reports_a_stage_outside_the_six_and_reads_it_as_a_source( snapshot = json.loads(out.read_text(encoding="utf-8")) [entry] = [d for d in snapshot["documents"] if d["path"] == document] assert entry["stage"] == "source" - assert "someday" not in set(_string_values(snapshot)) + carried = [text for text in _string_values(snapshot) if "someday" in text] + assert carried == [], ( + f"the undeclared value reached the snapshot, inside {carried}") assert {d["stage"] for d in snapshot["documents"]} <= set(ROLE_KEYS) From 183b40fc5a87d2bd2332014a8c06fcd124678772 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Wed, 30 Sep 2026 15:21:23 +0000 Subject: [PATCH 35/38] T055: a regenerate passes the project register only when one is set Copilot at c2a8ad9f (r4146219833) is right. SnapshotSource._regenerate always passed project_register_source to the generator, even when no register was configured. So an injected generator that takes only the core arguments raised TypeError before a refresh could write. The input is now passed only when it is set, as the generator seam omits an unset input. A generator given a register still receives it. The regression case is test_an_injected_generator_is_handed_the_register_only_when_one_is_set. It fails against c2a8ad9f's registry. The whole suite: selected=2988 passed=2977 skipped=11, which is +1. Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- src/opendox/default_registry.py | 9 +++++++-- tests/test_projection_seams.py | 29 +++++++++++++++++++++++++++++ 2 files changed, 36 insertions(+), 2 deletions(-) diff --git a/src/opendox/default_registry.py b/src/opendox/default_registry.py index 80bc49b..1059076 100644 --- a/src/opendox/default_registry.py +++ b/src/opendox/default_registry.py @@ -622,8 +622,13 @@ def _regenerate(self, *, repository: str | None, ref: str | None) -> dict: if root is None or not Path(root).is_dir(): raise ValueError(f"{entry.key_id}: no served checkout to regenerate from") generate = self._generator or generator_seam.generate - snapshot = generate(Path(root), entry.repository, - project_register_source=self.project_register) + # AN UNSET INPUT IS NOT PASSED (Copilot at openDox-code#59 c2a8ad9f, + # r4146219833), as the generator seam omits one. An injected + # generator that takes only the core arguments then regenerates, and + # one given a register still receives it. + inputs = ({} if self.project_register is None else + {"project_register_source": self.project_register}) + snapshot = generate(Path(root), entry.repository, **inputs) target = Path(entry.snapshot_path) boundary = OutputBoundary(target.parent, [target.name]) projection_seams.writer.current().write_snapshot(snapshot, target, boundary) diff --git a/tests/test_projection_seams.py b/tests/test_projection_seams.py index 24e8e96..2f4e932 100644 --- a/tests/test_projection_seams.py +++ b/tests/test_projection_seams.py @@ -854,6 +854,35 @@ def generator(root, repository, *, project_register_source=None): {"source_revision": "s1"} +def test_an_injected_generator_is_handed_the_register_only_when_one_is_set(tmp_path) -> None: + """An unset project register is not passed to an injected generator, as + the generator seam omits an unset input, so one that takes only the core + arguments regenerates. A set register is still handed over (Copilot at + openDox-code#59 c2a8ad9f, r4146219833).""" + ps.register_defaults() + calls = [] + + def core_only(root, repository): + calls.append(("core", repository)) + return {"schema_version": 1, "kind": NEUTRAL, "repository": repository, + "generation": {"source_revision": "s1"}} + + def with_register(root, repository, *, project_register_source): + calls.append(("register", project_register_source)) + return core_only(root, repository) + + register = tmp_path / "register.yaml" + for generator, project_register in ((core_only, None), (with_register, register)): + source = default_registry.SnapshotSource( + checkout_root=tmp_path, generator=generator, + project_register=project_register) + source.registry.register(default_registry.SnapshotEntry( + "garden", snapshot_path=tmp_path / "main.json", source_root=tmp_path)) + source.refresh(repository="garden") + assert calls == [("core", "garden"), ("register", register), + ("core", "garden")] + + # --------------------------------------------------------------------------- # 5b — the core `/snapshot.json` arm, answered from the registered source # --------------------------------------------------------------------------- From caa01caf4a47b4caca50f19cfaf0ef5bbc45b839 Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Wed, 30 Sep 2026 15:38:04 +0000 Subject: [PATCH 36/38] T056: the "serving until interrupted" line is read while the server runs Copilot at openDox-code#66 e3574774 (r4146289331). The generate-and-open case looked for "serving until interrupted" only after the interrupt. By then Python's exit flush delivers an unflushed line anyway, so dropping that line's flush=True (cli.py) would still have passed. The case now waits for the line while the child runs, before any request and before the interrupt. Checked with the mutant Copilot names (the URL line flushed, the serving line not): the case now fails, "never printed a line matching ... serving until interrupted". Arc: neutral-product-standalone-operability Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/test_standalone_generate_path.py | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/tests/test_standalone_generate_path.py b/tests/test_standalone_generate_path.py index 58fd505..8eaab14 100644 --- a/tests/test_standalone_generate_path.py +++ b/tests/test_standalone_generate_path.py @@ -157,6 +157,7 @@ def _assert_the_port_is_closed(base: tuple[str, int]) -> None: _URL = re.compile(r"^(http://([0-9.]+):([0-9]+))/index\.html$") +_SERVING = re.compile(r"^ serving until interrupted \(Ctrl-C to stop\)$") _SERVE_URL = re.compile(r"^serving ideation dashboard at " r"(http://([0-9.]+):([0-9]+))/index\.html$") @@ -249,7 +250,13 @@ def test_the_unedited_fixture_declares_that_document_a_candidate(tmp_path) -> No def test_generate_and_open_starts_a_server_that_answers_with_no_sibling(tmp_path) -> None: """`python -m opendox.cli generate-and-open --no-open`, with no `--no-serve`: the server starts, says where on a buffered pipe, answers - the core routes, and stops on an interrupt with status 0.""" + the core routes, and stops on an interrupt with status 0. + + Both lines it prints before blocking in `serve_forever()`, the URL and + "serving until interrupted", are read WHILE IT RUNS, before the + interrupt: after it, Python's exit flush would deliver an unflushed line + anyway, and the case would not tell (Copilot at openDox-code#66 + e3574774, r4146289331).""" repo = _fresh_repository(tmp_path) run_dir = tmp_path / "run" child = Child(tmp_path, "opendox.cli", "generate-and-open", @@ -259,13 +266,14 @@ def test_generate_and_open_starts_a_server_that_answers_with_no_sibling(tmp_path match = child.wait_for_line(_URL) base = (match.group(2), int(match.group(3))) assert child.process.poll() is None, "the server exited after printing its URL" + child.wait_for_line(_SERVING) + assert child.process.poll() is None, "the server exited after saying it serves" _assert_the_server_answers(base, run_dir / "snapshot.json", repo) assert child.interrupt() == 0, child.stderr_text() finally: child.kill() _assert_the_port_is_closed(base) assert child.refused() == [], child.refused() - assert "serving until interrupted" in "".join(child.stdout) # --------------------------------------------------------------------------- From 149d729567c0008869f7ca697f655adf22a168aa Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Wed, 30 Sep 2026 17:57:24 +0000 Subject: [PATCH 37/38] T056: the harness gives the child Ctrl-C as at a terminal, whatever the runner ignores Measured at a6e953ce (this branch with main merged): the whole suite, run as `nohup pytest ... &`, failed cases 3 and 4 of tests/test_standalone_generate_path.py with a TimeoutExpired at the interrupt. The same module passed 6 of 6 in the foreground. The cause is the runner, not the server. POSIX starts an asynchronous command with SIGINT ignored when job control is off (a probe under `nohup ... &` reads signal.getsignal(SIGINT) == 1, SIG_IGN). An ignored signal survives exec, and Python installs its KeyboardInterrupt handler only where SIGINT was not ignored. So every server the harness started ignored the interrupt, and both cases reported how the suite was launched. The estate runs long suites exactly this way, so any lane that runs this module under nohup would read a false red. The fix is in the harness, not the product. A program started with SIGINT ignored should keep it ignored. tests/standalone_child.py's sitecustomize now sets SIGINT back to signal.default_int_handler, which models the case's claim: a Ctrl-C at a terminal. A child that ignores SIGINT itself, after startup, still does, and case 5 still kills it at the deadline. Falsifier: test_a_child_stops_on_the_interrupt_even_when_the_runner_ignores_it builds that runner in process. It ignores SIGINT while the child starts and restores it at once, then requires the child to stop on the interrupt with exit 0 and the parent's handler to be back. Red before the fix (TimeoutExpired at 10 s, in a foreground run). Green after. The module under `nohup ... &` went from 2 failed / 4 passed to 7 passed. Mutant M14 (the reset removed) is killed by the new case, and mutants are 14/14 killed. Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/standalone_child.py | 13 +++++++ tests/test_standalone_generate_path.py | 48 +++++++++++++++++++++++++- 2 files changed, 60 insertions(+), 1 deletion(-) diff --git a/tests/standalone_child.py b/tests/standalone_child.py index 5362f2a..37d1fe5 100644 --- a/tests/standalone_child.py +++ b/tests/standalone_child.py @@ -20,6 +20,14 @@ * It is read on threads while it runs (`Child.wait_for_line`), so a server that never exits can still be asked where it serves, and then interrupted (`Child.interrupt`, SIGINT, as Ctrl-C sends). +* CTRL-C REACHES IT AS IT WOULD AT A TERMINAL, whatever the runner's own + disposition. The same `sitecustomize` sets SIGINT back to Python's + KeyboardInterrupt handler. A runner started as a background job + (`nohup pytest ... &`) has SIGINT ignored, and every child would inherit + that, so an interrupted server would time out and the case would report the + runner, not the server (measured at openDox-code#66 a6e953ce). A child that + ignores SIGINT ITSELF, after startup, still does, and is killed at the + deadline. `fresh_repository()` is #1144's preamble: a fixture copied into a FRESH git repository and committed as the fixture's own identity. @@ -61,8 +69,13 @@ #: The `sitecustomize` every child loads. _BLOCKER = f'''\ import os +import signal import sys +# Ctrl-C as at a terminal: a runner started as a background job ignores +# SIGINT, and an ignored signal is inherited across exec (tests/standalone_child.py). +signal.signal(signal.SIGINT, signal.default_int_handler) + _SIBLINGS = {SIBLINGS!r} for _name in list(sys.modules): diff --git a/tests/test_standalone_generate_path.py b/tests/test_standalone_generate_path.py index 8eaab14..79a554d 100644 --- a/tests/test_standalone_generate_path.py +++ b/tests/test_standalone_generate_path.py @@ -24,7 +24,10 @@ the same way. 5. The harness itself: a child that ignores the interrupt is killed at the deadline, and the timeout is raised, so a server that will not stop is - reported rather than waited out. + reported rather than waited out. And a child stops on the interrupt even + when the RUNNER ignores SIGINT, as a suite started as a background job + (`nohup ... &`) does, so cases 3 and 4 test the server, not how the suite + was launched. HOW "NEITHER SIBLING IS IMPORTABLE" IS MADE TRUE. Each run is a real child process, `python -m ...`, built by `tests/standalone_child.py`. The child's @@ -54,6 +57,7 @@ import http.client import json import re +import signal import socket import subprocess import textwrap @@ -330,3 +334,45 @@ def test_a_child_that_ignores_the_interrupt_is_killed_at_the_deadline( assert time.monotonic() - started < 10 finally: child.kill() + + +def test_a_child_stops_on_the_interrupt_even_when_the_runner_ignores_it( + tmp_path, monkeypatch) -> None: + """A runner started as a background job ignores SIGINT: POSIX starts an + asynchronous command with SIGINT and SIGQUIT ignored when job control is + off, and `nohup ... &` from a script is such a command. An ignored signal + stays ignored across `exec`, and Python installs its KeyboardInterrupt + handler only where SIGINT was not ignored. So every child such a runner + starts would ignore the interrupt, and cases 3 and 4 would time out + having tested nothing about the server. Measured at openDox-code#66 + a6e953ce: both failed that way under `nohup pytest ... &`, and passed + run in the foreground. + + The case builds that runner in process. SIGINT is ignored while the child + is started, and restored at once. The child, a module that sleeps until + interrupted, must still stop on the interrupt and exit 0.""" + monkeypatch.setattr(standalone_child, "STOP_DEADLINE_SECONDS", 10.0) + blocker = tmp_path / "sibling-blocker" + blocker.mkdir() + (blocker / "t056_waits_for_sigint.py").write_text(textwrap.dedent(""" + import time + print("ready", flush=True) + try: + time.sleep(600) + except KeyboardInterrupt: + print("interrupted", flush=True) + raise SystemExit(0) + """), encoding="utf-8") + previous = signal.signal(signal.SIGINT, signal.SIG_IGN) + try: + child = Child(tmp_path, "t056_waits_for_sigint") + finally: + if previous is not None: + signal.signal(signal.SIGINT, previous) + try: + child.wait_for_line(re.compile(r"^ready$")) + assert child.interrupt() == 0, child.stderr_text() + finally: + child.kill() + assert child.stdout_text().splitlines() == ["ready", "interrupted"] + assert signal.getsignal(signal.SIGINT) is previous From a7bda066f0de8e3dc93497a23305cf6666a644cb Mon Sep 17 00:00:00 2001 From: Brett Heap <1513478+brettheap@users.noreply.github.com> Date: Wed, 30 Sep 2026 18:06:49 +0000 Subject: [PATCH 38/38] T056: the stage notice's six keys are checked as one rendered list Copilot at 149d7295 (r4147767447): the case checked each role key as a word anywhere in the notice. `candidate` is in the document's own name, candidate-toolshed-rebuild.md, and `source` is in "read as a source". So a notice whose parenthesized list left both out still passed. Measured: mutant M15, the list rendered without candidate and source, SURVIVED the old case (1 passed). The case now requires the whole list, "(source, grouping, candidate, selection, submission, completion)", as one substring of the notice. M15 is killed (1 failed), and so are M5, M6 and M12, the other stage mutants. The module still passes 7 of 7. Lane: openxfactory-4 (openXfactory-4-openDox_extraction) Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/test_standalone_generate_path.py | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/tests/test_standalone_generate_path.py b/tests/test_standalone_generate_path.py index 79a554d..0a95efc 100644 --- a/tests/test_standalone_generate_path.py +++ b/tests/test_standalone_generate_path.py @@ -218,8 +218,12 @@ def test_the_verb_reports_a_stage_outside_the_six_and_reads_it_as_a_source( notice = notices[0] assert document in notice assert "'someday'" in notice - for key in ROLE_KEYS: - assert re.search(rf"\b{key}\b", notice), (key, notice) + # The six keys as ONE rendered list: `candidate` is in the document's name + # and `source` in "read as a source", so a word-by-word check passed a + # list that left both out (Copilot at openDox-code#66 149d7295, + # r4147767447). + listed = "(" + ", ".join(ROLE_KEYS) + ")" + assert listed in notice, (listed, notice) assert "read as a source" in notice snapshot = json.loads(out.read_text(encoding="utf-8")) [entry] = [d for d in snapshot["documents"] if d["path"] == document]