diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e6cb5c2..025507c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -16,7 +16,8 @@ jobs: with: python-version: "3.x" - run: npm ci - - run: python3 design/tools/check.py + - run: PYTHONDONTWRITEBYTECODE=1 python3 design/tools/check.py + - run: PYTHONDONTWRITEBYTECODE=1 python3 -m unittest design/tools/test_decisions.py - run: npm run typecheck - run: npm run lint - run: npm run format:check diff --git a/AGENTS.md b/AGENTS.md index b27e4bf..c4990db 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -24,7 +24,7 @@ The Convex transactional domain platform. `docs/` holds the inputs. `design/` ho ## The advisor panel and the roadmap - `design/ROADMAP.md` is the aggregated roadmap: the next units in rounds, what each needs first and shows, what is deferred and why. The owner rules on it; the main thread keeps it current at each close. The slices of `design/STATE.md` keep their names, and the roadmap orders the units inside and between them. -- Four advisors, `advisor-convex`, `advisor-domain`, `advisor-operator` and `advisor-product`, are Fable agents defined under `.claude/agents/`. Each is launched with a task file under `design/advisors/`: `task-sort.md` sorts the open decisions of its lens, `task-fork.md` writes one owner decision as a fork, `task-roadmap.md` proposes the milestones of one subject. The four run at once, with disjoint subjects, and the main thread merges or cuts. +- Four advisors, `advisor-convex`, `advisor-domain`, `advisor-operator` and `advisor-product`, are Fable agents defined under `.claude/agents/`. Each is launched with a task file under `design/advisors/`: `task-sort.md` sorts the open decisions of its lens, `task-fork.md` writes one owner decision as a fork, `task-roadmap.md` proposes the milestones of one subject, `task-pass.md` reads one capability's Pack from its lens before the capability is built. The four run at once, with disjoint subjects, and the main thread merges or cuts. - Every lean, fork or memo is checked by a `gpt-6.1-sol` job of the other family before the owner sees it, on a copy that carries one planted false citation. `design/advisors/README.md` says how each run goes and what the first runs showed; `design/advisors/protocol.md` binds every sentence an advisor writes. - The memos, briefs and plants of a run live outside the repository, in the unit folder the project notes name. A check job's report stays in its job directory. diff --git a/CONTEXT.md b/CONTEXT.md index a877d38..af265f0 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -272,7 +272,8 @@ _Avoid_: failed, dead letter, stuck Words for how the platform is specified and proven. They are lasting and may appear in a Spec. They do not name things inside the platform, so no type, table, event or error code is named after them. -- **Layer**: one of the seven steps of capability, each usable before the next exists. +- **Layer**: one of the seven steps of capability, each usable before the next exists. _Avoid_: architectural layer. +- **Architectural layer**: the Protocol's `layer` on a component anchor, one of edge, application, domain, adapter or infrastructure, always written in full; a bare numbered Layer n is a step of capability. - **Profile**: a group of layers an application takes together: transactional, durable, workflow or agent. - **Trigger**: the concrete need that activates a later capability. A possible future need is not one. - **Law**: one of the twelve rules every decision is reviewed against. diff --git a/design/PLAN.md b/design/PLAN.md index 8b69e96..74d1eac 100644 --- a/design/PLAN.md +++ b/design/PLAN.md @@ -92,6 +92,31 @@ Detail levels follow the doc's rule "Detail follows the build" (decision method, Merges and splits against the task's list: idempotency and receipts stay one component because D6 defines both together; the rejection boundary is folded into the outcome model's boundary contract because D4 and D7 describe one mapping; tenancy and authority stay one component (D11); operations are split between Package D (what applies from Layer 1) and Package E (what applies to obligations), following the doc's sentence that security, bounded resource use and useful errors start in Layer 1. +### 2.1 The code's components + +The code declares its architecture where it is realized: twelve `component:` anchors, each with an architectural layer and a bounded context, and the `uses` between them. Recipe 28 reads them from the graph and `design/tools/check.py` prints them. An architectural layer is the Protocol's `layer` on a component anchor, and a numbered Layer of the table above is a step of capability; the two are different things (`CONTEXT.md`). + +| Component | Code | Architectural layer | Context | +|---|---|---|---| +| `component:platform.kernel` | `src/kernel/` | domain | platform | +| `component:platform.context` | `src/context/` | adapter | platform | +| `component:platform.command` | `src/command/` | application | platform | +| `component:platform.read-model` | `src/read-model/` | application | platform | +| `component:platform.gate` | `src/gate/` | infrastructure | platform | +| `component:platform.audit` | `src/audit/` | infrastructure | platform | +| `component:platform.operations` | `src/operations/` | infrastructure | platform | +| `component:example.orders-domain` | `example/domain/order.ts` | domain | orders | +| `component:example.inventory-domain` | `example/domain/stockItem.ts` | domain | inventory | +| `component:example.orders-context` | `example/convex/orders/` | adapter | orders | +| `component:example.inventory-context` | `example/convex/inventory/` | adapter | inventory | +| `component:example.parent` | `example/convex/` | application | orders-inventory | + +The layer rule, from the owner's rulings of 2026-10-10: `domain` is business meaning with no runtime import; `application` coordinates a use case; `adapter` realizes a kernel or command contract against Convex; `edge` is transport in; `infrastructure` is a shared mechanism that owns no use case. The gate supplies a pause and owns no use case, so it is `infrastructure`. The context library and the two context components realize the kernel's contract against Convex, so they are `adapter`. + +The context rule, from the same rulings: `platform` stays on the seven platform components, because the platform is the key bounded context, with its own language (`CONTEXT.md`). The example keeps `orders` and `inventory` on its domain and context components and `orders-inventory` on the parent composition: orders-inventory is the supporting context and development tool, the composition of the two business contexts it models. Nothing in the platform's core carries orders-inventory specifics, and that is read from the graph: `design/tools/check.py` fails when a component in the context `platform` has a `uses` edge to a component in any other context, each end of the edge resolved to its component directly or through `memberOf`. The check reads the declared `uses`, not the imports. + +`example/` is architecture: it is the production composition and the only place a composition and contexts exist as code, so it keeps its five components. `fixture/` has none. + ## 3. SDP mapping ### 3.1 IDs, files, families @@ -231,7 +256,7 @@ Fact slugs: `f01-serializable-mutations-under-occ`, `f02-component-calls-commit- | `spec:application.parent-use-cases` | behavior | feature | defined | refines epic; decidedBy D1, D10; constrainedBy Law 1, 2, 9, F2, F3, F13, `spec:constraints.one-call-per-context-per-use-case`; dependsOn `spec:context.context-component`, `spec:context.batch-shaped-api`, `spec:command.command-pipeline` | Contexts never read each other or call each other; cross-context rules in the parent, same transaction, or in an obligation; no handler issues commands; one receipt and outcome per use case; PlaceOrder is one command; grouping by flow; too-large operations rejected or a separate import command; OQ3 max lines `[non-blocking]`. Example space for L2-1 | | `spec:application.read-models` | behavior | feature | defined | refines epic; decidedBy D8; constrainedBy Law 9, F7, F15, `spec:constraints.zero-core-projection-jobs`; dependsOn `spec:context.queries` | The read-need table; zero projection jobs; projection properties; committed-or-later visibility; versions in results; reactive queries replace event-to-socket layers; CQRS as separate contracts. Example space for L2-2 | | `spec:application.projection-contract` | contract | story | defined | refines read-models | E-9: the projection signature shared by live update and rebuild; row conventions (tenant, generation, key, source version) | -| `spec:application.rebuild` | workflow | feature | defined | refines read-models; decidedBy D9; constrainedBy Law 10, F1, F17 | The four online steps; the missing-row rule for incremental updates; version-compare backfill; verify, switch, keep old; per-entity markers for counts and sums; the write pause for cross-stream history views (gate, fence, resume, abort); never commands or effects. Probe 6 `[non-blocking]`. Example space for L2-5, L2-6 | +| `spec:application.rebuild` | workflow | feature | defined | refines read-models; decidedBy D9; constrainedBy Law 10, F1, F17 | The four online steps; the missing-row rule for incremental updates; version-compare backfill; verify, switch, keep old; per-entity markers for counts and sums; the write pause for cross-stream history views (gate, fence, resume, abort), in `spec:application.history-rebuild`; never commands or effects. Probe 6 `[non-blocking]`. Example space for L2-5, L2-6 | | `spec:application.generation-registry` | contract | story | defined | refines rebuild; constrainedBy F13 | E-8: the generations table and states, the marker document, the migrations-component binding | | `spec:application.write-pause` | contract | story | defined | refines rebuild | E-8: the maintenance gate document, the writer check, fencing of background writers, operator resume and abort | | `spec:application.restore` | workflow | story | defined | refines `spec:operations.baseline-operations`; decidedBy D19; constrainedBy F12 | Restore with matching code and configuration; invariant checks for domain, journal and read models; what the recovery plan contains; restore is tested, not only backup. Example space for L2-8 | @@ -298,7 +323,7 @@ Every row of the doc's acceptance table becomes at least one `example` Spec. Whe | L2-3 | Orders of 1 line, 10 lines and the maximum | `spec:application.first-experiment` | `.order-of-1-line`, `.order-of-10-lines`, `.order-of-max-lines` | native | D | | L2-4 | Rename a typed handler or break its argument contract | `spec:command.command-declaration` | `.renamed-handler-fails-build` | build | C | | L2-5 | Rebuild a per-entity read model online while commands run; interrupt and resume | `spec:application.rebuild` | `.online-rebuild-interrupt-resume` | native | D | -| L2-6 | Rebuild a cross-stream history view under a write pause; interrupt | `spec:application.rebuild` | `.write-pause-rebuild-interrupt` | native | D | +| L2-6 | Rebuild a cross-stream history view under a write pause; interrupt | `spec:application.history-rebuild` | `.write-pause-rebuild-interrupt` | native | D | | L2-7 | Rebuild a stream that has a baseline event | `spec:context.journal` | `.rebuild-from-baseline` | native | B | | L2-8 | Restore a representative dataset with matching code and configuration | `spec:application.restore` | `.restore-representative-dataset` | end to end | D | | L2-9 | Run native acceptance with production configuration | `spec:application.first-experiment` | `.native-acceptance-production-configuration` | end to end | D | @@ -568,6 +593,23 @@ The design never resolves an inconsistency it finds in the doc. It records the t Use the doc's terms as `spec:platform.vocabulary` fixes them. Stream version, never revision. Obligation for the record, effect for the change it makes, attempt for one try. Receipt for the stored idempotency outcome. Generation for a numbered read-model build. Where a Convex term and a doc term differ, the doc's word names the design concept and the Convex word names the mechanism (`obligation` is the record, `scheduled function` is the dispatch). +### 6.7 A design ahead of its code + +A capability is designed in its Specs before anyone writes its code, at the rung its design has reached, and the graph shows which parts are realized, which are designed and which are deferred. These conventions keep it so. + +1. **One maturity per Spec.** When part of a Spec's design is realized, deferred or reviewed apart from the rest, it moves into a Spec of its own, with its own readiness, open questions, relations, anchors and verifiers. This is the Protocol's promotion rule, and the content moves out. What the parts share keeps one home, and each part `dependsOn` it. The projection contracts show the three cases side by side: `spec:application.projection-contract` holds the per-entity form, which code realizes, and the conventions every form shares; `spec:application.history-projection` holds the history form, designed and checked by one pure test; `spec:application.aggregate-projection` holds the aggregate form at `scoped`, deferred with its trigger. +2. **A capability Pack.** A Pack gathers, in reading order and across packages, the Specs one capability rests on, such as `pack:history-view`. Its page in the Design Review (`npx sdp view`, then `generated/design-review/pack/`) presents the design: each member's stated rung, the floor it reaches, its implementation binding and its verifier binding. Recipes 5, 19, 20 and 21 read a Pack's members for review, planning, open questions and dependency footing. +3. **Pinned declarations compile.** A Design entry whose value opens with a code span is a pinned declaration (6.3), and recipe 24 lists them. `scripts/pinned.mjs` emits the declarations of one scope by key prefix into `generated/pinned/.ts`, with the declarations other Specs pin that the scope uses; `npm run sdp:build` runs it and `npm run test:types` compiles it, so a name used and never declared, or declared twice, fails. A name the design uses and no Spec pins is listed in the emitter's preamble, one line each, until a Spec pins it. Where code realizes a pin, `tests/types/pinned.test-d.ts` asserts that the code's type equals it, so a changed pin, or code that drifts from it, fails the typecheck. Where the design leads its code on purpose, the test lists the pair as ahead, with what closes it, and asserts that the two still differ, so the build that catches up fails the test until it moves the pair to the bound list. The module states shapes only; a rule in prose is still read by a person. +4. **An entry is cited by its address.** `spec:#design.` names one Design entry, as in `spec:application.history-projection#design.fnApplyHistoryProjection`. Validate checks an address in a Spec's prose, and `check.py` checks one in the code, the tests, the register, the ledger and the work documents. A line number is a pointer that the next edit moves; it never stands for an entry. +5. **A mention is a relation or a listed reason.** A Spec id in prose with no relation between the two Specs is a validate warning. Declare the relation when the mentioning Spec could not be built or read correctly without the other. Otherwise list the pair in `design/tools/prose-mentions.json` with a one-line reason; `check.py` refuses a pair that is neither, and a listed pair that no longer warns. +6. **A design is read from several perspectives before its code.** A pass reads a capability's Pack from one lens and changes Spec text only through the main thread's rulings; what it cannot settle becomes an open question or a register row. A pass needs no status: git holds the passes and the Pack shows the design as it stands. The questions each lens asks of a design, as a start: + - domain: does every interleaving of the sources give the same row, what survives a subject's end, what a migrated stream's baseline means to the design; + - Convex: what one transaction reads and writes, whether the stated bounds hold on the pinned release, what changes atomically and what across transactions; + - operator: what an interruption, a resume and an abort leave, who closes and opens a pause, what a deploy or a restore does to work in flight; + - implementation and verification: the pinned module compiles and the code agrees with it, each dependency stands at its rung (recipe 21), and each example names its tier and has a planned verifier. +7. **A Spec is satisfied only when its code realizes all of it.** Anchor targets name whole Specs, as `spec:decisions.anchor-binding-grain` rules, so the units that satisfy a Spec together realize every entry it promises. A pinned declaration is no exception: a pin that a designed form widens, which the pinned module lists as ahead (item 3), is a promise the code has not built, and the Spec that holds it is not satisfied while the pair is ahead. `typeReadModelBinding` takes a history projection beside a per-entity read model while the code takes the per-entity form alone, so the code references `spec:command.command-declaration` and does not satisfy it. For a Spec with a part no unit builds, a coherent part with its own trigger or step moves into a child Spec that refines it, its entries moved verbatim and its readiness stated on its own, and the parent keeps `satisfies` (item 1); otherwise the code references the Spec until it realizes what the Spec promises. A reference runs one way: the code answers to the design, because it follows it or realizes part of it. A design that builds on existing code says so by its own `dependsOn` or `refines` to the Spec that code satisfies, never by a reference from the code: `spec:application.history-rebuild` holds the paused path of a history view and depends on `spec:application.rebuild`, the online path's Spec, and the online rebuild's code carries no reference to it. A deferred form states in its own Spec how it extends the Specs it would widen when its trigger comes, and those Specs do not depend on it: `spec:application.aggregate-projection` says what the declaration's binding, the pipeline's step 9 and the registry's tables take when OD-075's trigger comes. +8. **`ready` states approved to build.** This is the Protocol's reading. The owner states `ready` on a Spec when its design is approved for its build, so recipe 1, the Specs that state `ready` and have no implementation binding, is the build backlog, and a Spec leaves it when an anchor binds its code. A designed Spec states `ready` after the passes of item 6 have read its Pack, as the history projection may once the domain and operator passes have run. No agent states `ready` (section 3.2). + ## 7. Review rubric Reviewers judge the merged corpus. Each item is checkable; an item that fails names the Spec and line. diff --git a/design/README.md b/design/README.md index 52db332..8a1d620 100644 --- a/design/README.md +++ b/design/README.md @@ -45,11 +45,11 @@ python3 design/tools/check.py Expected result on the integrated corpus: ``` -237 specs · 5 packs · 142 anchors → 384 nodes · 1251 edges (0 errors, 0 warnings) -validate: 0 errors · 0 warnings (conformance + honesty over the one graph) +243 specs · 6 packs · 180 anchors → 429 nodes · 1499 edges (0 errors, 0 warnings) +validate: 0 errors · 12 warnings (conformance + honesty over the one graph) ``` -Any warning and any error is a defect. Every example declares `verifies` toward its parent for fidelity to the acceptance contract. An example that a test's `specTest` anchor binds gives its parent a verifier; an example that no anchor binds is data below `ready`, and the Protocol does not warn; recipe 10 lists each such example as a declared verifier that is not enabled. `npx sdp validate --watch` is the authoring loop. +Any error is a defect, and so is any warning but the twelve `conformance/prose-mentions` warnings, one for each pair of Specs listed in [`tools/prose-mentions.json`](tools/prose-mentions.json) with the reason no typed relation fits; `check.py` refuses a mention pair that is neither declared nor listed, and an entry address (`spec:#design.`) in the code, the tests or the records that does not resolve. Every example declares `verifies` toward its parent for fidelity to the acceptance contract. An example that a test's `specTest` anchor binds gives its parent a verifier; an example that no anchor binds is data below `ready`, and the Protocol does not warn; recipe 10 lists each such example as a declared verifier that is not enabled. `npx sdp validate --watch` is the authoring loop. ## The component map @@ -68,8 +68,8 @@ Detail follows the build (decision method, rule 4). Full means interfaces, TypeS | 9 | Tenancy and authority: tenant on everything, actor vocabulary, grants, namespaces, worker provenance | 1 | full | [`tenancy-and-authority`](specs/command/tenancy-and-authority.sdp.md), [`actor-and-scope`](specs/command/actor-and-scope.sdp.md) | | 10 | Command declaration and bindings: one declaration per command, two static exports, generation later | 1 to 2 | full | [`command-declaration`](specs/command/command-declaration.sdp.md) | | 11 | Parent use cases: contexts meet only in the parent, in one transaction, one call per context | 2 | full | [`parent-use-cases`](specs/application/parent-use-cases.sdp.md) | -| 12 | Read models and projection contract: reads inside the command, the read-need table, one projection for live update and rebuild | 2 | full | [`read-models`](specs/application/read-models.sdp.md), [`projection-contract`](specs/application/projection-contract.sdp.md) | -| 13 | Rebuild: generations, backfill, write pause | 2 | full | [`rebuild`](specs/application/rebuild.sdp.md), [`generation-registry`](specs/application/generation-registry.sdp.md), [`write-pause`](specs/application/write-pause.sdp.md) | +| 12 | Read models and projection contract: reads inside the command, the read-need table, one projection for live update and rebuild | 2 | full | [`read-models`](specs/application/read-models.sdp.md), [`projection-contract`](specs/application/projection-contract.sdp.md), [`history-projection`](specs/application/history-projection.sdp.md), [`aggregate-projection`](specs/application/aggregate-projection.sdp.md) | +| 13 | Rebuild: generations, backfill, write pause | 2 | full | [`rebuild`](specs/application/rebuild.sdp.md), [`history-rebuild`](specs/application/history-rebuild.sdp.md), [`generation-registry`](specs/application/generation-registry.sdp.md), [`write-pause`](specs/application/write-pause.sdp.md) | | 14 | Restore and baseline operations: the restore drill, audit fails closed, diagnostics never abort, bounds, retention | 2 | full | [`restore`](specs/application/restore.sdp.md), [`baseline-operations`](specs/operations/baseline-operations.sdp.md), [`events-stay-small`](specs/constraints/events-stay-small.sdp.md), [`bulk-operations-bounded`](specs/constraints/bulk-operations-bounded.sdp.md) | | 15 | First experiment and cost budgets: Orders and Inventory, the measurements, the six cost targets | 2 | full | [`first-experiment`](specs/application/first-experiment.sdp.md), [`orders-inventory-example`](specs/application/orders-inventory-example.sdp.md), the six cost constraints in [`specs/constraints/`](specs/constraints/) | | 16 | Obligations: the record, lifecycle, wrapper, sweeper, operator operations, fan-out, retention, restore, the do-nothing check | 3 | full where D13 rules | [`obligation-module`](specs/obligations/obligation-module.sdp.md) and its eight children in [`specs/obligations/`](specs/obligations/) | @@ -85,12 +85,14 @@ Detail follows the build (decision method, rule 4). Full means interfaces, TypeS | all | `platform`, `laws`, `decisions`, `facts` | [`foundation`](specs/foundation.pack.sdp.md) | 98 | | 0 | `kernel` | [`kernel-and-context`](specs/kernel-and-context.pack.sdp.md) | 7 | | 1 | `context`, `command` | [`kernel-and-context`](specs/kernel-and-context.pack.sdp.md) (19 with the kernel), [`command-pipeline`](specs/command-pipeline.pack.sdp.md) | 12 + 24 | -| 2 | `application`, `operations`, `constraints` | [`application`](specs/application.pack.sdp.md) | 48 | +| 2 | `application`, `operations`, `constraints` | [`application`](specs/application.pack.sdp.md) | 51 | | 3 | `obligations`, `effects` | [`durable-and-later`](specs/durable-and-later.pack.sdp.md) | 22 (of the pack's 43) | | 4 | `processes` | [`durable-and-later`](specs/durable-and-later.pack.sdp.md) | 6 | | 5 | `agents` | [`durable-and-later`](specs/durable-and-later.pack.sdp.md) | 7 | | 6 | `advanced` | [`durable-and-later`](specs/durable-and-later.pack.sdp.md) | 8 | +One pack follows a capability rather than a layer. [`history-view`](specs/history-view.pack.sdp.md) gathers, in reading order and across packages, the sixteen Specs the history view rests on, so its Design Review page shows that design at the rung each part has reached (`PLAN.md` 6.7). + Counts by kind and by stated readiness are derived, not kept here: `npx sdp census` writes the census, and the session check prints the readiness tally. Every Layer 4 to 6 non-example Spec states `scoped`, held by its deferral; F17 states `defined` with its native Probe 6 evidence. Nothing states `ready`. Every pack carries `modelRefs: [spec:platform.vocabulary]`; the application pack also refs the example domain's model. What each layer fixes, in one paragraph each. @@ -153,7 +155,7 @@ Every row of the doc's acceptance table is an example Spec that `refines` and `v | L2-3 | Orders of 1 line, 10 lines and the maximum | [`spec:application.first-experiment`](specs/application/first-experiment.sdp.md) | [`.order-of-1-line`](specs/application/first-experiment.order-of-1-line.sdp.md), [`.order-of-10-lines`](specs/application/first-experiment.order-of-10-lines.sdp.md), [`.order-of-max-lines`](specs/application/first-experiment.order-of-max-lines.sdp.md) | native | | L2-4 | Rename a typed handler or break its argument contract | [`spec:command.command-declaration`](specs/command/command-declaration.sdp.md) | [`.renamed-handler-fails-build`](specs/command/command-declaration.renamed-handler-fails-build.sdp.md), [`.broken-argument-contract-fails-build`](specs/command/command-declaration.broken-argument-contract-fails-build.sdp.md) | build | | L2-5 | Rebuild a per-entity read model online while commands run; interrupt and resume | [`spec:application.rebuild`](specs/application/rebuild.sdp.md) | [`.online-rebuild-interrupt-resume`](specs/application/rebuild.online-rebuild-interrupt-resume.sdp.md) | native | -| L2-6 | Rebuild a cross-stream history view under a write pause; interrupt | [`spec:application.rebuild`](specs/application/rebuild.sdp.md) | [`.write-pause-rebuild-interrupt`](specs/application/rebuild.write-pause-rebuild-interrupt.sdp.md), [`.write-pause-rebuild-abort`](specs/application/rebuild.write-pause-rebuild-abort.sdp.md) | native | +| L2-6 | Rebuild a cross-stream history view under a write pause; interrupt | [`spec:application.history-rebuild`](specs/application/history-rebuild.sdp.md) | [`.write-pause-rebuild-interrupt`](specs/application/history-rebuild.write-pause-rebuild-interrupt.sdp.md), [`.write-pause-rebuild-abort`](specs/application/history-rebuild.write-pause-rebuild-abort.sdp.md) | native | | L2-7 | Rebuild a stream that has a baseline event | [`spec:context.journal`](specs/context/journal.sdp.md) | [`.rebuild-from-baseline`](specs/context/journal.rebuild-from-baseline.sdp.md) | native | | L2-8 | Restore a representative dataset with matching code and configuration | [`spec:application.restore`](specs/application/restore.sdp.md) | [`.restore-representative-dataset`](specs/application/restore.restore-representative-dataset.sdp.md) | end to end | | L2-9 | Run native acceptance with production configuration | [`spec:application.first-experiment`](specs/application/first-experiment.sdp.md) | [`.native-acceptance-production-configuration`](specs/application/first-experiment.native-acceptance-production-configuration.sdp.md), [`.native-contention`](specs/application/first-experiment.native-contention.sdp.md) | end to end; native | @@ -191,8 +193,8 @@ An extension is a design claim the doc does not make. Each one is three things t | E-5 | [`command.outcome-boundary`](specs/command/outcome-boundary.sdp.md) | `RejectionData` and `TransientData`; the platform codes `invalidInput`, `unauthenticated`, `forbidden`, `idempotencyConflict`, `staleVersion`, `entityExists`, `operationTooLarge`, `unsupportedContractVersion` and the transient `rateLimited`, `capacity`, `writePaused`; `normalizeThrown`, which turns a code the declaration's `rejections` does not list into a technical failure; a 16 KiB bound on `details` that `reject` measures; the engine's OCC-retry exhaustion as a retryable technical failure; the conditional dispatcher; a parent query's refusal as the same `RejectionData`, with the query's name in `commandType` and only the codes `unauthenticated` and `forbidden`; a use case's own rejection thrown as the kernel's bare `Rejection`, as a context throws one | | E-6 | [`command.actor-and-scope`](specs/command/actor-and-scope.sdp.md), [`command.tenancy-and-authority`](specs/command/tenancy-and-authority.sdp.md) | `Actor`, `ActorRef`, `TenantScope`, `CallerNamespace` (`public`, `service`, `worker`, `agent`, `system`), the `authorize` signature, the `grants` table; the tenant scope of an operation, a command and a query is the `tenantId` argument alone, with no separate scope argument; a parent query establishes and authorizes its actor through `authorizeQuery`, generic over the parent's data model, and its refusal is the command's `RejectionData` with the query's name in `commandType`; the worker's two modes as one `Authority` shape, `{ actor, scope, mode }` with `recheckDelegator` and `serviceAuthority`, stored on the obligation record and never restated; a derived command carries `worker`, and `system` is reserved for the platform's own maintenance chains and operator operations | | E-7 | [`command.command-declaration`](specs/command/command-declaration.sdp.md) | `CommandDeclaration`, `publicCommand` and `internalCommand`, the executor that takes the minted `OperationRef` and returns the kernel's outcome plus `streams`, the read-model binding with its `source` and read model, what generation emits, and no standing map from command type to declaration, because the type check at a typed caller is the check of Sc L2-4; the cost of two registered mutations per command and the single-export dispatcher alternative it beat | -| E-8 | [`application.generation-registry`](specs/application/generation-registry.sdp.md), [`application.write-pause`](specs/application/write-pause.sdp.md), [`application.rebuild`](specs/application/rebuild.sdp.md) | The `generations` table with seven states, every transition with who makes it, and one generation being built per read model; the `generationProgress` row that holds the cursor, the batch size and the counts, read by batches and operator entries only; the `projectionMarkers` table; a read model installed through its first rebuild, generation 1 starting as `building`, with no transition from nothing to active, a command that finds no generation to write and a query that finds no active generation each failing as a technical failure; `generationsToWrite` answering the active generation and the one being built as a list tagged by role, `active`, `building` or `verifying`; a 7-day rollback period in which the retired generation is kept and not written, and a rollback that is one verify pass, presence and absence, and then the switch of one write, a single mutation that patches both generation rows; one generation per read model spanning every tenant, so the registry's indexes lead with the read model name and not `tenantId`, a confirmed deviation from Law 11's index rule, and no rebuild for one tenant; one gate document per deployment with scope keys for the deployment, a tenant and a source stream type, read by the pipeline's step 7 after the restore door, with no fence or drain of its own; a paused rebuild closes the view's source stream types, one `source:` scope per binding; an exemption only for the generation a closed entry names; gate changes treated as mandatory audit; no acceptable length stated for a write pause | -| E-9 | [`application.projection-contract`](specs/application/projection-contract.sdp.md) | A per-entity read model declared as `ReadModel` with `name`, `table`, `rowBudgetBytes` and its `projections`, each a `Projection` of `version`, `keyOf` and `project`, bound on a declaration with its source; `AggregateProjection` and `HistoryProjection`, the last folding events into the prior row for a view that depends on history across streams; `applyProjection` with modes `live-created`, `live-updated`, `backfill`, writing each generation by its role, where a live write to the active generation always lands and the rule that a newer row is never overwritten is the backfill path's, `applyAggregate` and `applyHistoryProjection`, called by the pipeline's step 9 and the rebuild's batches only; every row measured with `getConvexSize` against `rowBudgetBytes`; step 9 writes a per-entity read model from the DTO of each matching `streams` entry, never from the events; a read model declares its projection versions as a list, the latest first, and each generation is written with the projection whose version its row records; row conventions `tenantId`, `generation`, `key`, `projectionVersion`, `sourceVersions`; the `by_key` index; the created flag and the appended events read from the operation outcome's `streams` entry | +| E-8 | [`application.generation-registry`](specs/application/generation-registry.sdp.md), [`application.write-pause`](specs/application/write-pause.sdp.md), [`application.rebuild`](specs/application/rebuild.sdp.md), [`application.history-rebuild`](specs/application/history-rebuild.sdp.md), [`application.aggregate-projection`](specs/application/aggregate-projection.sdp.md) | The `generations` table with seven states, every transition with who makes it, and one generation being built per read model; the `generationProgress` row that holds the cursor, the batch size and the counts, read by batches and operator entries only; the `projectionMarkers` table; a read model installed through its first rebuild, generation 1 starting as `building`, with no transition from nothing to active, a command that finds no generation to write and a query that finds no active generation each failing as a technical failure; `generationsToWrite` answering the active generation and the one being built as a list tagged by role, `active`, `building` or `verifying`; a 7-day rollback period in which the retired generation is kept and not written, and a rollback that is one verify pass, presence and absence, and then the switch of one write, a single mutation that patches both generation rows; one generation per read model spanning every tenant, so the registry's indexes lead with the read model name and not `tenantId`, a confirmed deviation from Law 11's index rule, and no rebuild for one tenant; one gate document per deployment with scope keys for the deployment, a tenant and a source stream type, read by the pipeline's step 7 after the restore door, with no fence or drain of its own; a paused rebuild closes the view's source stream types, one `source:` scope per binding; an exemption only for the generation a closed entry names; gate changes treated as mandatory audit; no acceptable length stated for a write pause | +| E-9 | [`application.projection-contract`](specs/application/projection-contract.sdp.md), [`application.history-projection`](specs/application/history-projection.sdp.md), [`application.aggregate-projection`](specs/application/aggregate-projection.sdp.md) | A per-entity read model declared as `ReadModel` with `name`, `table`, `rowBudgetBytes` and its `projections`, each a `Projection` of `version`, `keyOf` and `project`, bound on a declaration with its source; `AggregateProjection` and `HistoryProjection`, the last folding events into the prior row for a view that depends on history across streams; `applyProjection` with modes `live-created`, `live-updated`, `backfill`, writing each generation by its role, where a live write to the active generation always lands and the rule that a newer row is never overwritten is the backfill path's, `applyAggregate` and `applyHistoryProjection`, called by the pipeline's step 9 and the rebuild's batches only; every row measured with `getConvexSize` against `rowBudgetBytes`; step 9 writes a per-entity read model from the DTO of each matching `streams` entry, never from the events; a read model declares its projection versions as a list, the latest first, and each generation is written with the projection whose version its row records; row conventions `tenantId`, `generation`, `key`, `projectionVersion`, `sourceVersions`; the `by_key` index; the created flag and the appended events read from the operation outcome's `streams` entry | | E-10 | [`obligations.lifecycle-transitions`](specs/obligations/lifecycle-transitions.sdp.md), [`obligations.record-contract`](specs/obligations/record-contract.sdp.md) | The allowed transitions between the six states and who causes each; the `obligations` and `obligationRepairs` tables with seven and two indexes; a 64 KiB inline payload bound; ten late-evidence entries; two attention reasons beyond D13's four, `recoveryFailing` and `chainBound` | | E-11 | [`effects.claim-call-settle`](specs/effects/claim-call-settle.sdp.md) | The signatures of `claimAttempt`, `readForCall`, `callProvider`, `reconcileAttempt`, `settleAttempt`, `expireLease`; the `RepetitionPolicy` union with a fourth value `none` | | E-12 | [`constraints.events-stay-small`](specs/constraints/events-stay-small.sdp.md), [`context.event-envelope`](specs/context/event-envelope.sdp.md) | A 16 KiB payload bound and a 4,096-byte envelope bound measured with `getConvexSize` by the journal's `append`, so a fold of 500 stored events of at most 20,541 bytes each stays under 9.8 MiB under the 16 MiB read ceiling; the reserved `baseline` event's payload is exempt and bounded by its stream type's `budgetBytes`, and the stored baseline event is at most `budgetBytes + 4,096 + 61` bytes; the 1 second query timeout binds first and the first experiment measures it | @@ -218,7 +220,7 @@ An extension is a design claim the doc does not make. Each one is three things t | E-37 | [`command.tenancy-and-authority`](specs/command/tenancy-and-authority.sdp.md) | Identity to actor mapping from `tokenIdentifier`, `subject` and `issuer`; grants written by the plain helpers `insertGrant` and `revokeGrant`, a tenant's first grant created with admin access through the composition's own internal grant mutation, with no grant function, bootstrap, first-admin flow or invitation in the library; revocation by deleting the grant row, with no cache; a 500-grant read bound | | E-38 | [`command.command-declaration`](specs/command/command-declaration.sdp.md) | The declaration also carries `contractVersion`, `retention`, `irreversible`, `admission`, `bounds`, `rejections`, `audit`, `diagnosticSink` and `writes`; `bounds` is the one input bound, checked at the pipeline's step 1; `writes` names the context and stream type pairs the executor may write, from which step 7 derives the write pause's source scopes and which step 9 proves; the declaration has no reactions field | | E-39 | [`command.idempotency-and-receipts`](specs/command/idempotency-and-receipts.sdp.md) | The second UI submit of a create is answered with the rejection `entityExists`, not an idempotent success, because the UI path writes no receipt to replay from and the client holds the entity ID | -| E-40 | [`application.rebuild`](specs/application/rebuild.sdp.md), [`application.generation-registry`](specs/application/generation-registry.sdp.md) | A batch is one mutation with one cursor on the progress row, at most 100 entities capped by the row budget and by the context's byte-derived page bound, a design number not attributed to the component; a self-scheduled internal mutation drives every batch, the backfill, the verify pass and the purge, and the rebuild uses `@convex-dev/migrations` for nothing; one batch names one tenant, taken from the tenant list; a fence that every interrupt, resume, rollback, abort and purge start raises | +| E-40 | [`application.rebuild`](specs/application/rebuild.sdp.md), [`application.history-rebuild`](specs/application/history-rebuild.sdp.md), [`application.generation-registry`](specs/application/generation-registry.sdp.md) | A batch is one mutation with one cursor on the progress row, at most 100 entities capped by the row budget and by the context's byte-derived page bound, a design number not attributed to the component; a self-scheduled internal mutation drives every batch, the backfill, the verify pass and the purge, and the rebuild uses `@convex-dev/migrations` for nothing; one batch names one tenant, taken from the tenant list; a fence that every interrupt, resume, rollback, abort and purge start raises | | E-41 | [`application.parent-use-cases`](specs/application/parent-use-cases.sdp.md) | The caller-driven import command with an `imports` progress row; the input bound itself is the declaration's `bounds.maxItems` under E-38 | | E-42 | [`application.restore`](specs/application/restore.sdp.md), [`application.generation-registry`](specs/application/generation-registry.sdp.md) | Four checks (journal, domain, read model, receipt) as checkpointed batches, the stream check bounded through the `rebuild` query's `maxEvents` so its worst case is one fold, and a refused fold counted as a journal finding; a fifth step that inventories the closed gate entries, the generations in flight and the running imports the backup archive restored; the `restoreRuns` record; an accepted branch that resumes every stranded generation, proven by the generation row's fence; `MAINTENANCE_MODE=restore` as the writer door, read by the write pause's check before the gate document and cleared before the durable profile's schedule rebuild runs; a failed check keeps the whole deployment closed with no override; the drill before every release, its cadence reconsidered on the first drill's measured check times | | E-43 | [`operations.baseline-operations`](specs/operations/baseline-operations.sdp.md) | The `auditRecords` table that fails closed, with `AuditRecordInput` and the source of each field in the command; the `operatorAudit` table for the gate changes an operator entry makes; Convex's own audit logging recorded as the do-nothing option; a 4 KiB diagnostic record through a sink that never throws; the `personalData` reference table | @@ -296,11 +298,11 @@ Four example IDs were added beyond the plan's section 4, each because the doc's | Sc | Added example | Reason | |---|---|---| | L2-1 | `spec:application.parent-use-cases.second-context-throws` | the row says "rejects or throws"; the throw case is a technical failure, not the rejection | -| L2-6 | `spec:application.rebuild.write-pause-rebuild-abort` | the row's pass condition says "resume or abort safely" | +| L2-6 | `spec:application.history-rebuild.write-pause-rebuild-abort` | the row's pass condition says "resume or abort safely" | | L3-2 | `spec:obligations.obligation-module.wrapper-failed-rearmed` | the row says "kill a dispatch before work, or fail the scheduled wrapper" | | L3-6 | `spec:effects.external-effects.stale-worker-after-cancellation` | the row says "after a new attempt or a cancellation" | -No inventory ID was renamed or split. Package D therefore has 13 examples instead of 11, Package E has 22 instead of 20, and the corpus has 54. +No inventory ID was renamed or split at integration; the history rebuild's split below later moved the two Sc L2-6 examples. Package D therefore has 13 examples instead of 11, Package E has 22 instead of 20, and the corpus has 54. Slice S1 added three more for the same reason, when its scenario tests bound the Layer 1 rows: @@ -312,7 +314,7 @@ Slice S1 added three more for the same reason, when its scenario tests bound the Package B therefore has 7 examples, Package C has 15, and the corpus has 57. -The Layer 2 design added two more for the same reason, and ten examples that verify no doc row: +The Layer 2 design added two more for the same reason, and thirteen examples that verify no doc row: | Sc | Added example | Reason | |---|---|---| @@ -324,6 +326,9 @@ The Layer 2 design added two more for the same reason, and ten examples that ver | none | `spec:application.read-models.query-refused-without-grant` | verifies Law 5 for a parent query called with no grant in the tenant | | none | `spec:application.read-models.list-pages-by-cursor` | verifies a parent list paged by a cursor pair, under E-24 | | none | `spec:application.write-pause.restore-door-without-deploy` | shows on the pinned release that the restore door's environment switch needs no deploy, under E-42 | +| none | `spec:application.history-rebuild.write-pause-rebuild-version-change` | verifies the history rebuild's handoff for a deploy that changes a history projection's version, under E-8 | +| none | `spec:application.history-rebuild.write-pause-rebuild-installation` | verifies the history rebuild's handoff for a deploy that adds a history view's first binding, under E-8 | +| none | `spec:application.history-rebuild.write-pause-rebuild-source-order-mismatch` | verifies the history rebuild's compatibility check and its recovery for sources a deploy reordered, under E-8 | | none | `spec:facts.f15-parent-query-over-component-query-stays-reactive.probe-5-built-in-paginate-throws-in-component` | Probe 5: the built-in `paginate` throws inside a component | | none | `spec:facts.f15-parent-query-over-component-query-stays-reactive.probe-5-full-first-page-not-split` | Probe 5: a full first page under a row cap of twice the page size is not split | | none | `spec:facts.f15-parent-query-over-component-query-stays-reactive.probe-5-full-first-page-split-at-equal-cap` | Probe 5: the same page under a row cap equal to the page size is split | @@ -370,6 +375,21 @@ The bound on a stock item ID added three examples that verify no doc row, each a Package D therefore has 30 examples. +The history form and the aggregate form of `spec:application.projection-contract` moved into contracts of their own, so that each Spec carries one maturity: + +| Sc | Added Spec | Reason | +|---|---|---| +| L2-6 | [`spec:application.history-projection`](specs/application/history-projection.sdp.md) | the history form, its fold and the order allocation history, checked by one pure test of the fold | +| none | [`spec:application.aggregate-projection`](specs/application/aggregate-projection.sdp.md) | the aggregate form and its marker, deferred until a read model needs an aggregate row (OD-075) | + +The paused path of `spec:application.rebuild` moved into a workflow of its own, so that the rebuild's implementation binding claims the online path alone, and the two Sc L2-6 examples moved under it with the parent they verify. The marker table, the aggregate binding and the aggregate call of step 9 moved from the registry, the declaration and the pipeline into `spec:application.aggregate-projection`, which states how the form extends them when OD-075's trigger comes: + +| Sc | Added or renamed Spec | Reason | +|---|---|---| +| L2-6 | [`spec:application.history-rebuild`](specs/application/history-rebuild.sdp.md) | the paused path of a history view: the source scopes the start closes, the history batches with their cursor and bound, and the verify pass under the pause | +| L2-6 | [`spec:application.history-rebuild.write-pause-rebuild-interrupt`](specs/application/history-rebuild.write-pause-rebuild-interrupt.sdp.md), formerly `spec:application.rebuild.write-pause-rebuild-interrupt` | it verifies the paused path, so it refines and verifies the history rebuild | +| L2-6 | [`spec:application.history-rebuild.write-pause-rebuild-abort`](specs/application/history-rebuild.write-pause-rebuild-abort.sdp.md), formerly `spec:application.rebuild.write-pause-rebuild-abort` | the same, for the abort case | + ## Integration notes What the integration pass changed after the five packages landed, and what it left for the owner or the plan. diff --git a/design/ROADMAP.md b/design/ROADMAP.md index 3bf2057..5193ee9 100644 --- a/design/ROADMAP.md +++ b/design/ROADMAP.md @@ -1,6 +1,6 @@ # Roadmap: the next units -Written on 2026-10-02 by the main thread, from four advisor memos written the same day under `design/advisors/task-roadmap.md`. It is a work document: it orders units and names what each needs and shows, and it says nothing a Spec should say. The owner rules on it. The memos, the plants and the check are in the roadmap unit's folder, which the project notes name. It is kept current at each close, last at the close of round 3 on 2026-10-03. +Written on 2026-10-02 by the main thread, from four advisor memos written the same day under `design/advisors/task-roadmap.md`. It is a work document: it orders units and names what each needs and shows, and it says nothing a Spec should say. The owner rules on it. The memos, the plants and the check are in the roadmap unit's folder, which the project notes name. It is kept current at each close, last at the close of pull request #12 on 2026-10-10, before its merge into `main`. ## How to read it @@ -31,12 +31,12 @@ The slices of `STATE.md` keep their names. S3 is round 1's attribution and round | 4 | a | A command's audit fails closed and its diagnostic never aborts, Sc ALL-1. Done in the close of round 3: both Sc ALL-1 examples pass natively, the proof written apart from the fix | mixed | operator | an evening | 3a | | 4 | c | The history view's fold, its memory and its rollback, on paper. Done in the close of round 3: ten choices ruled, ten ledger findings on paper, one pure fold with its test | design | domain | an evening | 2c | | 4 | d | The receipts sweep, one tenant per run. Done in the close of round 3: `receipts.sweep` and `sweepNext` in both compositions, at `convex-test` | build | operator | under an evening | the tenant list from 2b | -| 4 | e | `ready` on the built Specs, by family | decide | product | a session, one owner unit | 1c, sitting two | +| 4 | e | `ready` as approval to build: the history view's Specs after their passes, then the built Specs by family. The four passes are done; the owner's statements are next | decide | product | a session, one owner unit | 1c, sitting two, the four passes over `pack:history-view` (done) | | 4 | f | The language in the code: the refusal field `entry`, the tier names in the examples' opening lines and the harness parser, the harness's backup archive names. Done in the close of round 3 | build | product | under an evening | sitting two | | 5 | a | Baselines, migration and the fold bound, Sc L2-7 at the native tier; the history-reading sweep with its transient refusal and the restore check's pending-work inventory, at `convex-test` for the interleavings and natively for the sweep | mixed | domain | a night or more | OD-041 (ruled), 3b | | 5 | b | The restore procedure and the drill, Sc L2-8 | mixed | operator | unknown until 4b | 4b, 5a | | 5 | c | The first adopter-facing cut: README, the example walkthrough, the package shape | mixed | product | a night | OD-047, OD-048, Sc L2-5 to L2-8 bound | -| 5 | d | The build of the order allocation history view, Sc L2-6 at the native tier, by ruling F10 | build | domain | a night | 3b, 4c | +| 5 | d | The build of the order allocation history view, Sc L2-6 at the native tier, by ruling F10. Its design is `pack:history-view`, read in four passes: the implementation and Convex passes in the unit `design-in-the-graph`, the domain and operator passes in `sdp-round-2`. The build waits for the owner's `ready` | build | domain | a night | 3b, 4c, the four passes, round 5 of their fold-in and the third read of the history Specs (all done), the owner's `ready` on the history projection, and on the history rebuild once OD-157 is ruled | ## Round 1: what blocks nothing @@ -97,7 +97,7 @@ Every item is done but the hosted tier's first runs, which are the owner's after **4d. The receipts sweep.** One internal mutation per tenant over the expiry index, at most the bound, returning what it deleted and whether more remains, with the loop over tenants from 2b's ruling. Done in the close of round 3, as `receipts.sweep` and `sweepNext`. -**4e. `ready` by family.** The fork for OD-024 lists the built Specs with their floors and their open questions and asks for `ready` by family: kernel and context first, then command, then application without the generation registry. A Spec with an unruled `owner` row stays `defined`. +**4e. `ready` as approval to build.** By OD-163, the owner's ruling of 2026-10-04, `ready` states that a Spec is approved to build, the Protocol's reading: recipe 1 is the build backlog, and a `ready` Spec leaves it when an anchor binds its code. 4e is therefore no longer a stamp on what is built. Its first use is the history view. The four passes over `pack:history-view`, round 5 of their fold-in and the third read of the history Specs have run, and the read found the history projection and the version-change and installation examples ready to state. So the owner may state `ready` on `spec:application.history-projection`, which OD-174 left with no unruled `owner` row, with nothing else in the way, and on the paused path, `spec:application.history-rebuild` with its five examples, once OD-157 is ruled; those statements approve 5d. The fork for OD-024 still lists the built Specs with their floors and their open questions and asks for `ready` by family: kernel and context first, then command, then application. Since OD-162 the generation registry, the command declaration and the command pipeline no longer rest on the aggregate Spec, so their floors reach `ready` and the registry no longer stands apart. A Spec with an unruled `owner` row stays `defined`. ## Round 5: S5 @@ -107,6 +107,16 @@ Every item is done but the hosted tier's first runs, which are the owner's after **5c. The adopter-facing cut.** The root README's first hour, an `example/README.md` that walks `PlaceOrder` from its declaration to the order summary, and the package kept private with nothing of the first-release list. No name, export or version is a contract yet. +**5d. The history view's build.** Its design is `pack:history-view`, nineteen Specs, read before the build in four passes, one lens each (`PLAN.md` 6.7, `design/advisors/task-pass.md`). The implementation and Convex passes ran in the unit `design-in-the-graph`, the domain and operator passes in `sdp-round-2`, and their rulings are OD-156 to OD-161 and OD-164 to OD-175. The second read of the history Specs found the release procedure uneven, and round 5 of the unit's fold-in stated it whole (L1 to L8 of its rulings); the third read found the projection and the version-change and installation examples ready to state, and its four seams at the protected abort's edges are folded in as round 6 (M1 to M5). The build waits for the owner's `ready` on `spec:application.history-projection`, which nothing else gates, and on `spec:application.history-rebuild`, which also waits for OD-157. It then binds the two Specs to their code: `applyHistoryProjection` at step 9, checking each generation's version first; the paused path with `historyHandoff`, the protected abort and the source check of OD-175; the order allocation history bound on `PlaceOrder`, `CancelOrder` and `ReceiveStock`, with the pure test over the declarations that OD-172 asks for; and `eventSchemaVersion` on the history event, OD-173. It moves the pairs it closes from the ahead list of `tests/types/pinned.test-d.ts` to the bound list, and binds the history rebuild's five examples at the native tier: the two Sc L2-6 examples, interrupt then resume and interrupt then abort, a version-changing deploy and a first installation through the handoff, and a source-order mismatch. OD-164 to OD-170 are sorted by their lenses before or beside it. + +## Beside the rounds + +**The Protocol's round 2** is merged: `feature/adopter-round-2` of `libar-dev/software-delivery-protocol` as its pull request #28 (`415b095`) on 2026-10-10, followed by the annotations arc, #29 (`822aea4`). It brought authored entry order rendered, a Pack's member order, the next rung, question keys, recipes 25 and 26, and the Studio slice, `pack:spec-studio-v1`. This corpus is pinned at `822aea4`, the Protocol's `main` after both merges. The owner's `ready` on the four Specs the round implemented, and the Studio Pack's 25 departures, keyed questions for the owner, are in the Owner queue of `STATE.md`. + +**The dedicated pull request of O16**, a ruling of the unit `sdp-round-2`, comes after 5d or beside it, as the owner chooses. The owner: "Now it is perfect ground for demonstation in a dedicated PR. I am preparing the ground for that." and "The Graph, when properly implemented, gives all of these as projections." The Pack page, recipes 25 and 26, the question keys, the next-rung data and the Studio slice are its ground. Its subject lives in a private repository, and nothing from it is quoted here without the owner's word. + +**The rest of the Protocol's original design** is captured as Specs in a later unit, as O8 of the same rulings left it; the owner's later decision on annotating every design pattern in code (O14) bears on how. + ## Deferred, with the trigger - Layers 3 to 6: at their trigger, after Probe 7, as the doc and tactical decision 3 say. OD-050 to OD-054 and OD-056 stay waiting until an activation record names a consumer. diff --git a/design/SESSIONS.md b/design/SESSIONS.md index b53376d..40fbd77 100644 --- a/design/SESSIONS.md +++ b/design/SESSIONS.md @@ -41,14 +41,14 @@ The two families are one team. The owner ruled on 2026-10-01 that Claude models Proof and fix are written apart. Once a designer's Spec text is ruled, astra writes the tests and a builder writes the code, each from the Spec text alone, in a worktree of its own, and an integrator merges them. A test that fails at that merge is a gap in the Spec, a wrong test or a wrong fix, and the integrator says which. So the designer writes into the Spec everything the two must agree on. -A GPT job is started as an agent of type `gpt:codex`, with its brief in a file, and a job that ended is continued on its own thread with the next rulings. Each reviewer that mutates code gets its own frozen copy. The mechanics, the briefing of each family and the calibration of a reviewer are in the orchestration guide, and this project's commands, brief fragments, plant results and job log are in its notes file. `AGENTS.md` names both. +A GPT job is started as an agent of type `relay:job`, with its brief in a file, and a job that ended is continued on its own thread with the next rulings. Each reviewer that mutates code gets its own frozen copy. The mechanics, the briefing of each family and the calibration of a reviewer are in the orchestration guide, and this project's commands, brief fragments, plant results and job log are in its notes file. `AGENTS.md` names both. Four rules from the stop of 2026-09-30, when ten parallel Fable agents hit the usage limit and one left 109 unreported edits: - One unit per session. - One write lane per tree. A second writer gets its own git worktree and files the first does not touch, and an integrator agent or the main thread merges its branch. Two writers share one tree only when the two briefs split the files and each names the other's files as forbidden, and then nothing is committed or checked until both are done. - The ledger is written ahead: each finding and each status before the next begins. -- A GPT job's report stays in its job directory and a Claude agent answers in the thread. The rule to save every report to a file is gone, because a job's report now survives in `~/.codex-jobs`. +- A GPT job's report stays in its job directory and a Claude agent answers in the thread. The rule to save every report to a file is gone, because a job's report now survives in `~/.relay-jobs`. ## Open @@ -101,7 +101,7 @@ Paper only, for findings no slice will build soon: Layer 3 and later, and polish ### adopt -For a new Protocol version. Change the pin in `package.json`, run the check, apply what the version changes to the corpus, and write what the adoption found into `docs/sdp-feedback.md` for the Protocol's maintainers. +For a new Protocol version. Change the pin in `package.json`, run the check, apply what the version changes to the corpus, and write what the adoption found into `docs/feedback/sdp-feedback-01.md`, the running file for the Protocol's maintainers. ### recover @@ -126,7 +126,7 @@ Consensus holds for Layers 0 to 2 when the first experiment passes on a native b A build slice is too long for one conversation. The main thread hands over to a fresh main thread at a seam, before its own context gets in the way: after a commit, with no delegated job running. -1. Reach the seam. Wait for or stop every delegated job. A Claude agent dies with the conversation and leaves no report. A GPT job survives it, and its report, events and thread id are in its directory under `~/.codex-jobs`. +1. Reach the seam. Wait for or stop every delegated job. A Claude agent dies with the conversation and leaves no report. A GPT job survives it, and its report, events and thread id are in its directory under `~/.relay-jobs`. 2. Commit what is in the tree on the unit's branch, as work in progress if it is not reviewed, and say so in the message. 3. Write the handover note beside the unit's rulings page, in the folder the project notes name. It says what is committed, which gates fail at the last commit, which findings are ruled and where the rulings are, what is drafted and not applied, the next steps in order, and what waits for the owner. It lists every instruction the owner gave during the conversation, in the owner's words. 4. Put a short "In progress" block at the top of `STATE.md` that names the unit, the seam and the note. The rest of `STATE.md` waits for the close. diff --git a/design/STATE.md b/design/STATE.md index 02dc6ea..e3442b9 100644 --- a/design/STATE.md +++ b/design/STATE.md @@ -1,42 +1,79 @@ # State of the design corpus -Written on 2026-10-03 at the close of round 3 and of `feature/round-2`, which followed sitting two (round 3's unit 3d), unit 3b, the tooling unit, round 2, round 1, the roadmap unit, the decision register unit, the platform review unit and slice S2. `SESSIONS.md` says how to use this file. Every close rewrites it. `ROADMAP.md` orders the next units and is kept current at each close. +Written on 2026-10-04 at the close of the unit `sdp-round-2`, which extends the pull request of the unit `design-in-the-graph` (#12) after the unit `pack-page`, and refreshed on 2026-10-10 at the close of that pull request, before its merge into `main`. That pull request was stacked on `feature/round-2`, which followed the close of round 3, sitting two (round 3's unit 3d), unit 3b, the tooling unit, round 2, round 1, the roadmap unit, the decision register unit, the platform review unit and slice S2. `SESSIONS.md` says how to use this file. Every close rewrites it. `ROADMAP.md` orders the next units and is kept current at each close. ## Measured at close +Re-measured on 2026-10-10 with `python3 design/tools/check.py` at the close of pull request #12, at the pin `822aea4` (`b5fb659`), the Protocol's `main` after the merge of its #29, after the rulings of 2026-10-10 (`ecba393`), the fold-in of their second read, the merge of `feature/round-2` with its review repair (`09224b5`), the fold of the sol check of the review repairs with the merge of its follow-up on the parent (`ab7a13f`), and the merge of #11 into `main` (`a758ba4`). The re-pin moved nothing in it: + ``` -237 specs · 5 packs · 142 anchors → 384 nodes · 1251 edges (0 errors, 0 warnings) -validate: 0 errors · 0 warnings; readiness divergence: [] -open questions: 69 Specs, 153 questions, 9 blocking; extensions registered: 54 -stated readiness: {'defined': 227, 'idea': 1, 'scoped': 9} +243 specs · 6 packs · 180 anchors → 429 nodes · 1499 edges (0 errors, 0 warnings) +validate: 0 errors · 12 warnings, 12 prose mentions listed; readiness divergence: [] +entry addresses outside the Specs: 497; dependsOn cycles: 0 +architecture: 76 code units; roles: codec 1, contract 6, decider 2, event-store 1, operator-entry 1, projection 2, query 5, read-model 1, schema 6, service 10, use-case 2; layers: adapter 3, application 3, domain 3, infrastructure 3; contexts: inventory 2, orders 2, orders-inventory 1, platform 7; 49 references edges from 41 units; platform isolation: 7 platform components, 21 uses edges from them, 0 into another context +open questions: 72 Specs, 166 questions, 10 blocking; extensions registered: 54 +stated readiness: {'defined': 232, 'idea': 1, 'scoped': 10} ledger: 278 findings, {'fixed': 159, 'open': 80, 'owner': 1, 'partially-fixed': 38} open findings by slice: {'L3': 30, 'P': 29, 'S3': 1, 'S4': 4, 'S5': 16} -corpus digest: d27b24dde358fe7e -protocol: file:vendor/libar-dev-software-delivery-protocol-0.0.0-5993da7.tgz +corpus digest: b8d5b77dcf27d47c +Pack pages current: 1 of 1 +protocol: file:vendor/libar-dev-software-delivery-protocol-0.0.0-822aea4.tgz ``` -`python3 design/tools/decisions.py`, after this unit's two register commits, `19738b4` and `c1bdf80`: +`python3 design/tools/decisions.py`, after this unit, which adds OD-164 to OD-175 and decides OD-172 to OD-175: ``` -total: 155 -status: unsorted=0, sorted=80, waiting=21, decided=39, folded=15 -class: tactical=3, delegated=86, owner=51, null=15 -advisor: convex=37, domain=45, operator=34, product=39 +total: 175 +status: unsorted=12, sorted=82, waiting=21, decided=45, folded=15 +class: tactical=3, delegated=89, owner=56, null=27 +advisor: convex=43, domain=51, operator=41, product=40 ``` -**Round 2, the tooling unit, round 3's unit 3b, sitting two and the close of round 3 are on `feature/round-2`, not on `main`.** `main` is still at `592f1dc`, the close of round 1. The close of round 3 did the rest of the mini roadmap (4f, the hosted tier, 4c, 4a and 4d) and ends the branch: `feature/round-2` goes to `main` as one pull request, which the main thread opens at this close for the owner's review and merge. The owner's words at the opening of the close: "forks have been resolved and mini roadmap unlocked. please complete the work to wrap up the feature branch and PR. use Opus, GPT 6 Astra and 6.1 Sol and orchestrate from the main thread". The main thread was Opus 5.5; no Fable agent and no advisor ran. Its head is this close's commit, on top of the fold-in's follow-up `651fcc3`. Tactical decision 42 holds the branch and its one pull request. +**Round 2, the tooling unit, round 3's unit 3b, sitting two and the close of round 3 reached `main` through pull request #11, `feature/round-2`, merged on 2026-10-10 as `a758ba4` with the receipt-sweep repair.** Until then `main` stood at `592f1dc`, the close of round 1. The close of round 3 did the rest of the mini roadmap (4f, the hosted tier, 4c, 4a and 4d) and ended the branch: `feature/round-2` went to `main` as that one pull request, which the main thread opened at the close for the owner's review and merge. The owner's words at the opening of the close: "forks have been resolved and mini roadmap unlocked. please complete the work to wrap up the feature branch and PR. use Opus, GPT 6 Astra and 6.1 Sol and orchestrate from the main thread". The main thread was Opus 5.5; no Fable agent and no advisor ran. Its head is this close's commit, on top of the fold-in's follow-up `651fcc3`. Tactical decision 42 holds the branch and its one pull request. CI passed on the branch's pushes in round 2 (`1b92bd8`, `a5d339d`) and in 3b (`314c265`, `2283083`, `de285bb`, and its close `840b32c` in 7 m 12 s). It failed on the push of sitting two, `d2a8d1b` (run 37105650743): two native files of 166, Probe 4's nested-call example and Sc L2-5's `rebuild.online-rebuild-interrupt-resume`, whose `interruptGeneration` lost every rerun to the backfill batches on `generationProgress` (`OptimisticConcurrencyControlFailure`), so acceptance read 31 passed, 1 failed and 6 missing (Leads 43). Nothing was pushed after it until this close. CI on the close's push, `1b3ff58` (run 37112192453), failed one file of 175, Probe 4's nested file on its 1 s limit, and passed L2-5. Two fixes followed before the pull request, each read by sol: Probe 4's documents hold an ASCII string in place of bytes, so the step reaches the bytes-written limit in about 0.06 s where it took 0.44 s (`4ff759d`); and `interruptGeneration` reads and writes the generation row only, marking `interruptedFence`, while a batch that finds its fence stale records the interruption on the progress row, so the interrupt no longer conflicts with the batches on `generationProgress` (`2a44655`, `131b48a`). CI on `131b48a` (run 37113652222, 7 m 17 s) passed: `npm test` 69 files; the whole tier 175 files, 174 passed and 1 skipped, 1,201 tests passed and 5 skipped; acceptance 34 passed, 0 failed, 0 absent, 4 missing, "run passed · tree clean". The local whole-tier runs of this unit ran under load and are no proof of the whole tier; CI's runs are. A review of the pull request at `9e58019` found three things, and the commit after it fixes them. The timing command forced the path of the owner's Linux backend binary, so a fresh clone, another checkout or a Mac failed. It now leaves the choice to the harness, which downloads the pinned release into `.cache/`. Probe 7's hosted in-place example skipped `depot/migrations`, the migrations component the depot mounts, because the archive names its tables `_components/depot/_components/migrations/` and the parser read only one level. The example now reads, compares and asserts on that scope too. A throwaway run of it on a local native backend running the hosted copy passed, with `depot/migrations/migrations` among the compared tables; the hosted deployment has not run it. And this file kept the corpus digest of `1b3ff58`, from before the two fixes and the close edited Specs. -A review of 2026-10-10 across the four open pull requests found one contract break on this branch, and the commit after `77acdf2` repairs it. The receipt sweep deleted expired receipts without reading the maintenance gate, and `sweepNext` reached it, while `spec:application.write-pause` promises that every writer in a scope reads the gate inside its own mutation, names the mutations that skip the read with the sweep among none of them, and says the restore door keeps every writer out. The Spec side was right. `sweep` now reads the gate after its argument check and before its first receipt read, the restore door first, for the scopes `all` and the tenant's and no source scope, and `sweepNext` reads it through `sweep` after the tenant-list read; a closed scope refuses the run with `writePaused` and it deletes nothing. The three Specs that describe the sweep, the gate's header comment, the `convex-test` suite of the sweep on both compositions and one native test of the restore door against the sweep changed in the same commit, and the corpus digest below is re-measured. Drift repair, not a decision: no register row. A `gpt-6.1-sol` read of the repair (`job-mv27ccog-b3a71a66`) found the `typeScopeKey` entry of the write pause still naming the builders of a scope without the sweep, which builds its tenant scope through `tenantScope`; the commit after `bb82f33` adds it, and the digest below is re-measured again. +A review of 2026-10-10 across the four open pull requests found one contract break on this branch, and the commit after `77acdf2` repairs it. The receipt sweep deleted expired receipts without reading the maintenance gate, and `sweepNext` reached it, while `spec:application.write-pause` promises that every writer in a scope reads the gate inside its own mutation, names the mutations that skip the read with the sweep among none of them, and says the restore door keeps every writer out. The Spec side was right. `sweep` now reads the gate after its argument check and before its first receipt read, the restore door first, for the scopes `all` and the tenant's and no source scope, and `sweepNext` reads it through `sweep` after the tenant-list read; a closed scope refuses the run with `writePaused` and it deletes nothing. The three Specs that describe the sweep, the gate's header comment, the `convex-test` suite of the sweep on both compositions and one native test of the restore door against the sweep changed in the same commit, and the corpus digest above is re-measured. Drift repair, not a decision: no register row. The merge of that repair into this branch (`09224b5`) met the mention policy of `PLAN.md` 6.7 at the pin `f23e358`: the sweep's gate read is a `dependsOn` on `spec:application.write-pause` in `spec:command.receipt-table` and `spec:command.idempotency-and-receipts`, and the Pack page follows the gate's header comment to its new line. A `gpt-6.1-sol` read of the repair (`job-mv27ccog-b3a71a66`) found the `typeScopeKey` entry of the write pause still naming the builders of a scope without the sweep, which builds its tenant scope through `tenantScope`; `44a89c1` on `feature/round-2` adds it, merged here, and the digest above is re-measured again. + +The same `gpt-6.1-sol` read held two more cases of F5 in `scripts/pinned.mjs`, neither failing the current corpus. A name a conditional type infers was in scope in its false branch, where TypeScript does not see it, so `T extends Promise ? Outcome : Outcome` pulled no outer `Outcome` in and emitted a module with a TS2304. A value parameter hid a type reference of its name, so `run(Outcome: Outcome): Outcome` pulled nothing in and emitted two TS2749. The emitter now resolves a name in its own space, values apart from types, and gives an inferred name to the true branch only; `tests/pure/pinned.test.ts` holds both cases, compiled with and without the pulled-in pin, and the pinned module is byte-identical. The read also found that the merge `09224b5` had joined two clauses of the write pause's exemption list into `parkand`, restored in the fold, the commit after `ab7a13f`, with the digest above re-measured. Drift repair and a merge repair, not a decision: no register row. The default branch is `main`, published at `github.com/libar-dev/libar-platform`. Slice S0 reached it through a pull request; from S1 on the session fast-forwarded it once CI was green, on the owner's mandate of 2026-10-01, until the merge of slice S2 and the platform review unit on 2026-10-02, when the owner said: "Please merge to main and push. This is the last change we will do without formal PRs since things are starting to shape up." Round 1's four lanes then reached it through pull requests #6 to #9 (decision 32). The platform review as it was received is in commit `70c15bc`, with paths of the owner's machine in its links; its files left the tree at that unit's close, and the owner said the deleted review may stay in the history. +**The units `design-in-the-graph`, `pack-page` and `sdp-round-2` are on `unit/design-in-the-graph`, pull request #12**, opened stacked on `feature/round-2` and retargeted to `main` on 2026-10-10, after #11 merged there as `a758ba4`. CI passed at `36bc5a0`, the native job included, and `main` brings nothing the branch lacks: the merge tree is the branch head's tree. The Protocol's side of the unit, `feature/adopter-round-2` of `libar-dev/software-delivery-protocol` from `5e29c03`, is merged as its pull request #28 (`415b095`), and the annotations arc, `feature/annotations-v0`, as #29 (`822aea4`), so the Protocol's `main` is `822aea4` and this corpus pins it there (`b5fb659`). This close refreshes the state at that pin; #12 merges after it. The gates at `b5fb659`, run locally at this close: lint, format and typecheck clean; `npm test` 75 files, 1,151 passed and 10 skipped, at the compiled, pure test and `convex-test` tiers; `check.py` OK; and the native tier on the cached local backend, 106 files, 105 passed and 1 skipped, 152 tests passed and 5 skipped. + ## Where the work stands -Slices S0, S1 and S2 are built, one unit after them took in an independent review of the whole, one built the decision register and the advisor panel, one asked the panel for a roadmap, and round 1 of that roadmap is on `main`. On `feature/round-2` follow round 2, a design unit for the tooling, round 3's unit 3b (the online rebuild of the order summary), sitting two (the owner's decisions of round 3) and the close of round 3, which did the rest of round 3's mini roadmap. +**The unit `sdp-round-2`**, on 2026-10-04, in two repositories: this one on `unit/design-in-the-graph` from `15f3d04`, extending pull request #12, and the Protocol on a new branch `feature/adopter-round-2` from `5e29c03`, the merge of its pull request 27. Main thread Opus 5.5; no Fable agent ran. The owner: "Please help me plan and execute a logical but sizeable part of this scope: docs/feedback/sdp-feedback-03-selection.md ... Target is extension of this stacked PR, where I am attempting to take application of SDP and SPD to a level of usable tool for managing design-heavy project like the Platform. ... Given the capable suite of models at your disposal, i sugges you lean towards addrewssing a lerger scope of work vs. smaller." From the main thread's options the owner chose "Passes + SDP round 2 + adopt" for the scope and "One branch, one PR" for the Protocol. Every ruling and every word of the owner (O1 to O17, R1 to R8, F, G, H, K, L and A) is on `rulings.md` in the unit folder the project notes name, `application-platform-sdp-round-2`, with the briefs, memos, plants and designs. The owner's machine restarted once, at a seam after `0c7b7aa`, with a handover note beside the rulings. What each part showed, at the tier named: + +- The two passes over `pack:history-view`, each reading the page first. By the owner's "Opus domain, Astra operator", the domain pass ran as Opus 5.5 on the `advisor-domain` definition (6.4 min, eight findings) and the operator pass as `gpt-6-astra` at high on the `advisor-operator` definition (`job-mut5gte2-10a2467a`, 10.9 min, seven findings). Each had five page tasks: find the history contract and where each part is pinned, find OD-157 and its question, say what still rests on the aggregate form after OD-162, find one declaration's type comparison, and say whether a paused-rebuild example has native-backend evidence. The domain pass did three in full and two in part; the operator pass did all five, the third through the linked register. The page lacked OD-162, a link to the pinned module, the Contract bullets on order independence and the version rule, and the command declarations that bind the view. Each memo and each set of leans was checked by the other family on a copy with one planted citation: sol on the domain memo (`job-mut5plf6-cc622f9f`) and leans (`job-mut64uxp-2eb47318`), an Opus agent on the operator's. Every check caught its plant. +- The passes' fold-in, in three rounds, in Spec text, register rows and the pinned pairs, with no code. Round 1 (`ee8846a`): the last sentences of the built Specs that named the aggregate form point at `spec:application.aggregate-projection`; `spec:application.history-rebuild#design.historyVerify` says the pass finds a missing or lagging row and never a fold that disagrees with itself; seven non-blocking questions, OD-164 to OD-170; and OD-171, the owner's row on the terms of the history form. Round 2 (`c3637e9`), two rows decided by the domain advisor's checked leans. OD-172 says every command declaration whose `writes` names a source of a history projection binds it, and a pure test over the declarations finds one that does not; `PlaceOrder`, `CancelOrder` and `ReceiveStock` name the order allocation history in the example's Spec. OD-173 says a history projection reads every `eventSchemaVersion` its sources hold, so the pinned `HistoryEvent` carries it, and three pinned pairs moved from bound to ahead (compiled tier). Round 3 (`e54ae38`): OD-174, the owner's ruling, "One version + handoff": a history projection keeps one version, and a version-changing deploy, a first binding, an aborted rebuild and an aborted installation follow the handoff under the `all` scope (`spec:application.history-rebuild#design.historyHandoff` and `spec:application.history-rebuild#design.historyAbortHandoff`), so callers see `writePaused` where they saw technical failures. OD-175, decided by the operator advisor's checked lean: a history batch, the resume helper and `switchGeneration` compare the target's ordered sources with the gate entries naming the generation, and the projection version with the generation row, keep the pause on a mismatch, and refuse a restore's finish until compatible code is back or the generation is aborted (`spec:application.history-rebuild#design.historyCompatibility`). The write pause's list of mutations that skip the gate read names the rebuild's operator entries. +- The Protocol's round 2, on its branch. The owner ruled the model: "Ratify now" (`spec:decisions.authored-entry-order` states `ready`), "Members list on Pack node", "Key in the marker" and "Show the key"; the rest are the main thread's design rulings, in plan 39's new section. The Spec text came first (`ab024d0`), then the five changes, each package with mutations its own tests caught: open-section entries as a list in authored order (`2d5d851`, sol); a Pack's members in authored order through the graph, schema `0.7.0` (`d82d36d`, sol); the floor evaluated for a target rung, the next rung's unmet clauses on `SpecContext.nextRungFailures` and a typed-dependency failure naming its targets (`e945811`, Opus); an open question's optional key in its marker, `[blocking #key]`, with a `question` section in the entry address and the key on the Design Review (`d6c6b21`, `65edf75` and `e1e38d3`, merged at `61e0e54`, Opus, the one package in a worktree); recipes 25, address resolution in a batch, and 26, `dependsOn` cycles (`944a5a6`, Opus). A sol integrator re-derived the self-hosting pins at `27b896a`: 225 Specs, 2 Packs, 233 anchors, 460 nodes, 1,055 edges. +- The Studio slice. On the Protocol's original design the owner said: "That was more of FYI, but it is actually a pefect case what to showcase in this PR, so I will build towards that. To have a representative slice." An astra inventory of the original design against the Specs (`job-mut66y1s-f87c6a5d`, 13.6 min) proposed three slices, and the owner chose "Studio review workbench" and ruled how it is held: "the patterns should be in the implemented code and tests. We should not try to make a prose story." An Opus agent captured chapter 07's sections 1 to 3, 11 and 12 as `pack:spec-studio-v1`, twelve Specs at `scoped` (`3a0ed5c`). Where a ruled record departs from the original, the entry follows the record and a keyed question keeps the original's claim whole: 25 such questions, five blocking. Read back, four members stop at `scoped` on their blocking questions, and eight reach `defined` and are held below `ready` by the typed-dependency clause. Its 81-row claim map (5 realized, 25 in part, 25 not built, 25 departed, 1 placed later) goes in the pull request as evidence for the later decision on annotations (Owner queue). +- The Protocol's review. `gpt-6-astra` at xhigh (`job-mut8z07h-a0b05af1`, 22.4 min) read the branch against `5e29c03` on a frozen copy of `27b896a` with one plant: the authoring skill teaching the key outside the marker, which every test passed. It caught the plant and found eight more findings, folded in at `644a70f` and `dfa1bd9`: the import emitter wrote an omitted blocking flag wrongly, a repeated Pack member is kept only where the carrier admits one, four new tests, the Studio Specs' remaining homes, and plan 39. `npm run check` passed in full at `dfa1bd9`, `preflight` included. +- The adoption. A sol job (`job-mut8z0co-8575223c`, 5.4 min) moved the pin to `27b896a` and applied 166 question keys that another sol job drafted (`job-mut7blut-e5bf6617`), in one script (`8c217aa`). Every open question carries a key, the register's 152 open question sources cite their question by address, and the seven closed sources keep their text. An Opus agent then moved the page and `check.py` onto the Protocol's data (`0c7b7aa`, decision 67). Member order comes from the reader's Pack context and the next rung's clauses from `SpecContext.nextRungFailures`, and a register row joins its question by address. The Pack-file read, the floor walk and the below-defined recomputation are gone. `check.py` resolves the 489 entry addresses outside the Specs through recipe 25 and fails on any cycle recipe 26 reports. The page gained each question's key and address, a link to the pinned module and the 22 register rows whose sources lie in the members. The re-pin to `dfa1bd9` (`f04f44f`) moved nothing the corpus reads. `docs/feedback/sdp-feedback-01.md` has the dated section for the Protocol's maintainers. +- The adopter review and its fold-in. `gpt-6-astra` at xhigh (`job-mut9wpn7-575a7c07`, 35.1 min, resumed after the restart) read this side on a frozen copy of `0c7b7aa` with one plant: the deploy moved before the close of `all` in `historyHandoff`, which `check.py` and every test passed. It caught the plant as its first finding and found nine more, four P1 and five P2, ruled K1 to K8 and folded in by the main thread at `3545e9a`. `applyHistoryProjection` checks each generation's version before it reads anything, so a command whose events fold nothing fails alike on a mismatch. The abort example runs the protected abort. Three new examples give a 5d test something to bind: a version-changing deploy and a first installation through the handoff, and a source-order mismatch that parks, refuses the resume, aborts and starts afresh, so the Pack has 19 members. The restore's list of mutations that skip the gate read says what the write pause's says, and the history Specs carry `decidedBy` edges for D4, D5 and D19. New tests fail when `check.py` stops acting on an address, a cycle, a question without a key or the generator's exit, when the page's `--check` passes a stale page (pure test), and when the pinned `HistoryEvent` drifts from the code's `EventEnvelope` (compiled tier). The review retracted the `defineList` gap round 1 had listed: `defineList` refuses a newer saved schema version through `metaOf` (`convex-test`). +- The second read and its fold-in. `gpt-6-astra` at high (`job-mutbtzmh-06e6d440`) read the whole history Specs after `3545e9a` and found the release procedure still uneven, because it had been folded in from leans piece by piece: three P1 and five P2 seams. The main thread ruled them L1 to L8, stating the procedure whole, and an Opus agent applied them in round 5. What callers see is stated by case: `writePaused` while `all` is closed and, for writers of a source, while the source scopes are closed; technical failures only for an operator who skips the handoff, and only where they arise (L1). A replacement generation after an abort enters `historyHandoff` after its closing step, so no step closes a scope the procedure already holds (L2). The restore's read-model check holds for a per-entity read model, and a history view's check is OD-166's question (L3). The gate-read exemptions read alike in the restore and the write pause (L4). The interrupt, abort and source-order examples state a world in which the backfill has not ended when it is interrupted (L5). The version-change and installation examples wait until the first batch has parked under `all` before `all` reopens (L6), and the installation example installs the order summary first (L7). A test runs the page generator as a process with `--check` and expects a non-zero exit on a stale page (L8). A short third read follows before the owner's `ready` on the history Specs. +- The third read and the stop. `gpt-6-astra` at high (`job-mutch8yk-df8cfad1`) found the history projection and the version-change and installation examples ready, every new test failing under its mutation, and four seams at the protected abort's edges: an abort inside a handoff or a restore's recovery, the passes it parks, and an interrupt window a native backend could close early. Ruled M1 to M4 and folded in as round 6 (`7535f44`): the abort continues with `all` already held, resumes the parked passes just after it reopens `all`, keeps `all` closed in a restore until `finishRestoreRun` accepts, and the three examples hold the backfill before they interrupt. A `gpt-6.1-sol` check of those four alone followed, not another full read (`job-mutcyem1-1b9fe429`): M1 to M3 held, and the hold still raced, so it became a tenant scope closed before the start, which parks the backfill at the second tenant by construction (`29d31ca`, M5). The stopping rule is decision 66's. +- The gates at `29d31ca`, run locally at this close, after round 6 and M5: `check.py` OK; `test_decisions.py` 40 tests; `npm test` 74 files, 1,139 passed and 10 skipped, at the compiled, pure test and `convex-test` tiers; pinned pairs 49 bound, 17 ahead and 10 not compared. No file under `src/`, `example/`, `fixture/` or `harness/` changed in this unit, so the native tier has nothing new to show. + +**The unit `pack-page`**, on 2026-10-04, on `unit/design-in-the-graph` from `746782c`, extending pull request #12. Main thread Opus 5.5. The owner: "Please help me plan and execute a logical but sizeable part of this scope: docs/feedback/sdp-feedback-03-selection.md", and chose round 1 of the selection, steps 1, 3 and 4, from the main thread's options. The unit folder is `application-platform-pack-page`. + +- The owner's rulings: OD-162, the generation registry, the command declaration and the command pipeline pin the per-entity and history forms only ("Drop the aggregate form"); OD-163, a stated `ready` means approved to build ("Approved to build"); and the move of the feedback file into `docs/feedback/`. +- The page. `scripts/pack-page.mjs` writes `design/pages/history-view.md` from the graph in memory (`npm run pages`), deterministic and committed, and `check.py` fails when it is behind the tree, missing or unlisted (`PACK_PAGES`). The pinned pairs became data in `tests/types/pinned-pairs.json`, and the source files of the 22 anchors that then lived in tests were named in `design/tools/anchor-sites.json`, kept in step by a pure test, until the comment form moved those anchors into the source (decision 60). The page's own reads of the Pack file and of the floors gave way to the Protocol's data in `sdp-round-2`. +- One maturity per Spec, applied (`PLAN.md` 6.7). `spec:application.history-rebuild` took the rebuild's history path and its two paused examples verbatim, the rebuild and the write pause carry implementation anchors, and the aggregate form left the three built Specs for `spec:application.aggregate-projection`, so their floors reach `ready`. +- The review. `gpt-6-astra` at high on a frozen copy with one plant (`job-mut4hpxq-53aad487`, 10.0 min) caught the plant, found one more blocker, a deleted page that left nothing to check, and confirmed every member's floor. An astra reader (`job-mut4hpww-482df8d2`, 2.9 min) did four of the five page tasks from the page alone. The fold-in `0f6e91b` closed both. Two sol lanes and an Opus taste lane were stopped by the owner as excessive (decision 63). + +**The unit `design-in-the-graph`**, on 2026-10-03, on `unit/design-in-the-graph` from `77acdf2`, a pull request stacked on `feature/round-2`. The owner, after the Protocol's pull request 27 merged: "My primary motivation was to unlock management of specs at different levels of maturity and ability to capture and present designs for upcoming implementations as SDP first-class citizens and enable iterative design with iterations focusing on different perspectives of the system, before committing to implementation.", then "please investigate opportunities to better use SDP sepecs and designs (stubs) and propose a stacked PR for platform to showcase that." The main thread was Opus 5.5 at xhigh effort. It showed, on the history view as the subject: + +- The adoption of pull request 27 at `5e29c03`, with the graph's shape and `npm test` unchanged. Of the 83 prose-mention warnings, 70 were dependencies and became 56 declared relations, 12 stay prose and are listed with their reason in `design/tools/prose-mentions.json`, and one moved with its text. Two `dependsOn` edges of the event envelope ran backwards and closed nine Specs into cycles; they are reversed, and `check.py` now fails on a cycle (decision 67). +- One maturity per Spec. The projection contract became three Specs: `spec:application.projection-contract` (per-entity form and shared conventions, implemented), `spec:application.history-projection` (`defined`, nothing implements it) and `spec:application.aggregate-projection` (`scoped`, deferred with OD-075 as its trigger). `pack:history-view` gathers the Specs the history view rests on. +- The pinned module. `scripts/pinned.mjs` emits the Pack's pinned declarations into `generated/pinned/history-view.ts` on every `sdp:build`, and `tests/types/pinned.test-d.ts` keeps three lists: pairs bound where the code equals the pin, pairs where the design leads the code, and pairs not compared because they differ in the ctx they take. A name pinned in two entries stops the emitter. Thirteen names the design uses and no Spec pins stand in the emitter's preamble (Leads 44). +- Two passes, before the build. The implementation pass, read through the module, ruled six Spec changes. The Convex pass (`advisor-convex` with the new `design/advisors/task-pass.md`, 11.7 min) wrote ten findings; `gpt-6.1-sol` checked its facts on a copy with one plant and caught it (`job-musctnbl-f5337dfb`, 4.6 min). The rulings added OD-156 to OD-161, OD-157 the owner's, three F13 entries from the limits page, and a pure test that holds the largest history row at 55,803 bytes. +- What it found beside, each since closed: the Design Review sorted a Pack's members by ID (the Protocol's round 2 keeps authored order), the rebuild and the write pause showed no implementation binding (`pack-page`), and three implemented Specs depended on the deferred aggregate Spec (OD-162). + +Slices S0, S1 and S2 are built, one unit after them took in an independent review of the whole, one built the decision register and the advisor panel, one asked the panel for a roadmap, and round 1 of that roadmap is on `main`. So, since #11 merged on 2026-10-10, are round 2, a design unit for the tooling, round 3's unit 3b (the online rebuild of the order summary), sitting two (the owner's decisions of round 3) and the close of round 3, which did the rest of round 3's mini roadmap. On `unit/design-in-the-graph`, pull request #12 against `main`, follow `design-in-the-graph`, `pack-page` and `sdp-round-2`, which put the history view's design in the graph and read it in four passes before its build. The close of round 3, `application-platform-round-3-close`, on 2026-10-03 from about 09:15 to 11:15, on the ThinkPad, on `feature/round-2` from `d2a8d1b`. The cut was the list "Round 3, as it now stands" of `ROADMAP.md`, whose items 1 and 3 (3b, sitting two) were done: the hosted tier, 4a, 4c, 4d and lane 4f, which sitting two created, then an integration, a review, one fold-in, one read of it and this close. Five lanes ran at once, each in a worktree from `d2a8d1b`: language (4f, an Opus agent), hosted (an Opus lead), history (4c on paper, an Opus agent), audit (4a, an Opus designer, then an Opus builder with astra writing the proof apart) and sweep (4d, sol). `rulings-1-close.md` and `foldin-list.md` in the unit folder hold every ruling. What each showed, at the tier named: @@ -165,7 +202,7 @@ What passed, and at which tier, with backend release `precompiled-2026-09-28-5c7 Every Layer 1 scenario runs on the fixture composition. The Layer 2 scenarios run on the production composition, except the one that needs a fault. -What the code has and what it does not have yet, at the close of round 3. The Specs pin all of it and say nothing about which part is built: the owner's direction of 2026-10-01 is that a Spec carries no temporal information. +What the code has and what it does not have yet, at this close; no library, composition or harness code changed after the close of round 3. The Specs pin all of it and say nothing about which part is built: the owner's direction of 2026-10-01 is that a Spec carries no temporal information. - Pipeline: both entries, steps 1 to 6 with the ID length bound, the gate read of step 7 through `assertWritable` before the `OperationRef` is minted, step 8, step 9 with the `writes` check, each writable generation written with the projection its row's version selects, the row budget and the row count, and the return of step 11. Step 10 inserts the receipt, then, for a declaration that sets `audit`, the audit record of an applied command or a business failure only; a failed audit insert throws, and the command rolls back as a technical failure with nothing stored. The diagnostic (`src/operations/diagnostic.ts`): one line per execution of a command entry's handler, written at the end through the declaration's `diagnosticSink` (`consoleSink` when it names none), with `relayFailure` in the outermost catch, at most 4,096 bytes with `versions` cut by a binary search for the longest prefix that fits, no duration, and one gap line, never a throw, when the sink throws or returns a thenable. Step 9 has no history or aggregate path. - Declaration: `writes` is required, `readModels`, `audit` and `diagnosticSink` are optional. No `irreversible` field yet, and `retention.afterExpiry` is `"delete"` only. There is no `commandRegistry` and no registration check: the Specs dropped them, and the compiler over the generated api is the check of Sc L2-4. @@ -210,7 +247,7 @@ Review round 3 stands as it was for the rest of the corpus: no lens approved. Th | Slice | Builds | Open findings | Blocker, major, minor | |---|---|---|---| | S3 | Measurement: 1, 10 and 100 lines, without and with contention. Repeated after S4 and S5 | 1 | 0, 0, 1 | -| S4 | Generations and rebuild, the history view and the write pause, with the tenant list, operator entry points and gate audit the rebuild uses. The paper, the gate, Probe 6, the per-entity online rebuild and the history view on paper are done | 4 | 0, 0, 4 | +| S4 | Generations and rebuild, the history view and the write pause, with the tenant list, operator entry points and gate audit the rebuild uses. The paper, the gate, Probe 6, the per-entity online rebuild, the history view on paper and its four passes are done | 4 | 0, 0, 4 | | S5 | Baseline migration, restore, the rest of operators and audit, diagnostics, the receipts sweep, maintenance jobs. Probe 7's local part, command audit, the diagnostic and the sweep are done | 16 | 0, 9, 7 | | L3 | Layers 3 to 6, written from what the experiment shows | 30 | 0, 12, 18 | | P | Polish with no effect on what is built: citations, markers, stale sentences, README | 29 | 0, 0, 29 | @@ -219,11 +256,15 @@ Thirty-eight more findings are `partially-fixed`, each waiting for its slice. Re ## Next unit -`ROADMAP.md` is the order, ruled as cut by decision 36. Rounds 1 and 2 are done, and so is round 3's mini roadmap but for the hosted tier's first runs. The next unit is the owner's: the review and merge of the pull request of `feature/round-2` into `main`, which the main thread opens at this close. After the merge: +`ROADMAP.md` is the order, ruled as cut by decision 36. Rounds 1 and 2 are done, round 3's mini roadmap is done but for the hosted tier's first runs, and the four passes over `pack:history-view` are done. The next unit is the owner's: -- The hosted tier's first runs, the owner's steps 10 and 11 of `design-hosted-tier.md`: first from the owner's machine with `npm run test:hosted`, then the workflow on `main`, which the `hosted` environment admits, since GitHub runs a schedule only from the default branch. They settle the design's eight "built when" items, OD-036's count, and Leads 39 and 43's assumptions about a hosted deployment. +- The merge of #12, `unit/design-in-the-graph`, into `main`, after this close. The other two pull requests of this unit are merged: #11, `feature/round-2` into `main`, on 2026-10-10 as `a758ba4` with the receipt-sweep repair, and the Protocol's #28, `feature/adopter-round-2`, as `415b095`, followed by its #29, the annotations arc, as `822aea4`. The adopter's pin is a tarball of the Protocol's `main` at `822aea4` (`b5fb659`), made the way `docs/feedback/sdp-feedback-01.md` says under "2026-10-01, adopting pull request 26". +- The owner's `ready` statements, each an approval to build (OD-163). `spec:application.history-projection` carries no unruled `owner` row since OD-174, and the third read of the history Specs, after round 5 of the unit's fold-in, found it ready to state with the version-change and installation examples; nothing gates its `ready` but the owner's word. `spec:application.history-rebuild` and its five examples wait for OD-157 (`spec:application.history-rebuild#question.verifyPassUnderPause`). In the Protocol, the four Specs this round implemented, `spec:extraction.pack-member-order`, `spec:model.open-question-keys`, `spec:validation.next-rung-floor` and `spec:consumers.agent-surface.address-and-cycle-recipes`, still state `defined` at `822aea4` and sit with their examples in the drift alarm (recipe 2) until the owner states `ready`, as R5 expects. +- 5d, the build of the history view, once the history projection and the history rebuild state `ready`; `ROADMAP.md` says what it binds. +- The dedicated pull request of O16, after 5d or beside it, as the owner chooses. The owner: "Now it is perfect ground for demonstation in a dedicated PR. I am preparing the ground for that." It would show the graph giving, as projections, the views the owner once made of the earlier platform with a 20-hour workflow of Opus agents, a snapshot "Outdated the moment it was created.": "The Graph, when properly implemented, gives all of these as projections." This unit's Pack page, recipes 25 and 26, question keys, next-rung data and Studio slice are its ground. Nothing from that private repository is quoted in a public one without the owner's word. +- The hosted tier's first runs, the owner's steps 10 and 11 of `design-hosted-tier.md`, after the merge of #11: first from the owner's machine with `npm run test:hosted`, then the workflow on `main`, which the `hosted` environment admits, since GitHub runs a schedule only from the default branch. They settle the design's eight "built when" items, OD-036's count, and Leads 39 and 43's assumptions about a hosted deployment. - The dashboard check of `clean-capybara-16`: a production deployment, reference `production`, never deployed, with no tables and no keys, made by the project's default. The design wants `dev/hosted` only, so the owner deletes it if nothing uses it. Deletion is permanent and an agent does not do it. -- Then the roadmap's remaining units, 4e (`ready` by family, the owner's), 5a, 5b, 5c and 5d, discussed with the owner before they are prepared (decision 42). +- Then the roadmap's remaining units, 4e (`ready` by family, the owner's), 5a, 5b and 5c, discussed with the owner before they are prepared (decision 42). Under OD-163, 4e is the owner stating `ready` on a family approved for its build. The setup is done: the project `libar-platform-hosted` and its development deployment `dev/hosted` (`wandering-mule-403`, `HOSTED_DEPLOYMENT_URL` in the regional form the CLI printed), the GitHub environment `hosted` with its `main` branch policy and its three variables (`HOSTED_PLAN` is `Professional`, `HOSTED_DEPLOY_KEY_NAME` is `hosted-ci-scoped`), and the key `hosted-ci-scoped` with exactly the fourteen actions as the secret `HOSTED_DEPLOY_KEY`, which passed the key-only check (Owner queue); the first key, with 25 actions, is revoked. The local backend stays primary, by the owner's words of 2026-10-03: "Local backend should be primary, IMO. But happy to take the hosted as default if easier for start." @@ -240,7 +281,7 @@ What S2 left, as it stands now: Decisions each later slice needs on paper before its code: -- S4, the history view: taken on paper in 4c (`design/specs/application/**`, ten ledger findings); what is left is the build, 5d, with Leads 43's step 9 gap. The rest of S4's paper was taken in round 2: `design-s4-paper.md` sections 1 and 7. +- S4, the history view: taken on paper in 4c (`design/specs/application/**`, ten ledger findings) and in the four passes over `pack:history-view`, whose rulings are OD-156 to OD-161 and OD-164 to OD-175; what is left is OD-157, the owner's, the sort of OD-164 to OD-170 by their lenses, and the build, 5d, with Leads 43's step 9 gap. The rest of S4's paper was taken in round 2: `design-s4-paper.md` sections 1 and 7. - S5: the history-reading sweep's transient refusal and the restore check's pending-work inventory, ruled at sitting two (OD-041) and designed with the baseline unit, including a read for the journal check that `list` does not refuse with `writePaused` (restore's open question from F5); what a restore in place does with the schedules it keeps (`probe-7-architecture-restore-kept-schedules-not-reconciled`); how the restore check detects a stream state that is not the fold of its events (`s3-measurement-architecture-current-state-not-checked-against-event-fold`); the read-only comparison of the restore's read-model check (`s4-paper-sdp-restore-read-model-check-compare-mode-undefined`). What the platform review of 2026-10-02 asks of the later units. The ledger owns each finding's counterexample and what its build must show. This is the order and the guidance that fits no single finding. @@ -260,7 +301,7 @@ What the platform review of 2026-10-02 asks of the later units. The ledger owns ## Tactical decisions taken on 2026-10-01 -The owner accepted the first nine on the session's recommendation. The session took 10 to 19 during S0, 20 to 23 during S1, 25 to 27 during S2 and 28 to 31 during the platform review unit, on 2026-10-02. Decisions 32 and 33 are the owner's words of the same day. Decisions 40 to 45 were taken during round 2, on 2026-10-02 and 2026-10-03, most of them on the owner's words, 46 and 47 during the tooling unit on 2026-10-03, 48 and 49 during round 3's unit 3b the same day, 50 and 51 at sitting two the same day, and 52 at the close of round 3 the same day. Any of them can be reopened. +The owner accepted the first nine on the session's recommendation. The session took 10 to 19 during S0, 20 to 23 during S1, 25 to 27 during S2 and 28 to 31 during the platform review unit, on 2026-10-02. Decisions 32 and 33 are the owner's words of the same day. Decisions 40 to 45 were taken during round 2, on 2026-10-02 and 2026-10-03, most of them on the owner's words, 46 and 47 during the tooling unit on 2026-10-03, 48 and 49 during round 3's unit 3b the same day, 50 and 51 at sitting two the same day, 52 at the close of round 3 the same day, 53 to 57 during the unit `design-in-the-graph` the same day, 58 to 63 during `pack-page` on 2026-10-04, and 64 to 68 during `sdp-round-2` the same day. Any of them can be reopened. 1. Slices replace the eight fix units of the previous plan. 2. A slice is built on Specs at `defined` with no open blocker inside the slice. The owner states `ready` after the slice passes. @@ -315,33 +356,61 @@ The owner accepted the first nine on the session's recommendation. The session t 51. A paper sitting renames nothing in the code. The words sitting two ruled leave the refusal field, the tier names in the examples' opening lines and the harness parser, and the harness's backup archive names disagreeing with the Specs on purpose, as roadmap lane 4f, so that the branch the owner reviews stays green and small. 52. How the close of round 3 ran, at the owner's word in 42. The main thread was Opus 5.5 and no Fable agent ran: Opus agents designed on paper where a Fable designer did before (4c, and 4a's Spec text), and the main thread ruled every sentence they wrote. 4a ran proof apart from fix (decision 29): an Opus builder and astra wrote from the ruled text alone, and the integrator merged them. The integration ran in four stages as the lanes finished (decision 49). The review over the branch since `840b32c` was four line reads by sol and one behavior review by astra at xhigh, with an Opus agent reading the GPT-written code and sol the Claude-written code (decision 30). One fold-in and one read of it followed, then the stop (decision 19). +53. A prose mention of a Spec is a declared relation or a listed reason. The Protocol's validator warns once per pair with no relation; `check.py` accepts a warning only for a pair in `design/tools/prose-mentions.json` with its one-line reason, and refuses a listed pair that no longer warns. A relation is declared when the mentioning Spec could not be built or read correctly without the other; `constrainedBy` stays for laws, facts and constraints and a component Spec of kind `rule` is depended on with `dependsOn` (`PLAN.md` 3.3); the corpus keeps no `dependsOn` cycle. +54. One maturity per Spec (`PLAN.md` 6.7). The projection contract is split into the per-entity, history and aggregate projections, the moved text verbatim. The new IDs follow the code's names, `HistoryProjection` and `AggregateProjection`, while the language rows "history view" and "history projection" wait for the owner; a ruling of other words renames them. +55. The pinned module is this corpus's recipe for the Protocol's declarations module: `scripts/pinned.mjs` over one Pack, run by `sdp:build`, and `tests/types/pinned.test-d.ts` with three lists named by entry address, bound, ahead and not compared. A pin that leads its code is listed ahead with what closes it, and a duplicate pin fails the compile. The scope grows a Pack at a time. +56. A Design entry is cited by its entry address, `spec:#design.`, in code comments, test names, the register, the ledger and the advisors' memos; `check.py` resolves each one outside the Specs. A line pointer into a Spec is replaced when its file is next touched. +57. A pass reads one capability's Pack from one lens before its build, with `design/advisors/task-pass.md`; its memo is checked like a roadmap memo, on a copy with a plant, before the main thread rules. The implementation pass is the main thread's, read through the pinned module. +58. A capability Pack has a committed page, `design/pages/.md`, written by `scripts/pack-page.mjs` from the graph in memory, with no commit, timestamp or absolute path; `check.py` keeps the list of pages (`PACK_PAGES`) and fails on one that is behind, missing or unlisted. An edit to a member Spec is followed by `npm run pages`. A pass reads the page first. +59. The pinned pairs are data, `tests/types/pinned-pairs.json`, beside the type test, which a pure test keeps in step; the page reads them and parses no test. The pinned module is formatted by Prettier. +60. Amended by decision 69, which rules the mapping and makes the coverage test read the anchors. An implementation anchor sits in the source file it binds, as a comment-form block of reserved `@sdp` tags that imports nothing from the Protocol (`57ed4f8`, at the pin `da56a34`). There is one identity per realizing unit. A unit satisfies a Spec its own code realizes and references a design it is written against, and a reference confers no delivery fact. A component anchor states its layer and context from the provisional mapping until the owner rules on it (Owner queue). No test file keeps an implementation anchor, and the harness keeps its 18 constant-form anchors. `design/tools/anchor-sites.json`, the join that stood in while anchors lived in tests, is deleted with its test, and `tests/pure/byfile-coverage.test.ts` asserts that `byFile` names each of the 50 file and Spec pairs the join listed. +61. Ended by decision 65. The register cited an open question by its Spec and the opening of its text, never by position, and `check.py` verified it. +62. Ended by decision 69. A built Spec may pin a type that a designed form widens, listed ahead; a behavior the code lacks lives in a Spec of its own (`PLAN.md` 6.7). The rebuild is split by it. +63. The owner, 2026-10-04, on the four worktree lanes and five review lanes of `pack-page`: "This approach is appropriate if for implementation of a large full-week epic. ... it is excessive, very."; on worktrees: "I would say no benefit at all."; and "situation dictates actions. And intelligence of the orchestrator decides. Guides are recommendations to consider (unless project has hard rules)." Until the owner rules on the proposal (Owner queue), the main thread sizes review and isolation to the work, and one review that runs things is the usual start. +64. An advisor runs on the model the owner names for it. For the passes over `pack:history-view` the owner chose "Opus domain, Astra operator", so the domain pass ran as Opus 5.5 on the `advisor-domain` definition and the operator pass as `gpt-6-astra` at high, briefed with the `advisor-operator` definition's three paths and `design/advisors/task-pass.md`. `design/advisors/protocol.md` bound both alike, and each memo and each set of leans was checked by the other family on a copy with one planted citation. With no model named, an advisor is a Fable agent (decision 33). +65. Every open question carries a key, and a tool cites it by its address. The owner's ruling of 2026-10-04 on the Protocol's model: "Key in the marker". So the marker reads `[blocking #key]` or `[non-blocking #key]`, the key unique among one Spec's open questions, and the question's entry address is its Spec's ID, then `#question.` and the key. `check.py` fails on an open question without a key. The register cites an open question by address; a source whose question left its Spec keeps its Spec and last text with `open: false`, on a decided or folded row only. `decisions.py`, `check.py` and their tests resolve an address through recipe 25, so a removed or renamed key fails the check. In prose a question is cited by its OD number. +66. One review per repository, by `gpt-6-astra` at xhigh, on a frozen copy with one plant, running what it reads, as decision 63 sizes it. In this unit the Protocol's branch had one (`job-mut8z07h-a0b05af1`, 22.4 min) and this repository's side one (`job-mut9wpn7-575a7c07`, 35.1 min), and both caught their plant. A second read of the history Specs (`job-mutbtzmh-06e6d440`) followed the fold-in, because the release procedure had been folded in piece by piece, and a short third read follows its own fold-in, round 5. What the plants showed: the builders' own tests caught every code plant tried, and two candidate plants for the Protocol's copy were withdrawn for that reason. The plants that tested a reviewer sat where no gate reads: in taught text (the authoring skill teaching the key outside the marker, with every test passing) and in Spec prose (the deploy moved before the close of `all` in `historyHandoff`, with `check.py` and every test passing). So a plant goes in taught text, Spec prose or a citation, and a code plant is first tried against the tests. +67. The Pack page and `check.py` read the Protocol's data, not this corpus's copies of it (A2, A4). The page takes member order from the reader's Pack context and the next rung's unmet clauses and typed-dependency targets from `SpecContext.nextRungFailures`. It keeps its own meaning of the next rung, the rung above the stated one, and says nothing blocks it when the floor is above the stated rung. `check.py` sends every entry address outside the Specs to recipe 25 in one call and fails on any set recipe 26 reports; finding the addresses in files and naming each one's line stay this corpus's. The adopter's joins stay: the pinned pairs and the register. The anchor sites went when the Protocol gave the comment form (decision 60). A workaround goes when the Protocol gives the thing, and what the Protocol still lacks goes to `docs/feedback/sdp-feedback-01.md`. +68. A worktree only where two builds would race. The rulings of 2026-10-04: "No worktree unless two lanes would break each other's gates." In this unit one package had one: Q, the question key, whose build would have raced package G's on the Protocol's `dist/`. The other packages ran in the main checkout, two lanes at once where their files were disjoint. Decision 39 says how a worktree is set up when one is made, not that every lane gets one. +69. The owner's rulings of 2026-10-10 on the annotations arc, applied at the pin `f23e358` (`ecba393`). A reference runs one way: code answers to a design it follows or realizes in part, and a design that builds on existing code says so by its own relation to the Spec that code satisfies; the online rebuild's reference to the history rebuild came out, and so did a reference kept only for a `byFile` pair. Anchor targets name whole Specs, so the units that satisfy a Spec together realize every entry it promises, a pinned declaration ahead of its code included (`PLAN.md` 6.7 item 7). The pass over the twelve Specs the anchor check named, and `spec:command.receipt-table`, found in each a part no unit builds running through entries the code realizes, so their units reference them. A second read by `gpt-6.1-sol` found three more, whose units reference them too: `spec:application.parent-use-cases` (the business failure policy and the import command), `spec:application.projection-contract` (the history writer's callers) and `spec:kernel.outcome-model`, whose reserved codes the adapter's `invalidInput` contradicts. A component satisfies its seam's design Spec or binds nothing its members bind. The layer and context mapping stands, with the owner's rules in `PLAN.md` 2.1, and `check.py` fails when a platform component uses a component of another context. `tests/pure/byfile-coverage.test.ts` reads its pairs from the anchors in the tree. + ## Owner queue Every open decision is a row of `design/decisions/register.json`, and `python3 design/tools/decisions.py` filters it: `--owner` lists what waits for the owner, forks first, `--owner --before S4` what the owner must decide before S4, `--decided --by advisor --since ` what agents decided. The register at the close: ``` -total: 155 -status: unsorted=0, sorted=80, waiting=21, decided=39, folded=15 -class: tactical=3, delegated=86, owner=51, null=15 -advisor: convex=37, domain=45, operator=34, product=39 +total: 175 +status: unsorted=12, sorted=82, waiting=21, decided=45, folded=15 +class: tactical=3, delegated=89, owner=56, null=27 +advisor: convex=43, domain=51, operator=41, product=40 ``` The key check is done ("Next unit"). For the record, the procedure that worked with CLI 1.46.0, which refuses a directory with no `package.json`: in a fresh temporary directory, write `{"name":"libar-hosted-key-check","private":true,"dependencies":{"convex":"1.46.0"}}` as `package.json`, install nothing, paste the key at a silent `read -rs CONVEX_DEPLOY_KEY`, export it, run the repository's `node_modules/.bin/convex env list --names-only` under a fresh temporary `HOME`, then unset the variable. An empty list with no error means the key authenticates. -What waits for the owner at the close of round 3: - -- The pull request of `feature/round-2` into `main`, which the main thread opens at this close: the owner reviews and merges it. +What waits for the owner at this close: + +- The `ready` statements, each an approval to build (OD-163): `spec:application.history-projection`, which the third read found ready to state, with the version-change and installation examples; `spec:application.history-rebuild` and its five examples once OD-157 is ruled; the Protocol's four implemented Specs and their examples. No Spec of this corpus states `ready` yet. +- OD-157, the verify pass under the write pause (convex lens). Under the pause no source changes, so the pass finds nothing over a covered generation and doubles the pause; the option kept is the pass, as D9 and Sc L2-6 read, and whether a paused generation is `verified` when its backfill ends is the owner's. The domain pass added what the pass can find: a missing or lagging row, never a fold that disagrees with itself (`spec:application.history-rebuild#design.historyVerify`). +- OD-171, the terms of the history form: history projection or history view, source version with its meaning on each form, and binding, and whether Read model and Fold widen to rows fed by events. +- Annotations. The owner ruled on the annotations arc on 2026-10-10; the Protocol's `main` carries the rulings since the merge of its #29 (`822aea4`), the pin since `b5fb659`, after the branch `feature/annotations-v0` at `f23e358`, the pin from `04a6aff`. Every item this queue held for it closed: the layers, the contexts, `example/` as architecture, the reference direction and the whole-Spec satisfies (decision 69; `docs/feedback/sdp-feedback-01.md`, section "2026-10-10, the rulings at `f23e358`", holds the per-Spec table). One item remains: + - The capability steps' name. "Layer" names one of the seven steps of capability in this corpus and, as the Protocol's `layer`, a component's architectural layer; `CONTEXT.md` now writes the second in full as "architectural layer". Renaming the capability steps, with "stage" as the candidate, is the one-word-one-meaning answer. It touches a Spec identity (`spec:platform.layers-and-profiles`), 143 numbered uses across 65 carriers and the decisions document the rubric binds this corpus to follow, so it is an adopter-language decision for the owner. +- The 25 departures of `pack:spec-studio-v1`, each a keyed open question, five of them blocking, where a ruled record of the Protocol goes another way than the original design: among them the patch loop against `spec:consumers.edit-model`, HTML first against the Markdown Design Review, and seven rungs against four. The Protocol's pull request lists them. +- Words this unit used that the language lacks. In this corpus's `CONTEXT.md`: handoff and protected abort (OD-174's procedure), question key, next rung, target rung, dependency cycle. In the Protocol's: the same four, and from the Studio slice Spec Studio, lens, panel, shell, custom element, gathered intent, hand-off, build stamp, hosted preview, harness. The history view's "handoff" and the Studio's "hand-off" name two different things. +- Words the units `design-in-the-graph` and `pack-page` used and `CONTEXT.md` lacks: pinned declaration (the Protocol's "declaration", which meets this corpus's "command declaration"), entry address, mention, pass, capability Pack, history rebuild (a Spec's ID), Pack page, stated rung, floor, pair with bound, ahead and not compared, an anchor's site, register row; and from the Convex pass, batch, source, consistent cut, history view. +- The proposal on sizing review and isolation to the work, from the owner's words of 2026-10-04, is `proposal-review-sizing.md` in the unit folder of `pack-page`. It would change step 6 of `slice` in `SESSIONS.md`, the guide's run pattern and decision 39's default, once the owner rules; decisions 66 and 68 are how this unit sized them meanwhile. - The doc's two hunks, from OD-008 and OD-044, in `patch-doc-sitting-two.diff` of sitting two's folder: `git apply ~/dev-libar-supporting-context/calibration/application-platform-sitting-two/patch-doc-sitting-two.diff` from the repository root, then a commit. Only the owner edits the doc. - The hosted tier's first run, from the owner's machine and then by the workflow on `main`, and the dashboard check of `clean-capybara-16` ("Next unit"). OD-036, Probe 3's quota half, is `waiting` for that run. -- The untracked `tmp-convex-findings.txt` at the repository root, the owner's notes of the hosted key rotation, makes every local run record "not clean" until the owner moves it. +- The untracked `tmp-convex-findings.txt`, the owner's notes of the hosted key rotation, at the root of the ThinkPad's checkout, makes every local run record there "not clean" until the owner moves it. The Mac's checkout, where this unit ran, has none. - OD-135 and OD-095 wait for their advisors when the owner next asks for a sort: OD-135's lean held 9 of 19 claims and needs its advisor's correction, and OD-095's held 6 of 6 but is a `do` alone and needs its lean in full. -- The language rows still open, which keep the code's names until the owner rules: interrupt, park, stall, batch chain, fill, the aggregate's marker, history view, history projection, tombstone, drill, door, gate, audit record, diagnostic record, diagnostic gap, adopter, package, activation record; and from this unit Probe 3's "set of calls", the one "preview deployment", the Vitest project names (`types`, `pure`, `simulator`, `native` and now `hosted`) beside the four tier names, and the design keys `tierDomain` and its siblings, which keep their IDs. +- The language rows still open, which keep the code's names until the owner rules: interrupt, park, stall, batch chain, fill, the aggregate's marker, history view, history projection, tombstone, drill, door, gate, audit record, diagnostic record, diagnostic gap, adopter, package, activation record; and from the close of round 3 Probe 3's "set of calls", the one "preview deployment", the Vitest project names (`types`, `pure`, `simulator`, `native` and `hosted`) beside the four tier names, and the design keys `tierDomain` and its siblings, which keep their IDs. + +Decided since the last close: OD-174, the history projection's one version, by the owner from the main thread's options ("One version + handoff"); OD-172 and OD-173 by the domain advisor's checked leans; OD-175 by the operator advisor's checked lean. `decisions.py --decided --since 2026-10-04` lists them with OD-162 and OD-163, the owner's rulings of `pack-page`. The seven questions of the passes, OD-164 to OD-170, are `unsorted` and wait for their lens's sort, OD-164 and OD-165 the domain's and OD-166 to OD-170 the operator's. -Decided since the last close: OD-149, the tenant created behind a pass's cursor, by its operator advisor's checked lean (`19738b4`); and OD-155, the hosted tier's money, as the owner's, from the owner's words of the tooling unit on the advisor's eight hosted recommendations, applied by the main thread's register patch (`c1bdf80`), which also set OD-036 `waiting`. `decisions.py --decided --since 2026-10-03` lists them with sitting two's sixteen `owner` rows (decision 50) and the advisor rows of 3b and the tooling unit (OD-057 by the convex advisor; OD-010, OD-014, OD-071, OD-075 and OD-080 by the domain advisor). At sitting one twelve `owner` rows were recorded as the owner's (decision 40) and OD-007 was decided by the convex advisor. OD-154, the grant read, is a `delegated` row with its measured cost. OD-093's provisional reading still quotes the old E-9 text, a field the tool does not patch (Leads 43). +Before that, at the close of round 3: OD-149, the tenant created behind a pass's cursor, by its operator advisor's checked lean (`19738b4`); and OD-155, the hosted tier's money, as the owner's, from the owner's words of the tooling unit on the advisor's eight hosted recommendations, applied by the main thread's register patch (`c1bdf80`), which also set OD-036 `waiting`. `decisions.py --decided --since 2026-10-03` lists them with sitting two's sixteen `owner` rows (decision 50) and the advisor rows of 3b and the tooling unit (OD-057 by the convex advisor; OD-010, OD-014, OD-071, OD-075 and OD-080 by the domain advisor). At sitting one twelve `owner` rows were recorded as the owner's (decision 40) and OD-007 was decided by the convex advisor. OD-154, the grant read, is a `delegated` row with its measured cost. OD-093's provisional reading still quotes the old E-9 text, a field the tool does not patch (Leads 43). `delegated` rows that wait for the next sort of their lens, each with the round's result on its lean: OD-066 (3N + 5 with every term attributed; the journal tail read is 10 documents and 6,010 bytes, 29 percent of the documents read on ten lines), OD-120 (Probe 7's states) and OD-154 (3 documents and 918 bytes for the grant read) for the convex lens, OD-081 (Probe 7's restore answers) and OD-135 (its check) for the operator lens, and OD-095 (its check) for the domain lens. The session may not decide a `delegated` row (`decisions.py`: "session may decide only tactical rows"). -The 86 `delegated` rows carry the leans of the first panel run, five of them replaced in the tooling unit. A `gpt-6.1-sol` check read the first 23: twelve held on every checked claim and carry `checkedBy`; eight hold with a gap and three do not hold (OD-066, OD-103, OD-109), each waiting for its advisor's correction with the check's findings in the unit folder the project notes name. Later checks added `checkedBy` to OD-057, OD-071, OD-075 and OD-080 in the tooling unit and to OD-095, OD-135 and OD-149 in the close of round 3, so eighteen rows carry it, and eight of them are decided. The other leans are unchecked. A checked lean is applied to the Specs as a ruling by a later unit, one Spec family at a time, and the row becomes `decided`. The rest carry a one-line lean or wait for a probe, a measurement or a build (`--waiting`). +Of the 89 `delegated` rows, 86 carry the leans of the first panel run, five of them replaced in the tooling unit, and the three this unit added, OD-172, OD-173 and OD-175, carry checked leans and are decided. A `gpt-6.1-sol` check read the first 23: twelve held on every checked claim and carry `checkedBy`; eight hold with a gap and three do not hold (OD-066, OD-103, OD-109), each waiting for its advisor's correction with the check's findings in the unit folder the project notes name. Later checks added `checkedBy` to OD-057, OD-071, OD-075 and OD-080 in the tooling unit to OD-095, OD-135 and OD-149 in the close of round 3, and to OD-172 to OD-175 in this unit, so twenty-two rows carry it, and twelve of them are decided. The other leans are unchecked. A checked lean is applied to the Specs as a ruling by a later unit, one Spec family at a time, and the row becomes `decided`. The rest carry a one-line lean or wait for a probe, a measurement or a build (`--waiting`). ## Leads @@ -373,7 +442,7 @@ A session confirms or drops each. None is a finding yet. 7. Four type names used in pinned signatures and declared by no Spec: `HistoryArgs`, `WriteBaselineArgs`, `WriteBaselineResult`, `BatchCursor`. Recipe 23 finds where each is used. 8. `dispatchId` on the obligation row is `v.id("_scheduled_functions")`. Nothing shows whether such an ID validates after a restore into another deployment. It belongs in Probe 7. 9. The mention audit (recipe 22) gives 78 unbacked pairs, 50 of them declared from neither side. Each slice checks the ones in its families. -10. Using the Protocol more, from a side thread's read on 2026-10-01. A local query for the build backlog at `defined` (recipe 1 reads stated `ready`, which no Spec here has before its slice passes; the same question at `defined` with a clear floor gave 80 Specs). A Pack per slice, so that "which Specs does S1 build" is a graph read. `check.py` counting open questions through recipe 20 in place of its own expression. The carrier grammar restated in `PLAN.md` 6.2, which can drift from the authoring skill's table. +10. Using the Protocol more, from a side thread's read on 2026-10-01. A local query for the build backlog at `defined` (recipe 1 reads stated `ready`, which no Spec here has until the owner approves its build, OD-163; the same question at `defined` with a clear floor gave 80 Specs). A Pack per slice, so that "which Specs does S1 build" is a graph read. The carrier grammar restated in `PLAN.md` 6.2, which can drift from the authoring skill's table. 11. Small things in the harness that no finding carries. The sweep signals by process id after reading the process table, and the Spec says the interval is not closed. A `convex dev` child killed with SIGKILL leaves a temporary directory behind, which a `TMPDIR` inside the watch mode's own home would remove. The scripts' hand-over of the interrupt to executable resolution is tested by reading their source. The admin read's page size of 10 in the shared-limits helper is not needed for the read to succeed. Inside a Codex sandbox `ps` cannot inspect other processes, so the sweep tests fail there and pass outside it. 12. Left by the last review of S0 on purpose. Redaction of a failed child's output scans a character at a time: 901 ms and about 725 MiB at peak for a 16 MiB output, where the version before the overlapping-secret fix took 3 ms. No pure test covers the accepting path of the download's hash check; it needs a real archive as a fixture, and the path runs for real whenever the cache is cold. Four mutations survive with no test: the interrupt not passed to `unzip`, the SQLite-path match dropped from the ownership check, SIGTERM mapped to SIGINT in `scripts/dev.mjs`, and a global teardown that does nothing. The test that both scripts hand their interrupt to executable resolution parses their source, and a shadowed variable or a later spread that overrides the step passes it. A failed amendment of a run record leaves a `.next` file beside the whole record. @@ -382,7 +451,7 @@ A session confirms or drops each. None is a finding yet. 14. Left by the review of S1 on purpose. Tests of fixture code that would not catch a removal: the evaluate-twice test compares its two outputs only after the second evaluation, states that `evolve` produces are never frozen, the reference stream has no purity or rebuild example, and the submitted event's schema stamp is pinned by no test. `tests/native/depot-operation-commits.test.ts` sends its calls with admin access, and the pure guard that forbids it looks only for the token `adminKey`. The rule for repeated document items in one call (identical items merge, items that differ are `invalidInput`) is the fixture's and no Spec states a general one. 15. The native tests of S1 read function-log fields no Convex page promises: `willRetry` and `occInfo`, the absence of a record for a component sub-call, `usageStats.databaseReadDocuments`. They are `observed` entries of the native harness Spec and are read again when the backend pin moves. The two race tests rerun on a fresh key until the log shows an engine retry, up to twenty rounds. 16. About ten older Spec sentences still name a slice or a date, such as "slice S2 decides" in `spec:context.queries` and "ran on 2026-10-01" in open questions. The owner's direction is against them; a unit that touches those Specs moves them out, and a sweep waits for the owner's word. -17. The kernel's code anchor lives in `tests/pure/kernel.test.ts`, because the kernel may not import the Protocol package, so the graph cannot say what a change under `src/kernel` reaches. `npx sdp build` writes `*.test.generated.ts` files beside a suite that binds an example whose parent has an example space; they are ignored by git, Prettier and ESLint. Both are in `docs/sdp-feedback.md`. +17. Closed in part on 2026-10-09. The kernel's anchors sit in `src/kernel/` as comment-form blocks that import nothing (decision 60), so `byFile` on `src/kernel/decider.ts` names `spec:kernel.decider-contract`. What stands: `npx sdp build` writes `*.test.generated.ts` files beside a suite that binds an example whose parent has an example space; they are ignored by git, Prettier and ESLint. Both are in `docs/feedback/sdp-feedback-01.md`, and its section "2026-10-09, anchors in the source at `da56a34`" closes the first. 18. `append` refuses a stream row whose version has no event in the journal. A later audit-only context that imports rows with incomplete history would trip it. 19. Dropped at the close of the roadmap unit: the root `README.md` now names every `_generated` directory under both compositions. What remains: nobody ran `npm run dev` to see that it rewrites them, and `scripts/dev.mjs` names only `annex/_generated` in a comment. @@ -412,7 +481,6 @@ A session confirms or drops each. None is a finding yet. 28. From round 1. `design/specs/application/rebuild.sdp.md` near line 45 promises a one-write rollback and near line 84 a reverify and switch: the fork of OD-068 shows both, and S4's paper rules it. The pipeline accepts a stock stream row at version zero with positive stock and no event, which no command makes and the version-zero comparison planted. The journal tail read and the grant read stay, by C3 and C4 of round 2, and removal needs the writer-invariant proof first. OD-066 waits for its lens with 3N + 5 attributed. Probe 7 closed three items of this lead: `writeTable` is exercised against component tables, a ZIP reader replaced `unzip`, and `BackendFacts.composition` is `null` after a temporary deploy. The f15 remainder names a work stage and a run date, which overlaps the temporal sweep of leads 16 and 22. `design/PLAN.md` near lines 297 and 299 still lists one example for Sc L2-2 and L2-4. The harness's child runner now has a both-streams output whose timeout reports through the bounded error and not the "did not finish" message. 29. From round 2. Each is a test gap, a hole in a failure path, a Spec line about what is not built, or a number still to take. - - `design/tools/test_decisions.py` has one test that fails at `main` too: `test_real_register_has_unique_rows_and_exact_recipe_20_coverage` ends in an `IndexError`. It is not one of the gates. - Journal inspection. `orders-inventory-example.sdp.md` near line 70 names the context queries that return a stream's events and its current version; the Orders and Inventory contexts register only `get` and `list`, no `history` query of `spec:context.queries`, and the native tests read journals through admin table reads. The Spec line stays as intended truth. - The Probe 6 context callback at `fixture/convex/migrations.ts:198-203` writes a `draft` summary beside the source's version whatever the source's status. Fixture probe code that nothing rests on; fixed only if that fixture is reused. - A partial tenant list passes `startGeneration`'s refusal: a list that holds a tenant whose first grant came after the table, and misses an older one, gets through. The rebuild Spec states it as an open question and tells the operator to run `fillTenants` first. `limitTenantFillBatch` is 8 because a grant row is bounded only by the 1 MiB document limit. @@ -471,6 +539,33 @@ Leads 30 to 38 are lane data's of the tooling unit, as `leads-data.md` in its fo - OD-093's provisional reading quotes the old E-9 text, and the register tool does not patch that field. - The interrupt accepts a `retired` or `aborted` generation, because its row does not show whether a purge's batches run; a fence bump with nothing running is harmless, and a purge marker on the row would tell the two apart if an interrupt of a purge must ever be refused. +44. From the unit `design-in-the-graph`. Each is a gap a later unit takes when it touches its subject. + - Nine pairs of `tests/types/pinned.test-d.ts` are not compared because they differ in the ctx they take: the pinned `MutationCtx` is the emitter's generic stand-in, so it never equals a composition's ctx. Five of them (`typeCommandDeclaration`, `fnPublicCommand`, `fnInternalCommand`, `fnRunPipeline`, `fnRelayFailure`) also lead the code and carry the ctx inside the executor type, so nothing flips when their build lands. Either the notation pins a ctx generic over the data model, or the emitter compares with the ctx erased throughout. + - Thirteen names the design uses and no Spec pins stand in the emitter's preamble, each pointing at the code's type: `ReadModelDataModel`, `RowDataModel`, `GateDataModel`, `ListArgs`, `BatchRef`, `FillBatchRef`, `AppendResult`, `SubjectRef`, `DiagnosticSink`, `FailedCall`, `outcomeKindValidator`, `WriteBaselineArgs`, `WriteBaselineResult`. The Spec that uses one may pin it when next touched. + - The emitter takes no Convex registration (`export const name = mutation({ ... })`, 51 `fn*` entries in the corpus), no `index*` fragment and no `limit*` formula; the module covers shapes, not the registered surface. + - The Convex pass left questions to two lenses, and their passes took them in `sdp-round-2`. The domain pass said what `verified` asserts under the pause, a missing or lagging row found and never a fold that disagrees with itself (`spec:application.history-rebuild#design.historyVerify`). The operator pass kept the acceptable pause length for the owner, not a timeout derived from a batch estimate, and asked how the operating instructions measure the closed interval (OD-170); who takes the batch time stays open. + +45. From the unit `pack-page`. + - `readModelTables` in `src/read-model/tables.ts` still installs `projectionMarkers`, which only the aggregate form uses and no Pack Spec pins after OD-162; its pair is not compared. The build that next touches the read-model tables drops it, or OD-075's build keeps it. + - The rebuild keeps the whole example space, because a one-value slot is refused; `spec:application.history-rebuild` has none, and its five examples' steps are free-form. + - Closed on 2026-10-09: the anchor sites lived beside the graph. Each source file now carries its own anchor (decision 60). `byFile` on `src/read-model/rebuild.ts` names `spec:application.rebuild` and `spec:application.history-rebuild`, and on each file under `src/gate/` it names `spec:application.write-pause`. The Pack page reads the units from the graph alone. See `docs/feedback/sdp-feedback-01.md`, section "2026-10-09, anchors in the source at `da56a34`". + - The page is about 186 KB at this close, most of it the pinned-declaration tables. The reader's remaining obstacles: the test link has no line anchor, and the aggregate answer is spread over three sections. + - Every edit to a member Spec moves line links, so `check.py` fails until `npm run pages` runs. + +46. From the unit `sdp-round-2`. Each is a gap this unit found and did not fix. + - Decision citations against `decidedBy`. The adopter review found new bullets of the history Specs citing D4, D5 and D19 with no `decidedBy` edge, which `PLAN.md` 6.4 asks for (K6). The fold-in added those edges by hand; nothing checks the rest of the corpus. A check in `check.py` that every decision record a Spec cites is reached by its `decidedBy` edges would find the others. + - The declarations that bind a history view are stated only in prose, in the three model terms of `spec:application.orders-inventory-example` that OD-172 changed, and no declaration in the code binds the view yet. The page reads no prose, so it shows nothing for them. A keyed Design entry pinning each binding would reach the page through recipe 24; this corpus's gap, recorded in `docs/feedback/sdp-feedback-01.md`, section "2026-10-04, adopting round 2 at `27b896a`". + - An entry's line in the graph. The page reads each member's carrier to find the line of every Design entry and open question, because the graph records the line of a binding and not of an entry; for the Protocol, in the same section's "What still needs the Protocol". + - A recipe's parameter as data. Recipe 25 takes its addresses on its opening `const` line, which `design/tools/decisions.py` rewrites with a regular expression, 489 addresses and 28,788 characters of JSON on the command line. An input read from the environment, as recipe 4 reads `SDP_CHANGED_FILES_JSON`, would let the body run as shipped; for the Protocol, in the same place. + - Closed on 2026-10-09: anchors that lived in tests. The Protocol's comment form put them in the source and the join is deleted (decision 60); `docs/feedback/sdp-feedback-01.md`, section "2026-10-09, anchors in the source at `da56a34`", records the adoption. What stands, for the Protocol in the same section: a block binds by file and line, so a block whose declaration moved or went still binds (`#attachmentIdentity` on `spec:decisions.anchor-comment-form`). + - Two of the Protocol's carrier checks, `check-carrier-rule.mjs` and `check-carrier-truth.mjs` at its root, fail on `feature/adopter-round-2` and fail identically at `5e29c03`. Neither is a `package.json` gate, and the Protocol's pull request says so; its maintainers rule whether they become gates or go. + - Retracted: the gap the passes' first round listed, `defineList` mapping saved state through `toDto` without the schema-version checks of `spec:context.queries#design.fnList`. The adopter review showed that `defineList` refuses a newer saved schema version through `metaOf` (`convex-test`), so no lead stands for it. + +47. From the rulings of 2026-10-10. Each is a gap the pass found and did not fix. + - Splits by narrowing. The journal (baseline migration, stream rebuild), the tables, the persistence adapter and the context component (derived mapping, migrate on load, maintenance writer), the queries (journal inspection), and the receipts and the receipt table (tombstones) each hold a part with its own trigger or step, but the part runs through entries the code realizes, mostly pinned declarations. A split narrows those entries to the code and moves the widening into a child Spec, as `spec:application.aggregate-projection` does; it changes realized Spec text, so it waits for a pass the main thread rules. + - Two Spec corrections, from the second read. The outcome model reserves three rejection codes for the adapter and gives every other code to a context, while the adapter's step 1 also raises the boundary's `invalidInput`, as `spec:context.persistence-adapter` states; either the model admits it or the adapter stops raising it. The projection contract's `callers` entry names the callers of the history writer that `spec:application.history-projection` holds. Until the main thread rules each, the code references the Spec. + - Worked examples in the platform's Specs. `check.py` now shows that no platform component uses the example's components, but the platform's own Specs carry orders-inventory worked examples in Design entries: `ordersExample`, `parentPassThrough` and `parentGet` in `spec:context.queries`, `operationExample` and `parentCall` in `spec:context.context-component`, `contextCall` in `spec:command.command-pipeline`. Moving them to `spec:application.orders-inventory-example` would make the platform's design read without its example, as its code does. + The reports behind leads 1 to 3 and 6 are outside the repository, because they quote a private repository. The project notes file named in `AGENTS.md` lists them, with the reports, briefs and rulings of S0. ## Inputs @@ -483,7 +578,7 @@ The doc is `docs/convex-transactional-domain-platform-decisions.md`, unchanged s - First release: dual ESM and CJS, `publint` and the type-resolution check, packed-consumer fixtures, the Node and TypeScript matrix, changesets, trusted publishing, the compatibility policy for error codes, handler versions and event schema versions. - Not taken: its repository blueprint, which puts journal, receipts and obligations in one component against D2; the Node 18 floor and the TypeScript 5 promise, which came from its prompt; its sample types and error codes, which differ from what the corpus pins; test functions shipped behind a runtime guard; its sample CI, which builds before component codegen. -`docs/sdp-development-from-application-platform.md` is the first report to the Protocol's maintainers. `docs/sdp-feedback.md` is the running file that follows it; the `adopt` unit and each slice add to it. +`docs/sdp-development-from-application-platform.md` is the first report to the Protocol's maintainers. `docs/feedback/sdp-feedback-01.md` is the running file that follows it; the `adopt` unit and each slice add to it. Beside it, `sdp-feedback-02.md` and `sdp-feedback-03-selection.md` are the two reads of 2026-10-04 that chose the next work from it; `pack-page` did round 1 of the selection and `sdp-round-2` the rest, the passes and the Protocol's round 2. ## Leftovers diff --git a/design/advisors/README.md b/design/advisors/README.md index 20d3316..4a850c7 100644 --- a/design/advisors/README.md +++ b/design/advisors/README.md @@ -11,7 +11,7 @@ An advisor is three layers, kept apart so that a change to one leaves the other | Layer | File | Changes when | |---|---|---| | The lens: the questions a field asks of any decision, its hazards, its reading list | `lenses/.md` | The field's questions get sharper | -| The protocol: how an advisor stands on the corpus, what a lean is, the limits | `protocol.md`, with `task-sort.md` and `task-fork.md` | The way of working changes | +| The protocol: how an advisor stands on the corpus, what a lean is, the limits | `protocol.md`, with `task-sort.md`, `task-fork.md`, `task-roadmap.md` and `task-pass.md` | The way of working changes | | The project: what the corpus holds and where to look | `project-context.md` | A close moves a path | | Advisor | Slug | Lens | Definition | @@ -44,6 +44,17 @@ The main thread merges each patch with `python3 design/tools/decisions.py apply `python3 design/tools/decisions.py --owner --before ` lists what the owner must decide first. One launch for each fork, or one for a few forks of the same advisor, with `design/advisors/task-fork.md`, the ids and a patch path. Forks of different advisors run at once: each writes `design/decisions/forks/.md` and its own patch. +## How the main thread runs a pass + +The owner asked on 2026-10-03 for designs of coming work to be first-class in the corpus and iterated from several perspectives before the work is committed to. A pass is one advisor reading one capability's Pack from its lens, with `task-pass.md`, before the capability is built; `design/PLAN.md` 6.7 says how such a design sits in the graph and what each lens asks of it. Passes of different lenses over one Pack may run at once, since each writes only its memo. + +``` +Agent(subagent_type: "advisor-convex", + prompt: "Your task is design/advisors/task-pass.md. Your slug is convex. Your Pack is pack:history-view. Write your memo to /pass-convex.md.") +``` + +The memo's facts are checked like a roadmap memo's, on a copy with a plant. The main thread then rules each finding: a Spec change written by an Opus agent, an open question, a register row, or nothing. The pass closes as `design: , the pass`, and the next pass reads the Specs as that one left them. The implementation and verification pass needs no advisor: `npm run test:types` compiles the Pack's pinned module, recipe 21 gives its dependency footing, and the main thread rules on what they show. + ## How the main thread runs a roadmap The owner asked on 2026-10-02 for the panel to propose the upcoming milestones as a short-term roadmap. The task is `task-roadmap.md`, and the aggregated result is `design/ROADMAP.md`, a work document the main thread cuts from the four memos and the owner rules on. @@ -58,12 +69,12 @@ Agent(subagent_type: "advisor-domain", Then, in this order: 1. The memos are copied to `/check-copy/`, and the main thread plants one false citation in each copy and records the plants in `/plants.md`, as tactical decision 31 of `STATE.md` asks for a review copy. -2. One `gpt:codex` job, `gpt-6.1-sol` at medium with the fast tier, checks the copies: every cited claim held, not held or not checked, every unlabelled fact listed. Its brief is `/check-brief.md`, and its report stays in its job directory under `~/.codex-jobs`. +2. One `relay:job` job, `gpt-6.1-sol` at medium with the fast tier, checks the copies: every cited claim held, not held or not checked, every unlabelled fact listed. Its brief is `/check-brief.md`, and its report stays in its job directory under `~/.relay-jobs`. 3. While the check runs, the main thread writes `design/ROADMAP.md`: the cut as one table, the rounds, what is deferred with its trigger, where the advisors differed and what was ruled, the questions that are not rows yet, the findings beyond the roadmap, and the words the advisors needed that `CONTEXT.md` lacks. 4. The claims the check marked not held are corrected in the roadmap, or the advisor is continued with the check's findings where the memo must change. The plants caught go into `plants.md` and the project notes. 5. The unit closes as any unit does: the check prints `OK`, `STATE.md` points at the roadmap, the commit goes up through a pull request. -The next run is lighter, by the owner's words of 2026-10-02, quoted in tactical decision 37 of `STATE.md`. One Fable advisor and one astra advisor are launched for input, on the subjects where the roadmap is in doubt; astra runs as a `gpt:codex` job whose brief carries the protocol, the lens and the task by path, at high effort. The product advisor, a Fable agent, cuts the roadmap as its product manager, with the same task file and the same check. The main thread rules on the tactical questions and asks the owner only for the critical forks. Forks are drafted by `gpt-6.1-sol` from the register's leans and the Spec lines they cite, read by an Opus agent for balance, and an advisor rewrites one only when the owner names the row: Fable is spent on the main thread and on the design of a slice on paper. +The next run is lighter, by the owner's words of 2026-10-02, quoted in tactical decision 37 of `STATE.md`. One Fable advisor and one astra advisor are launched for input, on the subjects where the roadmap is in doubt; astra runs as a `relay:job` job whose brief carries the protocol, the lens and the task by path, at high effort. The product advisor, a Fable agent, cuts the roadmap as its product manager, with the same task file and the same check. The main thread rules on the tactical questions and asks the owner only for the critical forks. Forks are drafted by `gpt-6.1-sol` from the register's leans and the Spec lines they cite, read by an Opus agent for balance, and an advisor rewrites one only when the owner names the row: Fable is spent on the main thread and on the design of a slice on paper. What the first run showed, on 2026-10-02. Four advisors took five to six and a half minutes each, at 186 to 231 thousand tokens, and each returned a memo of 76 to 86 lines in the task's shape. The disjoint subjects held: no advisor leaned on another's decision, and the needs between subjects came back as one-line requests, as the task asks. Two advisors put the same case against the order, S3 before S4, which is the main thread's to rule. Every advisor listed the words it lacked; the union is in the roadmap. The memos state far fewer unlabelled facts than the first sort did: the task repeats the rules of a sentence, and that appears to be enough. The sol check took ten minutes on the fast tier for 291 claims and caught all four plants; its real findings were line citations past the end of a Spec, which an advisor can avoid by opening the line, and sentences that said more than their source. diff --git a/design/advisors/protocol.md b/design/advisors/protocol.md index 5a5c492..5144b9d 100644 --- a/design/advisors/protocol.md +++ b/design/advisors/protocol.md @@ -21,6 +21,7 @@ Your field is not evidence. What you know from practice says where to look and w 1. **A sentence about the platform, about Convex or about the code carries its citation, or it opens with its basis.** No third kind exists. 2. **Five kinds of evidence, each written one way.** - A Spec line: `[design/specs/command/command-pipeline.sdp.md:61]`. It shows what is intended. + - A Design entry: its entry address, `[spec:application.history-projection#design.fnApplyHistoryProjection]`. It shows what is intended, like a Spec line, and it still points at the entry after an edit moves the lines. Prefer it whenever the claim rests on one Design entry. - The doc: its token, `[D7]`, `[Law 3]`, `[F13]`, `[Sc L2-4]`, `[OQ1]`, with the line of the doc when you quote it. - Code: `[src/command/receipts.ts:40]`. It shows what is written, not what happens. - A measured fact: the tier, the pins and the record, `[native backend, convex 1.46.0, backend precompiled-2026-09-28-5c7cb5b, evidence/.json]`. diff --git a/design/advisors/register.md b/design/advisors/register.md index dc458e6..cff6fc8 100644 --- a/design/advisors/register.md +++ b/design/advisors/register.md @@ -58,7 +58,7 @@ An advisor that finds a row plainly another's leaves it unsorted and says so in "id": "OD-014", "title": "Admission is an optional per-command policy in the declaration, absent by default", "sources": [ - { "kind": "specQuestion", "ref": "spec:command.command-pipeline#3", "file": "design/specs/command/command-pipeline.sdp.md", "line": 61 }, + {"kind": "specQuestion", "address": "spec:command.command-pipeline#question.commandAdmissionPolicy", "file": "design/specs/command/command-pipeline.sdp.md"}, { "kind": "extension", "ref": "E-32", "file": "design/README.md", "line": 211 } ], "families": ["command"], @@ -81,13 +81,15 @@ An advisor that finds a row plainly another's leaves it unsorted and says so in } ``` +`check.py` resolves each open question source through the Protocol's address-resolution recipe and checks that `file` names its Spec. Every open question carries a key in its marker, such as `[non-blocking #commandAdmissionPolicy]`, addressed as `spec:#question.`. A closed source keeps its Spec and opening text. The check compares that text with the open questions' first sentences, normalizing whitespace only, and refuses a removed question that resolves again. + `units` is the order of the units a decision can block. The tool reads "before S4" from it, and a close keeps it current. | Field | What it holds | Filled by | |---|---|---| | `id` | `OD-` and three digits, given in source order and never reused | the load | | `title` | The first sentence of the first source, cut at 120 characters | the load | -| `sources` | Where the decision came from, every place once. `kind` is `ownerQueue`, `specQuestion`, `extension`, `planAmbiguity`, `productDecision` or `specLine`. `ref` is the lasting name: `000/3` for the third bullet of owner queue item 000, `#` for a Spec's n-th open question in authored order, `E-32`, `11.3` for an ambiguity of `PLAN.md`, `product/1`, `product/2` or `product/3` for a product decision, and `#` for a named Spec bullet that is not an open question. `file` and `line` say where it stood at the load. | the load | +| `sources` | Where the decision came from, every place once. `kind` is `ownerQueue`, `specQuestion`, `extension`, `planAmbiguity`, `productDecision` or `specLine`. An open `specQuestion` uses `address`, in the form `spec:#question.`, and `file`. A closed source keeps `spec`, the question's opening text in `question`, `file` and `open: false`, only for a removed question on a decided or folded row. Other source kinds use `ref` as the lasting name: `000/3` for the third bullet of owner queue item 000, `E-32`, `11.3` for an ambiguity of `PLAN.md`, `product/1`, `product/2` or `product/3` for a product decision, and `#` for a named Spec bullet that is not an open question. `file` and `line` on those other kinds say where it stood at the load. | the load | | `families` | The Spec families it touches | the load | | `cites` | The tokens its sources cite: E, D, F, OQ, Sc, Probe, Law | the load | | `provisionalReading` | What the code or the Specs rest on today, word for word from the source, or `null` | the load | diff --git a/design/advisors/task-pass.md b/design/advisors/task-pass.md new file mode 100644 index 0000000..8c38103 --- /dev/null +++ b/design/advisors/task-pass.md @@ -0,0 +1,32 @@ +# Task: read one capability's design from your lens + +Your launch names your slug, a Pack and the path of the memo you write, written below as ``, `` and ``. Work for about thirty minutes. + +## Why this work exists + +A capability is designed in its Specs before anyone builds it. The owner asked on 2026-10-03 for designs of coming work to be first-class in the corpus and to be iterated from several perspectives before the work is committed to. `design/PLAN.md` 6.7 says how such a design sits in the graph: one maturity per Spec, a Pack that gathers what the capability rests on, pinned declarations that compile, entries cited by address. A pass is one reading of that design from one lens. It finds what the design leaves an implementer to guess, what it gets wrong, and what it cannot settle yet. The main thread rules on what you find, and the next pass reads the result. + +## Before you read + +Read `design/advisors/protocol.md`, then `design/advisors/project-context.md`, then `design/advisors/lenses/.md`. Then read the Pack's page, `design/pages/.md` for `pack:`: the members in reading order with their rungs, bindings, pairs and open questions on one page. Your memo ends with one line on what you needed that the page did not give. Then read the Pack through the graph where the page is not enough, from the repository root, with bodies copied verbatim from `node_modules/@libar-dev/software-delivery-protocol/docs/agent-surface/recipes.md`: + +- recipe 5, the Pack review backbone, with ``: the members, their rungs and bindings; +- recipe 20, the open questions, and recipe 21, the dependency footing, for each member that pins what the capability builds; +- recipe 24, the pinned declarations, filtered to the members, and `generated/pinned/` after `npm run sdp:build` has run, for the shapes as one module; +- the member Specs themselves, where a question turns on their text. + +## What you look for + +The questions of your lens, asked of this design: what a specialist of your field would want settled before the first line of code. `design/PLAN.md` 6.7 lists a few per lens; they are a start, not a limit. Keep to your lens. A question of another lens is one line in your return. + +## The memo + +One Markdown file at ``, at most fifteen findings, the most consequential first. Each finding is one short paragraph under a heading of a few words, and it says: + +- **Where:** the entry address of the Design entry it concerns, `spec:#design.`, or the Spec id and the section when no entry holds it. +- **What:** the gap, the error or the open point, every sentence on its basis as `protocol.md` writes it. +- **Proposed:** one of four things. Exact text for a Spec, with the entry address it replaces or follows, in the words of `CONTEXT.md` and none of the work's. An open question for a Spec, with `[blocking]` or `[non-blocking]`. A row for the decision register, with the class you would give it. Or nothing, when the finding only informs. + +## What you return + +Text only, at most twelve lines: the path of the memo, the findings by kind of proposal, the three you are least sure of, every question you left to another lens, and each word you needed that `CONTEXT.md` does not have. diff --git a/design/decisions/forks/OD-025.md b/design/decisions/forks/OD-025.md index ace9e54..75879e1 100644 --- a/design/decisions/forks/OD-025.md +++ b/design/decisions/forks/OD-025.md @@ -1,6 +1,6 @@ # OD-025. Who may create the first grant of a tenant and change its grants? -Drafted by sol from the register, 2026-10-02. Sources: owner queue 0/2, spec:command.tenancy-and-authority#3. Blocks: S4. +Drafted by sol from the register, 2026-10-02. Sources: owner queue 0/2, spec:command.tenancy-and-authority. Blocks: S4. ## The fork diff --git a/design/decisions/forks/OD-064.md b/design/decisions/forks/OD-064.md index 7ad79f2..014471c 100644 --- a/design/decisions/forks/OD-064.md +++ b/design/decisions/forks/OD-064.md @@ -1,6 +1,6 @@ # OD-064. Is 100 lines the supported maximum of PlaceOrder or only its measurement size? -Drafted by sol from the register, 2026-10-02. Sources: spec:application.first-experiment#2, spec:application.parent-use-cases#1, spec:context.batch-shaped-api#1, spec:decisions.d10-contexts-meet-in-parent-use-cases#1. Blocks: S3. +Drafted by sol from the register, 2026-10-02. Sources: spec:application.first-experiment, spec:application.parent-use-cases, spec:context.batch-shaped-api, spec:decisions.d10-contexts-meet-in-parent-use-cases. Blocks: S3. ## The fork diff --git a/design/decisions/forks/OD-068.md b/design/decisions/forks/OD-068.md index d741d91..1a7deb3 100644 --- a/design/decisions/forks/OD-068.md +++ b/design/decisions/forks/OD-068.md @@ -1,6 +1,6 @@ # OD-068. Should rollback reverify a retired generation or switch to one kept current? -Drafted by sol from the register, 2026-10-02. Sources: spec:application.generation-registry#1, spec:application.rebuild#3, E-8. Blocks: S4. +Drafted by sol from the register, 2026-10-02. Sources: spec:application.generation-registry, spec:application.rebuild, E-8. Blocks: S4. ## The fork diff --git a/design/decisions/forks/OD-069.md b/design/decisions/forks/OD-069.md index 4c3a6e8..c1ed93c 100644 --- a/design/decisions/forks/OD-069.md +++ b/design/decisions/forks/OD-069.md @@ -1,6 +1,6 @@ # OD-069. Should a generation span the deployment or belong to one tenant? -Drafted by sol from the register, 2026-10-02. Sources: spec:application.generation-registry#2. Blocks: S4. +Drafted by sol from the register, 2026-10-02. Sources: spec:application.generation-registry. Blocks: S4. ## The fork diff --git a/design/decisions/forks/OD-123.md b/design/decisions/forks/OD-123.md index 98e69ca..dd95ed1 100644 --- a/design/decisions/forks/OD-123.md +++ b/design/decisions/forks/OD-123.md @@ -1,6 +1,6 @@ # OD-123. Should one document per stream remain the default, and what largest order should it support? -Drafted by sol from the register, 2026-10-02. Sources: spec:kernel.state-document-mapping#2, 11.6. Blocks: S3. +Drafted by sol from the register, 2026-10-02. Sources: spec:kernel.state-document-mapping, 11.6. Blocks: S3. ## The fork diff --git a/design/decisions/register.json b/design/decisions/register.json index 7c77a94..958e01b 100644 --- a/design/decisions/register.json +++ b/design/decisions/register.json @@ -149,9 +149,8 @@ }, { "kind": "specQuestion", - "ref": "spec:platform.acceptance-contract#2", - "file": "design/specs/platform/acceptance-contract.sdp.md", - "line": 25 + "address": "spec:platform.acceptance-contract#question.scenarioAcceptanceCheck", + "file": "design/specs/platform/acceptance-contract.sdp.md" }, { "kind": "extension", @@ -200,9 +199,8 @@ }, { "kind": "specQuestion", - "ref": "spec:application.first-experiment#7", - "file": "design/specs/application/first-experiment.sdp.md", - "line": 65 + "address": "spec:application.first-experiment#question.requiredAcceptanceScenarios", + "file": "design/specs/application/first-experiment.sdp.md" } ], "families": [ @@ -241,9 +239,10 @@ }, { "kind": "specQuestion", - "ref": "spec:application.first-experiment.native-acceptance-production-configuration#1", + "spec": "spec:application.first-experiment.native-acceptance-production-configuration", + "question": "The row's pass condition names concurrency and this example binds no concurrency claim: no native observation shows that the disposable backend's concurrency matches a release's, and the test shows only that the deployment's environment holds the three variables its `auth.config.ts` reads and no other; the owner rules whether that reading is enough for the row or the row needs an example that runs commands at once on the production composition (Sc L2-9, E-15)", "file": "design/specs/application/first-experiment.native-acceptance-production-configuration.sdp.md", - "line": 21 + "open": false } ], "families": [ @@ -358,9 +357,8 @@ }, { "kind": "specQuestion", - "ref": "spec:application.generation-registry#4", - "file": "design/specs/application/generation-registry.sdp.md", - "line": 36 + "address": "spec:application.generation-registry#question.readModelInstallation", + "file": "design/specs/application/generation-registry.sdp.md" } ], "families": [ @@ -486,9 +484,8 @@ }, { "kind": "specQuestion", - "ref": "spec:platform.native-harness#2", - "file": "design/specs/platform/native-harness.sdp.md", - "line": 25 + "address": "spec:platform.native-harness#question.fixtureCompositionContract", + "file": "design/specs/platform/native-harness.sdp.md" }, { "kind": "extension", @@ -532,9 +529,8 @@ }, { "kind": "specQuestion", - "ref": "spec:application.projection-contract#2", - "file": "design/specs/application/projection-contract.sdp.md", - "line": 34 + "address": "spec:application.projection-contract#question.projectionGenerationVersions", + "file": "design/specs/application/projection-contract.sdp.md" } ], "families": [ @@ -578,9 +574,8 @@ }, { "kind": "specQuestion", - "ref": "spec:command.command-declaration#2", - "file": "design/specs/command/command-declaration.sdp.md", - "line": 35 + "address": "spec:command.command-declaration#question.commandDeclarationLookup", + "file": "design/specs/command/command-declaration.sdp.md" } ], "families": [ @@ -656,15 +651,13 @@ }, { "kind": "specQuestion", - "ref": "spec:application.read-models#3", - "file": "design/specs/application/read-models.sdp.md", - "line": 51 + "address": "spec:application.read-models#question.parentQueryRefusal", + "file": "design/specs/application/read-models.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:command.outcome-boundary#2", - "file": "design/specs/command/outcome-boundary.sdp.md", - "line": 39 + "address": "spec:command.outcome-boundary#question.parentQueryRefusalShape", + "file": "design/specs/command/outcome-boundary.sdp.md" } ], "families": [ @@ -745,15 +738,13 @@ }, { "kind": "specQuestion", - "ref": "spec:application.first-experiment#3", - "file": "design/specs/application/first-experiment.sdp.md", - "line": 61 + "address": "spec:application.first-experiment#question.lifecycleCommandSummary", + "file": "design/specs/application/first-experiment.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:application.orders-inventory-example#1", - "file": "design/specs/application/orders-inventory-example.sdp.md", - "line": 27 + "address": "spec:application.orders-inventory-example#question.orderInventoryCommandChoices", + "file": "design/specs/application/orders-inventory-example.sdp.md" }, { "kind": "extension", @@ -800,9 +791,8 @@ }, { "kind": "specQuestion", - "ref": "spec:application.projection-contract#3", - "file": "design/specs/application/projection-contract.sdp.md", - "line": 35 + "address": "spec:application.projection-contract#question.backfillVersionOverwrite", + "file": "design/specs/application/projection-contract.sdp.md" } ], "families": [ @@ -976,9 +966,8 @@ }, { "kind": "specQuestion", - "ref": "spec:command.tenancy-and-authority#3", - "file": "design/specs/command/tenancy-and-authority.sdp.md", - "line": 39 + "address": "spec:command.tenancy-and-authority#question.grantCreationEntries", + "file": "design/specs/command/tenancy-and-authority.sdp.md" } ], "families": [ @@ -1018,9 +1007,8 @@ }, { "kind": "specQuestion", - "ref": "spec:command.actor-and-scope#2", - "file": "design/specs/command/actor-and-scope.sdp.md", - "line": 32 + "address": "spec:command.actor-and-scope#question.commandQueryTenantScope", + "file": "design/specs/command/actor-and-scope.sdp.md" } ], "families": [ @@ -1064,9 +1052,8 @@ }, { "kind": "specQuestion", - "ref": "spec:facts.f11-components-have-no-ctx-auth#1", - "file": "design/specs/facts/f11-components-have-no-ctx-auth.sdp.md", - "line": 21 + "address": "spec:facts.f11-components-have-no-ctx-auth#question.contextActorIdentity", + "file": "design/specs/facts/f11-components-have-no-ctx-auth.sdp.md" } ], "families": [ @@ -1105,33 +1092,28 @@ }, { "kind": "specQuestion", - "ref": "spec:command.command-pipeline#4", - "file": "design/specs/command/command-pipeline.sdp.md", - "line": 62 + "address": "spec:command.command-pipeline#question.publicRequestKey", + "file": "design/specs/command/command-pipeline.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:command.idempotency-and-receipts#1", - "file": "design/specs/command/idempotency-and-receipts.sdp.md", - "line": 42 + "address": "spec:command.idempotency-and-receipts#question.httpCallerRequestKey", + "file": "design/specs/command/idempotency-and-receipts.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:command.idempotency-and-receipts#2", - "file": "design/specs/command/idempotency-and-receipts.sdp.md", - "line": 43 + "address": "spec:command.idempotency-and-receipts#question.publicReceiptChoice", + "file": "design/specs/command/idempotency-and-receipts.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:decisions.d06-idempotency-client-and-receipts#1", - "file": "design/specs/decisions/d06-idempotency-client-and-receipts.sdp.md", - "line": 33 + "address": "spec:decisions.d06-idempotency-client-and-receipts#question.httpCallerReceipts", + "file": "design/specs/decisions/d06-idempotency-client-and-receipts.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:facts.probe-plan#1", - "file": "design/specs/facts/probe-plan.sdp.md", - "line": 38 + "address": "spec:facts.probe-plan#question.httpClientRequestKey", + "file": "design/specs/facts/probe-plan.sdp.md" }, { "kind": "extension", @@ -1209,9 +1191,8 @@ }, { "kind": "specQuestion", - "ref": "spec:platform.native-harness#3", - "file": "design/specs/platform/native-harness.sdp.md", - "line": 26 + "address": "spec:platform.native-harness#question.fixtureCallerNamespace", + "file": "design/specs/platform/native-harness.sdp.md" } ], "families": [ @@ -1282,9 +1263,8 @@ }, { "kind": "specQuestion", - "ref": "spec:platform.native-harness#1", - "file": "design/specs/platform/native-harness.sdp.md", - "line": 24 + "address": "spec:platform.native-harness#question.nativeCallerIdentity", + "file": "design/specs/platform/native-harness.sdp.md" }, { "kind": "extension", @@ -1329,9 +1309,8 @@ }, { "kind": "specQuestion", - "ref": "spec:facts.probe-plan#6", - "file": "design/specs/facts/probe-plan.sdp.md", - "line": 43 + "address": "spec:facts.probe-plan#question.probeExpectationContract", + "file": "design/specs/facts/probe-plan.sdp.md" }, { "kind": "extension", @@ -1449,9 +1428,8 @@ }, { "kind": "specQuestion", - "ref": "spec:facts.probe-plan#2", - "file": "design/specs/facts/probe-plan.sdp.md", - "line": 44 + "address": "spec:facts.probe-plan#question.hostedContextCallQuota", + "file": "design/specs/facts/probe-plan.sdp.md" } ], "families": [ @@ -1691,7 +1669,7 @@ "reason": "Owner test 5: whether a failed check keeps every tenant's writers out with no override, and how often the drill runs, trade availability against safety and a standing operator duty against assurance, with no evidence on which weighs more, and the doc says none of them [design/reviews/consensus-ledger.json:2301]; it stands apart from OD-082, which is the mechanism and names these policies as outside it.", "lean": { "do": "Keep fail-closed with no override and the deployment-wide door as the provisional reading, mark the three policies as extensions on `spec:application.restore`, and let the owner set the override, the failure scope and the drill cadence.", - "restsOn": "[design/specs/application/restore.sdp.md:68] and [design/specs/application/restore.sdp.md:85] keep a failed run closed with no reopening; [design/specs/application/restore.sdp.md:62] and [design/specs/application/restore.sdp.md:112] set the drill at once per release; [design/specs/application/write-pause.sdp.md:61] refuses every write with scope `all` while the door is set; [design/specs/application/restore.sdp.md:96] already lets a check run for one tenant, so a per-tenant failure scope is half a mechanism and the door is what makes it deployment-wide. **Judgment.** For the person restoring at night, no override means a finding in one tenant's stream keeps every tenant down until a repair or another backup; what each level would mean is: no override, safest and longest; a tenant-scoped door, one tenant down; an override recorded on the run, shortest and a decision a person makes under pressure.", + "restsOn": "[design/specs/application/restore.sdp.md:68] and [design/specs/application/restore.sdp.md:85] keep a failed run closed with no reopening; [design/specs/application/restore.sdp.md:62] and [design/specs/application/restore.sdp.md:112] set the drill at once per release; [spec:application.write-pause#design.fnGateAllows] refuses every write with scope `all` while the door is set; [design/specs/application/restore.sdp.md:96] already lets a check run for one tenant, so a per-tenant failure scope is half a mechanism and the door is what makes it deployment-wide. **Judgment.** For the person restoring at night, no override means a finding in one tenant's stream keeps every tenant down until a repair or another backup; what each level would mean is: no override, safest and longest; a tenant-scoped door, one tenant down; an override recorded on the run, shortest and a decision a person makes under pressure.", "settles": "The owner's word on the three policies." }, "checkedBy": null, @@ -1935,9 +1913,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:advanced.coalesced-recomputation#1", - "file": "design/specs/advanced/coalesced-recomputation.sdp.md", - "line": 29 + "address": "spec:advanced.coalesced-recomputation#question.recomputationStalenessRecords", + "file": "design/specs/advanced/coalesced-recomputation.sdp.md" } ], "families": [ @@ -1969,9 +1946,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:advanced.cross-stream-online-rebuild#1", - "file": "design/specs/advanced/cross-stream-online-rebuild.sdp.md", - "line": 29 + "address": "spec:advanced.cross-stream-online-rebuild#question.crossStreamRebuildCut", + "file": "design/specs/advanced/cross-stream-online-rebuild.sdp.md" } ], "families": [ @@ -2003,9 +1979,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:advanced.ordered-consumer#1", - "file": "design/specs/advanced/ordered-consumer.sdp.md", - "line": 28 + "address": "spec:advanced.ordered-consumer#question.orderedConsumerPosition", + "file": "design/specs/advanced/ordered-consumer.sdp.md" } ], "families": [ @@ -2037,9 +2012,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:advanced.trigger-table#1", - "file": "design/specs/advanced/trigger-table.sdp.md", - "line": 22 + "address": "spec:advanced.trigger-table#question.capabilityActivationTrigger", + "file": "design/specs/advanced/trigger-table.sdp.md" } ], "families": [ @@ -2071,9 +2045,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:agents.agent-runs#1", - "file": "design/specs/agents/agent-runs.sdp.md", - "line": 36 + "address": "spec:agents.agent-runs#question.agentProposalAttempts", + "file": "design/specs/agents/agent-runs.sdp.md" } ], "families": [ @@ -2105,21 +2078,18 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:agents.agent-runs#2", - "file": "design/specs/agents/agent-runs.sdp.md", - "line": 37 + "address": "spec:agents.agent-runs#question.agentProposalRejectionRecord", + "file": "design/specs/agents/agent-runs.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:command.outcome-boundary#4", - "file": "design/specs/command/outcome-boundary.sdp.md", - "line": 41 + "address": "spec:command.outcome-boundary#question.refusedCommandRecord", + "file": "design/specs/command/outcome-boundary.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:decisions.d07-rejections-thrown-not-stored#1", - "file": "design/specs/decisions/d07-rejections-thrown-not-stored.sdp.md", - "line": 31 + "address": "spec:decisions.d07-rejections-thrown-not-stored#question.refusedCommandHistory", + "file": "design/specs/decisions/d07-rejections-thrown-not-stored.sdp.md" }, { "kind": "planAmbiguity", @@ -2159,9 +2129,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:agents.budget-accounting#1", - "file": "design/specs/agents/budget-accounting.sdp.md", - "line": 23 + "address": "spec:agents.budget-accounting#question.agentBudgetAccounting", + "file": "design/specs/agents/budget-accounting.sdp.md" } ], "families": [ @@ -2193,27 +2162,23 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:facts.f17-migrations-fits-generation-backfill#1", - "file": "design/specs/facts/f17-migrations-fits-generation-backfill.sdp.md", - "line": 22 + "address": "spec:facts.f17-migrations-fits-generation-backfill#question.rebuildBackendCoverage", + "file": "design/specs/facts/f17-migrations-fits-generation-backfill.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:application.rebuild#1", - "file": "design/specs/application/rebuild.sdp.md", - "line": 57 + "address": "spec:application.rebuild#question.rebuildBatchDriver", + "file": "design/specs/application/rebuild.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:decisions.d09-rebuild-online-by-default#1", - "file": "design/specs/decisions/d09-rebuild-online-by-default.sdp.md", - "line": 30 + "address": "spec:decisions.d09-rebuild-online-by-default#question.rebuildContentionCoverage", + "file": "design/specs/decisions/d09-rebuild-online-by-default.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:facts.probe-plan#4", - "file": "design/specs/facts/probe-plan.sdp.md", - "line": 41 + "address": "spec:facts.probe-plan#question.rebuildNativeCoverage", + "file": "design/specs/facts/probe-plan.sdp.md" } ], "families": [ @@ -2237,10 +2202,10 @@ "lean": { "do": "Close on Probe 6's result: the rebuild drives every batch, backfill, verify and purge, with its own self-scheduled internal mutation, uses `@convex-dev/migrations` for nothing, keeps the progress row as the one checkpoint, and keeps the component pinned as a dev dependency for the fixture's probe only.", "restsOn": "Lean of the domain lens, under the owner's one-advisor word of 2026-10-03, on a row whose advisor is `convex`. Probe 6 ran at the native backend tier on `precompiled-2026-09-28-5c7cb5b` with `convex` 1.46.0, `convex-helpers` 0.1.124 and `@convex-dev/migrations` 0.3.6 [design/specs/facts/f17-migrations-fits-generation-backfill.sdp.md:13], and F17 is probed [design/specs/facts/f17-migrations-fits-generation-backfill.sdp.md:11]. On those pins the component reaches a context's rows only through a parent mutation written by hand against a contract its README does not document and the compiled tier needs a cast for, its status and the generation's state are two vocabularies, and a failure past the read limit leaves no error on its row [design/specs/facts/f17-migrations-fits-generation-backfill.sdp.md:21,44,45,49]. The Specs already take the consequence: every batch is driven by a self-scheduled internal mutation under one batch contract and the rebuild uses the component for nothing [design/specs/application/rebuild.sdp.md:58,59,86; design/specs/decisions/d09-rebuild-online-by-default.sdp.md:42]. The round 2 close records the pin as a dev dependency for the fixture's probe only [design/STATE.md:38, a pointer]. **Judgment.** The earlier lean kept a split of two drivers \"until it passes\"; the probe did not pass the component, so that condition was not met but replaced: the split is gone and one driver remains, the do-nothing option the earlier `against` named. Nothing in this row changes a Spec: the ruling records what the Specs say.", - "against": "**Judgment.** The component gives checkpointing, resume and a status for free where it fits, and a later release might fit; the probe shows 0.3.6 on the pinned backend and nothing about a later one. The cost of the Spec's choice is that the platform owns its batch driver and its stall answer, which is `abortGeneration` and a new `startGeneration` [design/specs/application/rebuild.sdp.md:144].", + "against": "**Judgment.** The component gives checkpointing, resume and a status for free where it fits, and a later release might fit; the probe shows 0.3.6 on the pinned backend and nothing about a later one. The cost of the Spec's choice is that the platform owns its batch driver and its stall answer, which is `abortGeneration` and a new `startGeneration` [spec:application.rebuild#design.stallSignal].", "settles": "Settled by Probe 6 at the native backend tier on the pins named; what the six examples do not run, hosted quotas, sustained contention, automatic stall detection, timeout and retry exhaustion [design/specs/facts/f17-migrations-fits-generation-backfill.sdp.md:31,49], stays open on F17 and is not this row's.", "changes": "No Spec: `spec:application.rebuild`, `spec:decisions.d09-rebuild-online-by-default` and `spec:facts.f17-migrations-fits-generation-backfill` already carry the ruling. The register row alone. The alternative, a driver on a later release of the component, would edit `spec:application.rebuild` at its E-40 question and `fnBackfillBatch` and add a dependency.", - "ifWrong": "A later release that fits is a new fact with its own probe; the batch contract is one internal mutation per batch [design/specs/application/rebuild.sdp.md:111,114], so a driver swap is an edit to the rebuild Spec and the batch functions, and the cursor and the bound are the design's numbers either way [design/specs/application/rebuild.sdp.md:88,146]." + "ifWrong": "A later release that fits is a new fact with its own probe; the batch contract is one internal mutation per batch [spec:application.rebuild#design.transactionBoundary,114], so a driver swap is an edit to the rebuild Spec and the batch functions, and the cursor and the bound are the design's numbers either way [design/specs/application/rebuild.sdp.md:88,146]." }, "checkedBy": "gpt-6.1-sol, medium, 2026-10-03: every checked claim held", "fork": null, @@ -2254,9 +2219,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:processes.approvals#1", - "file": "design/specs/processes/approvals.sdp.md", - "line": 22 + "address": "spec:processes.approvals#question.approvalCommandDeclaration", + "file": "design/specs/processes/approvals.sdp.md" } ], "families": [ @@ -2287,9 +2251,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:processes.start-checkout-example#1", - "file": "design/specs/processes/start-checkout-example.sdp.md", - "line": 26 + "address": "spec:processes.start-checkout-example#question.checkoutProcessDefinition", + "file": "design/specs/processes/start-checkout-example.sdp.md" } ], "families": [ @@ -2320,9 +2283,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:processes.start-checkout-example#2", - "file": "design/specs/processes/start-checkout-example.sdp.md", - "line": 27 + "address": "spec:processes.start-checkout-example#question.latePaymentPolicy", + "file": "design/specs/processes/start-checkout-example.sdp.md" }, { "kind": "productDecision", @@ -2359,9 +2321,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:processes.workflow-processes#1", - "file": "design/specs/processes/workflow-processes.sdp.md", - "line": 37 + "address": "spec:processes.workflow-processes#question.processObligationWait", + "file": "design/specs/processes/workflow-processes.sdp.md" } ], "families": [ @@ -2392,9 +2353,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:processes.workflow-processes#2", - "file": "design/specs/processes/workflow-processes.sdp.md", - "line": 38 + "address": "spec:processes.workflow-processes#question.processDefinitionRetention", + "file": "design/specs/processes/workflow-processes.sdp.md" }, { "kind": "productDecision", @@ -2431,27 +2391,23 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.first-experiment#1", - "file": "design/specs/application/first-experiment.sdp.md", - "line": 59 + "address": "spec:application.first-experiment#question.contextIsolationCost", + "file": "design/specs/application/first-experiment.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:context.context-component#1", - "file": "design/specs/context/context-component.sdp.md", - "line": 49 + "address": "spec:context.context-component#question.contextIsolationMeasurement", + "file": "design/specs/context/context-component.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:decisions.d02-context-owns-state-and-journal#2", - "file": "design/specs/decisions/d02-context-owns-state-and-journal.sdp.md", - "line": 31 + "address": "spec:decisions.d02-context-owns-state-and-journal#question.contextIsolationChoice", + "file": "design/specs/decisions/d02-context-owns-state-and-journal.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:obligations.obligation-module#3", - "file": "design/specs/obligations/obligation-module.sdp.md", - "line": 55 + "address": "spec:obligations.obligation-module#question.contextReactionIsolation", + "file": "design/specs/obligations/obligation-module.sdp.md" }, { "kind": "planAmbiguity", @@ -2495,27 +2451,31 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.first-experiment#2", + "spec": "spec:application.first-experiment", + "question": "OQ3: the largest order `PlaceOrder` supports is a product decision; the experiment measures with a provisional maximum of 100 lines and reports the numbers per line so the owner can choose (OQ3)", "file": "design/specs/application/first-experiment.sdp.md", - "line": 60 + "open": false }, { "kind": "specQuestion", - "ref": "spec:application.parent-use-cases#1", + "spec": "spec:application.parent-use-cases", + "question": "OQ3: the largest order `PlaceOrder` supports is a product decision; the limit bullet below carries a placeholder and the first experiment measures with a provisional maximum (OQ3, D10)", "file": "design/specs/application/parent-use-cases.sdp.md", - "line": 54 + "open": false }, { "kind": "specQuestion", - "ref": "spec:context.batch-shaped-api#1", + "spec": "spec:context.batch-shaped-api", + "question": "OQ3: the largest order the placement command supports is a product decision; each operation's `maxStreams` is set from it and the default ceiling of 256 stands until then (OQ3, D10)", "file": "design/specs/context/batch-shaped-api.sdp.md", - "line": 26 + "open": false }, { "kind": "specQuestion", - "ref": "spec:decisions.d10-contexts-meet-in-parent-use-cases#1", + "spec": "spec:decisions.d10-contexts-meet-in-parent-use-cases", + "question": "OQ3: the largest order the placement command supports is a product decision; until it is made the limit is a placeholder in the first experiment (OQ3, D10)", "file": "design/specs/decisions/d10-contexts-meet-in-parent-use-cases.sdp.md", - "line": 31 + "open": false } ], "families": [ @@ -2551,9 +2511,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.first-experiment#4", - "file": "design/specs/application/first-experiment.sdp.md", - "line": 62 + "address": "spec:application.first-experiment#question.commandMeasurementRecord", + "file": "design/specs/application/first-experiment.sdp.md" }, { "kind": "extension", @@ -2595,9 +2554,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.first-experiment#5", - "file": "design/specs/application/first-experiment.sdp.md", - "line": 63 + "address": "spec:application.first-experiment#question.commandDocumentReads", + "file": "design/specs/application/first-experiment.sdp.md" } ], "families": [ @@ -2634,9 +2592,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.first-experiment#6", - "file": "design/specs/application/first-experiment.sdp.md", - "line": 64 + "address": "spec:application.first-experiment#question.compositionInitialGrants", + "file": "design/specs/application/first-experiment.sdp.md" } ], "families": [ @@ -2677,15 +2634,13 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.generation-registry#1", - "file": "design/specs/application/generation-registry.sdp.md", - "line": 33 + "address": "spec:application.generation-registry#question.generationRollbackPolicy", + "file": "design/specs/application/generation-registry.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:application.rebuild#3", - "file": "design/specs/application/rebuild.sdp.md", - "line": 59 + "address": "spec:application.rebuild#question.rebuildGenerationPauseContract", + "file": "design/specs/application/rebuild.sdp.md" }, { "kind": "extension", @@ -2728,9 +2683,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.generation-registry#2", - "file": "design/specs/application/generation-registry.sdp.md", - "line": 34 + "address": "spec:application.generation-registry#question.generationTenantReach", + "file": "design/specs/application/generation-registry.sdp.md" } ], "families": [ @@ -2766,9 +2720,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.generation-registry#3", - "file": "design/specs/application/generation-registry.sdp.md", - "line": 35 + "address": "spec:application.generation-registry#question.restoreGenerationFence", + "file": "design/specs/application/generation-registry.sdp.md" } ], "families": [ @@ -2787,7 +2740,7 @@ "reason": "Delegated test: a mechanism, the meaning of a field, inside what D19 and F12 rule; it is the registry half of OD-082's accepted branch and is decided with it.", "lean": { "do": "Keep the generation row's fence at inventory time as the restore's proof that a generation is stranded, and `resumeChain`'s bump as the proof that it is not.", - "restsOn": "[design/specs/application/generation-registry.sdp.md:47] makes the row the checkpoint and [design/specs/application/generation-registry.sdp.md:50] gives a building or verifying generation exactly one batch identified by the fence, dropped by a restore with every pending scheduled function, which [design/specs/facts/f12-backups-exclude-pending-scheduled-functions.sdp.md:11] carries as documented; [design/specs/application/restore.sdp.md:104] resumes only a row still in `building` or `verifying` at the inventoried fence; [design/specs/application/write-pause.sdp.md:43] makes a batch whose fence differs return without writing. **Judgment.** A proof that lives on the row is one the operator reads in `getGenerations` [design/specs/application/generation-registry.sdp.md:77], and a bump that fences every older batch means a resume can be repeated safely.", + "restsOn": "[design/specs/application/generation-registry.sdp.md:47] makes the row the checkpoint and [design/specs/application/generation-registry.sdp.md:50] gives a building or verifying generation exactly one batch identified by the fence, dropped by a restore with every pending scheduled function, which [design/specs/facts/f12-backups-exclude-pending-scheduled-functions.sdp.md:11] carries as documented; [design/specs/application/restore.sdp.md:104] resumes only a row still in `building` or `verifying` at the inventoried fence; [design/specs/application/write-pause.sdp.md:43] makes a batch whose fence differs return without writing. **Judgment.** A proof that lives on the row is one the operator reads in `getGenerations` [spec:application.rebuild#design.fnGetGenerations], and a bump that fences every older batch means a resume can be repeated safely.", "against": "The fence proves that no older batch will write; it does not prove that none is running, and a `verified` row needs no batch [design/specs/application/restore.sdp.md:83], so there the fence proves nothing; an operator resume between the inventory and `finishRestoreRun` changes the fence and the finish then skips that row, which is right but silent. Nothing is built: no file under `src`, `harness`, `fixture`, `example` or `tests` contains `restoreRuns`, so the proof is intent at no tier.", "settles": "The S5 build at the `convex-test` tier with a generation restored mid-build, then the drill on the native tier [design/specs/application/restore.sdp.md:86].", "changes": "For the lean: nothing; [design/specs/application/generation-registry.sdp.md:50] and [design/specs/application/restore.sdp.md:104] as they stand. For the alternative, a `stranded` state or flag on the row: the states at [design/specs/application/generation-registry.sdp.md:42] and the transitions at [design/specs/application/generation-registry.sdp.md:44] gain one, and the restore's inventory writes it.", @@ -2805,9 +2758,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.generation-registry#5", - "file": "design/specs/application/generation-registry.sdp.md", - "line": 37 + "address": "spec:application.generation-registry#question.rebuildProgressRow", + "file": "design/specs/application/generation-registry.sdp.md" } ], "families": [ @@ -2826,11 +2778,11 @@ "reason": "Rule 3, the shape of a table and which row holds a cursor; the doc binds batching to the component and fixes no checkpoint [design/specs/application/generation-registry.sdp.md:37]; an edit to one Spec and one file takes it back. Doubt: the lean goes against the built reading of [design/specs/application/generation-registry.sdp.md:47] on a contention judgment that only the native tier shows, and S4's guidance [design/STATE.md:160], a pointer, asks for progress away from rows every command reads.", "lean": { "do": "Keep the two rows as the Specs pin them: the generation row carries state, projection version, fence and who started and last changed it, and a batch patches it only in the mutation that ends its pass; the progress row carries the cursor, the batch size and the counts and is the one checkpoint of every batch; no state of the migrations component exists beside them.", - "restsOn": "The E-40 question takes the progress row because a batch that wrote its checkpoint on the row every command reads would re-run every command in flight once per batch [design/specs/application/generation-registry.sdp.md:38]; the generation row carries nothing a batch writes on its healthy path except the state the batch ending a pass sets, and the progress row is written in the same mutation as the rows [design/specs/application/generation-registry.sdp.md:53,74]; `generationsToWrite` and `activeGeneration` read the generation row through `by_read_model_state` and never the progress row [design/specs/application/generation-registry.sdp.md:82,83,94]; a build costs the commands one re-run at the end of the backfill, one at the end of the verify pass and one per operator entry [design/specs/application/generation-registry.sdp.md:86]; the rebuild uses `@convex-dev/migrations` for nothing [design/specs/application/rebuild.sdp.md:86], so the earlier lean's clause that derived the component's state from the progress row names nothing. **Judgment.** One checkpoint with one owner is the rule of my lens: the progress row is the source of where a pass is, the generation row the source of what a generation is, and neither is computed a second way.", + "restsOn": "The E-40 question takes the progress row because a batch that wrote its checkpoint on the row every command reads would re-run every command in flight once per batch [design/specs/application/generation-registry.sdp.md:38]; the generation row carries nothing a batch writes on its healthy path except the state the batch ending a pass sets, and the progress row is written in the same mutation as the rows [design/specs/application/generation-registry.sdp.md:53,74]; `generationsToWrite` and `activeGeneration` read the generation row through `by_read_model_state` and never the progress row [spec:application.generation-registry#design.fnGenerationsToWrite,83,94]; a build costs the commands one re-run at the end of the backfill, one at the end of the verify pass and one per operator entry [spec:application.generation-registry#design.transitionCost]; the rebuild uses `@convex-dev/migrations` for nothing [design/specs/application/rebuild.sdp.md:86], so the earlier lean's clause that derived the component's state from the progress row names nothing. **Judgment.** One checkpoint with one owner is the rule of my lens: the progress row is the source of where a pass is, the generation row the source of what a generation is, and neither is computed a second way.", "against": "**Judgment.** Two rows cost a second table, a second read per batch and a second write at the end of a pass; and whether a batch's checkpoint re-runs no command is shown only at the native tier, since `convex-test` orders nothing concurrently. The ledger still asks for a contention run against a run with no rebuild [design/reviews/consensus-ledger.json:3946].", "settles": "The `convex-test` obligation that the generation row is untouched by a batch except at the end of a pass (`design-s4-paper.md:104` in the round 2 folder) and the native L2-5 example's required assertions that no command failed for reading the registry while batches checkpointed and that the generation rows changed only at the start, the end of each pass and each operator entry [design/specs/application/rebuild.online-rebuild-interrupt-resume.sdp.md:36]; neither has run. The comparison run is a later measurement.", "changes": "No Spec: `spec:application.generation-registry` `tableGenerations`, `tableGenerationProgress` and `transitionCost` and `spec:application.rebuild` `step1`, `step6` and `occRace` carry it. The register row alone. The alternative, one row, would edit the same entries and the ledger's checkpoint finding.", - "ifWrong": "If the registry read set of a command still meets a batch's write, each pass end re-runs commands once, which `transitionCost` already states [design/specs/application/generation-registry.sdp.md:86]; a meeting beyond that would be found by a contention run that records each refused execution's `occInfo` tables as the measurement does [design/specs/application/first-experiment.sdp.md:114], and an edit to the index use or the row split takes it back." + "ifWrong": "If the registry read set of a command still meets a batch's write, each pass end re-runs commands once, which `transitionCost` already states [spec:application.generation-registry#design.transitionCost]; a meeting beyond that would be found by a contention run that records each refused execution's `occInfo` tables as the measurement does [design/specs/application/first-experiment.sdp.md:114], and an edit to the index use or the row split takes it back." }, "checkedBy": "gpt-6.1-sol, medium, 2026-10-03: every checked claim held", "fork": null, @@ -2844,9 +2796,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.parent-use-cases#2", - "file": "design/specs/application/parent-use-cases.sdp.md", - "line": 55 + "address": "spec:application.parent-use-cases#question.importCommandProgress", + "file": "design/specs/application/parent-use-cases.sdp.md" }, { "kind": "extension", @@ -2886,9 +2837,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.parent-use-cases#3", - "file": "design/specs/application/parent-use-cases.sdp.md", - "line": 56 + "address": "spec:application.parent-use-cases#question.contextBusinessFailurePolicy", + "file": "design/specs/application/parent-use-cases.sdp.md" }, { "kind": "extension", @@ -2930,9 +2880,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.parent-use-cases#4", - "file": "design/specs/application/parent-use-cases.sdp.md", - "line": 57 + "address": "spec:application.parent-use-cases#question.useCaseContextOrder", + "file": "design/specs/application/parent-use-cases.sdp.md" } ], "families": [ @@ -2966,9 +2915,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.projection-contract#1", - "file": "design/specs/application/projection-contract.sdp.md", - "line": 33 + "address": "spec:application.projection-contract#question.projectionRowContract", + "file": "design/specs/application/projection-contract.sdp.md" }, { "kind": "extension", @@ -2997,7 +2945,7 @@ "reason": "Rule 3, signatures and the shape of a row: the doc gives no signature, row shape or input for the three forms [design/specs/application/projection-contract.sdp.md:33]; an edit to one Spec and the read-model files takes it back. The order in which the forms are bound is the main thread's [design/STATE.md:160], a pointer.", "lean": { "do": "Keep the per-entity form as the Specs pin it: a `ReadModel` with its `projections`, the latest first, and a `Projection` of `version`, `keyOf` and `project` alone; `applyProjection` writing each generation by its role, `active`, `building` or `verifying`, with the projection whose version the generation records; and every row measured against `rowBudgetBytes` at every write. The history form stays as declared and bound for the order allocation history, whose build is not 3b's, and the aggregate form stays a signature with no read model of the example bound to it.", - "restsOn": "The E-9 question names the three forms and the three points the owner has not ruled, the split, the write by role and the measured row [design/specs/application/projection-contract.sdp.md:33]; the declaration and the helper are pinned [design/specs/application/projection-contract.sdp.md:47,66,67,68]; the write rule by mode and role is pinned, live-updated in the role `building` skipping a missing row because the backfill has not reached it, and a missing row in `verifying` meaning a subject that projected to none [design/specs/application/projection-contract.sdp.md:50,69,92; design/specs/application/rebuild.sdp.md:69,70]; the row is measured with `getConvexSize` at every write [design/specs/application/projection-contract.sdp.md:51]. The history form is declared as `orderAllocationHistory` [design/specs/application/projection-contract.sdp.md:78] and bound once per source stream type in the example [design/specs/application/orders-inventory-example.sdp.md:69], and 3b does not build the paused path (`design-s4-paper.md:107` in the round 2 folder); the aggregate form's table is given as a shape and not as a read model the first experiment builds [design/specs/application/projection-contract.sdp.md:85], and the online rebuild is pinned for a per-entity read model with one source stream type while an aggregate gets a rebuild when a view needs it [design/specs/application/rebuild.sdp.md:61]. **Judgment.** The roles exist because the building generation has rows the backfill has not reached and the verifying one does not; one rule for both would read a missing row two ways.", + "restsOn": "The E-9 question names the three forms and the three points the owner has not ruled, the split, the write by role and the measured row [design/specs/application/projection-contract.sdp.md:33]; the declaration and the helper are pinned [design/specs/application/projection-contract.sdp.md:47,66,67,68]; the write rule by mode and role is pinned, live-updated in the role `building` skipping a missing row because the backfill has not reached it, and a missing row in `verifying` meaning a subject that projected to none [design/specs/application/projection-contract.sdp.md:50,69,92; design/specs/application/rebuild.sdp.md:69,70]; the row is measured with `getConvexSize` at every write [design/specs/application/projection-contract.sdp.md:51]. The history form is declared as `orderAllocationHistory` [spec:application.history-projection#design.tableOrderAllocationHistory] and bound once per source stream type in the example [design/specs/application/orders-inventory-example.sdp.md:69], and 3b does not build the paused path (`design-s4-paper.md:107` in the round 2 folder); the aggregate form's table is given as a shape and not as a read model the first experiment builds [spec:application.aggregate-projection#design.tableOrderTotals], and the online rebuild is pinned for a per-entity read model with one source stream type while an aggregate gets a rebuild when a view needs it [design/specs/application/rebuild.sdp.md:61]. **Judgment.** The roles exist because the building generation has rows the backfill has not reached and the verifying one does not; one rule for both would read a missing row two ways.", "against": "**Judgment.** The write rule varies by mode, by role and by the row's state, with the `null` candidate beside them, so its pure table is wide (`design-s4-paper.md:103,144` in the round 2 folder), where a rule without roles would be narrower; it is not taken because it cannot tell a row the backfill has not written from a subject that projects to none [design/specs/application/rebuild.sdp.md:70]. Measuring every row at every write costs a `getConvexSize` per write.", "settles": "The pure write-rule table of mode by role by row state, with the `null` row (`design-s4-paper.md:103,144`), and the `convex-test` obligations of a live create and a live update against `building` and against `verifying` and of two projection versions at once (`design-s4-paper.md:104`), none yet run; the owner's confirmation of the three forms, which the E-9 question asks for, stays the owner's and this ruling does not take it.", "changes": "No Spec: `spec:application.projection-contract` `typeReadModel`, `typeProjection`, `typeWritableGeneration`, `step1` to `step4` and `limitRowSize` carry the per-entity rules, and `tableOrderAllocationHistory` and the example's binding carry the history form. The register row alone. The alternative, one write rule without roles, would edit `step3` and the rebuild's rule on a missing row in `building`.", @@ -3015,9 +2963,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.read-models#1", - "file": "design/specs/application/read-models.sdp.md", - "line": 49 + "address": "spec:application.read-models#question.parentListGenerationSwitch", + "file": "design/specs/application/read-models.sdp.md" } ], "families": [ @@ -3055,9 +3002,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.read-models#2", - "file": "design/specs/application/read-models.sdp.md", - "line": 50 + "address": "spec:application.read-models#question.readModelListBounds", + "file": "design/specs/application/read-models.sdp.md" } ], "families": [ @@ -3096,21 +3042,18 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.read-models#4", - "file": "design/specs/application/read-models.sdp.md", - "line": 52 + "address": "spec:application.read-models#question.contextQueryCost", + "file": "design/specs/application/read-models.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:context.context-component#2", - "file": "design/specs/context/context-component.sdp.md", - "line": 50 + "address": "spec:context.context-component#question.contextQueryLatencyQuota", + "file": "design/specs/context/context-component.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:decisions.d02-context-owns-state-and-journal#1", - "file": "design/specs/decisions/d02-context-owns-state-and-journal.sdp.md", - "line": 30 + "address": "spec:decisions.d02-context-owns-state-and-journal#question.contextCallLatencyQuota", + "file": "design/specs/decisions/d02-context-owns-state-and-journal.sdp.md" }, { "kind": "planAmbiguity", @@ -3153,9 +3096,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.read-models#5", - "file": "design/specs/application/read-models.sdp.md", - "line": 53 + "address": "spec:application.read-models#question.parentQueryReadBounds", + "file": "design/specs/application/read-models.sdp.md" }, { "kind": "extension", @@ -3194,9 +3136,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.rebuild#2", - "file": "design/specs/application/rebuild.sdp.md", - "line": 58 + "address": "spec:application.rebuild#question.rebuildBatchBounds", + "file": "design/specs/application/rebuild.sdp.md" }, { "kind": "extension", @@ -3227,11 +3168,11 @@ "reason": "Rule 3, a bound and its number and which mechanism drives a batch: the doc gives no batch size [design/specs/application/rebuild.sdp.md:58], and the Spec itself leaves the component's reach to Probe 6, so a probe comes first.", "lean": { "do": "Keep the batch bound as the Specs pin it: at most `rebuildBatchCeiling`, 100 entities capped by the row budget, and never more than the context's `list` page admits, a field of the progress row that `startGeneration` sets and nothing changes after, so every pass and every resumed pass runs at it; every batch, over a context's enumeration or a parent table, driven by the platform's own self-scheduled internal mutation; a stalled pass answered by `abortGeneration` and a new `startGeneration` with a smaller size, and no automatic stall check.", - "restsOn": "The bound and its two caps [design/specs/application/rebuild.sdp.md:88,116,146], the purge cap [design/specs/application/rebuild.sdp.md:147], the one driver for every batch [design/specs/application/rebuild.sdp.md:59,86], `resumeGeneration` taking no batch size [design/specs/application/rebuild.sdp.md:132; design/specs/application/generation-registry.sdp.md:46] and the stall answer [design/specs/application/rebuild.sdp.md:144]; the main thread's ruling 9 after the paper's check fixed the resumed size (`design-s4-paper.md:150` in the round 2 folder), and the paper excludes an automatic stall check from 3b (`design-s4-paper.md:107`). Probe 6 settled the driver: the component walks only the tables of the component it is defined in [design/specs/facts/f17-migrations-fits-generation-backfill.sdp.md:21], so the row's provisional reading, which had the component drive parent-table batches and `resumeGeneration` lower the size, is superseded on both points. **Judgment.** A size that never changes makes the progress row the one source of a pass's shape and the batch count a function of the data, and a stall that needs a smaller size is a new generation, not a mutated one.", + "restsOn": "The bound and its two caps [design/specs/application/rebuild.sdp.md:88,116,146], the purge cap [spec:application.rebuild#design.limitPurgeBatch], the one driver for every batch [design/specs/application/rebuild.sdp.md:59,86], `resumeGeneration` taking no batch size [spec:application.rebuild#design.fnResumeGeneration; design/specs/application/generation-registry.sdp.md:46] and the stall answer [spec:application.rebuild#design.stallSignal]; the main thread's ruling 9 after the paper's check fixed the resumed size (`design-s4-paper.md:150` in the round 2 folder), and the paper excludes an automatic stall check from 3b (`design-s4-paper.md:107`). Probe 6 settled the driver: the component walks only the tables of the component it is defined in [design/specs/facts/f17-migrations-fits-generation-backfill.sdp.md:21], so the row's provisional reading, which had the component drive parent-table batches and `resumeGeneration` lower the size, is superseded on both points. **Judgment.** A size that never changes makes the progress row the one source of a pass's shape and the batch count a function of the data, and a stall that needs a smaller size is a new generation, not a mutated one.", "against": "**Judgment.** A fixed size means a hot key range forces an abort and a whole new backfill rather than a slower continuation of the same generation, and the numbers 100, 16 and `limitPurgeBatch` are design choices under F13 that no run has pressed. Sustained contention over a hot range is not run [design/specs/facts/f17-migrations-fits-generation-backfill.sdp.md:31].", "settles": "Probe 6 settled the driver at the native backend tier; the pure test of `batchSizeFor` at 0, -1, 1.5, above the ceiling and at both row budgets (`design-s4-paper.md:103`) settles the bound's arithmetic once run; the native L2-5 example specifies generation 2 at batch size 1 [design/specs/application/rebuild.online-rebuild-interrupt-resume.sdp.md:33] and has not run; contention over a hot range is a later measurement on the quiet machine.", "changes": "No Spec: `spec:application.rebuild` `fnBatchSizeFor`, `limitBatchSize`, `limitPurgeBatch`, `stallSignal` and the E-40 question carry it. The register row alone; its `provisionalReading` text is stale and the ruling supersedes it. The alternative, a size an operator lowers on resume, would edit `fnResumeChain`, `fnResumeGeneration` and the transition Contract bullet of the registry.", - "ifWrong": "The ceiling and the caps are constants an edit changes [design/specs/application/rebuild.sdp.md:146,147]; a resumed size would be one optional argument on `resumeGeneration` and one patch of the progress row, and no stored row changes meaning." + "ifWrong": "The ceiling and the caps are constants an edit changes [spec:application.rebuild#design.limitBatchSize,147]; a resumed size would be one optional argument on `resumeGeneration` and one patch of the progress row, and no stored row changes meaning." }, "checkedBy": "gpt-6.1-sol, medium, 2026-10-03: every checked claim held", "fork": null, @@ -3245,9 +3186,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.restore#1", - "file": "design/specs/application/restore.sdp.md", - "line": 55 + "address": "spec:application.restore#question.restoreBackupCoverage", + "file": "design/specs/application/restore.sdp.md" } ], "families": [ @@ -3282,9 +3222,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.restore#2", - "file": "design/specs/application/restore.sdp.md", - "line": 56 + "address": "spec:application.restore#question.restoreInvariantChecks", + "file": "design/specs/application/restore.sdp.md" }, { "kind": "extension", @@ -3313,7 +3252,7 @@ "restsOn": "[design/specs/application/restore.sdp.md:63] and [design/specs/facts/f12-backups-exclude-pending-scheduled-functions.sdp.md:11] carry as documented that a backup excludes pending scheduled functions and environment variables, which is why the door is a variable [design/specs/application/write-pause.sdp.md:48] and why a generation in flight is stranded [design/specs/application/restore.sdp.md:70]; [design/specs/application/restore.sdp.md:68] and [design/specs/application/restore.sdp.md:85] keep a failed run closed with the check and the subject named; [design/specs/application/restore.sdp.md:81] makes every check resumable from its cursor; [design/specs/application/restore.sdp.md:99] and [design/specs/application/restore.sdp.md:104] refuse `startRestoreRun` outside the door and `finishRestoreRun` while the door is still set or the inventory unfinished; [design/specs/application/restore.sdp.md:106] orders the checks so a broken journal stops the fold check. **Judgment.** In the practice of restoring a deployment, a record of what was checked and found, a run that can be interrupted and resumed, and a door a second operator cannot open by accident are what make a restore safe to run at night, and this design has each.", "against": "The operator clears `MAINTENANCE_MODE` before calling `finishRestoreRun` [design/specs/application/restore.sdp.md:84], so between the two steps writers run while the restored closed entries still hold their scopes [design/specs/application/write-pause.sdp.md:49]; if `finishRestoreRun` is never called those scopes stay closed, against what [design/specs/application/restore.sdp.md:70] promises, and only `getGate` shows why. The door is a variable that no table records: who set it and when is outside the deployment's data. Nothing is built: no file under `src`, `harness`, `fixture`, `example` or `tests` contains `restoreRuns` or `MAINTENANCE_MODE`, so every step is intent at no tier; and whether the snapshot replaces the components' tables is OD-081.", "settles": "The S5 build, then the drill on a disposable deployment on the native tier [design/specs/application/restore.sdp.md:86], with Probe 7's transactional half for what the snapshot leaves [design/specs/application/restore.sdp.md:55]; whether a failed check admits an override and how often the drill runs are policies the owner rules, outside this row.", - "changes": "For the lean: nothing in `spec:application.restore`; `spec:application.write-pause` at [design/specs/application/write-pause.sdp.md:48] and [design/specs/application/write-pause.sdp.md:61], and `spec:application.generation-registry` for `resumeChain` [design/specs/application/restore.sdp.md:104], as they stand, built in S5. For the alternative, a restore with no record and no inventory that reopens by hand: [design/specs/application/restore.sdp.md:76], [design/specs/application/restore.sdp.md:82] to [design/specs/application/restore.sdp.md:84] and [design/specs/application/restore.sdp.md:93] to [design/specs/application/restore.sdp.md:95], and a stranded rebuild's gate entries would stay closed until an operator reads the gate.", + "changes": "For the lean: nothing in `spec:application.restore`; `spec:application.write-pause` at [design/specs/application/write-pause.sdp.md:48] and [spec:application.write-pause#design.fnGateAllows], and `spec:application.generation-registry` for `resumeChain` [design/specs/application/restore.sdp.md:104], as they stand, built in S5. For the alternative, a restore with no record and no inventory that reopens by hand: [design/specs/application/restore.sdp.md:76], [design/specs/application/restore.sdp.md:82] to [design/specs/application/restore.sdp.md:84] and [design/specs/application/restore.sdp.md:93] to [design/specs/application/restore.sdp.md:95], and a stranded rebuild's gate entries would stay closed until an operator reads the gate.", "ifWrong": "A missing check is found by the drill before a production restore, and an added check is an edit plus one internal mutation; a wrong order of the two steps loses nothing, because the door refuses writes and `finishRestoreRun` refuses until the door is clear [design/specs/application/restore.sdp.md:104]; **Judgment.** the one cost an edit does not take back is a restore accepted on a dataset the drill did not represent, which is why [design/specs/application/restore.sdp.md:86] records the dataset." }, "checkedBy": null, @@ -3328,9 +3267,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.restore#3", - "file": "design/specs/application/restore.sdp.md", - "line": 57 + "address": "spec:application.restore#question.restoreStreamBatchBounds", + "file": "design/specs/application/restore.sdp.md" } ], "families": [ @@ -3364,9 +3302,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:application.write-pause#1", - "file": "design/specs/application/write-pause.sdp.md", - "line": 34 + "address": "spec:application.write-pause#question.writePauseSourceScopes", + "file": "design/specs/application/write-pause.sdp.md" } ], "families": [ @@ -3390,10 +3327,10 @@ "reason": "Delegated test: a Convex mechanism, a table shape and a refusal code inside what D9 and D4 rule; the one trade-off, sources against the read model as the closed scope, has a documented reason on its side; the acceptable length of a pause is the owner's policy and not this row.", "lean": { "do": "Keep one gate document per deployment with closed scopes in one `closed` state, no drain, the `writePaused` transient refusal, and a rebuild closing the paused view's sources.", - "restsOn": "[design/specs/application/write-pause.sdp.md:38] pins the one document and its two readers, `assertWritable` at pipeline step 7 and `gateAllows` for every background batch; [design/specs/application/write-pause.sdp.md:43] and [design/specs/application/write-pause.sdp.md:71] rest the consistent cut on optimistic concurrency, which [design/specs/facts/f01-serializable-mutations-under-occ.sdp.md:11] carries as documented; [design/specs/application/write-pause.sdp.md:42] closes the sources because a use case that writes a source stream without maintaining the view would otherwise pass the gate; [design/specs/application/write-pause.sdp.md:40] and [src/command/outcome-boundary.ts:31] make `writePaused` a transient code the boundary already reserves; [design/specs/application/write-pause.sdp.md:50] writes every close, resume and abort as audit that fails closed. **Judgment.** In the practice of running a deployment, one state in one document is what a person at night can read and reason about: `getGate` [design/specs/application/write-pause.sdp.md:70] shows every closed scope with its reason and who changed it, and nothing reopens by itself [design/specs/application/write-pause.sdp.md:23].", - "against": "Every command pays one point read on one hot document and a close re-runs every concurrent command once [design/specs/application/write-pause.sdp.md:29]; `gateAllows` reads `process.env.MAINTENANCE_MODE` on every command [design/specs/application/write-pause.sdp.md:61], so a variable set by mistake refuses every write with no row to find it in. The gate is not built: no file under `src`, `harness`, `fixture`, `example` or `tests` contains `assertWritable` or `gateAllows`, so the cut and the one-read cost are intent at no tier.", - "settles": "The S4 build at the `convex-test` tier for the refusal and the fence, and a native backend run of two writers racing a close, which [design/specs/application/write-pause.sdp.md:71] describes and nobody has run.", - "changes": "For the lean: nothing in `spec:application.write-pause`; `spec:command.command-pipeline` step 7 and `spec:application.rebuild` as they stand, built in S4. For the alternative, closing the tenant or `all` for every paused rebuild [design/specs/application/write-pause.sdp.md:34]: [design/specs/application/write-pause.sdp.md:41], [design/specs/application/write-pause.sdp.md:42] and `scopesOfUseCase` [design/specs/application/write-pause.sdp.md:64], and every writer of an unrelated stream in the tenant is held for the length of the rebuild.", + "restsOn": "[design/specs/application/write-pause.sdp.md:38] pins the one document and its two readers, `assertWritable` at pipeline step 7 and `gateAllows` for every background batch; [design/specs/application/write-pause.sdp.md:43] and [spec:application.write-pause#design.occSerialization] rest the consistent cut on optimistic concurrency, which [design/specs/facts/f01-serializable-mutations-under-occ.sdp.md:11] carries as documented; [design/specs/application/write-pause.sdp.md:42] closes the sources because a use case that writes a source stream without maintaining the view would otherwise pass the gate; [design/specs/application/write-pause.sdp.md:40] and [src/command/outcome-boundary.ts:31] make `writePaused` a transient code the boundary already reserves; [design/specs/application/write-pause.sdp.md:50] writes every close, resume and abort as audit that fails closed. **Judgment.** In the practice of running a deployment, one state in one document is what a person at night can read and reason about: `getGate` [spec:application.write-pause#design.fnGetGate] shows every closed scope with its reason and who changed it, and nothing reopens by itself [design/specs/application/write-pause.sdp.md:23].", + "against": "Every command pays one point read on one hot document and a close re-runs every concurrent command once [design/specs/application/write-pause.sdp.md:29]; `gateAllows` reads `process.env.MAINTENANCE_MODE` on every command [spec:application.write-pause#design.fnGateAllows], so a variable set by mistake refuses every write with no row to find it in. The gate is not built: no file under `src`, `harness`, `fixture`, `example` or `tests` contains `assertWritable` or `gateAllows`, so the cut and the one-read cost are intent at no tier.", + "settles": "The S4 build at the `convex-test` tier for the refusal and the fence, and a native backend run of two writers racing a close, which [spec:application.write-pause#design.occSerialization] describes and nobody has run.", + "changes": "For the lean: nothing in `spec:application.write-pause`; `spec:command.command-pipeline` step 7 and `spec:application.rebuild` as they stand, built in S4. For the alternative, closing the tenant or `all` for every paused rebuild [design/specs/application/write-pause.sdp.md:34]: [design/specs/application/write-pause.sdp.md:41], [design/specs/application/write-pause.sdp.md:42] and `scopesOfUseCase` [spec:application.write-pause#design.fnScopesOfUseCase], and every writer of an unrelated stream in the tenant is held for the length of the rebuild.", "ifWrong": "If a writer slips the cut, the read-model check compares each row's source versions with the current versions and finds it [design/specs/application/restore.sdp.md:79], so the cost is a rebuild rerun and not lost events, and an edit to `scopesOfUseCase` takes the scope back; if the hot document costs more than one read's worth under contention, the S3 measurement repeated after S4 shows it." }, "checkedBy": "gpt-6.1-sol, medium, 2026-10-02: every checked claim held", @@ -3408,15 +3345,13 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.actor-and-scope#1", - "file": "design/specs/command/actor-and-scope.sdp.md", - "line": 31 + "address": "spec:command.actor-and-scope#question.actorCallerNamespaceContract", + "file": "design/specs/command/actor-and-scope.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:command.tenancy-and-authority#2", - "file": "design/specs/command/tenancy-and-authority.sdp.md", - "line": 38 + "address": "spec:command.tenancy-and-authority#question.actorCallerNamespaceShapes", + "file": "design/specs/command/tenancy-and-authority.sdp.md" }, { "kind": "extension", @@ -3460,9 +3395,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.actor-and-scope#3", - "file": "design/specs/command/actor-and-scope.sdp.md", - "line": 33 + "address": "spec:command.actor-and-scope#question.parentQueryAuthorization", + "file": "design/specs/command/actor-and-scope.sdp.md" } ], "families": [ @@ -3496,9 +3430,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.command-declaration#1", - "file": "design/specs/command/command-declaration.sdp.md", - "line": 34 + "address": "spec:command.command-declaration#question.commandEntrySignatures", + "file": "design/specs/command/command-declaration.sdp.md" }, { "kind": "extension", @@ -3538,9 +3471,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.command-declaration#3", - "file": "design/specs/command/command-declaration.sdp.md", - "line": 36 + "address": "spec:command.command-declaration#question.commandDeclarationFields", + "file": "design/specs/command/command-declaration.sdp.md" }, { "kind": "extension", @@ -3585,9 +3517,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.command-pipeline#1", - "file": "design/specs/command/command-pipeline.sdp.md", - "line": 59 + "address": "spec:command.command-pipeline#question.commandOutcomeWireShape", + "file": "design/specs/command/command-pipeline.sdp.md" }, { "kind": "extension", @@ -3627,9 +3558,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.command-pipeline#2", - "file": "design/specs/command/command-pipeline.sdp.md", - "line": 60 + "address": "spec:command.command-pipeline#question.operationIdentityCause", + "file": "design/specs/command/command-pipeline.sdp.md" }, { "kind": "extension", @@ -3670,9 +3600,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.command-pipeline#3", - "file": "design/specs/command/command-pipeline.sdp.md", - "line": 61 + "address": "spec:command.command-pipeline#question.commandAdmissionPolicy", + "file": "design/specs/command/command-pipeline.sdp.md" }, { "kind": "extension", @@ -3712,9 +3641,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.command-pipeline#5", - "file": "design/specs/command/command-pipeline.sdp.md", - "line": 63 + "address": "spec:command.command-pipeline#question.contextCallBudgetReservation", + "file": "design/specs/command/command-pipeline.sdp.md" } ], "families": [ @@ -3751,9 +3679,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.command-pipeline#6", - "file": "design/specs/command/command-pipeline.sdp.md", - "line": 64 + "address": "spec:command.command-pipeline#question.commandReadModelWrites", + "file": "design/specs/command/command-pipeline.sdp.md" } ], "families": [ @@ -3790,9 +3717,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.command-pipeline#7", - "file": "design/specs/command/command-pipeline.sdp.md", - "line": 65 + "address": "spec:command.command-pipeline#question.commandTextBounds", + "file": "design/specs/command/command-pipeline.sdp.md" } ], "families": [ @@ -3826,9 +3752,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.idempotency-and-receipts#3", - "file": "design/specs/command/idempotency-and-receipts.sdp.md", - "line": 44 + "address": "spec:command.idempotency-and-receipts#question.receiptRetentionBounds", + "file": "design/specs/command/idempotency-and-receipts.sdp.md" }, { "kind": "extension", @@ -3868,9 +3793,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.idempotency-and-receipts#4", - "file": "design/specs/command/idempotency-and-receipts.sdp.md", - "line": 45 + "address": "spec:command.idempotency-and-receipts#question.businessInputFingerprint", + "file": "design/specs/command/idempotency-and-receipts.sdp.md" }, { "kind": "extension", @@ -3909,9 +3833,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.idempotency-and-receipts#5", - "file": "design/specs/command/idempotency-and-receipts.sdp.md", - "line": 46 + "address": "spec:command.idempotency-and-receipts#question.duplicateSubjectCreation", + "file": "design/specs/command/idempotency-and-receipts.sdp.md" }, { "kind": "extension", @@ -3950,9 +3873,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.outcome-boundary#1", - "file": "design/specs/command/outcome-boundary.sdp.md", - "line": 38 + "address": "spec:command.outcome-boundary#question.rejectionShapeCodes", + "file": "design/specs/command/outcome-boundary.sdp.md" }, { "kind": "extension", @@ -3991,9 +3913,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.outcome-boundary#3", - "file": "design/specs/command/outcome-boundary.sdp.md", - "line": 40 + "address": "spec:command.outcome-boundary#question.writePauseTransientRefusal", + "file": "design/specs/command/outcome-boundary.sdp.md" }, { "kind": "extension", @@ -4032,9 +3953,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.receipt-table#1", - "file": "design/specs/command/receipt-table.sdp.md", - "line": 28 + "address": "spec:command.receipt-table#question.receiptFieldsIndexes", + "file": "design/specs/command/receipt-table.sdp.md" }, { "kind": "extension", @@ -4072,9 +3992,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:command.tenancy-and-authority#1", - "file": "design/specs/command/tenancy-and-authority.sdp.md", - "line": 37 + "address": "spec:command.tenancy-and-authority#question.actorGrantRevocation", + "file": "design/specs/command/tenancy-and-authority.sdp.md" }, { "kind": "extension", @@ -4117,9 +4036,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:constraints.events-stay-small#1", - "file": "design/specs/constraints/events-stay-small.sdp.md", - "line": 26 + "address": "spec:constraints.events-stay-small#question.eventPayloadBound", + "file": "design/specs/constraints/events-stay-small.sdp.md" }, { "kind": "extension", @@ -4167,9 +4085,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:context.context-component#3", - "file": "design/specs/context/context-component.sdp.md", - "line": 51 + "address": "spec:context.context-component#question.contextOperationRegistration", + "file": "design/specs/context/context-component.sdp.md" }, { "kind": "extension", @@ -4213,9 +4130,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:context.event-envelope#1", - "file": "design/specs/context/event-envelope.sdp.md", - "line": 32 + "address": "spec:context.event-envelope#question.eventEnvelopeFields", + "file": "design/specs/context/event-envelope.sdp.md" }, { "kind": "extension", @@ -4253,9 +4169,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:context.event-envelope#2", - "file": "design/specs/context/event-envelope.sdp.md", - "line": 33 + "address": "spec:context.event-envelope#question.envelopeTextBounds", + "file": "design/specs/context/event-envelope.sdp.md" } ], "families": [ @@ -4289,9 +4204,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:context.journal#1", - "file": "design/specs/context/journal.sdp.md", - "line": 45 + "address": "spec:context.journal#question.baselineMigrationContract", + "file": "design/specs/context/journal.sdp.md" }, { "kind": "extension", @@ -4337,9 +4251,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:context.persistence-adapter#1", - "file": "design/specs/context/persistence-adapter.sdp.md", - "line": 43 + "address": "spec:context.persistence-adapter#question.contextOperationOutcome", + "file": "design/specs/context/persistence-adapter.sdp.md" }, { "kind": "extension", @@ -4379,21 +4292,18 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:context.queries#1", - "file": "design/specs/context/queries.sdp.md", - "line": 35 + "address": "spec:context.queries#question.contextPaginationSplit", + "file": "design/specs/context/queries.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:decisions.d08-read-models-in-command#1", - "file": "design/specs/decisions/d08-read-models-in-command.sdp.md", - "line": 30 + "address": "spec:decisions.d08-read-models-in-command#question.parentQueryPaginationSplit", + "file": "design/specs/decisions/d08-read-models-in-command.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:facts.probe-plan#3", - "file": "design/specs/facts/probe-plan.sdp.md", - "line": 40 + "address": "spec:facts.probe-plan#question.contextListSplit", + "file": "design/specs/facts/probe-plan.sdp.md" } ], "families": [ @@ -4429,9 +4339,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:context.queries#2", - "file": "design/specs/context/queries.sdp.md", - "line": 36 + "address": "spec:context.queries#question.contextQueryContract", + "file": "design/specs/context/queries.sdp.md" }, { "kind": "extension", @@ -4476,9 +4385,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:context.queries#3", - "file": "design/specs/context/queries.sdp.md", - "line": 37 + "address": "spec:context.queries#question.queryRefusalEntry", + "file": "design/specs/context/queries.sdp.md" } ], "families": [ @@ -4511,9 +4419,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:context.tables#1", - "file": "design/specs/context/tables.sdp.md", - "line": 29 + "address": "spec:context.tables#question.contextJournalTables", + "file": "design/specs/context/tables.sdp.md" }, { "kind": "extension", @@ -4556,21 +4463,18 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:decisions.d03-events-only-source-of-next-state#1", - "file": "design/specs/decisions/d03-events-only-source-of-next-state.sdp.md", - "line": 26 + "address": "spec:decisions.d03-events-only-source-of-next-state#question.initialStateChoice", + "file": "design/specs/decisions/d03-events-only-source-of-next-state.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:kernel.domain-kernel#2", - "file": "design/specs/kernel/domain-kernel.sdp.md", - "line": 38 + "address": "spec:kernel.domain-kernel#question.initialStateChoice", + "file": "design/specs/kernel/domain-kernel.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:kernel.initial-state#1", - "file": "design/specs/kernel/initial-state.sdp.md", - "line": 24 + "address": "spec:kernel.initial-state#question.initialStateChoice", + "file": "design/specs/kernel/initial-state.sdp.md" }, { "kind": "extension", @@ -4609,15 +4513,13 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:decisions.d03-events-only-source-of-next-state#2", - "file": "design/specs/decisions/d03-events-only-source-of-next-state.sdp.md", - "line": 27 + "address": "spec:decisions.d03-events-only-source-of-next-state#question.streamStateSizeBudget", + "file": "design/specs/decisions/d03-events-only-source-of-next-state.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:kernel.state-document-mapping#1", - "file": "design/specs/kernel/state-document-mapping.sdp.md", - "line": 27 + "address": "spec:kernel.state-document-mapping#question.streamStateBudget", + "file": "design/specs/kernel/state-document-mapping.sdp.md" }, { "kind": "extension", @@ -4661,15 +4563,13 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:decisions.d13-deferred-work-is-an-obligation#1", - "file": "design/specs/decisions/d13-deferred-work-is-an-obligation.sdp.md", - "line": 35 + "address": "spec:decisions.d13-deferred-work-is-an-obligation#question.obligationSchedulerCoverage", + "file": "design/specs/decisions/d13-deferred-work-is-an-obligation.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:obligations.do-nothing-check#1", - "file": "design/specs/obligations/do-nothing-check.sdp.md", - "line": 33 + "address": "spec:obligations.do-nothing-check#question.schedulerObligationEvidence", + "file": "design/specs/obligations/do-nothing-check.sdp.md" } ], "families": [ @@ -4707,15 +4607,13 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:decisions.d19-operations-travel-with-capability#1", - "file": "design/specs/decisions/d19-operations-travel-with-capability.sdp.md", - "line": 31 + "address": "spec:decisions.d19-operations-travel-with-capability#question.restoreObligationCoverage", + "file": "design/specs/decisions/d19-operations-travel-with-capability.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:obligations.retention-and-restore#3", - "file": "design/specs/obligations/retention-and-restore.sdp.md", - "line": 35 + "address": "spec:obligations.retention-and-restore#question.restoreDeferredWorkCoverage", + "file": "design/specs/obligations/retention-and-restore.sdp.md" } ], "families": [ @@ -4750,9 +4648,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:effects.claim-call-settle#1", - "file": "design/specs/effects/claim-call-settle.sdp.md", - "line": 41 + "address": "spec:effects.claim-call-settle#question.effectAttemptPolicies", + "file": "design/specs/effects/claim-call-settle.sdp.md" }, { "kind": "extension", @@ -4791,15 +4688,13 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:effects.claim-call-settle#2", - "file": "design/specs/effects/claim-call-settle.sdp.md", - "line": 42 + "address": "spec:effects.claim-call-settle#question.effectAttemptLeaseEvidence", + "file": "design/specs/effects/claim-call-settle.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:effects.external-effects#1", - "file": "design/specs/effects/external-effects.sdp.md", - "line": 42 + "address": "spec:effects.external-effects#question.effectAttemptRecoveryContract", + "file": "design/specs/effects/external-effects.sdp.md" }, { "kind": "extension", @@ -4841,15 +4736,13 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:effects.external-effects#2", - "file": "design/specs/effects/external-effects.sdp.md", - "line": 43 + "address": "spec:effects.external-effects#question.providerBusinessRejection", + "file": "design/specs/effects/external-effects.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:obligations.lifecycle-transitions#2", - "file": "design/specs/obligations/lifecycle-transitions.sdp.md", - "line": 26 + "address": "spec:obligations.lifecycle-transitions#question.obligationBusinessRejection", + "file": "design/specs/obligations/lifecycle-transitions.sdp.md" }, { "kind": "extension", @@ -4891,15 +4784,13 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:effects.external-effects#3", - "file": "design/specs/effects/external-effects.sdp.md", - "line": 44 + "address": "spec:effects.external-effects#question.effectAttemptDefaultBounds", + "file": "design/specs/effects/external-effects.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:obligations.obligation-module#4", - "file": "design/specs/obligations/obligation-module.sdp.md", - "line": 56 + "address": "spec:obligations.obligation-module#question.obligationDefaultBounds", + "file": "design/specs/obligations/obligation-module.sdp.md" }, { "kind": "planAmbiguity", @@ -4947,21 +4838,18 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:facts.f16-scheduled-functions-table-shows-failed-runs#1", - "file": "design/specs/facts/f16-scheduled-functions-table-shows-failed-runs.sdp.md", - "line": 21 + "address": "spec:facts.f16-scheduled-functions-table-shows-failed-runs#question.scheduledFunctionRetentionCoverage", + "file": "design/specs/facts/f16-scheduled-functions-table-shows-failed-runs.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:facts.probe-plan#5", - "file": "design/specs/facts/probe-plan.sdp.md", - "line": 42 + "address": "spec:facts.probe-plan#question.schedulerRestoreCoverage", + "file": "design/specs/facts/probe-plan.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:obligations.obligation-module#1", - "file": "design/specs/obligations/obligation-module.sdp.md", - "line": 53 + "address": "spec:obligations.obligation-module#question.obligationDispatchEvidence", + "file": "design/specs/obligations/obligation-module.sdp.md" } ], "families": [ @@ -5002,15 +4890,13 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:kernel.decider-contract#1", - "file": "design/specs/kernel/decider-contract.sdp.md", - "line": 29 + "address": "spec:kernel.decider-contract#question.deciderEventContract", + "file": "design/specs/kernel/decider-contract.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:kernel.domain-kernel#1", - "file": "design/specs/kernel/domain-kernel.sdp.md", - "line": 37 + "address": "spec:kernel.domain-kernel#question.deciderStateContract", + "file": "design/specs/kernel/domain-kernel.sdp.md" }, { "kind": "extension", @@ -5048,9 +4934,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:kernel.outcome-model#1", - "file": "design/specs/kernel/outcome-model.sdp.md", - "line": 35 + "address": "spec:kernel.outcome-model#question.outcomeRejectionContract", + "file": "design/specs/kernel/outcome-model.sdp.md" }, { "kind": "extension", @@ -5089,9 +4974,10 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:kernel.state-document-mapping#2", + "spec": "spec:kernel.state-document-mapping", + "question": "OQ3: one document per stream as the default, and the largest order the placement command supports, are product decisions; the design assumes one document and leaves the maximum lines as a placeholder in the first experiment (OQ3, D10)", "file": "design/specs/kernel/state-document-mapping.sdp.md", - "line": 28 + "open": false }, { "kind": "planAmbiguity", @@ -5129,9 +5015,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:obligations.fan-out-and-chains#1", - "file": "design/specs/obligations/fan-out-and-chains.sdp.md", - "line": 30 + "address": "spec:obligations.fan-out-and-chains#question.obligationFanOutBounds", + "file": "design/specs/obligations/fan-out-and-chains.sdp.md" }, { "kind": "extension", @@ -5175,9 +5060,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:obligations.lifecycle-transitions#1", - "file": "design/specs/obligations/lifecycle-transitions.sdp.md", - "line": 25 + "address": "spec:obligations.lifecycle-transitions#question.obligationStateTransitions", + "file": "design/specs/obligations/lifecycle-transitions.sdp.md" } ], "families": [ @@ -5210,9 +5094,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:obligations.local-reaction-wrapper#1", - "file": "design/specs/obligations/local-reaction-wrapper.sdp.md", - "line": 34 + "address": "spec:obligations.local-reaction-wrapper#question.obligationAttemptBackoff", + "file": "design/specs/obligations/local-reaction-wrapper.sdp.md" } ], "families": [ @@ -5246,9 +5129,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:obligations.obligation-module#2", - "file": "design/specs/obligations/obligation-module.sdp.md", - "line": 54 + "address": "spec:obligations.obligation-module#question.parentObligationPlacement", + "file": "design/specs/obligations/obligation-module.sdp.md" }, { "kind": "extension", @@ -5288,9 +5170,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:obligations.operator-operations#1", - "file": "design/specs/obligations/operator-operations.sdp.md", - "line": 32 + "address": "spec:obligations.operator-operations#question.operatorObligationRepair", + "file": "design/specs/obligations/operator-operations.sdp.md" }, { "kind": "extension", @@ -5335,9 +5216,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:obligations.record-contract#1", - "file": "design/specs/obligations/record-contract.sdp.md", - "line": 28 + "address": "spec:obligations.record-contract#question.obligationCompletionEvidence", + "file": "design/specs/obligations/record-contract.sdp.md" }, { "kind": "extension", @@ -5378,15 +5258,13 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:obligations.record-contract#2", - "file": "design/specs/obligations/record-contract.sdp.md", - "line": 29 + "address": "spec:obligations.record-contract#question.obligationTenantIndexes", + "file": "design/specs/obligations/record-contract.sdp.md" }, { "kind": "specQuestion", - "ref": "spec:obligations.sweeper#3", - "file": "design/specs/obligations/sweeper.sdp.md", - "line": 38 + "address": "spec:obligations.sweeper#question.tenantSweepFairness", + "file": "design/specs/obligations/sweeper.sdp.md" }, { "kind": "extension", @@ -5426,9 +5304,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:obligations.retention-and-restore#1", - "file": "design/specs/obligations/retention-and-restore.sdp.md", - "line": 33 + "address": "spec:obligations.retention-and-restore#question.obligationRetentionBounds", + "file": "design/specs/obligations/retention-and-restore.sdp.md" }, { "kind": "extension", @@ -5468,9 +5345,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:obligations.retention-and-restore#2", - "file": "design/specs/obligations/retention-and-restore.sdp.md", - "line": 34 + "address": "spec:obligations.retention-and-restore#question.restoreDispatchSwitch", + "file": "design/specs/obligations/retention-and-restore.sdp.md" }, { "kind": "extension", @@ -5516,9 +5392,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:obligations.sweeper#1", - "file": "design/specs/obligations/sweeper.sdp.md", - "line": 36 + "address": "spec:obligations.sweeper#question.obligationSweepBounds", + "file": "design/specs/obligations/sweeper.sdp.md" }, { "kind": "extension", @@ -5559,9 +5434,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:obligations.sweeper#2", - "file": "design/specs/obligations/sweeper.sdp.md", - "line": 37 + "address": "spec:obligations.sweeper#question.sweepContextCallCost", + "file": "design/specs/obligations/sweeper.sdp.md" } ], "families": [ @@ -5595,9 +5469,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:operations.baseline-operations#1", - "file": "design/specs/operations/baseline-operations.sdp.md", - "line": 42 + "address": "spec:operations.baseline-operations#question.commandAuditDiagnostics", + "file": "design/specs/operations/baseline-operations.sdp.md" }, { "kind": "extension", @@ -5640,9 +5513,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:operations.baseline-operations#2", - "file": "design/specs/operations/baseline-operations.sdp.md", - "line": 43 + "address": "spec:operations.baseline-operations#question.historicalClaimRetention", + "file": "design/specs/operations/baseline-operations.sdp.md" }, { "kind": "productDecision", @@ -5679,9 +5551,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:platform.acceptance-contract#1", - "file": "design/specs/platform/acceptance-contract.sdp.md", - "line": 24 + "address": "spec:platform.acceptance-contract#question.compiledAcceptanceTier", + "file": "design/specs/platform/acceptance-contract.sdp.md" }, { "kind": "extension", @@ -5724,9 +5595,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:platform.transactional-domain-platform#1", - "file": "design/specs/platform/transactional-domain-platform.sdp.md", - "line": 44 + "address": "spec:platform.transactional-domain-platform#question.decisionSpecMapping", + "file": "design/specs/platform/transactional-domain-platform.sdp.md" }, { "kind": "planAmbiguity", @@ -5763,9 +5633,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:platform.vocabulary#1", - "file": "design/specs/platform/vocabulary.sdp.md", - "line": 25 + "address": "spec:platform.vocabulary#question.receiptCompletionEvidence", + "file": "design/specs/platform/vocabulary.sdp.md" } ], "families": [ @@ -5796,9 +5665,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:platform.vocabulary#2", - "file": "design/specs/platform/vocabulary.sdp.md", - "line": 26 + "address": "spec:platform.vocabulary#question.generationObligationAttempt", + "file": "design/specs/platform/vocabulary.sdp.md" } ], "families": [ @@ -6067,10 +5935,10 @@ "status": "sorted", "foldedInto": [], "class": "delegated", - "reason": "Rule 3, a bound and its number: the doc says events stay small and names no number, both numbers are extensions [design/specs/command/command-pipeline.sdp.md:131] and [design/specs/command/outcome-boundary.sdp.md:98], a refused caller is a rejection that stores nothing, and an edit to two Spec lines and two constants takes either back.", + "reason": "Rule 3, a bound and its number: the doc says events stay small and names no number, both numbers are extensions [spec:command.command-pipeline#design.limitCallText] and [design/specs/command/outcome-boundary.sdp.md:98], a refused caller is a rejection that stores nothing, and an edit to two Spec lines and two constants takes either back.", "lean": { "do": "Keep `limitActorIdLength` at 512 bytes and the rejection `details` bound at 16,384 bytes.", - "restsOn": "[design/specs/command/command-pipeline.sdp.md:131] lists `actor.id` and `actor.onBehalfOf.id` at 512 among the caller-set text fields step 1 checks, with a field one byte over answered as rejection `invalidInput` with `{ field, length, limit }`, and says `limitEnvelopeBytes` is derived from these bounds; [design/specs/command/outcome-boundary.sdp.md:98] bounds a rejection's `details` to 16,384 bytes measured with `getConvexSize`, checked by `reject` and by `normalizeThrown`. [src/context/text.ts:3] exports 512 and [src/command/outcome-boundary.ts:100] holds 16384. The pure and `convex-test` tiers pass as a whole at 5bb227e, 438 tests, run by me; the test that binds these two lines I did not open. **Judgment.** The two numbers are inputs of a derived set, the envelope bound and the fold bound rest on them, and what a wrong number costs is a refused caller or a refused rejection, each a rejection that stores nothing (`CONTEXT.md:55`), so no stored event or receipt carries the mistake.", + "restsOn": "[spec:command.command-pipeline#design.limitCallText] lists `actor.id` and `actor.onBehalfOf.id` at 512 among the caller-set text fields step 1 checks, with a field one byte over answered as rejection `invalidInput` with `{ field, length, limit }`, and says `limitEnvelopeBytes` is derived from these bounds; [design/specs/command/outcome-boundary.sdp.md:98] bounds a rejection's `details` to 16,384 bytes measured with `getConvexSize`, checked by `reject` and by `normalizeThrown`. [src/context/text.ts:3] exports 512 and [src/command/outcome-boundary.ts:100] holds 16384. The pure and `convex-test` tiers pass as a whole at 5bb227e, 438 tests, run by me; the test that binds these two lines I did not open. **Judgment.** The two numbers are inputs of a derived set, the envelope bound and the fold bound rest on them, and what a wrong number costs is a refused caller or a refused rejection, each a rejection that stores nothing (`CONTEXT.md:55`), so no stored event or receipt carries the mistake.", "against": "**Judgment.** 512 for `actor.id` has to hold whatever `tokenIdentifier` an issuer mints, and no cited source gives the longest one the pinned Convex auth produces; that is a lead for the convex advisor. 16,384 for `details` equals the event payload bound at [src/context/journal.ts:68], so a rejection's details may be as large as an event, and the rule that error data carries no payload rests on the one-sentence message rule of the same line, not on the number.", "settles": "For the lead: the longest `tokenIdentifier` a Convex auth provider produces, from the docs page with the day it was read, which the convex advisor's lane cites. Otherwise nothing: either number changes by an edit, and a native run with a caller at the bound is the only tier that shows the refusal on the production composition.", "changes": "For the lean: nothing. For the alternative, other numbers: `design/specs/command/command-pipeline.sdp.md` (`limitCallText`), `design/specs/context/event-envelope.sdp.md` (`limitEnvelopeBytes`, which [design/specs/context/event-envelope.sdp.md:33] says the journal's fold bound moves with), `design/specs/command/outcome-boundary.sdp.md` (`limitErrorData`), `src/context/text.ts` and `src/command/outcome-boundary.ts`.", @@ -6113,7 +5981,7 @@ "reason": "Owner tests 5 and 6: who may run a rebuild or a restore across every tenant trades safety against ease with no evidence on which weighs more, and any grant for it needs a scope beyond the tenant, which rules on the term grant, a permission in a tenant [CONTEXT.md:128]; doubt stated, because admin access alone is a mechanism and D11's trust boundary already admits it.", "lean": { "do": "Keep admin access as the authority for a deployment-wide operation, with the operator actor stated as the argument and written on the generation row, the run record and the gate's audit, and read no grant for it until a scope beyond the tenant exists.", - "restsOn": "[design/specs/application/rebuild.sdp.md:94] makes every rebuild function internal and reached through operator functions that authorize an operator actor; [design/specs/application/rebuild.sdp.md:73] runs the rebuild across every tenant under one operator actor; [design/specs/command/actor-and-scope.sdp.md:43] and [design/specs/command/actor-and-scope.sdp.md:71] make every grant name one tenant and [docs/convex-transactional-domain-platform-decisions.md:251] make an absent tenant never a wildcard, so no grant the table can hold covers a pass over every tenant; what is recorded is the stated actor, `startedBy` on the row [design/specs/application/generation-registry.sdp.md:63], `operator` on the restore run [design/specs/application/restore.sdp.md:94] and `changedBy` on every gate change [design/specs/application/write-pause.sdp.md:50]; [design/specs/decisions/d11-tenant-scope-and-authority.sdp.md:32] rules the trust boundary as one trusted deployment with no claim against a malicious administrator. **Judgment.** In the practice of answering for who ran a restore, a stated actor is recorded and not checked: a person with admin access can state any actor, and the record then says what they chose to say.", + "restsOn": "[design/specs/application/rebuild.sdp.md:94] makes every rebuild function internal and reached through operator functions that authorize an operator actor; [design/specs/application/rebuild.sdp.md:73] runs the rebuild across every tenant under one operator actor; [design/specs/command/actor-and-scope.sdp.md:43] and [design/specs/command/actor-and-scope.sdp.md:71] make every grant name one tenant and [docs/convex-transactional-domain-platform-decisions.md:251] make an absent tenant never a wildcard, so no grant the table can hold covers a pass over every tenant; what is recorded is the stated actor, `startedBy` on the row [spec:application.generation-registry#design.tableGenerations], `operator` on the restore run [design/specs/application/restore.sdp.md:94] and `changedBy` on every gate change [design/specs/application/write-pause.sdp.md:50]; [design/specs/decisions/d11-tenant-scope-and-authority.sdp.md:32] rules the trust boundary as one trusted deployment with no claim against a malicious administrator. **Judgment.** In the practice of answering for who ran a restore, a stated actor is recorded and not checked: a person with admin access can state any actor, and the record then says what they chose to say.", "against": "[design/specs/application/rebuild.sdp.md:94] promises operator functions that authorize an operator actor, and with no grant to read that sentence is a promise nothing can keep as written; the alternatives are a grant in every tenant, which a pass of a hundred tenants reads a hundred times, or a deployment scope that is not a tenant, which only the owner can name.", "settles": "The owner's word on whether a deployment-wide operator holds a grant, in which scope, or admin access alone; until then the S4 build takes the stated actor and records it." }, @@ -6183,7 +6051,7 @@ "reason": "Owner test 5, and the doc makes acceptability itself a trigger: [docs/convex-transactional-domain-platform-decisions.md:354] activates online rebuild of cross-stream views when the write pause is unacceptable for them, so naming the acceptable length settles a trigger of the doc, which only the owner does; a policy by the operator lens's own rule.", "lean": { "do": "Keep the Spec silent on a bound, and give the owner the length as a sum: the backfill plus the verify pass of the view's sources, batch count times batch time, with the levels each would mean.", - "restsOn": "[design/specs/application/write-pause.sdp.md:23] reopens nothing by itself and [design/specs/application/write-pause.sdp.md:46] resumes a scope only after the rebuild it served is verified and switched or aborted, so the pause lasts the whole rebuild; [design/specs/application/rebuild.sdp.md:121] bounds a history batch at 128 events at the default row budget and [design/specs/application/rebuild.sdp.md:124] a backfill batch at 100 entities, so the batch count follows from the view's size; [docs/convex-transactional-domain-platform-decisions.md:229] says the pause costs more than the online path and is the exception. **Judgment.** Minutes admit only small views; an hour is a maintenance window the operator announces; a pause longer than the business tolerates is the trigger at [docs/convex-transactional-domain-platform-decisions.md:354] firing, and no batch time is known at any tier until S4 measures one.", + "restsOn": "[design/specs/application/write-pause.sdp.md:23] reopens nothing by itself and [design/specs/application/write-pause.sdp.md:46] resumes a scope only after the rebuild it served is verified and switched or aborted, so the pause lasts the whole rebuild; [spec:application.history-rebuild#design.limitHistoryBatch] bounds a history batch at 128 events at the default row budget and [spec:application.rebuild#design.limitBatchSize] a backfill batch at 100 entities, so the batch count follows from the view's size; [docs/convex-transactional-domain-platform-decisions.md:229] says the pause costs more than the online path and is the exception. **Judgment.** Minutes admit only small views; an hour is a maintenance window the operator announces; a pause longer than the business tolerates is the trigger at [docs/convex-transactional-domain-platform-decisions.md:354] firing, and no batch time is known at any tier until S4 measures one.", "settles": "S4's measurement of a history-view rebuild under a pause on the native tier, for the batch time, then the owner's word on the acceptable length." }, "checkedBy": null, @@ -6330,9 +6198,8 @@ "sources": [ { "kind": "specQuestion", - "ref": "spec:platform.native-harness#4", - "file": "design/specs/platform/native-harness.sdp.md", - "line": 27 + "address": "spec:platform.native-harness#question.hostedNativeTarget", + "file": "design/specs/platform/native-harness.sdp.md" } ], "families": [ @@ -6363,6 +6230,672 @@ "decidedBy": "owner", "decidedOn": "2026-10-03", "ruling": "The owner's words in the tooling unit, on the advisor's eight hosted recommendations: \"I agree with all recommendations. We have unused Convex Professional account. Let me know how to login.\" Taken as the lean: one hosted development deployment, dev/hosted in the project libar-platform-hosted of the owner's otherwise unused team on the Professional plan, with one development deploy key, hosted-ci-scoped, limited to the fourteen actions, held as the one secret HOSTED_DEPLOY_KEY by the GitHub environment hosted, which admits the default branch only; run twice a week and on dispatch. The owner made the key, the environment and the secret by hand on 2026-10-03." + }, + { + "id": "OD-156", + "title": "Extension E-40, the shape of a history batch: a batch takes one stream, so a pass costs at least one scheduled mutation ", + "sources": [ + { + "kind": "specQuestion", + "address": "spec:application.history-rebuild#question.historyBatchShape", + "file": "design/specs/application/history-rebuild.sdp.md" + } + ], + "families": [ + "application" + ], + "cites": [ + "E-40", + "E-9", + "D9", + "F13", + "F19" + ], + "provisionalReading": null, + "blocks": "S5", + "advisor": "convex", + "status": "unsorted", + "foldedInto": [], + "class": null, + "reason": null, + "lean": null, + "checkedBy": null, + "fork": null, + "decidedBy": null, + "decidedOn": null, + "ruling": null + }, + { + "id": "OD-157", + "title": "The verify pass under the write pause: a history view's generation is built while no source changes, so its verify pass ", + "sources": [ + { + "kind": "specQuestion", + "address": "spec:application.history-rebuild#question.verifyPassUnderPause", + "file": "design/specs/application/history-rebuild.sdp.md" + } + ], + "families": [ + "application" + ], + "cites": [ + "D9", + "Sc L2-6" + ], + "provisionalReading": "the option kept here is the pass, as D9's \"Verify coverage, then switch\" and Sc L2-6's \"writes reopen only after verification\" read", + "blocks": "S5", + "advisor": "convex", + "status": "sorted", + "foldedInto": [], + "class": "owner", + "reason": "Reads the doc's words of D9 and Sc L2-6 for one row of the acceptance table, which only the owner rules.", + "lean": null, + "checkedBy": null, + "fork": null, + "decidedBy": null, + "decidedOn": null, + "ruling": null + }, + { + "id": "OD-158", + "title": "Whether a history view's batch chain, at up to 2 MiB of rows per batch scheduled with `runAfter(0)`, is held by the depl", + "sources": [ + { + "kind": "specQuestion", + "address": "spec:application.history-rebuild#question.historyBatchWriteThroughput", + "file": "design/specs/application/history-rebuild.sdp.md" + } + ], + "families": [ + "application", + "facts" + ], + "cites": [ + "E-40", + "F13" + ], + "provisionalReading": null, + "blocks": "S5", + "advisor": "convex", + "status": "unsorted", + "foldedInto": [], + "class": null, + "reason": null, + "lean": null, + "checkedBy": null, + "fork": null, + "decidedBy": null, + "decidedOn": null, + "ruling": null + }, + { + "id": "OD-159", + "title": "Extension E-9, the cost of `sourceVersions`: a history row's entries repeat the row's tenant ID once per source stream, ", + "sources": [ + { + "kind": "specQuestion", + "address": "spec:application.projection-contract#question.historySourceVersionTenant", + "file": "design/specs/application/projection-contract.sdp.md" + } + ], + "families": [ + "application" + ], + "cites": [ + "E-9", + "F13", + "Law 11" + ], + "provisionalReading": null, + "blocks": "S5", + "advisor": "convex", + "status": "unsorted", + "foldedInto": [], + "class": null, + "reason": null, + "lean": null, + "checkedBy": null, + "fork": null, + "decidedBy": null, + "decidedOn": null, + "ruling": null + }, + { + "id": "OD-160", + "title": "The cost of the binding on the live path, one row read and one write of up to `rowBudgetBytes` per command and the measu", + "sources": [ + { + "kind": "specQuestion", + "address": "spec:application.history-projection#question.historyProjectionCost", + "file": "design/specs/application/history-projection.sdp.md" + } + ], + "families": [ + "application" + ], + "cites": [ + "E-9", + "E-47", + "F13" + ], + "provisionalReading": null, + "blocks": "S5", + "advisor": "convex", + "status": "unsorted", + "foldedInto": [], + "class": null, + "reason": null, + "lean": null, + "checkedBy": null, + "fork": null, + "decidedBy": null, + "decidedOn": null, + "ruling": null + }, + { + "id": "OD-161", + "title": "Extension E-8, the generation exemption: a batch reads the gate with the scopes `all` and its tenant, and no entry of ei", + "sources": [ + { + "kind": "specQuestion", + "address": "spec:application.write-pause#question.generationGateExemption", + "file": "design/specs/application/write-pause.sdp.md" + } + ], + "families": [ + "application" + ], + "cites": [ + "E-8", + "D9" + ], + "provisionalReading": null, + "blocks": "S5", + "advisor": "convex", + "status": "unsorted", + "foldedInto": [], + "class": null, + "reason": null, + "lean": null, + "checkedBy": null, + "fork": null, + "decidedBy": null, + "decidedOn": null, + "ruling": null + }, + { + "id": "OD-162", + "title": "Whether the generation registry, the command declaration and the command pipeline pin the aggregate form.", + "sources": [ + { + "kind": "ownerQueue", + "ref": "O1", + "file": "design/STATE.md", + "line": 331 + }, + { + "kind": "specLine", + "ref": "application.generation-registry#outcome", + "file": "design/specs/application/generation-registry.sdp.md", + "line": 29 + }, + { + "kind": "specLine", + "ref": "command.command-declaration#typeReadModelBinding", + "file": "design/specs/command/command-declaration.sdp.md", + "line": 70 + }, + { + "kind": "specLine", + "ref": "command.command-pipeline#readModelWrites", + "file": "design/specs/command/command-pipeline.sdp.md", + "line": 126 + } + ], + "families": [ + "application", + "command" + ], + "cites": [], + "provisionalReading": null, + "blocks": null, + "advisor": "domain", + "status": "decided", + "foldedInto": [], + "class": "owner", + "reason": "Owner ruling O1 of 2026-10-04, the aggregate form's reach.", + "lean": null, + "checkedBy": null, + "fork": null, + "decidedBy": "owner", + "decidedOn": "2026-10-04", + "ruling": "The three Specs pin the per-entity and history forms only, and the aggregate projection states how it extends them when OD-075 triggers, so their floor no longer waits on a scoped Spec. The owner chose, on 2026-10-04, from the main thread's options: \"Drop the aggregate form (Recommended)\": \"The three Specs pin the per-entity and history forms; the aggregate Spec states how it extends them when OD-075 triggers. Their floor stops waiting on a scoped Spec; history pairs stay 'ahead' until 5d.\"" + }, + { + "id": "OD-163", + "title": "What a stated ready means in this corpus.", + "sources": [ + { + "kind": "ownerQueue", + "ref": "O2", + "file": "design/STATE.md", + "line": 331 + } + ], + "families": [ + "platform" + ], + "cites": [], + "provisionalReading": null, + "blocks": null, + "advisor": "product", + "status": "decided", + "foldedInto": [], + "class": "owner", + "reason": "Owner ruling O2 of 2026-10-04, the meaning of ready.", + "lean": null, + "checkedBy": null, + "fork": null, + "decidedBy": "owner", + "decidedOn": "2026-10-04", + "ruling": "A stated ready means approved to build, the Protocol's reading: recipe 1 is the build backlog and recipe 2 the drift alarm. No Spec states ready until the owner rules on it; the history projection may after the domain and operator passes. The owner chose, on 2026-10-04, from the main thread's options: \"Approved to build (Recommended)\": \"The Protocol's reading. Recipe 1 becomes the build backlog; the history projection could state ready after the domain and operator passes; roadmap 4e ('ready by family') changes meaning. No Spec states ready in this unit.\"" + }, + { + "id": "OD-164", + "title": "Extension E-9, the check of the history fold: the order independence the Contract requires is checked for the order allo", + "sources": [ + { + "kind": "specQuestion", + "address": "spec:application.history-projection#question.historyFoldPureTest", + "file": "design/specs/application/history-projection.sdp.md" + } + ], + "families": [ + "application" + ], + "cites": [ + "E-9", + "D2", + "D18" + ], + "provisionalReading": "the option taken here is that every history projection of an application carries a pure test that folds a sample of one key's events in every interleaving of its streams and every split into calls, and a library helper that runs that test is added with a second history view", + "blocks": "S5", + "advisor": "domain", + "status": "unsorted", + "foldedInto": [], + "class": null, + "reason": null, + "lean": null, + "checkedBy": null, + "fork": null, + "decidedBy": null, + "decidedOn": null, + "ruling": null + }, + { + "id": "OD-165", + "title": "A history row keeps a deleted subject's history and every rebuild writes it again from the journal; whether a history vi", + "sources": [ + { + "kind": "specQuestion", + "address": "spec:application.history-projection#question.historySubjectErasure", + "file": "design/specs/application/history-projection.sdp.md" + } + ], + "families": [ + "application" + ], + "cites": [ + "E-9", + "D19", + "D9" + ], + "provisionalReading": null, + "blocks": "S5", + "advisor": "domain", + "status": "unsorted", + "foldedInto": [], + "class": null, + "reason": null, + "lean": null, + "checkedBy": null, + "fork": null, + "decidedBy": null, + "decidedOn": null, + "ruling": null + }, + { + "id": "OD-166", + "title": "Before a restore accepts an active history generation, how does its bounded check compare the projected fields and sourc", + "sources": [ + { + "kind": "specQuestion", + "address": "spec:application.restore#question.restoreHistoryFoldVerification", + "file": "design/specs/application/restore.sdp.md" + } + ], + "families": [ + "application" + ], + "cites": [], + "provisionalReading": null, + "blocks": "S5", + "advisor": "operator", + "status": "unsorted", + "foldedInto": [], + "class": null, + "reason": null, + "lean": null, + "checkedBy": null, + "fork": null, + "decidedBy": null, + "decidedOn": null, + "ruling": null + }, + { + "id": "OD-167", + "title": "Must every source's migration that reads history finish before `startGeneration` closes its source scopes, with abort, m", + "sources": [ + { + "kind": "specQuestion", + "address": "spec:application.history-rebuild#question.historyMigrationBeforePause", + "file": "design/specs/application/history-rebuild.sdp.md" + } + ], + "families": [ + "application" + ], + "cites": [], + "provisionalReading": null, + "blocks": "S5", + "advisor": "operator", + "status": "unsorted", + "foldedInto": [], + "class": null, + "reason": null, + "lean": null, + "checkedBy": null, + "fork": null, + "decidedBy": null, + "decidedOn": null, + "ruling": null + }, + { + "id": "OD-168", + "title": "Should the native-backend abort example inject failure in the second source's audit insert, assert unchanged generation,", + "sources": [ + { + "kind": "specQuestion", + "address": "spec:application.history-rebuild#question.abortSwitchAtomicity", + "file": "design/specs/application/history-rebuild.sdp.md" + } + ], + "families": [ + "application" + ], + "cites": [], + "provisionalReading": null, + "blocks": "S5", + "advisor": "operator", + "status": "unsorted", + "foldedInto": [], + "class": null, + "reason": null, + "lean": null, + "checkedBy": null, + "fork": null, + "decidedBy": null, + "decidedOn": null, + "ruling": null + }, + { + "id": "OD-169", + "title": "Should the native-backend example use two tenants with equal local order and stock item IDs, check their folds separatel", + "sources": [ + { + "kind": "specQuestion", + "address": "spec:application.history-rebuild#question.historyTenantPauseIsolation", + "file": "design/specs/application/history-rebuild.sdp.md" + } + ], + "families": [ + "application" + ], + "cites": [], + "provisionalReading": null, + "blocks": "S5", + "advisor": "operator", + "status": "unsorted", + "foldedInto": [], + "class": null, + "reason": null, + "lean": null, + "checkedBy": null, + "fork": null, + "decidedBy": null, + "decidedOn": null, + "ruling": null + }, + { + "id": "OD-170", + "title": "Should its operating instructions distinguish a stopped pass from a verified generation awaiting switch, using `getGener", + "sources": [ + { + "kind": "specQuestion", + "address": "spec:application.write-pause#question.writePauseOperatingInstructions", + "file": "design/specs/application/write-pause.sdp.md" + } + ], + "families": [ + "application" + ], + "cites": [], + "provisionalReading": null, + "blocks": "S5", + "advisor": "operator", + "status": "unsorted", + "foldedInto": [], + "class": null, + "reason": null, + "lean": null, + "checkedBy": null, + "fork": null, + "decidedBy": null, + "decidedOn": null, + "ruling": null + }, + { + "id": "OD-171", + "title": "The terms of the history form: history projection or history view, source version with its meaning on each form, and binding, and whether Read model and Fold widen to rows fed by events", + "sources": [ + { + "kind": "specLine", + "ref": "spec:application.projection-contract#source-versions", + "file": "design/specs/application/projection-contract.sdp.md", + "line": 47 + } + ], + "families": [ + "application" + ], + "cites": [], + "provisionalReading": null, + "blocks": null, + "advisor": "domain", + "status": "sorted", + "foldedInto": [], + "class": "owner", + "reason": "Rule 1, a term: it rules on terms of CONTEXT.md for the history form; raised by the domain pass over pack:history-view and made a row by its fold-in.", + "lean": null, + "checkedBy": null, + "fork": null, + "decidedBy": null, + "decidedOn": null, + "ruling": null + }, + { + "id": "OD-172", + "title": "Which declarations bind a history view", + "sources": [ + { + "kind": "specLine", + "ref": "spec:application.history-projection#declarations-bind", + "file": "design/specs/application/history-projection.sdp.md", + "line": 51 + } + ], + "families": [ + "application" + ], + "cites": [ + "E-9", + "E-7", + "D8" + ], + "provisionalReading": null, + "blocks": "S5", + "advisor": "domain", + "status": "decided", + "foldedInto": [], + "class": "delegated", + "reason": "Rule 3, a rule of the history form that an edit inside this repository takes back; raised by the domain pass over pack:history-view.", + "lean": { + "do": "Add to `spec:application.history-projection` a Contract rule that every command declaration whose `writes` names a source of a history projection binds the projection for each such source, whether or not its events carry a key the view folds. Check the rule in 5d with a pure test over the example's three declarations, and list the order allocation history in the `readModels` of `PlaceOrder`, `CancelOrder` and `ReceiveStock`.", + "restsOn": "A history projection is bound per declaration, once per source stream type [design/specs/application/history-projection.sdp.md:45] [spec:command.command-declaration#design.typeReadModelBinding]. The write pause holds every writer of a source \"whether or not it maintains the view\" [design/specs/application/history-rebuild.sdp.md:49]. The meaning of a row's recorded version, that every event of the stream with the row's key up to that version is folded, is stated under the pause only [design/specs/application/history-rebuild.sdp.md:48]. `applyHistoryProjection` drops every event whose version is not above the row's recorded version [spec:application.history-projection#design.fnApplyHistoryProjection]. A per-entity row repairs itself, because `project` sees the whole DTO and the next live write replaces the row [design/specs/application/projection-contract.sdp.md:45] [design/specs/application/projection-contract.sdp.md:50]. The CancelOrder declaration lists only the order summary [design/specs/application/orders-inventory-example.sdp.md:57], and `ReceiveStock` writes the stock item source [design/specs/application/orders-inventory-example.sdp.md:55]. A search of `design/specs` found no rule that a writer of a source binds the view. **Judgment.** On the specified paths, an event with a row's key that an unbound declaration writes is never folded into the live generation. Once a later bound event on the same stream raises the recorded version past it, no specified path folds it into that generation; of those paths, only a new generation does. The rule is kept by the set of all declarations, so only a check over that set keeps it. The wide rule can be checked from the declarations alone, by comparing each `writes` with its bindings. A pure test over the example's three declarations proves the rule for that set only. Each application that declares a history projection needs its own test over its own declarations.", + "against": "The do-nothing option states no rule and leaves each binding to the author of each declaration. In 5d the builder binds `PlaceOrder` and `CancelOrder`, and the L2-6 resume example compares each rebuilt row with the fold of the journal's events [design/specs/application/history-rebuild.write-pause-rebuild-interrupt.sdp.md:37] and with the row live commands wrote [design/specs/application/history-rebuild.write-pause-rebuild-interrupt.sdp.md:38], which would show a binding missing from that sample. **Judgment.** That catches a gap in the example's tested sample and nothing in an adopter's application, and nothing tells the 5d builder which declarations to bind. The narrower rule binds only the declarations that write a source stream with events the view folds, such as `CancelOrder`'s release of an order's stock. It is what must be true. It spares `ReceiveStock`, whose `StockReceived` has no key [spec:application.history-projection#design.fnOrderAllocationHistoryKeyOf], up to two indexed registry reads per command [spec:application.generation-registry#design.limitRegistryReadsPerCommand]. It also keeps `ReceiveStock` working wherever the wide rule makes it a technical failure. Those cases are while the view has no generation, before the first `startGeneration` and after the only generation was aborted [spec:application.generation-registry#design.errorNoGenerationToWrite], and after a deploy that changes the projection's version, until a generation of that version is switched [design/specs/application/history-projection.sdp.md:35] [spec:application.history-projection#design.fnApplyHistoryProjection]. **Judgment.** I still drop the narrow rule, with moderate confidence. Whether a declaration's events are folded depends on `keyOf` at run time, not on the declaration, so no test can read it from the declarations. A change to `keyOf` that folds one more event type would then silently require bindings on declarations nobody looks at. Under the pause `ReceiveStock` is refused either way, because it writes a closed source [design/specs/application/history-rebuild.sdp.md:53]. In each case where the wide rule makes it fail, `PlaceOrder` and `CancelOrder` fail as well. The cost is real: receiving stock then depends on a view it does not feed.", + "settles": "The build of 5d. The pure test of the rule passes over the example's three declarations (pure test). The L2-6 resume example, at the native backend tier, compares rebuilt rows with the journal's events and with the live rows [design/specs/application/history-rebuild.write-pause-rebuild-interrupt.sdp.md:37] [design/specs/application/history-rebuild.write-pause-rebuild-interrupt.sdp.md:38]. Neither has run.", + "changes": "For the lean: `design/specs/application/history-projection.sdp.md` gains a Contract bullet after line 45. `design/specs/application/orders-inventory-example.sdp.md` names the binding in the CancelOrder declaration at line 57 and states it for `PlaceOrder` and `ReceiveStock`. In 5d, the code's `readModels` at `example/convex/ordering.ts:48` and `:112` and the declaration of `ReceiveStock` change, and one pure test is added. The `PlaceOrder` counts of `design/specs/application/first-experiment.sdp.md:124` already add the history row and its registry read when the view is bound. `ReceiveStock` adds up to two indexed registry reads per command, and no row, because a `null` key reads none [spec:application.history-projection#design.fnApplyHistoryProjection]. For the narrow rule, the same Spec bullet names folded events instead of `writes`, `ReceiveStock` stays unbound, and nothing beyond the example is tested. For the do-nothing option, only the example's declarations change.", + "ifWrong": "If the wide rule's cost matters, removing `ReceiveStock`'s binding and narrowing the test is one edit. If no rule is stated and a binding is missing, the active generation loses events with no error. The loss shows only when its rows are compared with the journal's events or with a rebuilt generation [design/specs/application/history-rebuild.write-pause-rebuild-interrupt.sdp.md:37]. The specified repair is a new generation under a write pause. The Spec edit itself is easy to take back." + }, + "checkedBy": "gpt-6.1-sol, medium, 2026-10-04: plant caught; seven corrections to the reasons, applied by the advisor before the ruling; neither do changed (job-mut64uxp-2eb47318)", + "fork": null, + "decidedBy": "advisor:domain", + "decidedOn": "2026-10-04", + "ruling": "Add to `spec:application.history-projection` a Contract rule that every command declaration whose `writes` names a source of a history projection binds the projection for each such source, whether or not its events carry a key the view folds. Check the rule in 5d with a pure test over the example's three declarations, and list the order allocation history in the `readModels` of `PlaceOrder`, `CancelOrder` and `ReceiveStock`." + }, + { + "id": "OD-173", + "title": "A history projection reads every event schema version", + "sources": [ + { + "kind": "specLine", + "ref": "spec:application.history-projection#event-schema-versions", + "file": "design/specs/application/history-projection.sdp.md", + "line": 48 + } + ], + "families": [ + "application" + ], + "cites": [ + "E-9", + "E-25", + "D2", + "D5" + ], + "provisionalReading": null, + "blocks": "S5", + "advisor": "domain", + "status": "decided", + "foldedInto": [], + "class": "delegated", + "reason": "Rule 3, a rule of the history form and the shape of a type, which an edit inside this repository takes back; raised by the domain pass over pack:history-view.", + "lean": { + "do": "Add to `spec:application.history-projection` a Contract rule: `project` reads every `eventSchemaVersion` that its sources' journals hold of the event types it folds, so no baseline retires an older shape for it. Add `eventSchemaVersion: number` to `HistoryEvent`.", + "restsOn": "An older `eventSchemaVersion` is upcast in `evolve` \"until a baseline retires it\" [design/specs/context/journal.sdp.md:71], and a migration retires once every stream has its baseline [design/specs/context/journal.sdp.md:33]. `applyHistoryProjection` drops baselines and leaves out events whose key is `null` [design/specs/application/history-projection.sdp.md:52], and the backfill reads each stream with no `fromVersion` [spec:application.history-rebuild#design.fnHistoryPageCall]. A history query returns the events before a baseline [design/specs/context/queries.sdp.md:54] [design/specs/context/queries.sdp.md:81]. `eventType` and `eventSchemaVersion` are an event's historical contract [design/specs/context/event-envelope.sdp.md:39]. `HistoryEvent` carries no `eventSchemaVersion` [spec:application.history-projection#design.typeHistoryEvent] [example/domain/orderAllocationHistory.ts:11], and the fold chooses its case by `eventType` alone [example/domain/orderAllocationHistory.ts:66] and casts the payload [example/domain/orderAllocationHistory.ts:72]. **Judgment.** A history projection folds every event of its types from a stream's first version, so for those types it meets every shape ever stored, and it reads the raw payload, not an upcast one. The pinned input type gives its rules no way to tell those shapes apart.", + "against": "Do nothing yet. Each event type of the example has one schema version: the Orders and Inventory deciders write `eventSchemaVersion` 1 [example/domain/order.ts:50] [example/domain/stockItem.ts:37]. **Judgment.** The rule could wait for the first schema change of a folded type. D18's triggers do not name a schema change, so the waiting rests on the same reading of need and not on a trigger the doc lists. Meanwhile, adding the field moves a bound pair to `ahead` and adds a field no rule reads today. The field supplies a discriminator only; a projection still has to handle each old shape itself. A second option hands the projection the stream type's upcasters. Each context owns its current state, its journal and its stream metadata [D2]. **Judgment.** Sharing a context's upcasters with a view in the parent would reach into what the context owns; D2 says nothing about upcasters. I keep the lean, because the rule is one sentence and the field one line, while the case it guards comes late.", + "settles": "The build of 5d at the compiled tier: the type test of `typeHistoryEvent` is bound again once the code carries the field. The first schema change of a folded event type shows the rule in use. It has not been run.", + "changes": "For the lean: `design/specs/application/history-projection.sdp.md` gains a Contract bullet after line 43, and its `typeHistoryEvent` and `fnOrderAllocationHistoryKeyOf` entries, lines 54 and 57, add `eventSchemaVersion: number`. In the pinned module, `generated/pinned/history-view.ts`, the `HistoryEvent` declaration under `typeHistoryEvent` gains the field when the main thread regenerates it. The pair of `spec:application.history-projection#design.typeHistoryEvent` in `tests/types/pinned-pairs.json`, now `bound`, becomes `ahead` until 5d's `example/domain/orderAllocationHistory.ts` carries the field, and the page is regenerated. The journal's rules are unchanged. For the do-nothing option, nothing changes now, and the rule and the field arrive with the first schema change of a folded type.", + "ifWrong": "If the rule is unneeded, the cost is one sentence and one field, and an edit takes both back. **Judgment.** If it is needed and left out, the strongest case is a future schema change of a folded event type that leaves the stored older payloads unreadable by the projection, so it throws on them or misreads them, with no field to tell the shapes apart. The repair is explicit handling of the old shape in `project`, and a new projection version and so a new generation under a write pause when the projection's output changes [design/specs/application/history-projection.sdp.md:43]." + }, + "checkedBy": "gpt-6.1-sol, medium, 2026-10-04: plant caught; seven corrections to the reasons, applied by the advisor before the ruling; neither do changed (job-mut64uxp-2eb47318)", + "fork": null, + "decidedBy": "advisor:domain", + "decidedOn": "2026-10-04", + "ruling": "Add to `spec:application.history-projection` a Contract rule: `project` reads every `eventSchemaVersion` that its sources' journals hold of the event types it folds, so no baseline retires an older shape for it. Add `eventSchemaVersion: number` to `HistoryEvent`." + }, + { + "id": "OD-174", + "title": "Extension E-9, the history form's one version: a `HistoryProjection` declares one version, so a deploy that changes it l", + "sources": [ + { + "kind": "specQuestion", + "spec": "spec:application.history-projection", + "question": "Extension E-9, the history form's one version: a `HistoryProjection` declares one version, so a deploy that changes it leaves the active generation, which records the earlier version, unwritable, and every command whose events feed the view fails as a technical failure until a generation of the new version is switched; the option taken here is that one version, because a view rebuilt under the pause is never written by two versions at once, and a list of versions like the `projections` of a `ReadModel` is the form a view takes when its sources must stay writable across such a deploy (E-9, D8, D9, Law 6)", + "file": "design/specs/application/history-projection.sdp.md", + "open": false + } + ], + "families": [ + "application" + ], + "cites": [ + "E-9", + "D8", + "D9", + "Law 6" + ], + "provisionalReading": "the option taken here is that one version, because a view rebuilt under the pause is never written by two versions at once", + "blocks": "S5", + "advisor": "operator", + "status": "decided", + "foldedInto": [], + "class": "owner", + "reason": "Rule 1, it trades one value for another with no evidence which weighs more: unrelated commands and grant changes held during a handoff against technical failures or a list of versions; operator 1 of the pass over pack:history-view, merged with domain 6.", + "lean": { + "do": "Keep one history projection version and require an operator-controlled handoff from an all-scope pause to the generation's source scopes before exposing a changed version or a new binding to commands.", + "restsOn": "The do-nothing option retains the present contract: a mismatched projection version throws, and a bound read model with no generation also throws, both as technical failures; start then closes the sources, and abort removes those source closures without replacing the active generation. [spec:application.history-projection#design.fnApplyHistoryProjection] [spec:application.generation-registry#design.errorNoGenerationToWrite] [spec:application.history-rebuild#design.historyStart] [spec:application.rebuild#design.fnAbortGeneration] The existing operator entries can close and reopen all, while a batch encountering that closure parks without scheduling another batch. [spec:application.write-pause#design.fnCloseGate] [spec:application.write-pause#design.fnResumeGate] [spec:application.rebuild#design.step3]\n\n**Judgment.** Use those entries in this sequence, with maintenance control coordinated until the handoff and its possible abort are finished. Before the change, establish a compatible code and schema path back to the active projection, or back to commands without the new installation's bindings. Finish source migrations and the tenant fill first; finish or deliberately interrupt unrelated generation passes, retaining their IDs for later resume. Keep capacity for all beside every source closure, including capacity to close all again before an abort; do not reopen another maintenance procedure's closure. The gate's existing limit and the all closure mean a target that cannot fit with all closed must be refused before this procedure starts, not admitted by temporarily reopening writers. [spec:application.write-pause#design.limitClosedScopes] [spec:application.write-pause#design.fnCloseScope]\n\n**Judgment.** For a version-changing deploy or a first binding, close all with the reason and stated operator, confirm it through getGate, and only then deploy. Call startGeneration under the new code while all stays closed, and inspect both the new generation and the complete source closures naming it. If deployment or start fails, keep all closed until compatible code is restored or start succeeds. Once every required source scope is closed, resume only this procedure's all entry, then resumeGeneration for a pass still in backfill or verify; if it has already reached verified, proceed to switch instead. Reopening all alone is not resuming a parked chain. After verification, switchGeneration makes the generation active and removes its source closures. Explicitly resume unrelated passes this procedure interrupted or parked. [spec:application.write-pause#design.fnGetGate] [spec:application.rebuild#design.fnGetGenerations] [spec:application.rebuild#design.fnResumeGeneration] [spec:application.rebuild#design.fnSwitchGeneration]\n\n**Judgment.** Before aborting either a rebuild or an installation, close all again and confirm it, then abortGeneration. For a rebuild, leave all closed until the deployed projection matches the unchanged active generation; when that is already true, no replacement deploy is needed. Otherwise deploy the prepared compatible code, or start a fresh generation under the desired code and repeat the handoff. For an abandoned installation, remove its new command bindings and withhold its parent queries through a compatible deploy before reopening all; retain its table, rebuild target and operator entries until its aborted rows can be purged. Starting a replacement installation is the other exit. A compatible deploy is not automatically an old commit: newer stream data can make old code invalid. [spec:application.rebuild#design.fnAbortGeneration] [spec:application.rebuild#design.purgeSteps] [spec:context.journal#design.codeOlderThanData]\n\n**Judgment.** The caller contract for this sequence is ordinary command processing before the first close; transient writePaused for every new command reaching step 7 while all is closed; then writePaused only for commands covered by remaining closures during backfill and verification; then ordinary processing after switch or after compatible abort recovery. Earlier authentication, authorization, receipt and admission decisions keep their existing answers, so an authorized duplicate may still return its receipt during the pause. [design/specs/command/command-pipeline.sdp.md:92] [design/specs/command/command-pipeline.sdp.md:93] [design/specs/command/command-pipeline.sdp.md:95] [design/specs/command/command-pipeline.sdp.md:96] Skipping the sequence still exposes the existing technical failures; do not relabel every missing-generation or version defect as transient. A first installation has no active read model until switch, and the gate does not promise successful parent queries during that interval. [spec:application.generation-registry#design.fnActiveGeneration]", + "against": "**Judgment.** The do-nothing option is fewer operator steps but accepts an unbounded technical-failure window; the strongest alternative retains several history projection versions, so the active generation remains writable before start and after abort without closing unrelated commands. That alternative follows the per-entity ReadModel shape, but a first binding still needs a staged installation or a pause, and retaining versions does not remove the source pause during the history rebuild itself. [spec:application.projection-contract#design.typeReadModel] [spec:application.generation-registry#design.errorNoGenerationToWrite] [design/specs/application/history-rebuild.sdp.md:52] **Judgment.** The proposed handoff instead spends operator coordination and temporary refusal of unrelated commands to keep the single-version form; the evidence does not decide whether that cost is acceptable.", + "settles": "**Judgment.** Class this merged decision owner: the owner must choose whether the supported release procedure may hold unrelated commands during deployment and abort recovery, whether single-version history is sufficient, and whether omission of the procedure remains a technical failure rather than a supported transient installation state. After that ruling, native-backend tests should exercise version change, first binding, failed start, a parked first batch, same-version abort, version-changing abort and abandoned installation, checking callers at each handoff and recovery; test the gate audit failing before any reopen as well. These are proposed proofs, not results: no compiled, pure test, convex-test or native-backend test was run for this lean, and its new claims have not had the other-family check.", + "changes": "**Judgment.** For unit 5d, keep the signature in design/specs/application/history-projection.sdp.md and expand its one-version question into the ruled release contract; add the sequence to design/specs/application/history-rebuild.sdp.md and extend its interrupt and abort example Specs with version and installation cases. Clarify in design/specs/application/write-pause.sdp.md and design/specs/application/rebuild.sdp.md that the operator entries needed for this handoff may change their control rows under all while batches still park, and that an abort's source reopen is not a promise that commands can yet write; retain the missing-generation contract in design/specs/application/generation-registry.sdp.md. The alternative version list changes history-projection.sdp.md, the history start and writer selection, the command bindings' comparisons and the same examples in 5d; staged installation would need an additional owner-approved contract before its build.", + "ifWrong": "**Judgment.** If the coordinated all-scope pauses are unacceptable, adopt the version-list alternative and an installation contract; the proposed sequence changes no persisted shape and can be replaced by an edit. An operator who bypasses or breaks it can still expose technical failures, while a missing compatible recovery deploy can prolong the pause, so no acceptable duration or automatic reopen is promised by this lean." + }, + "checkedBy": "Opus 5.5, 2026-10-04: plant caught; every operator entry of the sequence exists with the arguments and effects the lean uses", + "fork": null, + "decidedBy": "owner", + "decidedOn": "2026-10-04", + "ruling": "Keep one history projection version and require an operator-controlled handoff from an all-scope pause to the generation's source scopes before exposing a changed version or a new binding to commands. The owner chose, on 2026-10-04, from the main thread's options: \"One version + handoff (Recommended)\": \"The advisor's lean. Keep one version. The operator closes the `all` scope before the deploy, starts the generation, then reopens `all` once the source scopes are closed; an abort runs under `all` again. Callers see `writePaused`, a transient refusal, instead of technical failures. The cost: unrelated commands, and grant changes, are refused during the handoff.\"" + }, + { + "id": "OD-175", + "title": "A saved cursor needs stable sources", + "sources": [ + { + "kind": "specLine", + "ref": "spec:application.history-rebuild#historyCompatibility", + "file": "design/specs/application/history-rebuild.sdp.md", + "line": 74 + } + ], + "families": [ + "application" + ], + "cites": [ + "E-8", + "E-9", + "D9", + "D19" + ], + "provisionalReading": null, + "blocks": "S5", + "advisor": "operator", + "status": "decided", + "foldedInto": [], + "class": "delegated", + "reason": "Rule 3, a mechanism and the order of steps inside what D9 and D19 rule, which an edit inside this repository takes back; operator 2 of the pass over pack:history-view.", + "lean": { + "do": "Compare each history generation's declared source order with the gate entries naming it, and its declared projection version with the generation row, before backfill, verify, their resume and switch; retain the pause on mismatch and block restore acceptance until compatible code is restored or the generation is deliberately aborted.", + "restsOn": "**Judgment.** The do-nothing option remains a release rule alone, preserving source order and query contracts while saved cursors refer to them. The cursor stores sourceIndex into the current target's sources, the generation row stores projectionVersion, and the resumeChain helper preserves the cursor. [spec:application.history-rebuild#design.historyCursor] [spec:application.history-rebuild#design.historySteps] [spec:application.generation-registry#design.tableGenerations] [spec:application.rebuild#design.fnResumeChain]\n\nhistoryStart closes the sources in order, maintenanceGates.closed retains closing order, and resumeGate cannot reopen a source scope; restored generation closures also stay closed until switch or abort. [spec:application.history-rebuild#design.historyStart] [spec:application.write-pause#design.tableMaintenanceGates] [spec:application.write-pause#design.fnResumeGate] [design/specs/application/write-pause.sdp.md:56] **Judgment.** Those entries provide the ordered identities for the period in which a history generation can backfill, verify or switch, so use them without adding a field to the generation row.\n\n**Judgment.** For a generation with pauseRequired, require a history target with nonempty, distinct sources, derive their source scope keys in order, and compare that complete array with closed entries naming this generation, filtered without sorting; also compare the declared projection version with the generation row. A missing gate, a missing target, or a difference in order, length, identity or version is incompatible. Read the stored gate document for this comparison, including during restore, rather than treating gateAllows as an inventory of its entries: its restore-door branch reads no document. [spec:application.write-pause#design.fnGateAllows] [spec:application.write-pause#design.indexGatesByKeyUse]\n\n**Judgment.** After ordinary state and fence checks, backfill and verify must check compatibility before interpreting any cursor or reading any source, including an empty tenant list. A mismatched batch keeps the cursor, counts, read-model rows, generation state and gate unchanged, records a bounded lastError naming the generation and incompatibility with updatedAt, and schedules nothing. The resumeChain helper checks backfill and verify before changing a fence or scheduling; switchGeneration checks before changing state or reopening scopes, and both refuse with a plain error on mismatch. Abort, purge and purge resume remain available without this check, since switch and abort remove the closures and purge reads the read model's table. [spec:application.rebuild#design.fnAbortGeneration] [spec:application.rebuild#design.fnSwitchGeneration] [spec:application.rebuild#design.purgeSteps]\n\nfinishRestoreRun calls resumeChain for each inventoried generation still building or verifying at its inventoried fence, and acceptance and rescheduling commit or fail as one mutation. [spec:application.restore#design.fnFinishRestoreRun] [spec:application.restore#design.transactionBoundary] **Judgment.** Therefore one compatibility refusal must refuse the whole finish, leaving the run unaccepted and committing none of its resumptions or resolutions; do not catch it and accept the remaining generations. Require compatible backup code and configuration, as the restore workflow already states, and retry finish after correcting a mismatch. [design/specs/application/restore.sdp.md:64]\n\n**Judgment.** Before opening the restore door for finish, establish and confirm an all closure using lean 1's capacity and recovery requirements, so a refused finish leaves callers paused. If compatible code cannot be restored, keep all closed, explicitly abort the incompatible generation after the checks have passed and the restore door has opened, complete lean 1's compatible abort recovery, and retry finish; its existing aborted resolution then accounts for the generation, rather than silently omitting it. Retain all until acceptance and safe command bindings and projection versions are established, then reopen it and explicitly resume any compatible chains parked under all. The entry already refuses acceptance while the restore door is closed and records aborted generations. [spec:application.restore#design.fnFinishRestoreRun] A verified generation has no backfill or verify chain for finish to resume; the proposed switch check still guards it. [spec:application.rebuild#design.verifySteps]\n\n**Judgment.** Preserve the meaning of saved list and history cursors for the lifetime of the projection version; a changed query contract or fold requires a new version, and equivalent code moves must preserve pending function handles and cursor meaning. Neither ordered scope keys nor a saved identity list detects changed query or fold semantics under unchanged identities and version. Outside restore, recover by compatible code and resume, or lean 1's protected abort and a fresh generation from a null cursor; do not translate sourceIndex or reuse incompatible partial rows. This release obligation follows D19's preserve, migrate or drain rule. [D19]", + "against": "**Judgment.** The strongest alternative is the saved ordered list on the generation row: while the gate follows its contract it detects the same source changes, but it also retains the original identities after switch or abort and provides an independent comparison if the gate's entries are lost, reordered or changed in a way that happens to match the current target. I prefer the gate for 5d because the guarded operations precede removal of those entries and a separate list adds persisted compatibility handling, while acknowledging that the gate alone cannot distinguish such an invalid gate change from the original closure order and neither choice detects an unbumped query or fold change.", + "settles": "**Judgment.** Treat the gate comparison as a delegated mechanism after the other-family check; ask the owner to confirm the restore consequence and recovery pause, alongside lean 1, if acceptance must instead remain possible with incompatible generations left parked. Proposed pure tests compare ordered scope arrays amid unrelated closures, and proposed native-backend tests cover source reorder, addition, removal, version change, missing closures, a restored interrupted cursor, and a deploy between verified and switch, including a restore with two resumable generations whose second is incompatible: it must accept neither, commit no fence or scheduling changes, retain all and the source closures, and succeed after compatible code or deliberate abort recovery; no compiled, pure test, convex-test or native-backend test was run for this revision, and its new claims await the other-family check.", + "changes": "**Judgment.** In unit 5d, pin the comparison and mismatch behavior in design/specs/application/history-rebuild.sdp.md and its interrupt example, preserve the relied-on order in design/specs/application/write-pause.sdp.md, guard resumeChain and switchGeneration in design/specs/application/rebuild.sdp.md while preserving abort and purge, state cursor compatibility in design/specs/context/queries.sdp.md and the version rule in design/specs/application/history-projection.sdp.md, and add the atomic refusal and protected retry or abort sequence to design/specs/application/restore.sdp.md and its pending-work example. The saved-list alternative changes those same checks and recovery rules in 5d and additionally changes design/specs/application/generation-registry.sdp.md and its declaration comparisons to save immutable ordered source identities on the generation row; the release-rule-only option needs the history release rules and examples but no stored shape or guard.", + "ifWrong": "**Judgment.** An overly strict comparison can block acceptance of the whole restore or cost an abort and another paused backfill; replacing it with a saved list later requires recovering original identities for existing generations or deliberately aborting them, since blindly copying the current target would hide the mismatch. If the gate is invalid in a way this check cannot see, or query semantics change without a version bump, an incorrect fold can survive the check and require a fresh generation from the journal; the check is reversible by an edit, but that edit cannot repair written rows." + }, + "checkedBy": "Opus 5.5, 2026-10-04: plant caught on the first version; the revision re-checked: 18 of 19 citations hold, write-pause.sdp.md:56 is :57; corrections to the reasons: `fnApplyHistoryProjection` already throws on a version mismatch before it reads a row, which ends a folding batch's chain with no lastError and does not reach the resume helper, `finishRestoreRun`, `switchGeneration` or a batch whose page is empty, so the version half adds those four; the do-nothing option includes that throw; the do stands", + "fork": null, + "decidedBy": "advisor:operator", + "decidedOn": "2026-10-04", + "ruling": "Compare each history generation's declared source order with the gate entries naming it, and its declared projection version with the generation row, before backfill, verify, their resume and switch; retain the pause on mismatch and block restore acceptance until compatible code is restored or the generation is deliberately aborted." } ], "possiblySame": [ diff --git a/design/pages/history-view.md b/design/pages/history-view.md new file mode 100644 index 0000000..4964ba7 --- /dev/null +++ b/design/pages/history-view.md @@ -0,0 +1,2725 @@ +# History view + + + +The Specs a history view rests on, in reading order. A history view is a read model whose row folds the events of several streams, such as the order allocation history of the example domain. The history projection comes first because it pins the form, its fold and that view; the projection contract and the read models follow with the row conventions and the read-model rules every view shares. The rebuild, the history rebuild with its five examples, the write pause and the generation registry say how such a view comes to exist and is rebuilt while the commands that write its sources are refused. The rebuild comes first because it holds the operator entries and the batch chain both paths share; the history rebuild holds the paused path alone and the five examples verify it, so the rebuild's implementation binding claims the online path and the history rebuild shows the paused path designed apart from it. The example domain names the events the view folds, and the command declaration and the command pipeline bind the view to the commands that feed it and write it inside each command. The event envelope, the journal and the queries are where the events come from on the live path and on the backfill, and D9 is the decision that a read model which depends on history across several streams rebuilds under a write pause. + +This page is generated from the Specs of `pack:history-view` by `scripts/pack-page.mjs`, with two inputs: the pinned pairs in [`tests/types/pinned-pairs.json`](../../tests/types/pinned-pairs.json) and the register rows in [`design/decisions/register.json`](../decisions/register.json). Do not edit it by hand. Change a Spec or an input, then run `npm run pages`; `npm run pages -- --check` fails while the page is behind the tree. The Protocol is pinned at `@libar-dev/software-delivery-protocol@file:vendor/libar-dev-software-delivery-protocol-0.0.0-822aea4.tgz`. The members speak the vocabulary of [`spec:platform.vocabulary`](../specs/platform/vocabulary.sdp.md). + +## How to read it + +Start with the members and with what holds a member below its next rung: together they say where the design stands and what keeps it there. Then come the Specs outside the Pack that the members rest on, the open questions with their register rows, the register rows whose sources lie in the members with their status and rulings, and each member in full with its code, its examples and verifiers, and its pinned declarations. Long lists are folded; open them where you need them. + +- The stated rung is the readiness a Spec's author states: idea, scoped, defined or ready. +- A register row blocks the first unit of work that needs its decision, a build slice such as S5 named in `design/STATE.md`. That is apart from a question marked blocking, which holds its Spec at scoped, and apart from the floor. +- The floor is the highest rung whose clauses all hold in the graph. It can sit above the stated rung. +- The next rung is the one above the stated rung, with the first floor clause that fails at it. A rung whose clauses all hold still waits for its author to state it. +- An open question is cited by its address, the Spec's id, `#question.` and the key its marker carries. +- An implementation anchor binds code to a Spec at its site, the file and line it is written at. Code that satisfies a Spec realizes it, and the graph derives implemented from it. Code that references a Spec answers to its design without claiming to realize it: the unit follows the design or realizes part of it, so a change to the design asks the code to follow. A design that builds on existing code says so by its own relation to the Spec that code satisfies. A reference confers nothing. Each unit gives its role and its component, with the component's layer and bounded context, where the anchor declares them. +- A pair is a pinned declaration and the code a type test compares it with: bound when the two are equal, ahead when the design leads the code, not compared when the test compares nothing. The test imports every pinned declaration of the Pack from [`generated/pinned/history-view.ts`](../../generated/pinned/history-view.ts), which `npm run sdp:build` emits and git does not keep. + +## Members + +19 Specs, in the order the Pack reads them. + +| # | Spec | Kind | Stated | Floor | Next rung | Code | Examples and verifiers | Pairs | Open questions | +|---|---|---|---|---|---|---|---|---|---| +| [1](#1-history-projection) | [`spec:application.history-projection`](../specs/application/history-projection.sdp.md) | contract | defined | ready | ready, which its floor holds | referenced by 1 unit | none | 2 bound, 3 ahead, 3 without a pair | 4, none blocking | +| [2](#2-projection-contract) | [`spec:application.projection-contract`](../specs/application/projection-contract.sdp.md) | contract | defined | ready | ready, which its floor holds | referenced by 2 units | none | 11 bound | 5, none blocking | +| [3](#3-read-models) | [`spec:application.read-models`](../specs/application/read-models.sdp.md) | behavior | defined | ready | ready, which its floor holds | referenced by 2 units | 5 of 5 examples with a verifier | 2 bound, 9 without a pair | 6, none blocking | +| [4](#4-rebuild-generations-backfill-and-the-write-pause) | [`spec:application.rebuild`](../specs/application/rebuild.sdp.md) | workflow | defined | ready | ready, which its floor holds | referenced by 1 unit | 1 of 1 example with a verifier | 2 bound, 3 ahead, 22 without a pair | 5, none blocking | +| [5](#5-rebuild-of-a-history-view-under-a-write-pause) | [`spec:application.history-rebuild`](../specs/application/history-rebuild.sdp.md) | workflow | defined | ready | ready, which its floor holds | none | 0 of 5 examples with a verifier | 3 without a pair | 6, none blocking | +| [6](#6-rebuild-a-cross-stream-history-view-under-a-write-pause-interrupt-then-resume) | [`spec:application.history-rebuild.write-pause-rebuild-interrupt`](../specs/application/history-rebuild.write-pause-rebuild-interrupt.sdp.md) | example | defined | ready | ready, which its floor holds | none | native backend tier, no verifier bound | no pinned declaration | none | +| [7](#7-rebuild-a-cross-stream-history-view-under-a-write-pause-interrupt-then-abort) | [`spec:application.history-rebuild.write-pause-rebuild-abort`](../specs/application/history-rebuild.write-pause-rebuild-abort.sdp.md) | example | defined | ready | ready, which its floor holds | none | native backend tier, no verifier bound | no pinned declaration | none | +| [8](#8-deploy-a-changed-history-projection-version-through-the-handoff) | [`spec:application.history-rebuild.write-pause-rebuild-version-change`](../specs/application/history-rebuild.write-pause-rebuild-version-change.sdp.md) | example | defined | ready | ready, which its floor holds | none | native backend tier, no verifier bound | no pinned declaration | none | +| [9](#9-install-a-history-view-through-the-handoff) | [`spec:application.history-rebuild.write-pause-rebuild-installation`](../specs/application/history-rebuild.write-pause-rebuild-installation.sdp.md) | example | defined | ready | ready, which its floor holds | none | native backend tier, no verifier bound | no pinned declaration | none | +| [10](#10-reorder-a-history-views-sources-while-its-generation-is-interrupted) | [`spec:application.history-rebuild.write-pause-rebuild-source-order-mismatch`](../specs/application/history-rebuild.write-pause-rebuild-source-order-mismatch.sdp.md) | example | defined | ready | ready, which its floor holds | none | native backend tier, no verifier bound | no pinned declaration | none | +| [11](#11-write-pause) | [`spec:application.write-pause`](../specs/application/write-pause.sdp.md) | contract | defined | ready | ready, which its floor holds | 4 anchors | 1 of 1 example with a verifier | 6 bound, 2 not compared, 6 without a pair | 3, none blocking | +| [12](#12-generation-registry) | [`spec:application.generation-registry`](../specs/application/generation-registry.sdp.md) | contract | defined | ready | ready, which its floor holds | referenced by 2 units | 2 of 2 examples with a verifier | 8 bound, 3 ahead, 2 without a pair | 6, none blocking | +| [13](#13-orders-and-inventory-the-example-domain) | [`spec:application.orders-inventory-example`](../specs/application/orders-inventory-example.sdp.md) | model | defined | ready | ready, which its floor holds | referenced by 14 units | 6 of 6 examples with a verifier | no pinned declaration | 1, none blocking | +| [14](#14-command-declaration-and-bindings) | [`spec:command.command-declaration`](../specs/command/command-declaration.sdp.md) | contract | defined | ready | ready, which its floor holds | referenced by 2 units | 2 of 2 examples with a verifier | 4 bound, 2 ahead, 5 not compared, 7 without a pair | 3, none blocking | +| [15](#15-the-command-pipeline) | [`spec:command.command-pipeline`](../specs/command/command-pipeline.sdp.md) | workflow | defined | ready | ready, which its floor holds | referenced by 2 units | 3 of 3 examples with a verifier | 3 bound, 1 ahead, 2 not compared, 12 without a pair | 7, none blocking | +| [16](#16-the-event-envelope) | [`spec:context.event-envelope`](../specs/context/event-envelope.sdp.md) | contract | defined | ready | ready, which its floor holds | 1 anchor | none | 7 bound, 3 without a pair | 2, none blocking | +| [17](#17-the-journal) | [`spec:context.journal`](../specs/context/journal.sdp.md) | behavior | defined | ready | ready, which its floor holds | referenced by 1 unit | 0 of 1 example with a verifier | 3 bound, 4 ahead, 11 without a pair | 1, none blocking | +| [18](#18-the-context-queries) | [`spec:context.queries`](../specs/context/queries.sdp.md) | contract | defined | ready | ready, which its floor holds | referenced by 1 unit | none | 3 bound, 2 ahead, 13 without a pair | 3, none blocking | +| [19](#19-rebuild-online-by-default-pause-writes-only-for-history-dependent-views) | [`spec:decisions.d09-rebuild-online-by-default`](../specs/decisions/d09-rebuild-online-by-default.sdp.md) | decision | defined | ready | ready, which its floor holds | none | none | no pinned declaration | 1, none blocking | + +## What holds a member below its next rung + +No member is held below its next rung. + +The floor holds at the next rung for [History projection](#1-history-projection), [Projection contract](#2-projection-contract), [Read models](#3-read-models), [Rebuild: generations, backfill and the write pause](#4-rebuild-generations-backfill-and-the-write-pause), [Rebuild of a history view under a write pause](#5-rebuild-of-a-history-view-under-a-write-pause), [Rebuild a cross-stream history view under a write pause; interrupt, then resume](#6-rebuild-a-cross-stream-history-view-under-a-write-pause-interrupt-then-resume), [Rebuild a cross-stream history view under a write pause; interrupt, then abort](#7-rebuild-a-cross-stream-history-view-under-a-write-pause-interrupt-then-abort), [Deploy a changed history projection version through the handoff](#8-deploy-a-changed-history-projection-version-through-the-handoff), [Install a history view through the handoff](#9-install-a-history-view-through-the-handoff), [Reorder a history view's sources while its generation is interrupted](#10-reorder-a-history-views-sources-while-its-generation-is-interrupted), [Write pause](#11-write-pause), [Generation registry](#12-generation-registry), [Orders and Inventory, the example domain](#13-orders-and-inventory-the-example-domain), [Command declaration and bindings](#14-command-declaration-and-bindings), [The command pipeline](#15-the-command-pipeline), [The event envelope](#16-the-event-envelope), [The journal](#17-the-journal), [The context queries](#18-the-context-queries) and [Rebuild online by default, pause writes only for history-dependent views](#19-rebuild-online-by-default-pause-writes-only-for-history-dependent-views). Each waits for its author to state that rung. + +## Outside the Pack + +### What the members rest on + +The members rest on 56 Specs outside the Pack through refines, dependsOn, constrainedBy or decidedBy. A Spec states ready only when every such target states at least defined, so a target below defined holds each member that rests on it below ready. + +No target outside the Pack holds a member below its next rung. + +
All 56 Specs outside the Pack that the members rest on + +| Spec | Stated | Floor | Rested on by | +|---|---|---|---| +| [`spec:application.first-experiment`](../specs/application/first-experiment.sdp.md) | defined | ready | `spec:application.orders-inventory-example` refines | +| [`spec:command.actor-and-scope`](../specs/command/actor-and-scope.sdp.md) | defined | ready | `spec:application.rebuild` dependsOn, `spec:application.write-pause` dependsOn, `spec:application.generation-registry` dependsOn, `spec:command.command-pipeline` dependsOn, `spec:context.event-envelope` dependsOn, `spec:context.queries` dependsOn | +| [`spec:command.idempotency-and-receipts`](../specs/command/idempotency-and-receipts.sdp.md) | defined | ready | `spec:command.command-pipeline` dependsOn | +| [`spec:command.outcome-boundary`](../specs/command/outcome-boundary.sdp.md) | defined | ready | `spec:command.command-declaration` dependsOn, `spec:command.command-pipeline` dependsOn, `spec:context.queries` dependsOn | +| [`spec:command.receipt-table`](../specs/command/receipt-table.sdp.md) | defined | ready | `spec:command.command-pipeline` dependsOn | +| [`spec:command.tenancy-and-authority`](../specs/command/tenancy-and-authority.sdp.md) | defined | ready | `spec:command.command-pipeline` dependsOn | +| [`spec:constraints.bulk-operations-bounded`](../specs/constraints/bulk-operations-bounded.sdp.md) | defined | ready | `spec:application.rebuild` constrainedBy | +| [`spec:constraints.events-stay-small`](../specs/constraints/events-stay-small.sdp.md) | defined | ready | `spec:context.event-envelope` constrainedBy, `spec:context.journal` constrainedBy | +| [`spec:constraints.no-queue-recovery-for-essential-reads`](../specs/constraints/no-queue-recovery-for-essential-reads.sdp.md) | defined | ready | `spec:application.read-models` constrainedBy | +| [`spec:constraints.zero-core-projection-jobs`](../specs/constraints/zero-core-projection-jobs.sdp.md) | defined | ready | `spec:application.read-models` constrainedBy | +| [`spec:context.batch-shaped-api`](../specs/context/batch-shaped-api.sdp.md) | defined | ready | `spec:application.orders-inventory-example` dependsOn | +| [`spec:context.context-component`](../specs/context/context-component.sdp.md) | defined | ready | `spec:command.command-pipeline` dependsOn, `spec:context.journal` refines, `spec:context.queries` refines | +| [`spec:context.persistence-adapter`](../specs/context/persistence-adapter.sdp.md) | defined | ready | `spec:command.command-declaration` dependsOn, `spec:command.command-pipeline` dependsOn | +| [`spec:context.tables`](../specs/context/tables.sdp.md) | defined | ready | `spec:context.journal` dependsOn | +| [`spec:decisions.d01-one-mutation-per-operation`](../specs/decisions/d01-one-mutation-per-operation.sdp.md) | defined | ready | `spec:application.read-models` decidedBy, `spec:command.command-pipeline` decidedBy | +| [`spec:decisions.d02-context-owns-state-and-journal`](../specs/decisions/d02-context-owns-state-and-journal.sdp.md) | defined | ready | `spec:application.history-projection` decidedBy, `spec:context.event-envelope` decidedBy, `spec:context.journal` decidedBy | +| [`spec:decisions.d03-events-only-source-of-next-state`](../specs/decisions/d03-events-only-source-of-next-state.sdp.md) | defined | ready | `spec:application.orders-inventory-example` decidedBy | +| [`spec:decisions.d04-four-outcomes`](../specs/decisions/d04-four-outcomes.sdp.md) | defined | ready | `spec:application.history-projection` decidedBy, `spec:application.history-rebuild` decidedBy, `spec:application.write-pause` decidedBy, `spec:application.orders-inventory-example` decidedBy, `spec:command.command-pipeline` decidedBy | +| [`spec:decisions.d05-rebuildable-history-with-baselines`](../specs/decisions/d05-rebuildable-history-with-baselines.sdp.md) | defined | ready | `spec:application.history-projection` decidedBy, `spec:context.journal` decidedBy | +| [`spec:decisions.d06-idempotency-client-and-receipts`](../specs/decisions/d06-idempotency-client-and-receipts.sdp.md) | defined | ready | `spec:application.orders-inventory-example` decidedBy, `spec:command.command-pipeline` decidedBy | +| [`spec:decisions.d07-rejections-thrown-not-stored`](../specs/decisions/d07-rejections-thrown-not-stored.sdp.md) | defined | ready | `spec:application.orders-inventory-example` decidedBy, `spec:command.command-pipeline` decidedBy | +| [`spec:decisions.d08-read-models-in-command`](../specs/decisions/d08-read-models-in-command.sdp.md) | defined | ready | `spec:application.history-projection` decidedBy, `spec:application.projection-contract` decidedBy, `spec:application.read-models` decidedBy, `spec:application.rebuild` decidedBy, `spec:command.command-pipeline` decidedBy, `spec:context.queries` decidedBy, `spec:decisions.d09-rebuild-online-by-default` dependsOn | +| [`spec:decisions.d10-contexts-meet-in-parent-use-cases`](../specs/decisions/d10-contexts-meet-in-parent-use-cases.sdp.md) | defined | ready | `spec:application.orders-inventory-example` decidedBy, `spec:command.command-pipeline` decidedBy | +| [`spec:decisions.d11-tenant-scope-and-authority`](../specs/decisions/d11-tenant-scope-and-authority.sdp.md) | defined | ready | `spec:application.read-models` decidedBy, `spec:application.rebuild` decidedBy, `spec:command.command-pipeline` decidedBy | +| [`spec:decisions.d12-one-declaration-per-command`](../specs/decisions/d12-one-declaration-per-command.sdp.md) | defined | ready | `spec:command.command-declaration` decidedBy, `spec:command.command-pipeline` decidedBy | +| [`spec:decisions.d18-everything-else-waits-for-trigger`](../specs/decisions/d18-everything-else-waits-for-trigger.sdp.md) | defined | ready | `spec:application.history-projection` decidedBy | +| [`spec:decisions.d19-operations-travel-with-capability`](../specs/decisions/d19-operations-travel-with-capability.sdp.md) | defined | ready | `spec:application.history-projection` decidedBy, `spec:application.rebuild` decidedBy, `spec:application.history-rebuild` decidedBy | +| [`spec:facts.f01-serializable-mutations-under-occ`](../specs/facts/f01-serializable-mutations-under-occ.sdp.md) | defined | ready | `spec:application.history-projection` constrainedBy, `spec:application.projection-contract` constrainedBy, `spec:application.rebuild` constrainedBy, `spec:application.history-rebuild` constrainedBy, `spec:application.write-pause` constrainedBy, `spec:application.generation-registry` constrainedBy, `spec:command.command-pipeline` constrainedBy, `spec:context.event-envelope` constrainedBy, `spec:context.journal` constrainedBy, `spec:decisions.d09-rebuild-online-by-default` constrainedBy | +| [`spec:facts.f02-component-calls-commit-with-caller`](../specs/facts/f02-component-calls-commit-with-caller.sdp.md) | defined | ready | `spec:command.command-pipeline` constrainedBy, `spec:context.journal` constrainedBy | +| [`spec:facts.f03-nested-run-mutation-partial-rollback`](../specs/facts/f03-nested-run-mutation-partial-rollback.sdp.md) | defined | ready | `spec:command.command-pipeline` constrainedBy | +| [`spec:facts.f04-nested-calls-cost-more-than-helpers`](../specs/facts/f04-nested-calls-cost-more-than-helpers.sdp.md) | defined | ready | `spec:application.read-models` constrainedBy | +| [`spec:facts.f07-queries-reactive-not-durable-delivery`](../specs/facts/f07-queries-reactive-not-durable-delivery.sdp.md) | defined | ready | `spec:application.read-models` constrainedBy, `spec:application.rebuild` constrainedBy | +| [`spec:facts.f08-scheduling-commits-with-mutation`](../specs/facts/f08-scheduling-commits-with-mutation.sdp.md) | defined | ready | `spec:application.rebuild` constrainedBy, `spec:context.journal` constrainedBy | +| [`spec:facts.f09-scheduled-mutation-and-action-retry-semantics`](../specs/facts/f09-scheduled-mutation-and-action-retry-semantics.sdp.md) | defined | ready | `spec:application.rebuild` constrainedBy, `spec:application.write-pause` constrainedBy, `spec:context.journal` constrainedBy | +| [`spec:facts.f11-components-have-no-ctx-auth`](../specs/facts/f11-components-have-no-ctx-auth.sdp.md) | defined | ready | `spec:application.read-models` constrainedBy, `spec:application.rebuild` constrainedBy, `spec:application.write-pause` constrainedBy, `spec:command.command-pipeline` constrainedBy, `spec:context.queries` constrainedBy | +| [`spec:facts.f12-backups-exclude-pending-scheduled-functions`](../specs/facts/f12-backups-exclude-pending-scheduled-functions.sdp.md) | defined | ready | `spec:application.rebuild` constrainedBy, `spec:application.write-pause` constrainedBy, `spec:application.generation-registry` constrainedBy | +| [`spec:facts.f13-transactions-have-limits`](../specs/facts/f13-transactions-have-limits.sdp.md) | defined | ready | `spec:application.history-projection` constrainedBy, `spec:application.projection-contract` constrainedBy, `spec:application.read-models` constrainedBy, `spec:application.rebuild` constrainedBy, `spec:application.history-rebuild` constrainedBy, `spec:application.write-pause` constrainedBy, `spec:application.generation-registry` constrainedBy, `spec:command.command-declaration` constrainedBy, `spec:command.command-pipeline` constrainedBy, `spec:context.event-envelope` constrainedBy, `spec:context.journal` constrainedBy, `spec:context.queries` constrainedBy | +| [`spec:facts.f14-convex-error-survives-nested-and-component-boundary`](../specs/facts/f14-convex-error-survives-nested-and-component-boundary.sdp.md) | defined | ready | `spec:command.command-pipeline` constrainedBy | +| [`spec:facts.f15-parent-query-over-component-query-stays-reactive`](../specs/facts/f15-parent-query-over-component-query-stays-reactive.sdp.md) | defined | ready | `spec:application.read-models` constrainedBy, `spec:application.rebuild` constrainedBy, `spec:context.queries` constrainedBy | +| [`spec:facts.f17-migrations-fits-generation-backfill`](../specs/facts/f17-migrations-fits-generation-backfill.sdp.md) | defined | ready | `spec:application.rebuild` constrainedBy, `spec:application.generation-registry` constrainedBy, `spec:decisions.d09-rebuild-online-by-default` constrainedBy | +| [`spec:facts.f18-commit-timestamps`](../specs/facts/f18-commit-timestamps.sdp.md) | defined | ready | `spec:context.journal` constrainedBy | +| [`spec:facts.f19-nested-calls-share-time-budgets`](../specs/facts/f19-nested-calls-share-time-budgets.sdp.md) | defined | ready | `spec:application.history-rebuild` constrainedBy | +| [`spec:facts.f20-pagination-checks-bytes-after-reading`](../specs/facts/f20-pagination-checks-bytes-after-reading.sdp.md) | defined | ready | `spec:context.queries` constrainedBy | +| [`spec:kernel.outcome-model`](../specs/kernel/outcome-model.sdp.md) | defined | ready | `spec:command.command-declaration` dependsOn, `spec:command.command-pipeline` dependsOn | +| [`spec:kernel.state-document-mapping`](../specs/kernel/state-document-mapping.sdp.md) | defined | ready | `spec:context.journal` decidedBy | +| [`spec:laws.law01-sanctioned-writes-only`](../specs/laws/law01-sanctioned-writes-only.sdp.md) | defined | ready | `spec:application.orders-inventory-example` constrainedBy, `spec:command.command-pipeline` constrainedBy | +| [`spec:laws.law02-state-and-events-commit-together`](../specs/laws/law02-state-and-events-commit-together.sdp.md) | defined | ready | `spec:command.command-pipeline` constrainedBy, `spec:context.journal` constrainedBy | +| [`spec:laws.law03-events-only-source-of-state`](../specs/laws/law03-events-only-source-of-state.sdp.md) | defined | ready | `spec:context.journal` constrainedBy | +| [`spec:laws.law04-server-scoped-idempotency-key`](../specs/laws/law04-server-scoped-idempotency-key.sdp.md) | defined | ready | `spec:command.command-pipeline` constrainedBy | +| [`spec:laws.law05-authorization-before-execution-and-disclosure`](../specs/laws/law05-authorization-before-execution-and-disclosure.sdp.md) | defined | ready | `spec:application.read-models` constrainedBy, `spec:command.command-pipeline` constrainedBy, `spec:context.queries` constrainedBy | +| [`spec:laws.law06-technical-failure-never-a-rejection`](../specs/laws/law06-technical-failure-never-a-rejection.sdp.md) | defined | ready | `spec:application.history-projection` constrainedBy, `spec:application.projection-contract` constrainedBy, `spec:application.read-models` constrainedBy, `spec:application.write-pause` constrainedBy, `spec:application.generation-registry` constrainedBy, `spec:command.command-declaration` constrainedBy, `spec:command.command-pipeline` constrainedBy, `spec:context.event-envelope` constrainedBy, `spec:context.journal` constrainedBy | +| [`spec:laws.law09-no-invariant-on-late-read-model`](../specs/laws/law09-no-invariant-on-late-read-model.sdp.md) | defined | ready | `spec:application.read-models` constrainedBy, `spec:application.orders-inventory-example` constrainedBy, `spec:command.command-pipeline` constrainedBy | +| [`spec:laws.law10-replay-never-runs-commands-or-effects`](../specs/laws/law10-replay-never-runs-commands-or-effects.sdp.md) | defined | ready | `spec:application.history-projection` constrainedBy, `spec:application.projection-contract` constrainedBy, `spec:application.read-models` constrainedBy, `spec:application.rebuild` constrainedBy, `spec:context.journal` constrainedBy | +| [`spec:laws.law11-tenant-scope-named`](../specs/laws/law11-tenant-scope-named.sdp.md) | defined | ready | `spec:application.projection-contract` constrainedBy, `spec:application.read-models` constrainedBy, `spec:application.rebuild` constrainedBy, `spec:application.history-rebuild` constrainedBy, `spec:application.generation-registry` constrainedBy, `spec:command.command-pipeline` constrainedBy, `spec:context.journal` constrainedBy, `spec:context.queries` constrainedBy | +| [`spec:operations.baseline-operations`](../specs/operations/baseline-operations.sdp.md) | defined | ready | `spec:application.write-pause` dependsOn, `spec:command.command-declaration` dependsOn, `spec:command.command-pipeline` dependsOn | +| [`spec:platform.transactional-domain-platform`](../specs/platform/transactional-domain-platform.sdp.md) | defined | ready | `spec:application.read-models` refines, `spec:command.command-declaration` refines, `spec:command.command-pipeline` refines, `spec:decisions.d09-rebuild-online-by-default` refines | + +
+ +### What rests on the members + +18 Specs outside the Pack rest on a member. An example that verifies a member is listed with that member instead. + +3 of them state less than defined: + +- [`spec:advanced.cross-stream-online-rebuild`](../specs/advanced/cross-stream-online-rebuild.sdp.md) states scoped and rests on [Rebuild: generations, backfill and the write pause](#4-rebuild-generations-backfill-and-the-write-pause). +- [`spec:agents.agent-runs`](../specs/agents/agent-runs.sdp.md) states scoped and rests on [The command pipeline](#15-the-command-pipeline). +- [`spec:application.aggregate-projection`](../specs/application/aggregate-projection.sdp.md) states scoped and rests on [Projection contract](#2-projection-contract), [Read models](#3-read-models), [Generation registry](#12-generation-registry), [Command declaration and bindings](#14-command-declaration-and-bindings), [The command pipeline](#15-the-command-pipeline) and [Rebuild online by default, pause writes only for history-dependent views](#19-rebuild-online-by-default-pause-writes-only-for-history-dependent-views). + +
The 18 Specs that rest on a member + +| Spec | Stated | Floor | Rests on | +|---|---|---|---| +| [`spec:advanced.cross-stream-online-rebuild`](../specs/advanced/cross-stream-online-rebuild.sdp.md) | scoped | scoped | dependsOn `spec:application.rebuild` | +| [`spec:agents.agent-runs`](../specs/agents/agent-runs.sdp.md) | scoped | scoped | dependsOn `spec:command.command-pipeline` | +| [`spec:application.aggregate-projection`](../specs/application/aggregate-projection.sdp.md) | scoped | scoped | dependsOn `spec:application.projection-contract`, refines `spec:application.read-models`, dependsOn `spec:application.generation-registry`, dependsOn `spec:command.command-declaration`, dependsOn `spec:command.command-pipeline`, decidedBy `spec:decisions.d09-rebuild-online-by-default` | +| [`spec:application.first-experiment`](../specs/application/first-experiment.sdp.md) | defined | ready | dependsOn `spec:application.history-projection`, dependsOn `spec:application.read-models`, dependsOn `spec:application.rebuild`, dependsOn `spec:application.history-rebuild`, dependsOn `spec:application.generation-registry`, dependsOn `spec:command.command-declaration`, dependsOn `spec:command.command-pipeline`, dependsOn `spec:context.journal`, dependsOn `spec:context.queries` | +| [`spec:application.parent-use-cases`](../specs/application/parent-use-cases.sdp.md) | defined | ready | dependsOn `spec:command.command-declaration`, dependsOn `spec:command.command-pipeline` | +| [`spec:application.restore`](../specs/application/restore.sdp.md) | defined | ready | dependsOn `spec:application.projection-contract`, dependsOn `spec:application.rebuild`, dependsOn `spec:application.history-rebuild`, dependsOn `spec:application.write-pause`, dependsOn `spec:application.generation-registry`, dependsOn `spec:context.event-envelope`, dependsOn `spec:context.journal`, dependsOn `spec:context.queries` | +| [`spec:command.idempotency-and-receipts`](../specs/command/idempotency-and-receipts.sdp.md) | defined | ready | dependsOn `spec:application.write-pause`, refines `spec:command.command-pipeline`, dependsOn `spec:context.queries` | +| [`spec:command.outcome-boundary`](../specs/command/outcome-boundary.sdp.md) | defined | ready | refines `spec:command.command-pipeline` | +| [`spec:command.receipt-table`](../specs/command/receipt-table.sdp.md) | defined | ready | dependsOn `spec:application.write-pause`, dependsOn `spec:context.queries` | +| [`spec:command.tenancy-and-authority`](../specs/command/tenancy-and-authority.sdp.md) | defined | ready | dependsOn `spec:application.write-pause` | +| [`spec:context.context-component`](../specs/context/context-component.sdp.md) | defined | ready | dependsOn `spec:context.event-envelope` | +| [`spec:context.persistence-adapter`](../specs/context/persistence-adapter.sdp.md) | defined | ready | dependsOn `spec:context.event-envelope`, dependsOn `spec:context.journal` | +| [`spec:context.tables`](../specs/context/tables.sdp.md) | defined | ready | dependsOn `spec:context.event-envelope` | +| [`spec:decisions.d18-everything-else-waits-for-trigger`](../specs/decisions/d18-everything-else-waits-for-trigger.sdp.md) | defined | ready | dependsOn `spec:decisions.d09-rebuild-online-by-default` | +| [`spec:obligations.fan-out-and-chains`](../specs/obligations/fan-out-and-chains.sdp.md) | defined | ready | dependsOn `spec:command.command-declaration`, dependsOn `spec:command.command-pipeline`, dependsOn `spec:context.event-envelope` | +| [`spec:obligations.local-reaction-wrapper`](../specs/obligations/local-reaction-wrapper.sdp.md) | defined | ready | dependsOn `spec:application.write-pause` | +| [`spec:obligations.obligation-module`](../specs/obligations/obligation-module.sdp.md) | defined | ready | dependsOn `spec:command.command-pipeline` | +| [`spec:operations.baseline-operations`](../specs/operations/baseline-operations.sdp.md) | defined | ready | dependsOn `spec:context.event-envelope` | + +
+ +## Open questions + +53 open questions on 14 members, none blocking. 47 join a register row by their address; 6 join none. A row gives its OD id, its class, its status, the advisor whose lens it falls under and what it blocks. Each question shows its address, which ends in its key, and its opening, at most its first sentence, and links to its line in the Spec for the rest. + +**[History projection](#1-history-projection)**, `spec:application.history-projection` + +- No register row. [`spec:application.history-projection#question.historyFoldOrder`](../specs/application/history-projection.sdp.md?plain=1#L37), [non-blocking]: Extension E-9, the history fold: D2 orders events only within a stream, and the two paths feed one history row in different orders, the live path command by command and the backfill stream by stream; the option taken … +- **OD-160** class none, status unsorted, advisor convex, blocks the unit S5. [`spec:application.history-projection#question.historyProjectionCost`](../specs/application/history-projection.sdp.md?plain=1#L38), [non-blocking]: The cost of the binding on the live path, one row read and one write of up to `rowBudgetBytes` per command and the measure of the row, is taken by the first experiment's latency cell with the order allocation history … +- **OD-164** class none, status unsorted, advisor domain, blocks the unit S5. [`spec:application.history-projection#question.historyFoldPureTest`](../specs/application/history-projection.sdp.md?plain=1#L39), [non-blocking]: Extension E-9, the check of the history fold: the order independence the Contract requires is checked for the order allocation history alone; the option taken here is that every history projection of an application … +- **OD-165** class none, status unsorted, advisor domain, blocks the unit S5. [`spec:application.history-projection#question.historySubjectErasure`](../specs/application/history-projection.sdp.md?plain=1#L40), [non-blocking]: A history row keeps a deleted subject's history and every rebuild writes it again from the journal; whether a history view falls under the retention and deletion policy of D19, and what its rebuild does after an erasure, … + +**[Projection contract](#2-projection-contract)**, `spec:application.projection-contract` + +- **OD-075** class delegated, status decided, advisor domain, blocks the unit S4. [`spec:application.projection-contract#question.projectionRowContract`](../specs/application/projection-contract.sdp.md?plain=1#L33), [non-blocking]: Extension E-9: the doc says a projection is named, versioned and deterministic and that live update and rebuild share its logic; it does not give the signature or the row shape, for counts and sums it names the marker … +- **OD-014** class delegated, status decided, advisor domain, blocks the unit S4. [`spec:application.projection-contract#question.projectionGenerationVersions`](../specs/application/projection-contract.sdp.md?plain=1#L34), [non-blocking]: Extension E-9, two projection versions: the doc says live update and rebuild share one projection and does not say how a command writes two generations built by two versions; the option taken here is that a read model … +- **OD-020** class delegated, status sorted, advisor domain, blocks no unit. [`spec:application.projection-contract#question.backfillVersionOverwrite`](../specs/application/projection-contract.sdp.md?plain=1#L35), [non-blocking]: The rule that a row with a newer version is never overwritten is read as the backfill path's rule only, so a live write to the active generation always lands, as `step3` has it; the owner has not confirmed that reading … +- No register row. [`spec:application.projection-contract#question.compositeCurrentStateRow`](../specs/application/projection-contract.sdp.md?plain=1#L36), [non-blocking]: Extension E-9, a composite row of current state: D8 lets a small cross-context view be a read model updated in the command, and no form here writes one row from the DTOs of two sources, because `project` builds a whole … +- **OD-159** class none, status unsorted, advisor convex, blocks the unit S5. [`spec:application.projection-contract#question.historySourceVersionTenant`](../specs/application/projection-contract.sdp.md?plain=1#L37), [non-blocking]: Extension E-9, the cost of `sourceVersions`: a history row's entries repeat the row's tenant ID once per source stream, about three quarters of the largest order allocation history row by … + +**[Read models](#3-read-models)**, `spec:application.read-models` + +- **OD-076** class delegated, status sorted, advisor convex, blocks no unit. [`spec:application.read-models#question.parentListGenerationSwitch`](../specs/application/read-models.sdp.md?plain=1#L49), [non-blocking]: Probe 5 showed that the built-in `.paginate` treats `maximumRowsRead` as advice once a page has an end cursor and that a cursor it issued is refused by any other query, so a subscription over the active generation threw … +- **OD-077** class delegated, status sorted, advisor convex, blocks no unit. [`spec:application.read-models#question.readModelListBounds`](../specs/application/read-models.sdp.md?plain=1#L50), [non-blocking]: Extension E-24, as it reaches read models: a list over a parent read-model table calls `paginator` of `convex-helpers` with the options of `boundedPage`, a row cap of twice the item cap, with each cursor of the client's … +- **OD-017** class owner, status decided, advisor domain, blocks no unit. [`spec:application.read-models#question.parentQueryRefusal`](../specs/application/read-models.sdp.md?plain=1#L51), [non-blocking]: Extension E-6, as it reaches queries: a parent query establishes and authorizes its actor through `authorizeQuery`, and its refusal is the command's rejection shape, `RejectionData` with the query's name in the field … +- **OD-078** class owner, status waiting, advisor domain, blocks no unit. [`spec:application.read-models#question.contextQueryCost`](../specs/application/read-models.sdp.md?plain=1#L52), [non-blocking]: Probe 3 pending: the cost of one component call in latency and function-call quota decides whether one entity's detail stays a component query or hot reads move, which is OQ1 (Probe 3, F4, OQ1) +- **OD-079** class delegated, status waiting, advisor domain, blocks no unit. [`spec:application.read-models#question.parentQueryReadBounds`](../specs/application/read-models.sdp.md?plain=1#L53), [non-blocking]: Extension E-45: the doc says a small cross-context view is a parent query over bounded component reads and does not give the bound; the option taken here is at most 8 component reads per parent query and at most 4 … +- No register row. [`spec:application.read-models#question.crossContextReadForm`](../specs/application/read-models.sdp.md?plain=1#L54), [non-blocking]: Extension E-9, as it reaches read models: D8 lets a small cross-context view be a parent query over bounded component reads or a read model updated in the command; the option taken here is the parent query for a view of … + +**[Rebuild: generations, backfill and the write pause](#4-rebuild-generations-backfill-and-the-write-pause)**, `spec:application.rebuild` + +- **OD-057** class delegated, status decided, advisor convex, blocks the unit S4. [`spec:application.rebuild#question.rebuildBatchDriver`](../specs/application/rebuild.sdp.md?plain=1#L59), [non-blocking]: `@convex-dev/migrations` walks only the tables of the component it is defined in, reaches a context's rows only through a parent internal mutation written by hand against a contract its README does not document and the … +- **OD-080** class delegated, status decided, advisor domain, blocks the unit S4. [`spec:application.rebuild#question.rebuildBatchBounds`](../specs/application/rebuild.sdp.md?plain=1#L60), [non-blocking]: Extension E-40: the doc binds resumable batching to `@convex-dev/migrations` and gives no batch size; the component iterates a parent table while the sources of a read model live in a context component. +- **OD-068** class owner, status decided, advisor domain, blocks the unit S4. [`spec:application.rebuild#question.rebuildGenerationPauseContract`](../specs/application/rebuild.sdp.md?plain=1#L61), [non-blocking]: Extension E-8: the registry states, the rollback period, what a rollback does, the gate document, the scope a paused rebuild closes and the reach of a generation are design choices the doc does not make; they are pinned … +- No register row. [`spec:application.rebuild#question.rebuildReadModelForms`](../specs/application/rebuild.sdp.md?plain=1#L62), [non-blocking]: Extension E-8, what this workflow does not offer: the online path is pinned for a per-entity read model with one source stream type and the paused path for a history view, a `HistoryRebuildTarget` with its sources, in … +- No register row. [`spec:application.rebuild#question.tenantListFill`](../specs/application/rebuild.sdp.md?plain=1#L63), [non-blocking]: Extension E-49, a deployment whose grants predate the tenant list: the tenant list is written by `insertGrant`, so on such a deployment it holds no row while `grants` holds many, and a pass over an empty list visits no … + +**[Rebuild of a history view under a write pause](#5-rebuild-of-a-history-view-under-a-write-pause)**, `spec:application.history-rebuild` + +- **OD-156** class none, status unsorted, advisor convex, blocks the unit S5. [`spec:application.history-rebuild#question.historyBatchShape`](../specs/application/history-rebuild.sdp.md?plain=1#L41), [non-blocking]: Extension E-40, the shape of a history batch: a batch takes one stream, so a pass costs at least one scheduled mutation per source stream and the pause lasts about twice the stream count times one batch; the alternative … +- **OD-157** class owner, status sorted, advisor convex, blocks the unit S5. [`spec:application.history-rebuild#question.verifyPassUnderPause`](../specs/application/history-rebuild.sdp.md?plain=1#L42), [non-blocking]: The verify pass under the write pause: a history view's generation is built while no source changes, so its verify pass repeats the backfill's batches and, by `spec:application.history-rebuild#design.historyVerify`, … +- **OD-158** class none, status unsorted, advisor convex, blocks the unit S5. [`spec:application.history-rebuild#question.historyBatchWriteThroughput`](../specs/application/history-rebuild.sdp.md?plain=1#L43), [non-blocking]: Whether a history view's batch chain, at up to 2 MiB of rows per batch scheduled with `runAfter(0)`, is held by the deployment class's write throughput limit, … +- **OD-167** class none, status unsorted, advisor operator, blocks the unit S5. [`spec:application.history-rebuild#question.historyMigrationBeforePause`](../specs/application/history-rebuild.sdp.md?plain=1#L44), [non-blocking]: Must every source's migration that reads history finish before `startGeneration` closes its source scopes, with abort, migration completion and a fresh generation as the recovery if it did not? If enumeration must … +- **OD-168** class none, status unsorted, advisor operator, blocks the unit S5. [`spec:application.history-rebuild#question.abortSwitchAtomicity`](../specs/application/history-rebuild.sdp.md?plain=1#L45), [non-blocking]: Should the native-backend abort example inject failure in the second source's audit insert, assert unchanged generation, progress, gate entries and audit records, then retry after removing the fault and assert one … +- **OD-169** class none, status unsorted, advisor operator, blocks the unit S5. [`spec:application.history-rebuild#question.historyTenantPauseIsolation`](../specs/application/history-rebuild.sdp.md?plain=1#L46), [non-blocking]: Should the native-backend example use two tenants with equal local order and stock item IDs, check their folds separately, and refuse a command that writes a paused source without a history binding in both tenants, while … + +**[Write pause](#11-write-pause)**, `spec:application.write-pause` + +- **OD-084** class delegated, status sorted, advisor operator, blocks the unit S4. [`spec:application.write-pause#question.writePauseSourceScopes`](../specs/application/write-pause.sdp.md?plain=1#L39), [non-blocking]: Extension E-8: the doc names a maintenance gate every writer reads, fencing or draining of background writers, and operator resume and abort; the option taken here is one gate document per deployment holding the closed … +- **OD-161** class none, status unsorted, advisor convex, blocks the unit S5. [`spec:application.write-pause#question.generationGateExemption`](../specs/application/write-pause.sdp.md?plain=1#L40), [non-blocking]: Extension E-8, the generation exemption: a batch reads the gate with the scopes `all` and its tenant, and no entry of either scope ever names a generation, so the exemption `gateAllows` makes for a given generation is … +- **OD-170** class none, status unsorted, advisor operator, blocks the unit S5. [`spec:application.write-pause#question.writePauseOperatingInstructions`](../specs/application/write-pause.sdp.md?plain=1#L41), [non-blocking]: Should its operating instructions distinguish a stopped pass from a verified generation awaiting switch, using `getGenerations` and `getGate`, and measure the closed interval through the successful switch or abort? Keep … + +**[Generation registry](#12-generation-registry)**, `spec:application.generation-registry` + +- **OD-068** class owner, status decided, advisor domain, blocks the unit S4. [`spec:application.generation-registry#question.generationRollbackPolicy`](../specs/application/generation-registry.sdp.md?plain=1#L37), [non-blocking]: Extension E-8: the doc names a generation registry, a marker and a gate without their shapes, and says to keep the old generation for a rollback period without saying whether live commands keep writing it; the option … +- **OD-069** class owner, status decided, advisor domain, blocks the unit S4. [`spec:application.generation-registry#question.generationTenantReach`](../specs/application/generation-registry.sdp.md?plain=1#L38), [non-blocking]: Extension E-8, reach of a generation: the doc says nothing about tenants and rebuild; one generation per read model spans every tenant, the registry has no tenant dimension, one row is active for all tenants and the … +- **OD-070** class delegated, status sorted, advisor operator, blocks no unit. [`spec:application.generation-registry#question.restoreGenerationFence`](../specs/application/generation-registry.sdp.md?plain=1#L39), [non-blocking]: Extension E-42, as it touches the registry: a restore brings back a generation row and its progress row at their checkpoint and drops the scheduled batch that was driving them, so the row's fence at inventory time is the … +- **OD-010** class delegated, status decided, advisor domain, blocks the unit S4. [`spec:application.generation-registry#question.readModelInstallation`](../specs/application/generation-registry.sdp.md?plain=1#L40), [non-blocking]: Extension E-8, the first generation: the doc says how a generation is rebuilt and not how a read model's first generation comes to exist; the option taken here is that a read model is installed through its first rebuild, … +- **OD-071** class delegated, status decided, advisor domain, blocks the unit S4. [`spec:application.generation-registry#question.rebuildProgressRow`](../specs/application/generation-registry.sdp.md?plain=1#L41), [non-blocking]: Extension E-40, the checkpoint: the cursor, the batch size and the counts live on the progress row and not on the generation row, because a batch that wrote its checkpoint on the row every command reads would re-run … +- No register row. [`spec:application.generation-registry#question.tenantListGenerationStart`](../specs/application/generation-registry.sdp.md?plain=1#L42), [non-blocking]: Extension E-49, as it touches the registry: on a deployment whose grants predate the tenant list the list is empty while grants exist, and a generation started then visits no tenant; the option taken here is that … + +**[Orders and Inventory, the example domain](#13-orders-and-inventory-the-example-domain)**, `spec:application.orders-inventory-example` + +- **OD-019** class delegated, status sorted, advisor convex, blocks no unit. [`spec:application.orders-inventory-example#question.orderInventoryCommandChoices`](../specs/application/orders-inventory-example.sdp.md?plain=1#L36), [non-blocking]: Extension E-46: the second lifecycle command, the essential summary, the command that creates stock, the unit price an order line carries, the bound on an order line's stock item ID, the rejection reasons, the stream … + +**[Command declaration and bindings](#14-command-declaration-and-bindings)**, `spec:command.command-declaration` + +- **OD-087** class delegated, status sorted, advisor domain, blocks no unit. [`spec:command.command-declaration#question.commandEntrySignatures`](../specs/command/command-declaration.sdp.md?plain=1#L38), [non-blocking]: Extension E-7: the doc says a small composition helper and shared schema values; the helper signatures, the executor signature and the read-model binding below are the design's; the cost is two registered mutations per … +- **OD-015** class delegated, status sorted, advisor domain, blocks no unit. [`spec:command.command-declaration#question.commandDeclarationLookup`](../specs/command/command-declaration.sdp.md?plain=1#L39), [non-blocking]: Extension E-7: the pipeline receives the declaration and takes the fingerprint's contract version from it, and the type check at a typed caller is the check of Sc L2-4, so no standing map from command type to declaration … +- **OD-088** class delegated, status sorted, advisor domain, blocks no unit. [`spec:command.command-declaration#question.commandDeclarationFields`](../specs/command/command-declaration.sdp.md?plain=1#L40), [non-blocking]: Extension E-38: the doc names five declaration fields; this design adds `contractVersion` for the fingerprint, `retention` and `irreversible` for receipts, `admission`, `bounds`, `rejections`, `audit`, `diagnosticSink` … + +**[The command pipeline](#15-the-command-pipeline)**, `spec:command.command-pipeline` + +- **OD-089** class delegated, status sorted, advisor domain, blocks no unit. [`spec:command.command-pipeline#question.commandOutcomeWireShape`](../specs/command/command-pipeline.sdp.md?plain=1#L68), [non-blocking]: Extension E-30: the doc names the outcome, the operation ID and the affected stream versions as what a command returns, but not a wire shape; the shape here is a union discriminated by `replayed`, whose executed member … +- **OD-090** class delegated, status sorted, advisor domain, blocks no unit. [`spec:command.command-pipeline#question.operationIdentityCause`](../specs/command/command-pipeline.sdp.md?plain=1#L69), [non-blocking]: Extension E-31: the doc says an operation's ID ties together everything one request caused but not how it is minted, and it says a derived command carries causation and an event's `causedBy` may be a triggering event … +- **OD-091** class delegated, status sorted, advisor domain, blocks no unit. [`spec:command.command-pipeline#question.commandAdmissionPolicy`](../specs/command/command-pipeline.sdp.md?plain=1#L70), [non-blocking]: Extension E-32: the doc places rate admission for new intent after the duplicate check and makes a rate or capacity refusal a retryable error, but names no admission mechanism; here admission is an optional per-command … +- **OD-028** class owner, status sorted, advisor domain, blocks no unit. [`spec:command.command-pipeline#question.publicRequestKey`](../specs/command/command-pipeline.sdp.md?plain=1#L71), [non-blocking]: Extension E-33: the doc lists the callers that need receipts and leaves UI commands to the client guarantee; here a public caller may still supply a request key to opt into a receipt, which Probe 1 showed a caller of … +- **OD-092** class delegated, status sorted, advisor convex, blocks no unit. [`spec:command.command-pipeline#question.contextCallBudgetReservation`](../specs/command/command-pipeline.sdp.md?plain=1#L72), [non-blocking]: Probe 4 showed on the pinned backend that the transaction limits are one budget across the parent and its component sub-transactions, so the per-command bounds are checked against one budget; `ctx.runMutation` accepts a … +- **OD-093** class delegated, status sorted, advisor domain, blocks no unit. [`spec:command.command-pipeline#question.commandReadModelWrites`](../specs/command/command-pipeline.sdp.md?plain=1#L73), [non-blocking]: Extension E-9: the doc says essential read models are updated inside the command and does not say from what or in what order; here step 9 writes a per-entity read model from the DTO of each matching `streams` entry, … +- **OD-094** class delegated, status sorted, advisor domain, blocks no unit. [`spec:command.command-pipeline#question.commandTextBounds`](../specs/command/command-pipeline.sdp.md?plain=1#L74), [non-blocking]: Extension E-34: step 1 bounds every text field a caller sets beside the input, the tenant ID, the request key, the correlation ID, the actor's fields and the fields of `causedBy`, in UTF-8 bytes as `limitCallText` lists … + +**[The event envelope](#16-the-event-envelope)**, `spec:context.event-envelope` + +- **OD-104** class delegated, status sorted, advisor domain, blocks no unit. [`spec:context.event-envelope#question.eventEnvelopeFields`](../specs/context/event-envelope.sdp.md?plain=1#L31), [non-blocking]: Extension E-26: the doc lists the fifteen fields and their meaning; this Spec fixes the validator shapes, the `CausedBy` union with a migration variant for baseline events, the `OperationRef` the parent passes, and the … +- **OD-105** class delegated, status sorted, advisor domain, blocks no unit. [`spec:context.event-envelope#question.envelopeTextBounds`](../specs/context/event-envelope.sdp.md?plain=1#L32), [non-blocking]: Extension E-12: the doc says events stay small and gives no number for the envelope; this Spec bounds the envelope beside the payload at `limitEnvelopeBytes`, 4,096 bytes, and each text field of it that a caller can set … + +**[The journal](#17-the-journal)**, `spec:context.journal` + +- **OD-106** class delegated, status sorted, advisor domain, blocks the unit S5. [`spec:context.journal#question.baselineMigrationContract`](../specs/context/journal.sdp.md?plain=1#L51), [non-blocking]: Extension E-25: the doc says a migration writes a baseline event holding the migrated state and that rebuild starts from the latest baseline, and it does not say who computes that state, who drives the migration, or what … + +**[The context queries](#18-the-context-queries)**, `spec:context.queries` + +- **OD-108** class none, status folded, advisor convex, blocks no unit. [`spec:context.queries#question.contextPaginationSplit`](../specs/context/queries.sdp.md?plain=1#L41), [non-blocking]: Probe 5 showed on the pinned backend with `convex-helpers` 0.1.124 that the parent query stays reactive, that pages pinned by their end cursors stay contiguous, that a capped page crosses the boundary as `SplitRequired` … +- **OD-109** class delegated, status sorted, advisor convex, blocks no unit. [`spec:context.queries#question.contextQueryContract`](../specs/context/queries.sdp.md?plain=1#L42), [non-blocking]: Extension E-24: the doc says component queries are authorized and return DTOs; this Spec fixes one definer per query, the one `tenantId` argument with no separate scope, the `toDto` mapping with the stream version on … +- **OD-110** class owner, status decided, advisor convex, blocks no unit. [`spec:context.queries#question.queryRefusalEntry`](../specs/context/queries.sdp.md?plain=1#L43), [non-blocking]: A parent query's refusal carries the name of the entry that refused in the field `entry` of the rejection data; the wire shape stays `RejectionData` of `spec:command.outcome-boundary` unchanged, and the owner confirmed … + +**[Rebuild online by default, pause writes only for history-dependent views](#19-rebuild-online-by-default-pause-writes-only-for-history-dependent-views)**, `spec:decisions.d09-rebuild-online-by-default` + +- **OD-057** class delegated, status decided, advisor convex, blocks the unit S4. [`spec:decisions.d09-rebuild-online-by-default#question.rebuildContentionCoverage`](../specs/decisions/d09-rebuild-online-by-default.sdp.md?plain=1#L30), [non-blocking]: Probe 6 ran a backfill batch racing a live command and the migrations component on the native backend, with its results on F17; a hosted deployment and sustained contention are not run (Probe 6, F1, F17, D9) + +## Register rows whose sources lie in the members + +22 register rows have a source in a member's file: every decided row, and every row not decided whose sources cite no open question; a row not decided that cites one is under the open questions above. 20 are decided and 2 are not. Each row gives its status and its sources in the members; a decided row gives the first sentence of its ruling, and a row not decided gives its title. A source other than a question links to the line where it stood when the row was loaded. `python3 design/tools/decisions.py --id ` prints a row in full. + +- **OD-010** class delegated, status decided, advisor domain, blocks the unit S4, from [`spec:application.generation-registry#question.readModelInstallation`](../specs/application/generation-registry.sdp.md?plain=1#L40) on [Generation registry](#12-generation-registry). Decided by advisor:domain on 2026-10-03. Ruling: Install every read model through its first rebuild: generation 1 starts as `building` through `startGeneration`, its backfill covers whatever subjects exist, and the switch makes it active; no transition leads from … +- **OD-014** class delegated, status decided, advisor domain, blocks the unit S4, from [`spec:application.projection-contract#question.projectionGenerationVersions`](../specs/application/projection-contract.sdp.md?plain=1#L34) on [Projection contract](#2-projection-contract). Decided by advisor:domain on 2026-10-03. Ruling: A read model declares its projection versions as a list, the latest first, and `applyProjection` writes each generation with the projection whose version the generation row records, failing as a technical failure when no … +- **OD-017** class owner, status decided, advisor domain, blocks no unit, from [`spec:application.read-models#question.parentQueryRefusal`](../specs/application/read-models.sdp.md?plain=1#L51) on [Read models](#3-read-models). Decided by owner on 2026-10-03. Ruling: The shared RejectionData with the codes unauthenticated and forbidden for a query stays, a use case's own rejection stays bare, and the field is renamed entry, the name of the entry that refused; the code follows in the … +- **OD-057** class delegated, status decided, advisor convex, blocks the unit S4, from [`spec:application.rebuild#question.rebuildBatchDriver`](../specs/application/rebuild.sdp.md?plain=1#L59) on [Rebuild: generations, backfill and the write pause](#4-rebuild-generations-backfill-and-the-write-pause) and [`spec:decisions.d09-rebuild-online-by-default#question.rebuildContentionCoverage`](../specs/decisions/d09-rebuild-online-by-default.sdp.md?plain=1#L30) on [Rebuild online by default, pause writes only for history-dependent views](#19-rebuild-online-by-default-pause-writes-only-for-history-dependent-views). Decided by advisor:convex on 2026-10-03. Ruling: By the advisor's checked lean, which the owner agreed to ("I agree with all recommendations", rulings 1 of round 3, Q8): the lean as the rebuild Specs build it, every batch its own self-scheduled internal mutation, … +- **OD-068** class owner, status decided, advisor domain, blocks the unit S4, from [`spec:application.generation-registry#question.generationRollbackPolicy`](../specs/application/generation-registry.sdp.md?plain=1#L37) on [Generation registry](#12-generation-registry) and [`spec:application.rebuild#question.rebuildGenerationPauseContract`](../specs/application/rebuild.sdp.md?plain=1#L61) on [Rebuild: generations, backfill and the write pause](#4-rebuild-generations-backfill-and-the-write-pause). Decided by owner on 2026-10-02. Ruling: A retired generation is not written. +- **OD-069** class owner, status decided, advisor domain, blocks the unit S4, from [`spec:application.generation-registry#question.generationTenantReach`](../specs/application/generation-registry.sdp.md?plain=1#L38) on [Generation registry](#12-generation-registry). Decided by owner on 2026-10-02. Ruling: One generation per read model spans every tenant. +- **OD-071** class delegated, status decided, advisor domain, blocks the unit S4, from [`spec:application.generation-registry#question.rebuildProgressRow`](../specs/application/generation-registry.sdp.md?plain=1#L41) on [Generation registry](#12-generation-registry). Decided by advisor:domain on 2026-10-03. Ruling: By the advisor's checked lean, which the owner agreed to ("I agree with all recommendations", rulings 1 of round 3, Q8): the lean as built, the generation row patched by a batch only at a pass end and the progress row … +- **OD-075** class delegated, status decided, advisor domain, blocks the unit S4, from [`spec:application.projection-contract#question.projectionRowContract`](../specs/application/projection-contract.sdp.md?plain=1#L33) on [Projection contract](#2-projection-contract). Decided by advisor:domain on 2026-10-03. Ruling: By the advisor's checked lean, which the owner agreed to ("I agree with all recommendations", rulings 1 of round 3, Q8): the lean as built, a ReadModel with its projections, the latest first, and applyProjection writing … +- **OD-080** class delegated, status decided, advisor domain, blocks the unit S4, from [`spec:application.rebuild#question.rebuildBatchBounds`](../specs/application/rebuild.sdp.md?plain=1#L60) on [Rebuild: generations, backfill and the write pause](#4-rebuild-generations-backfill-and-the-write-pause). Decided by advisor:domain on 2026-10-03. Ruling: By the advisor's checked lean, which the owner agreed to ("I agree with all recommendations", rulings 1 of round 3, Q8): the lean as built, the batch size the read model's by batchSizeFor and a field of the progress row, … +- **OD-110** class owner, status decided, advisor convex, blocks no unit, from [`spec:context.queries#question.queryRefusalEntry`](../specs/context/queries.sdp.md?plain=1#L43) on [The context queries](#18-the-context-queries). Decided by owner on 2026-10-03. Ruling: The field is entry on the wire, one validator and one word; the rename of the code follows in the language lane. +- **OD-147** class delegated, status sorted, advisor domain, blocks no unit, from [`spec:command.command-pipeline#limitCallText`](../specs/command/command-pipeline.sdp.md?plain=1#L131) on [The command pipeline](#15-the-command-pipeline). No ruling. Its title: The actor ID bound of 512 bytes and the rejection details bound of 16,384 bytes +- **OD-148** class owner, status decided, advisor operator, blocks the unit S4, from [`rebuild#73`](../specs/application/rebuild.sdp.md?plain=1#L73) on [Rebuild: generations, backfill and the write pause](#4-rebuild-generations-backfill-and-the-write-pause). Decided by owner on 2026-10-02. Ruling: Admin access is the authority of an operation over the whole deployment. +- **OD-149** class delegated, status decided, advisor operator, blocks the unit S4, from [`rebuild#104`](../specs/application/rebuild.sdp.md?plain=1#L104) on [Rebuild: generations, backfill and the write pause](#4-rebuild-generations-backfill-and-the-write-pause). Decided by advisor:operator on 2026-10-03. Ruling: By the lean of the domain lens on this operator row, under the owner's one-advisor word, its facts checked with the corrections in checkedBy, none of which touches its do: the Specs as written; a tenant whose row enters … +- **OD-150** class owner, status decided, advisor operator, blocks the unit S4, from [`write-pause#1`](../specs/application/write-pause.sdp.md?plain=1#L1) on [Write pause](#11-write-pause). Decided by owner on 2026-10-02. Ruling: The write-pause Spec states no acceptable length. +- **OD-151** class owner, status decided, advisor domain, blocks the unit S4, from [`application.orders-inventory-example#e46-release`](../specs/application/orders-inventory-example.sdp.md?plain=1#L27) on [Orders and Inventory, the example domain](#13-orders-and-inventory-the-example-domain). Decided by owner on 2026-10-02. Ruling: Stock state keeps totals only. +- **OD-152** class owner, status decided, advisor domain, blocks the unit S4, from [`application.orders-inventory-example#history-view`](../specs/application/orders-inventory-example.sdp.md?plain=1#L52) on [Orders and Inventory, the example domain](#13-orders-and-inventory-the-example-domain). Decided by owner on 2026-10-02. Ruling: The order allocation history view stays in the first experiment, because Sc L2-6 is a row of the doc and the experiment passes on every Layer 0 to 2 row. +- **OD-162** class owner, status decided, advisor domain, blocks no unit, from [`application.generation-registry#outcome`](../specs/application/generation-registry.sdp.md?plain=1#L29) on [Generation registry](#12-generation-registry), [`command.command-declaration#typeReadModelBinding`](../specs/command/command-declaration.sdp.md?plain=1#L70) on [Command declaration and bindings](#14-command-declaration-and-bindings) and [`command.command-pipeline#readModelWrites`](../specs/command/command-pipeline.sdp.md?plain=1#L126) on [The command pipeline](#15-the-command-pipeline). Decided by owner on 2026-10-04. Ruling: The three Specs pin the per-entity and history forms only, and the aggregate projection states how it extends them when OD-075 triggers, so their floor no longer waits on a scoped Spec. +- **OD-171** class owner, status sorted, advisor domain, blocks no unit, from [`spec:application.projection-contract#source-versions`](../specs/application/projection-contract.sdp.md?plain=1#L47) on [Projection contract](#2-projection-contract). No ruling. Its title: The terms of the history form: history projection or history view, source version with its meaning on each form, and binding, and whether Read model and Fold widen to rows fed by events +- **OD-172** class delegated, status decided, advisor domain, blocks the unit S5, from [`spec:application.history-projection#declarations-bind`](../specs/application/history-projection.sdp.md?plain=1#L51) on [History projection](#1-history-projection). Decided by advisor:domain on 2026-10-04. Ruling: Add to `spec:application.history-projection` a Contract rule that every command declaration whose `writes` names a source of a history projection binds the projection for each such source, whether or not its events carry … +- **OD-173** class delegated, status decided, advisor domain, blocks the unit S5, from [`spec:application.history-projection#event-schema-versions`](../specs/application/history-projection.sdp.md?plain=1#L48) on [History projection](#1-history-projection). Decided by advisor:domain on 2026-10-04. Ruling: Add to `spec:application.history-projection` a Contract rule: `project` reads every `eventSchemaVersion` that its sources' journals hold of the event types it folds, so no baseline retires an older shape for it. +- **OD-174** class owner, status decided, advisor operator, blocks the unit S5, from a question on [History projection](#1-history-projection) that the Spec no longer holds. Decided by owner on 2026-10-04. Ruling: Keep one history projection version and require an operator-controlled handoff from an all-scope pause to the generation's source scopes before exposing a changed version or a new binding to commands. +- **OD-175** class delegated, status decided, advisor operator, blocks the unit S5, from [`spec:application.history-rebuild#historyCompatibility`](../specs/application/history-rebuild.sdp.md?plain=1#L74) on [Rebuild of a history view under a write pause](#5-rebuild-of-a-history-view-under-a-write-pause). Decided by advisor:operator on 2026-10-04. Ruling: Compare each history generation's declared source order with the gate entries naming it, and its declared projection version with the generation row, before backfill, verify, their resume and switch; retain the pause on … + +## Each member + +### 1. History projection + +[`spec:application.history-projection`](../specs/application/history-projection.sdp.md) · contract · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [Read models](#3-read-models) by refines, [Projection contract](#2-projection-contract) by dependsOn, [The event envelope](#16-the-event-envelope) by dependsOn and [Rebuild online by default, pause writes only for history-dependent views](#19-rebuild-online-by-default-pause-writes-only-for-history-dependent-views) by decidedBy. + +**Code.** No implementation anchor binds it. + +**References.** 1 unit references this design. A reference says the code answers to the design without claiming to realize it, and it confers no delivery fact and moves no floor. + +- `impl:application.order-allocation-history` at [`example/domain/orderAllocationHistory.ts` line 8](../../example/domain/orderAllocationHistory.ts#L8): the order allocation history's keyOf and project rules. Its role is projection. It belongs to `component:example.parent`, in the application layer and the orders-inventory context. + +**Examples and verifiers.** None is bound to it. + +**Pinned declarations.** 8 Design entries open with a code span. Each pair comes from [`tests/types/pinned-pairs.json`](../../tests/types/pinned-pairs.json) and is the test named by its address in [`tests/types/pinned.test-d.ts`](../../tests/types/pinned.test-d.ts). + +| Design entry | Pair | Compared | Code | Why | +|---|---|---|---|---| +| [`spec:application.history-projection#design.typeHistoryProjection`](../specs/application/history-projection.sdp.md?plain=1#L61) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.history-projection#design.fnApplyHistoryProjection`](../specs/application/history-projection.sdp.md?plain=1#L62) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.history-projection#design.tableOrderAllocationHistory`](../specs/application/history-projection.sdp.md?plain=1#L63) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.history-projection#design.typeHistoryEvent`](../specs/application/history-projection.sdp.md?plain=1#L64) | ahead | the type | `HistoryEvent` in [`example/domain/orderAllocationHistory.ts`](../../example/domain/orderAllocationHistory.ts) | the design adds `eventSchemaVersion`; the code carries it in 5d | +| [`spec:application.history-projection#design.typeOrderAllocation`](../specs/application/history-projection.sdp.md?plain=1#L65) | bound | the type | `OrderAllocation` in [`example/domain/orderAllocationHistory.ts`](../../example/domain/orderAllocationHistory.ts) | | +| [`spec:application.history-projection#design.typeOrderAllocationHistoryFields`](../specs/application/history-projection.sdp.md?plain=1#L66) | bound | the type | `OrderAllocationHistoryFields` in [`example/domain/orderAllocationHistory.ts`](../../example/domain/orderAllocationHistory.ts) | | +| [`spec:application.history-projection#design.fnOrderAllocationHistoryKeyOf`](../specs/application/history-projection.sdp.md?plain=1#L67) | ahead | the type | `orderAllocationHistoryKeyOf` in [`example/domain/orderAllocationHistory.ts`](../../example/domain/orderAllocationHistory.ts) | the design adds `eventSchemaVersion`; the code carries it in 5d | +| [`spec:application.history-projection#design.fnProjectOrderAllocationHistory`](../specs/application/history-projection.sdp.md?plain=1#L68) | ahead | the type | `projectOrderAllocationHistory` in [`example/domain/orderAllocationHistory.ts`](../../example/domain/orderAllocationHistory.ts) | the design adds `eventSchemaVersion`; the code carries it in 5d | + +
The 8 declarations as the Spec pins them + +[`spec:application.history-projection#design.typeHistoryProjection`](../specs/application/history-projection.sdp.md?plain=1#L61) + +```ts +interface HistoryProjection { + name: string; + version: number; + table: string; + rowBudgetBytes?: number; + keyOf(tenantId: string, event: EventEnvelope): string | null; + project(tenantId: string, prior: Row | null, events: EventEnvelope[]): Row; +} +``` + +[`spec:application.history-projection#design.fnApplyHistoryProjection`](../specs/application/history-projection.sdp.md?plain=1#L62) + +```ts +applyHistoryProjection( + ctx: MutationCtx, + projection: HistoryProjection, + input: { + tenantId: string; + events: EventEnvelope[]; + mode: ApplyMode; + generations: readonly WritableGeneration[]; + }, +): Promise<{ key: string; results: ApplyResult[] }[]> +``` + +[`spec:application.history-projection#design.tableOrderAllocationHistory`](../specs/application/history-projection.sdp.md?plain=1#L63) + +```ts +orderAllocationHistory: defineTable({ + ...rowConventions, + orderId: v.string(), + placedAt: v.optional(v.number()), + cancelledAt: v.optional(v.number()), + allocations: v.array( + v.object({ + stockItemId: v.string(), + quantity: v.number(), + allocatedAt: v.number(), + releasedAt: v.optional(v.number()), + }), + ), +}).index("by_key", ["tenantId", "generation", "key"]) +``` + +[`spec:application.history-projection#design.typeHistoryEvent`](../specs/application/history-projection.sdp.md?plain=1#L64) + +```ts +type HistoryEvent = { + streamId: string; + eventType: string; + eventSchemaVersion: number; + recordedAt: number; + occurredAt?: number; + payload: unknown; +} +``` + +[`spec:application.history-projection#design.typeOrderAllocation`](../specs/application/history-projection.sdp.md?plain=1#L65) + +```ts +type OrderAllocation = { + stockItemId: string; + quantity: number; + allocatedAt: number; + releasedAt?: number; +} +``` + +[`spec:application.history-projection#design.typeOrderAllocationHistoryFields`](../specs/application/history-projection.sdp.md?plain=1#L66) + +```ts +type OrderAllocationHistoryFields = { + orderId: string; + placedAt?: number; + cancelledAt?: number; + allocations: OrderAllocation[]; +} +``` + +[`spec:application.history-projection#design.fnOrderAllocationHistoryKeyOf`](../specs/application/history-projection.sdp.md?plain=1#L67) + +```ts +orderAllocationHistoryKeyOf( + tenantId: string, + event: HistoryEvent, +): string | null +``` + +[`spec:application.history-projection#design.fnProjectOrderAllocationHistory`](../specs/application/history-projection.sdp.md?plain=1#L68) + +```ts +projectOrderAllocationHistory( + tenantId: string, + prior: OrderAllocationHistoryFields | null, + events: readonly HistoryEvent[], +): OrderAllocationHistoryFields +``` + +
+ +**Other Design entries.** 2 entries pin no declaration: [`orderAllocationHistoryFold`](../specs/application/history-projection.sdp.md?plain=1#L69), [`limitHistoryFold`](../specs/application/history-projection.sdp.md?plain=1#L70). + +### 2. Projection contract + +[`spec:application.projection-contract`](../specs/application/projection-contract.sdp.md) · contract · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [Read models](#3-read-models) by refines and [Rebuild online by default, pause writes only for history-dependent views](#19-rebuild-online-by-default-pause-writes-only-for-history-dependent-views) by decidedBy. + +**Code.** No implementation anchor binds it. + +**References.** 2 units reference this design. A reference says the code answers to the design without claiming to realize it, and it confers no delivery fact and moves no floor. + +- `impl:application.generation-registry.tables` at [`src/read-model/tables.ts` line 5](../../src/read-model/tables.ts#L5): the generations, progress, marker and tenant fill tables and rowConventions. Its role is schema. It belongs to `component:platform.read-model`, in the application layer and the platform context. +- `impl:application.projection-contract` at [`src/read-model/projection.ts` line 3](../../src/read-model/projection.ts#L3): ReadModel, Projection and applyProjection. Its role is projection. It belongs to `component:platform.read-model`, in the application layer and the platform context. + +**Examples and verifiers.** None is bound to it. + +**Pinned declarations.** 11 Design entries open with a code span. Each pair comes from [`tests/types/pinned-pairs.json`](../../tests/types/pinned-pairs.json) and is the test named by its address in [`tests/types/pinned.test-d.ts`](../../tests/types/pinned.test-d.ts). + +| Design entry | Pair | Compared | Code | Why | +|---|---|---|---|---| +| [`spec:application.projection-contract#design.typeProjection`](../specs/application/projection-contract.sdp.md?plain=1#L62) | bound | the type | `Projection` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.projection-contract#design.typeReadModel`](../specs/application/projection-contract.sdp.md?plain=1#L63) | bound | the type | `ReadModel` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.projection-contract#design.fnProjectionOf`](../specs/application/projection-contract.sdp.md?plain=1#L64) | bound | the type | `projectionOf` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.projection-contract#design.typeWritableGeneration`](../specs/application/projection-contract.sdp.md?plain=1#L65) | bound | the type | `WritableGeneration` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.projection-contract#design.typeRowWriter`](../specs/application/projection-contract.sdp.md?plain=1#L66) | bound | the type | `RowWriter` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.projection-contract#design.typeApplyMode`](../specs/application/projection-contract.sdp.md?plain=1#L67) | bound | the type | `ApplyMode` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.projection-contract#design.typeApplyResult`](../specs/application/projection-contract.sdp.md?plain=1#L68) | bound | the type | `ApplyResult` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.projection-contract#design.fnApplyProjection`](../specs/application/projection-contract.sdp.md?plain=1#L69) | bound | the type | `applyProjection` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.projection-contract#design.validatorRowConventions`](../specs/application/projection-contract.sdp.md?plain=1#L71) | bound | the type | `rowConventions` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.projection-contract#design.validatorOrderSummaryFields`](../specs/application/projection-contract.sdp.md?plain=1#L72) | bound | the type | `orderSummaryFields` in [`example/convex/orderSummary.ts`](../../example/convex/orderSummary.ts) | | +| [`spec:application.projection-contract#design.tableOrderSummaries`](../specs/application/projection-contract.sdp.md?plain=1#L73) | bound | the table's document and indexes | `default` in [`example/convex/schema.ts`](../../example/convex/schema.ts) | | + +
The 11 declarations as the Spec pins them + +[`spec:application.projection-contract#design.typeProjection`](../specs/application/projection-contract.sdp.md?plain=1#L62) + +```ts +type Projection> = { + version: number; + keyOf(tenantId: string, dto: D): string; + project(tenantId: string, dto: D, versions: StreamVersion[]): Row | null; +} +``` + +[`spec:application.projection-contract#design.typeReadModel`](../specs/application/projection-contract.sdp.md?plain=1#L63) + +```ts +type ReadModel> = { + name: string; + table: string; + rowBudgetBytes: number; + projections: readonly [Projection, ...Projection[]]; +} +``` + +[`spec:application.projection-contract#design.fnProjectionOf`](../specs/application/projection-contract.sdp.md?plain=1#L64) + +```ts +projectionOf>( + readModel: ReadModel, + version: number, +): Projection +``` + +[`spec:application.projection-contract#design.typeWritableGeneration`](../specs/application/projection-contract.sdp.md?plain=1#L65) + +```ts +type WritableGeneration = { + role: "active" | "building" | "verifying"; + generation: number; + projectionVersion: number; +} +``` + +[`spec:application.projection-contract#design.typeRowWriter`](../specs/application/projection-contract.sdp.md?plain=1#L66) + +```ts +type RowWriter = { + db: GenericDatabaseWriter; +} +``` + +[`spec:application.projection-contract#design.typeApplyMode`](../specs/application/projection-contract.sdp.md?plain=1#L67) + +```ts +type ApplyMode = "live-created" | "live-updated" | "backfill" +``` + +[`spec:application.projection-contract#design.typeApplyResult`](../specs/application/projection-contract.sdp.md?plain=1#L68) + +```ts +type ApplyResult = + | "inserted" + | "updated" + | "deleted" + | "skipped-missing" + | "skipped-newer" + | "unchanged" +``` + +[`spec:application.projection-contract#design.fnApplyProjection`](../specs/application/projection-contract.sdp.md?plain=1#L69) + +```ts +applyProjection< + DataModel extends GenericDataModel, + D, + Row extends Record, +>( + ctx: RowWriter, + readModel: ReadModel, + input: { + tenantId: string; + dto: D; + versions: StreamVersion[]; + mode: ApplyMode; + generations: readonly WritableGeneration[]; + }, +): Promise +``` + +[`spec:application.projection-contract#design.validatorRowConventions`](../specs/application/projection-contract.sdp.md?plain=1#L71) + +```ts +const rowConventions = { + tenantId: v.string(), + generation: v.number(), + key: v.string(), + projectionVersion: v.number(), + sourceVersions: v.array(streamVersionValidator), +} +``` + +[`spec:application.projection-contract#design.validatorOrderSummaryFields`](../specs/application/projection-contract.sdp.md?plain=1#L72) + +```ts +const orderSummaryFields = { + orderId: v.string(), + status: v.union(v.literal("placed"), v.literal("cancelled")), + lineCount: v.number(), + total: v.number(), + placedAt: v.number(), +} +``` + +[`spec:application.projection-contract#design.tableOrderSummaries`](../specs/application/projection-contract.sdp.md?plain=1#L73) + +```ts +orderSummaries: defineTable({ ...rowConventions, ...orderSummaryFields }) + .index("by_key", ["tenantId", "generation", "key"]) + .index("by_status", ["tenantId", "generation", "status", "placedAt", "key"]) +``` + +
+ +**Other Design entries.** 10 entries pin no declaration: [`callers`](../specs/application/projection-contract.sdp.md?plain=1#L70), [`indexByKeyUse`](../specs/application/projection-contract.sdp.md?plain=1#L74), [`indexByStatusUse`](../specs/application/projection-contract.sdp.md?plain=1#L75), [`step1`](../specs/application/projection-contract.sdp.md?plain=1#L76), [`step2`](../specs/application/projection-contract.sdp.md?plain=1#L77), [`step3`](../specs/application/projection-contract.sdp.md?plain=1#L78), [`step4`](../specs/application/projection-contract.sdp.md?plain=1#L79), [`limitRowSize`](../specs/application/projection-contract.sdp.md?plain=1#L80), [`limitGenerationsPerWrite`](../specs/application/projection-contract.sdp.md?plain=1#L81), [`determinismCheck`](../specs/application/projection-contract.sdp.md?plain=1#L82). + +### 3. Read models + +[`spec:application.read-models`](../specs/application/read-models.sdp.md) · behavior · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [The context queries](#18-the-context-queries) by dependsOn. + +**Code.** No implementation anchor binds it. + +**References.** 2 units reference this design. A reference says the code answers to the design without claiming to realize it, and it confers no delivery fact and moves no floor. + +- `impl:application.read-models` at [`src/read-model/write.ts` line 3](../../src/read-model/write.ts#L3): writeReadModels, step 9's read-model half. Its role is service. It belongs to `component:platform.read-model`, in the application layer and the platform context. +- `impl:application.read-models.queries` at [`src/read-model/queries.ts` line 3](../../src/read-model/queries.ts#L3): readModelView, pageInGeneration and limitReadModelList. Its role is query. It belongs to `component:platform.read-model`, in the application layer and the platform context. + +**Examples.** + +| Example | Tier | Verifier | Title | +|---|---|---|---| +| [`spec:application.read-models.committed-state-read-through-query`](../specs/application/read-models.committed-state-read-through-query.sdp.md) | native backend | bound | A successful command, then a query | +| [`spec:application.read-models.committed-state-visible`](../specs/application/read-models.committed-state-visible.sdp.md) | native backend | bound | A successful command, then a query or subscription | +| [`spec:application.read-models.list-pages-by-cursor`](../specs/application/read-models.list-pages-by-cursor.sdp.md) | native backend | bound | A client reads a tenant's order list in pages by cursor | +| [`spec:application.read-models.query-refused-before-disclosure`](../specs/application/read-models.query-refused-before-disclosure.sdp.md) | native backend | bound | A caller with no identity reads an order through the parent | +| [`spec:application.read-models.query-refused-without-grant`](../specs/application/read-models.query-refused-without-grant.sdp.md) | native backend | bound | A caller with no grant in the tenant reads an order through the parent | + +**Pinned declarations.** 11 Design entries open with a code span. Each pair comes from [`tests/types/pinned-pairs.json`](../../tests/types/pinned-pairs.json) and is the test named by its address in [`tests/types/pinned.test-d.ts`](../../tests/types/pinned.test-d.ts). + +| Design entry | Pair | Compared | Code | Why | +|---|---|---|---|---| +| [`spec:application.read-models#design.fnEntityDetail`](../specs/application/read-models.sdp.md?plain=1#L98) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.read-models#design.fnListOrSummary`](../specs/application/read-models.sdp.md?plain=1#L99) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.read-models#design.fnPageInGeneration`](../specs/application/read-models.sdp.md?plain=1#L100) | bound | the type | `pageInGeneration` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.read-models#design.fnReadModelView`](../specs/application/read-models.sdp.md?plain=1#L102) | bound | the type | `readModelView` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.read-models#design.validatorReadModelView`](../specs/application/read-models.sdp.md?plain=1#L103) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.read-models#design.fnCrossContextView`](../specs/application/read-models.sdp.md?plain=1#L105) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.read-models#design.typeVersionedResult`](../specs/application/read-models.sdp.md?plain=1#L106) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.read-models#design.indexReadModelByKey`](../specs/application/read-models.sdp.md?plain=1#L108) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.read-models#design.limitReadModelList`](../specs/application/read-models.sdp.md?plain=1#L111) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.read-models#design.limitComponentReadsPerQuery`](../specs/application/read-models.sdp.md?plain=1#L112) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.read-models#design.limitReadModelWritesPerCommand`](../specs/application/read-models.sdp.md?plain=1#L113) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | + +
The 11 declarations as the Spec pins them + +[`spec:application.read-models#design.fnEntityDetail`](../specs/application/read-models.sdp.md?plain=1#L98) + +```ts +getOrder +``` + +[`spec:application.read-models#design.fnListOrSummary`](../specs/application/read-models.sdp.md?plain=1#L99) + +```ts +export const listOrderSummaries = query({ + args: { + tenantId: v.string(), + status: v.union(v.literal("placed"), v.literal("cancelled")), + paginationOpts: paginationOptsValidator, + }, + returns: paginationResultValidator(orderSummaryView), + handler: async (ctx, { tenantId, status, paginationOpts }) => { + await authorizeQuery(ctx, { + name: "listOrderSummaries", + tenantId, + permission: "orders.read", + }); + const generation = await activeGeneration(ctx, "orderSummary"); + if (generation === undefined) + throw new Error("The read model orderSummary has no active generation"); + const result = await paginator(ctx.db, schema) + .query("orderSummaries") + .withIndex("by_status", (q) => + q + .eq("tenantId", tenantId) + .eq("generation", generation) + .eq("status", status), + ) + .paginate( + boundedPage( + pageInGeneration(paginationOpts, [tenantId, generation, status], 5), + limitReadModelList(orderSummary), + ), + ); + return { ...result, page: result.page.map(readModelView) }; + }, +}) +``` + +[`spec:application.read-models#design.fnPageInGeneration`](../specs/application/read-models.sdp.md?plain=1#L100) + +```ts +pageInGeneration( + opts: PaginationOptions, + prefix: readonly Value[], + indexFields: number, +): PaginationOptions +``` + +[`spec:application.read-models#design.fnReadModelView`](../specs/application/read-models.sdp.md?plain=1#L102) + +```ts +readModelView< + Row extends { _id: unknown; _creationTime: number; generation: number }, +>(row: Row): Omit +``` + +[`spec:application.read-models#design.validatorReadModelView`](../specs/application/read-models.sdp.md?plain=1#L103) + +```ts +orderSummaryView = v.object({ + tenantId: v.string(), + key: v.string(), + projectionVersion: v.number(), + sourceVersions: v.array(streamVersionValidator), + ...orderSummaryFields, +}) +``` + +[`spec:application.read-models#design.fnCrossContextView`](../specs/application/read-models.sdp.md?plain=1#L105) + +```ts +export const getOrderWithStock = query({ + args: { tenantId: v.string(), orderId: v.string() }, + returns: v.union( + v.object({ + data: orderWithStockDto, + sourceVersions: v.array(streamVersionValidator), + }), + v.null(), + ), + handler, +}) +``` + +[`spec:application.read-models#design.typeVersionedResult`](../specs/application/read-models.sdp.md?plain=1#L106), as the Spec writes it, which Prettier does not parse + +```ts +[extension] type VersionedResult = { data: T; sourceVersions: StreamVersion[] } +``` + +[`spec:application.read-models#design.indexReadModelByKey`](../specs/application/read-models.sdp.md?plain=1#L108) + +```ts +.index("by_key", ["tenantId", "generation", "key"]) +``` + +[`spec:application.read-models#design.limitReadModelList`](../specs/application/read-models.sdp.md?plain=1#L111) + +```ts +limitReadModelList(readModel) = { + items: limitListPage(readModel.rowBudgetBytes), + bytes: limitListBytes, +} +``` + +[`spec:application.read-models#design.limitComponentReadsPerQuery`](../specs/application/read-models.sdp.md?plain=1#L112) + +```ts +[extension] +``` + +[`spec:application.read-models#design.limitReadModelWritesPerCommand`](../specs/application/read-models.sdp.md?plain=1#L113) + +```ts +[extension] +``` + +
+ +**Other Design entries.** 9 entries pin no declaration: [`transactionBoundary`](../specs/application/read-models.sdp.md?plain=1#L96), [`convexSurface`](../specs/application/read-models.sdp.md?plain=1#L97), [`indexReadModelList`](../specs/application/read-models.sdp.md?plain=1#L101), [`noActiveGeneration`](../specs/application/read-models.sdp.md?plain=1#L104), [`readModelRowShape`](../specs/application/read-models.sdp.md?plain=1#L107), [`indexReadModelByKeyUse`](../specs/application/read-models.sdp.md?plain=1#L109), [`pagination`](../specs/application/read-models.sdp.md?plain=1#L110), [`readModelVisibility`](../specs/application/read-models.sdp.md?plain=1#L114), [`noProjectionJob`](../specs/application/read-models.sdp.md?plain=1#L115). + +### 4. Rebuild: generations, backfill and the write pause + +[`spec:application.rebuild`](../specs/application/rebuild.sdp.md) · workflow · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [Read models](#3-read-models) by refines, [History projection](#1-history-projection) by dependsOn, [Projection contract](#2-projection-contract) by dependsOn, [Write pause](#11-write-pause) by dependsOn, [Generation registry](#12-generation-registry) by dependsOn, [The context queries](#18-the-context-queries) by dependsOn and [Rebuild online by default, pause writes only for history-dependent views](#19-rebuild-online-by-default-pause-writes-only-for-history-dependent-views) by decidedBy. + +**Code.** No implementation anchor binds it. + +**References.** 1 unit references this design. A reference says the code answers to the design without claiming to realize it, and it confers no delivery fact and moves no floor. + +- `impl:application.rebuild` at [`src/read-model/rebuild.ts` line 2](../../src/read-model/rebuild.ts#L2): the online rebuild's operator entries, batch chains, tenant fill and batch bounds. Its role is service. It belongs to `component:platform.read-model`, in the application layer and the platform context. + +**Examples.** + +| Example | Tier | Verifier | Title | +|---|---|---|---| +| [`spec:application.rebuild.online-rebuild-interrupt-resume`](../specs/application/rebuild.online-rebuild-interrupt-resume.sdp.md) | native backend | bound | Rebuild a per-entity read model online while commands run; interrupt and resume | + +**Pinned declarations.** 27 Design entries open with a code span. Each pair comes from [`tests/types/pinned-pairs.json`](../../tests/types/pinned-pairs.json) and is the test named by its address in [`tests/types/pinned.test-d.ts`](../../tests/types/pinned.test-d.ts). + +| Design entry | Pair | Compared | Code | Why | +|---|---|---|---|---| +| [`spec:application.rebuild#design.convexSurface`](../specs/application/rebuild.sdp.md?plain=1#L107) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.typeRebuildTarget`](../specs/application/rebuild.sdp.md?plain=1#L108) | bound | the type | `RebuildTarget` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.rebuild#design.typeRebuildConfig`](../specs/application/rebuild.sdp.md?plain=1#L109) | ahead | the type | `RebuildConfig` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | the history view's build adds history targets | +| [`spec:application.rebuild#design.fnStartGeneration`](../specs/application/rebuild.sdp.md?plain=1#L110) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.fnBatchSizeFor`](../specs/application/rebuild.sdp.md?plain=1#L111) | ahead | the type | `batchSizeFor` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | the history view's build takes a history projection | +| [`spec:application.rebuild#design.fnBackfillBatch`](../specs/application/rebuild.sdp.md?plain=1#L112) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.fnVerifyBatch`](../specs/application/rebuild.sdp.md?plain=1#L113) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.fnPurgeBatch`](../specs/application/rebuild.sdp.md?plain=1#L114) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.fnEnumerateCall`](../specs/application/rebuild.sdp.md?plain=1#L115) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.step1`](../specs/application/rebuild.sdp.md?plain=1#L116) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.verifySteps`](../specs/application/rebuild.sdp.md?plain=1#L122) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.purgeSteps`](../specs/application/rebuild.sdp.md?plain=1#L123) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.fnInterruptGeneration`](../specs/application/rebuild.sdp.md?plain=1#L125) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.fnResumeChain`](../specs/application/rebuild.sdp.md?plain=1#L126) | ahead | the type, its ctx parameter erased | `resumeChain` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | through typeRebuildConfig, its ctx erased | +| [`spec:application.rebuild#design.fnResumeGeneration`](../specs/application/rebuild.sdp.md?plain=1#L127) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.fnSwitchGeneration`](../specs/application/rebuild.sdp.md?plain=1#L128) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.fnRollbackGeneration`](../specs/application/rebuild.sdp.md?plain=1#L129) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.fnAbortGeneration`](../specs/application/rebuild.sdp.md?plain=1#L130) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.fnPurgeGeneration`](../specs/application/rebuild.sdp.md?plain=1#L131) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.fnGetGenerations`](../specs/application/rebuild.sdp.md?plain=1#L132) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.tableTenantFill`](../specs/application/rebuild.sdp.md?plain=1#L133) | bound | the table's document and indexes | `readModelTables` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.rebuild#design.fnFillTenants`](../specs/application/rebuild.sdp.md?plain=1#L134) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.fnFillTenantsBatch`](../specs/application/rebuild.sdp.md?plain=1#L135) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.fillSteps`](../specs/application/rebuild.sdp.md?plain=1#L136) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.limitBatchSize`](../specs/application/rebuild.sdp.md?plain=1#L141) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.limitPurgeBatch`](../specs/application/rebuild.sdp.md?plain=1#L142) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.rebuild#design.limitRollbackPeriod`](../specs/application/rebuild.sdp.md?plain=1#L145) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | + +
The 27 declarations as the Spec pins them + +[`spec:application.rebuild#design.convexSurface`](../specs/application/rebuild.sdp.md?plain=1#L107) + +```ts +startGeneration +``` + +[`spec:application.rebuild#design.typeRebuildTarget`](../specs/application/rebuild.sdp.md?plain=1#L108) + +```ts +type RebuildTarget = { + readModel: ReadModel; + source: SourceRef; + list: FunctionReference<"query", "internal", ListArgs, PaginationResult>; +} +``` + +[`spec:application.rebuild#design.typeRebuildConfig`](../specs/application/rebuild.sdp.md?plain=1#L109) + +```ts +type RebuildConfig = { + targets: readonly (RebuildTarget | HistoryRebuildTarget)[]; + refs: { + backfillBatch: BatchRef; + verifyBatch: BatchRef; + purgeBatch: BatchRef; + fillTenantsBatch: FillBatchRef; + }; +} +``` + +[`spec:application.rebuild#design.fnStartGeneration`](../specs/application/rebuild.sdp.md?plain=1#L110) + +```ts +export const startGeneration = internalMutation({ + args: { + readModel: v.string(), + projectionVersion: v.optional(v.number()), + batchSize: v.optional(v.number()), + operator: v.string(), + }, + returns: v.id("generations"), + handler, +}) +``` + +[`spec:application.rebuild#design.fnBatchSizeFor`](../specs/application/rebuild.sdp.md?plain=1#L111) + +```ts +batchSizeFor( + readModel: ReadModel | HistoryProjection, + requested?: number, +): number +``` + +[`spec:application.rebuild#design.fnBackfillBatch`](../specs/application/rebuild.sdp.md?plain=1#L112) + +```ts +export const backfillBatch = internalMutation({ + args: { generationId: v.id("generations"), fence: v.number() }, + returns: v.null(), + handler, +}) +``` + +[`spec:application.rebuild#design.fnVerifyBatch`](../specs/application/rebuild.sdp.md?plain=1#L113) + +```ts +export const verifyBatch = internalMutation({ + args: { generationId: v.id("generations"), fence: v.number() }, + returns: v.null(), + handler, +}) +``` + +[`spec:application.rebuild#design.fnPurgeBatch`](../specs/application/rebuild.sdp.md?plain=1#L114) + +```ts +export const purgeBatch = internalMutation({ + args: { generationId: v.id("generations"), fence: v.number() }, + returns: v.null(), + handler, +}) +``` + +[`spec:application.rebuild#design.fnEnumerateCall`](../specs/application/rebuild.sdp.md?plain=1#L115) + +```ts +ctx.runQuery(target.list, { + tenantId, + includeDeleted: true, + paginationOpts: { numItems: batchSize, cursor: pageCursor }, +}) +``` + +[`spec:application.rebuild#design.step1`](../specs/application/rebuild.sdp.md?plain=1#L116) + +```ts +backfillBatch +``` + +[`spec:application.rebuild#design.verifySteps`](../specs/application/rebuild.sdp.md?plain=1#L122) + +```ts +verifyBatch +``` + +[`spec:application.rebuild#design.purgeSteps`](../specs/application/rebuild.sdp.md?plain=1#L123) + +```ts +purgeBatch +``` + +[`spec:application.rebuild#design.fnInterruptGeneration`](../specs/application/rebuild.sdp.md?plain=1#L125) + +```ts +export const interruptGeneration = internalMutation({ + args: { generationId: v.id("generations"), operator: v.string() }, + returns: v.null(), + handler, +}) +``` + +[`spec:application.rebuild#design.fnResumeChain`](../specs/application/rebuild.sdp.md?plain=1#L126) + +```ts +resumeChain( + ctx: MutationCtx, + config: RebuildConfig, + generationId: Id<"generations">, + operator: string, +): Promise +``` + +[`spec:application.rebuild#design.fnResumeGeneration`](../specs/application/rebuild.sdp.md?plain=1#L127) + +```ts +export const resumeGeneration = internalMutation({ + args: { generationId: v.id("generations"), operator: v.string() }, + returns: v.null(), + handler, +}) +``` + +[`spec:application.rebuild#design.fnSwitchGeneration`](../specs/application/rebuild.sdp.md?plain=1#L128) + +```ts +export const switchGeneration = internalMutation({ + args: { + generationId: v.id("generations"), + rollbackPeriodMs: v.optional(v.number()), + operator: v.string(), + }, + returns: v.null(), + handler, +}) +``` + +[`spec:application.rebuild#design.fnRollbackGeneration`](../specs/application/rebuild.sdp.md?plain=1#L129) + +```ts +export const rollbackGeneration = internalMutation({ + args: { generationId: v.id("generations"), operator: v.string() }, + returns: v.null(), + handler, +}) +``` + +[`spec:application.rebuild#design.fnAbortGeneration`](../specs/application/rebuild.sdp.md?plain=1#L130) + +```ts +export const abortGeneration = internalMutation({ + args: { + generationId: v.id("generations"), + reason: v.string(), + operator: v.string(), + }, + returns: v.null(), + handler, +}) +``` + +[`spec:application.rebuild#design.fnPurgeGeneration`](../specs/application/rebuild.sdp.md?plain=1#L131) + +```ts +export const purgeGeneration = internalMutation({ + args: { generationId: v.id("generations"), operator: v.string() }, + returns: v.null(), + handler, +}) +``` + +[`spec:application.rebuild#design.fnGetGenerations`](../specs/application/rebuild.sdp.md?plain=1#L132) + +```ts +export const getGenerations = internalQuery({ + args: { readModel: v.string() }, + returns: v.array( + v.object({ generation: generationDoc, progress: progressDoc }), + ), + handler, +}) +``` + +[`spec:application.rebuild#design.tableTenantFill`](../specs/application/rebuild.sdp.md?plain=1#L133) + +```ts +tenantFill: defineTable({ + pass: v.union(v.literal("fill"), v.literal("idle")), + fence: v.number(), + cursor: v.union(v.string(), v.null()), + batchesDone: v.number(), + tenantsRead: v.number(), + tenantsInserted: v.number(), + lastError: v.optional(v.string()), + startedAt: v.number(), + startedBy: v.string(), + changedAt: v.number(), + changedBy: v.string(), + updatedAt: v.number(), +}) +``` + +[`spec:application.rebuild#design.fnFillTenants`](../specs/application/rebuild.sdp.md?plain=1#L134) + +```ts +export const fillTenants = internalMutation({ + args: { operator: v.string() }, + returns: v.null(), + handler, +}) +``` + +[`spec:application.rebuild#design.fnFillTenantsBatch`](../specs/application/rebuild.sdp.md?plain=1#L135) + +```ts +export const fillTenantsBatch = internalMutation({ + args: { fence: v.number() }, + returns: v.null(), + handler, +}) +``` + +[`spec:application.rebuild#design.fillSteps`](../specs/application/rebuild.sdp.md?plain=1#L136) + +```ts +fillTenantsBatch +``` + +[`spec:application.rebuild#design.limitBatchSize`](../specs/application/rebuild.sdp.md?plain=1#L141) + +```ts +rebuildBatchCeiling(readModel) = min( + 100, + floor(4194304 / (2 * rowBudgetBytes)), +) +``` + +[`spec:application.rebuild#design.limitPurgeBatch`](../specs/application/rebuild.sdp.md?plain=1#L142) + +```ts +limitPurgeBatch( + readModel: ReadModel | HistoryProjection, +): number +``` + +[`spec:application.rebuild#design.limitRollbackPeriod`](../specs/application/rebuild.sdp.md?plain=1#L145) + +```ts +[extension] +``` + +
+ +**Other Design entries.** 15 entries pin no declaration: [`transactionBoundary`](../specs/application/rebuild.sdp.md?plain=1#L106), [`step2`](../specs/application/rebuild.sdp.md?plain=1#L117), [`step3`](../specs/application/rebuild.sdp.md?plain=1#L118), [`step4`](../specs/application/rebuild.sdp.md?plain=1#L119), [`step5`](../specs/application/rebuild.sdp.md?plain=1#L120), [`step6`](../specs/application/rebuild.sdp.md?plain=1#L121), [`purgeDriver`](../specs/application/rebuild.sdp.md?plain=1#L124), [`limitTenantFillBatch`](../specs/application/rebuild.sdp.md?plain=1#L137), [`errorUnknownGeneration`](../specs/application/rebuild.sdp.md?plain=1#L138), [`stallSignal`](../specs/application/rebuild.sdp.md?plain=1#L139), [`occRace`](../specs/application/rebuild.sdp.md?plain=1#L140), [`limitOutstandingBatches`](../specs/application/rebuild.sdp.md?plain=1#L143), [`limitBuildingGenerations`](../specs/application/rebuild.sdp.md?plain=1#L144), [`limitGenerationsListed`](../specs/application/rebuild.sdp.md?plain=1#L146), [`rebuildClass`](../specs/application/rebuild.sdp.md?plain=1#L147). + +### 5. Rebuild of a history view under a write pause + +[`spec:application.history-rebuild`](../specs/application/history-rebuild.sdp.md) · workflow · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [Read models](#3-read-models) by refines, [History projection](#1-history-projection) by dependsOn, [Rebuild: generations, backfill and the write pause](#4-rebuild-generations-backfill-and-the-write-pause) by dependsOn, [Write pause](#11-write-pause) by dependsOn, [Generation registry](#12-generation-registry) by dependsOn, [The event envelope](#16-the-event-envelope) by dependsOn, [The context queries](#18-the-context-queries) by dependsOn and [Rebuild online by default, pause writes only for history-dependent views](#19-rebuild-online-by-default-pause-writes-only-for-history-dependent-views) by decidedBy. + +**Code.** No implementation anchor binds it. + +**Examples.** + +| Example | Tier | Verifier | Title | +|---|---|---|---| +| [`spec:application.history-rebuild.write-pause-rebuild-abort`](../specs/application/history-rebuild.write-pause-rebuild-abort.sdp.md) | native backend | none bound | Rebuild a cross-stream history view under a write pause; interrupt, then abort | +| [`spec:application.history-rebuild.write-pause-rebuild-installation`](../specs/application/history-rebuild.write-pause-rebuild-installation.sdp.md) | native backend | none bound | Install a history view through the handoff | +| [`spec:application.history-rebuild.write-pause-rebuild-interrupt`](../specs/application/history-rebuild.write-pause-rebuild-interrupt.sdp.md) | native backend | none bound | Rebuild a cross-stream history view under a write pause; interrupt, then resume | +| [`spec:application.history-rebuild.write-pause-rebuild-source-order-mismatch`](../specs/application/history-rebuild.write-pause-rebuild-source-order-mismatch.sdp.md) | native backend | none bound | Reorder a history view's sources while its generation is interrupted | +| [`spec:application.history-rebuild.write-pause-rebuild-version-change`](../specs/application/history-rebuild.write-pause-rebuild-version-change.sdp.md) | native backend | none bound | Deploy a changed history projection version through the handoff | + +**Pinned declarations.** 3 Design entries open with a code span. Each pair comes from [`tests/types/pinned-pairs.json`](../../tests/types/pinned-pairs.json) and is the test named by its address in [`tests/types/pinned.test-d.ts`](../../tests/types/pinned.test-d.ts). + +| Design entry | Pair | Compared | Code | Why | +|---|---|---|---|---| +| [`spec:application.history-rebuild#design.typeHistorySource`](../specs/application/history-rebuild.sdp.md?plain=1#L65) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.history-rebuild#design.typeHistoryRebuildTarget`](../specs/application/history-rebuild.sdp.md?plain=1#L66) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.history-rebuild#design.fnHistoryPageCall`](../specs/application/history-rebuild.sdp.md?plain=1#L67) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | + +
The 3 declarations as the Spec pins them + +[`spec:application.history-rebuild#design.typeHistorySource`](../specs/application/history-rebuild.sdp.md?plain=1#L65) + +```ts +type HistorySource = { + source: SourceRef; + list: FunctionReference< + "query", + "internal", + ListArgs, + PaginationResult<{ version: StreamVersion }> + >; + history: FunctionReference< + "query", + "internal", + { + tenantId: string; + streamId: string; + fromVersion?: number; + paginationOpts: PaginationOptions; + }, + PaginationResult + >; +} +``` + +[`spec:application.history-rebuild#design.typeHistoryRebuildTarget`](../specs/application/history-rebuild.sdp.md?plain=1#L66) + +```ts +type HistoryRebuildTarget = { + readModel: HistoryProjection; + sources: readonly [HistorySource, ...HistorySource[]]; +} +``` + +[`spec:application.history-rebuild#design.fnHistoryPageCall`](../specs/application/history-rebuild.sdp.md?plain=1#L67) + +```ts +ctx.runQuery(source.history, { + tenantId, + streamId, + paginationOpts: { numItems: batchSize, cursor: eventCursor }, +}) +``` + +
+ +**Other Design entries.** 9 entries pin no declaration: [`historyCursor`](../specs/application/history-rebuild.sdp.md?plain=1#L68), [`historyStart`](../specs/application/history-rebuild.sdp.md?plain=1#L69), [`historySteps`](../specs/application/history-rebuild.sdp.md?plain=1#L70), [`historyVerify`](../specs/application/history-rebuild.sdp.md?plain=1#L71), [`historyHandoff`](../specs/application/history-rebuild.sdp.md?plain=1#L72), [`historyAbortHandoff`](../specs/application/history-rebuild.sdp.md?plain=1#L73), [`historyCompatibility`](../specs/application/history-rebuild.sdp.md?plain=1#L74), [`historyMismatch`](../specs/application/history-rebuild.sdp.md?plain=1#L75), [`limitHistoryBatch`](../specs/application/history-rebuild.sdp.md?plain=1#L76). + +### 6. Rebuild a cross-stream history view under a write pause; interrupt, then resume + +[`spec:application.history-rebuild.write-pause-rebuild-interrupt`](../specs/application/history-rebuild.write-pause-rebuild-interrupt.sdp.md) · example · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [Rebuild of a history view under a write pause](#5-rebuild-of-a-history-view-under-a-write-pause) by refines. + +**Tier and verifier.** Its opening line is `Sc L2-6 · native backend tier · the resume case.`, which names the native backend tier. No verifier is bound to it. + +### 7. Rebuild a cross-stream history view under a write pause; interrupt, then abort + +[`spec:application.history-rebuild.write-pause-rebuild-abort`](../specs/application/history-rebuild.write-pause-rebuild-abort.sdp.md) · example · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [Rebuild of a history view under a write pause](#5-rebuild-of-a-history-view-under-a-write-pause) by refines. + +**Tier and verifier.** Its opening line is `Sc L2-6 · native backend tier · the abort case, the doc's second enumerated case of the row.`, which names the native backend tier. No verifier is bound to it. + +### 8. Deploy a changed history projection version through the handoff + +[`spec:application.history-rebuild.write-pause-rebuild-version-change`](../specs/application/history-rebuild.write-pause-rebuild-version-change.sdp.md) · example · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [Rebuild of a history view under a write pause](#5-rebuild-of-a-history-view-under-a-write-pause) by refines. + +**Tier and verifier.** Its opening line is ``E-8 · native backend tier · no acceptance row; the example verifies the handoff of `spec:application.history-rebuild#design.historyHandoff` for a deploy that changes a history projection's version, an extension under E-8 and E-9.``, which names the native backend tier. No verifier is bound to it. + +### 9. Install a history view through the handoff + +[`spec:application.history-rebuild.write-pause-rebuild-installation`](../specs/application/history-rebuild.write-pause-rebuild-installation.sdp.md) · example · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [Rebuild of a history view under a write pause](#5-rebuild-of-a-history-view-under-a-write-pause) by refines. + +**Tier and verifier.** Its opening line is ``E-8 · native backend tier · no acceptance row; the example verifies the handoff of `spec:application.history-rebuild#design.historyHandoff` for a deploy that adds a history view's first binding, an extension under E-8 and E-9.``, which names the native backend tier. No verifier is bound to it. + +### 10. Reorder a history view's sources while its generation is interrupted + +[`spec:application.history-rebuild.write-pause-rebuild-source-order-mismatch`](../specs/application/history-rebuild.write-pause-rebuild-source-order-mismatch.sdp.md) · example · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [Rebuild of a history view under a write pause](#5-rebuild-of-a-history-view-under-a-write-pause) by refines. + +**Tier and verifier.** Its opening line is ``E-8 · native backend tier · no acceptance row; the example verifies the check of `spec:application.history-rebuild#design.historyCompatibility` and its recovery for a deploy that reorders a history view's sources while a generation of it is interrupted, an extension under E-8 and E-9.``, which names the native backend tier. No verifier is bound to it. + +### 11. Write pause + +[`spec:application.write-pause`](../specs/application/write-pause.sdp.md) · contract · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [Rebuild: generations, backfill and the write pause](#4-rebuild-generations-backfill-and-the-write-pause) by refines and [Rebuild online by default, pause writes only for history-dependent views](#19-rebuild-online-by-default-pause-writes-only-for-history-dependent-views) by decidedBy. + +**Code.** + +- `component:platform.gate` at [`src/gate/index.ts` line 3](../../src/gate/index.ts#L3): the maintenance gate: its table, the writer check, the two writers and the operator entries. It is a component in the infrastructure layer and the platform context. +- `impl:application.write-pause` at [`src/gate/gate.ts` line 8](../../src/gate/gate.ts#L8): gateAllows, assertWritable and the two scope writers. Its role is service. It belongs to `component:platform.gate`, in the infrastructure layer and the platform context. +- `impl:application.write-pause.entries` at [`src/gate/entries.ts` line 7](../../src/gate/entries.ts#L7): the four gate operator entries. Its role is operator-entry. It belongs to `component:platform.gate`, in the infrastructure layer and the platform context. +- `impl:application.write-pause.tables` at [`src/gate/tables.ts` line 4](../../src/gate/tables.ts#L4): the gate document's table fragment. Its role is schema. It belongs to `component:platform.gate`, in the infrastructure layer and the platform context. + +**Examples.** + +| Example | Tier | Verifier | Title | +|---|---|---|---| +| [`spec:application.write-pause.restore-door-without-deploy`](../specs/application/write-pause.restore-door-without-deploy.sdp.md) | native backend | bound | The restore door closes and opens with no deploy | + +**Pinned declarations.** 14 Design entries open with a code span. Each pair comes from [`tests/types/pinned-pairs.json`](../../tests/types/pinned-pairs.json) and is the test named by its address in [`tests/types/pinned.test-d.ts`](../../tests/types/pinned.test-d.ts). + +| Design entry | Pair | Compared | Code | Why | +|---|---|---|---|---| +| [`spec:application.write-pause#design.validatorClosedEntry`](../specs/application/write-pause.sdp.md?plain=1#L65) | bound | the type | `closedEntryValidator` in [`src/gate/index.ts`](../../src/gate/index.ts) | | +| [`spec:application.write-pause#design.tableMaintenanceGates`](../specs/application/write-pause.sdp.md?plain=1#L66) | bound | the table's document and indexes | `gateTables` in [`src/gate/index.ts`](../../src/gate/index.ts) | | +| [`spec:application.write-pause#design.typeScopeKey`](../specs/application/write-pause.sdp.md?plain=1#L68) | bound | the type | `ScopeKey` in [`src/gate/index.ts`](../../src/gate/index.ts) | | +| [`spec:application.write-pause#design.fnGateAllows`](../specs/application/write-pause.sdp.md?plain=1#L69) | bound | the type | `gateAllows` in [`src/gate/index.ts`](../../src/gate/index.ts) | | +| [`spec:application.write-pause#design.fnAssertWritable`](../specs/application/write-pause.sdp.md?plain=1#L70) | bound | the type | `assertWritable` in [`src/gate/index.ts`](../../src/gate/index.ts) | | +| [`spec:application.write-pause#design.errorCodeWritePaused`](../specs/application/write-pause.sdp.md?plain=1#L71) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.write-pause#design.restoreDoor`](../specs/application/write-pause.sdp.md?plain=1#L72) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.write-pause#design.fnScopesOfUseCase`](../specs/application/write-pause.sdp.md?plain=1#L73) | bound | the type | `scopesOfUseCase` in [`src/gate/index.ts`](../../src/gate/index.ts) | | +| [`spec:application.write-pause#design.fnCloseScope`](../specs/application/write-pause.sdp.md?plain=1#L74) | not compared | nothing | none | differs only in ctx | +| [`spec:application.write-pause#design.fnResumeScope`](../specs/application/write-pause.sdp.md?plain=1#L75) | not compared | nothing | none | differs only in ctx | +| [`spec:application.write-pause#design.fnCloseGate`](../specs/application/write-pause.sdp.md?plain=1#L76) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.write-pause#design.fnResumeGate`](../specs/application/write-pause.sdp.md?plain=1#L77) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.write-pause#design.fnGetGate`](../specs/application/write-pause.sdp.md?plain=1#L78) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.write-pause#design.fnGetGateAudit`](../specs/application/write-pause.sdp.md?plain=1#L79) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | + +
The 14 declarations as the Spec pins them + +[`spec:application.write-pause#design.validatorClosedEntry`](../specs/application/write-pause.sdp.md?plain=1#L65) + +```ts +const closedEntryValidator = v.object({ + scopeKey: v.string(), + reason: v.string(), + generationId: v.optional(v.id("generations")), + changedAt: v.number(), + changedBy: v.string(), +}) +``` + +[`spec:application.write-pause#design.tableMaintenanceGates`](../specs/application/write-pause.sdp.md?plain=1#L66) + +```ts +maintenanceGates: defineTable({ + key: v.literal("gates"), + closed: v.array(closedEntryValidator), + updatedAt: v.number(), +}).index("by_key", ["key"]) +``` + +[`spec:application.write-pause#design.typeScopeKey`](../specs/application/write-pause.sdp.md?plain=1#L68) + +```ts +type ScopeKey = "all" | (string & {}) +``` + +[`spec:application.write-pause#design.fnGateAllows`](../specs/application/write-pause.sdp.md?plain=1#L69) + +```ts +gateAllows( + ctx: { db: GenericDatabaseReader }, + scopes: readonly ScopeKey[], + allowGeneration?: Id<"generations">, +): Promise< + { allowed: true } | { allowed: false; scopeKey: ScopeKey; reason: string } +> +``` + +[`spec:application.write-pause#design.fnAssertWritable`](../specs/application/write-pause.sdp.md?plain=1#L70) + +```ts +assertWritable( + ctx: { db: GenericDatabaseReader }, + scopes: readonly ScopeKey[], +): Promise +``` + +[`spec:application.write-pause#design.errorCodeWritePaused`](../specs/application/write-pause.sdp.md?plain=1#L71) + +```ts +writePaused +``` + +[`spec:application.write-pause#design.restoreDoor`](../specs/application/write-pause.sdp.md?plain=1#L72) + +```ts +gateAllows +``` + +[`spec:application.write-pause#design.fnScopesOfUseCase`](../specs/application/write-pause.sdp.md?plain=1#L73) + +```ts +scopesOfUseCase( + tenantId: string, + writes: readonly SourceRef[], +): ScopeKey[] +``` + +[`spec:application.write-pause#design.fnCloseScope`](../specs/application/write-pause.sdp.md?plain=1#L74) + +```ts +closeScope( + ctx: MutationCtx, + scopeKey: ScopeKey, + reason: string, + operator: string, + generationId?: Id<"generations">, +): Promise +``` + +[`spec:application.write-pause#design.fnResumeScope`](../specs/application/write-pause.sdp.md?plain=1#L75) + +```ts +resumeScope( + ctx: MutationCtx, + scopeKey: ScopeKey, + operator: string, +): Promise +``` + +[`spec:application.write-pause#design.fnCloseGate`](../specs/application/write-pause.sdp.md?plain=1#L76) + +```ts +export const closeGate = internalMutation({ + args: { scopeKey: v.string(), reason: v.string(), operator: v.string() }, + returns: v.null(), + handler, +}) +``` + +[`spec:application.write-pause#design.fnResumeGate`](../specs/application/write-pause.sdp.md?plain=1#L77) + +```ts +export const resumeGate = internalMutation({ + args: { scopeKey: v.string(), operator: v.string() }, + returns: v.null(), + handler, +}) +``` + +[`spec:application.write-pause#design.fnGetGate`](../specs/application/write-pause.sdp.md?plain=1#L78) + +```ts +export const getGate = internalQuery({ + args: {}, + returns: v.object({ + restore: v.boolean(), + closed: v.array(closedEntryValidator), + }), + handler, +}) +``` + +[`spec:application.write-pause#design.fnGetGateAudit`](../specs/application/write-pause.sdp.md?plain=1#L79) + +```ts +export const getGateAudit = internalQuery({ + args: { scopeKey: v.string(), paginationOpts: paginationOptsValidator }, + returns: paginationResultValidator(operatorAuditDoc), + handler, +}) +``` + +
+ +**Other Design entries.** 6 entries pin no declaration: [`indexGatesByKeyUse`](../specs/application/write-pause.sdp.md?plain=1#L67), [`occSerialization`](../specs/application/write-pause.sdp.md?plain=1#L80), [`observedRestoreDoorWithoutDeploy`](../specs/application/write-pause.sdp.md?plain=1#L81), [`limitClosedScopes`](../specs/application/write-pause.sdp.md?plain=1#L82), [`limitGateReason`](../specs/application/write-pause.sdp.md?plain=1#L83), [`limitGateReadsPerCommand`](../specs/application/write-pause.sdp.md?plain=1#L84). + +### 12. Generation registry + +[`spec:application.generation-registry`](../specs/application/generation-registry.sdp.md) · contract · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [Rebuild: generations, backfill and the write pause](#4-rebuild-generations-backfill-and-the-write-pause) by refines, [Projection contract](#2-projection-contract) by dependsOn and [Rebuild online by default, pause writes only for history-dependent views](#19-rebuild-online-by-default-pause-writes-only-for-history-dependent-views) by decidedBy. + +**Code.** No implementation anchor binds it. + +**References.** 2 units reference this design. A reference says the code answers to the design without claiming to realize it, and it confers no delivery fact and moves no floor. + +- `impl:application.generation-registry` at [`src/read-model/generations.ts` line 2](../../src/read-model/generations.ts#L2): generationsToWrite and activeGeneration. Its role is query. It belongs to `component:platform.read-model`, in the application layer and the platform context. +- `impl:application.generation-registry.tables` at [`src/read-model/tables.ts` line 5](../../src/read-model/tables.ts#L5): the generations, progress, marker and tenant fill tables and rowConventions. Its role is schema. It belongs to `component:platform.read-model`, in the application layer and the platform context. + +**Examples.** + +| Example | Tier | Verifier | Title | +|---|---|---|---| +| [`spec:application.generation-registry.command-without-generation-fails`](../specs/application/generation-registry.command-without-generation-fails.sdp.md) | native backend | bound | A command whose read model has no generation fails | +| [`spec:application.generation-registry.first-rebuild-installs-read-model`](../specs/application/generation-registry.first-rebuild-installs-read-model.sdp.md) | native backend | bound | A read model is installed through its first rebuild | + +**Pinned declarations.** 13 Design entries open with a code span. Each pair comes from [`tests/types/pinned-pairs.json`](../../tests/types/pinned-pairs.json) and is the test named by its address in [`tests/types/pinned.test-d.ts`](../../tests/types/pinned.test-d.ts). + +| Design entry | Pair | Compared | Code | Why | +|---|---|---|---|---| +| [`spec:application.generation-registry#design.validatorGenerationState`](../specs/application/generation-registry.sdp.md?plain=1#L68) | bound | the type | `generationStateValidator` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.generation-registry#design.tableGenerations`](../specs/application/generation-registry.sdp.md?plain=1#L69) | bound | the table's document and indexes | `readModelTables` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.generation-registry#design.generationPauseRequired`](../specs/application/generation-registry.sdp.md?plain=1#L70) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.generation-registry#design.indexGenerationsByReadModel`](../specs/application/generation-registry.sdp.md?plain=1#L71) | bound | the indexes, through its table [`tableGenerations`](../specs/application/generation-registry.sdp.md?plain=1#L69) | `readModelTables` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.generation-registry#design.indexGenerationsByReadModelState`](../specs/application/generation-registry.sdp.md?plain=1#L73) | bound | the indexes, through its table [`tableGenerations`](../specs/application/generation-registry.sdp.md?plain=1#L69) | `readModelTables` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.generation-registry#design.indexGenerationsByReadModelStateUse`](../specs/application/generation-registry.sdp.md?plain=1#L74) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:application.generation-registry#design.validatorBatchCursor`](../specs/application/generation-registry.sdp.md?plain=1#L75) | ahead | the type | `batchCursorValidator` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | the history view's build adds sourceIndex | +| [`spec:application.generation-registry#design.validatorProgressPass`](../specs/application/generation-registry.sdp.md?plain=1#L76) | bound | the type | `progressPassValidator` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.generation-registry#design.tableGenerationProgress`](../specs/application/generation-registry.sdp.md?plain=1#L77) | ahead | the table's document and indexes | `readModelTables` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | its cursor, through validatorBatchCursor | +| [`spec:application.generation-registry#design.indexProgressByGeneration`](../specs/application/generation-registry.sdp.md?plain=1#L78) | ahead | the indexes, through its table [`tableGenerationProgress`](../specs/application/generation-registry.sdp.md?plain=1#L77) | `readModelTables` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | its cursor, through validatorBatchCursor | +| [`spec:application.generation-registry#design.typeRegistryReader`](../specs/application/generation-registry.sdp.md?plain=1#L80) | bound | the type | `RegistryReader` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.generation-registry#design.fnGenerationsToWrite`](../specs/application/generation-registry.sdp.md?plain=1#L82) | bound | the type | `generationsToWrite` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:application.generation-registry#design.fnActiveGeneration`](../specs/application/generation-registry.sdp.md?plain=1#L83) | bound | the type | `activeGeneration` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | + +
The 13 declarations as the Spec pins them + +[`spec:application.generation-registry#design.validatorGenerationState`](../specs/application/generation-registry.sdp.md?plain=1#L68) + +```ts +const generationStateValidator = v.union( + v.literal("building"), + v.literal("verifying"), + v.literal("verified"), + v.literal("active"), + v.literal("retired"), + v.literal("aborted"), + v.literal("purged"), +) +``` + +[`spec:application.generation-registry#design.tableGenerations`](../specs/application/generation-registry.sdp.md?plain=1#L69) + +```ts +generations: defineTable({ + readModel: v.string(), + generation: v.number(), + projectionVersion: v.number(), + state: generationStateValidator, + pauseRequired: v.boolean(), + fence: v.number(), + startedAt: v.number(), + startedBy: v.string(), + changedAt: v.number(), + changedBy: v.string(), + switchedAt: v.optional(v.number()), + retiredAt: v.optional(v.number()), + retireAfter: v.optional(v.number()), + interruptedFence: v.optional(v.number()), +}) +``` + +[`spec:application.generation-registry#design.generationPauseRequired`](../specs/application/generation-registry.sdp.md?plain=1#L70) + +```ts +pauseRequired +``` + +[`spec:application.generation-registry#design.indexGenerationsByReadModel`](../specs/application/generation-registry.sdp.md?plain=1#L71) + +```ts +.index("by_read_model", ["readModel", "generation"]) +``` + +[`spec:application.generation-registry#design.indexGenerationsByReadModelState`](../specs/application/generation-registry.sdp.md?plain=1#L73) + +```ts +.index("by_read_model_state", ["readModel", "state"]) +``` + +[`spec:application.generation-registry#design.indexGenerationsByReadModelStateUse`](../specs/application/generation-registry.sdp.md?plain=1#L74) + +```ts +generationsToWrite +``` + +[`spec:application.generation-registry#design.validatorBatchCursor`](../specs/application/generation-registry.sdp.md?plain=1#L75) + +```ts +const batchCursorValidator = v.object({ + tenantId: v.string(), + pageCursor: v.union(v.string(), v.null()), + sourceIndex: v.optional(v.number()), + streamId: v.optional(v.string()), + eventCursor: v.optional(v.union(v.string(), v.null())), +}) +``` + +[`spec:application.generation-registry#design.validatorProgressPass`](../specs/application/generation-registry.sdp.md?plain=1#L76) + +```ts +const progressPassValidator = v.union( + v.literal("backfill"), + v.literal("verify"), + v.literal("purge"), + v.literal("idle"), +) +``` + +[`spec:application.generation-registry#design.tableGenerationProgress`](../specs/application/generation-registry.sdp.md?plain=1#L77) + +```ts +generationProgress: defineTable({ + generationId: v.id("generations"), + pass: progressPassValidator, + batchSize: v.number(), + cursor: v.union(batchCursorValidator, v.null()), + batchesDone: v.number(), + rowsWritten: v.number(), + rowsSkipped: v.number(), + misses: v.number(), + rowsPurged: v.number(), + lastError: v.optional(v.string()), + updatedAt: v.number(), +}) +``` + +[`spec:application.generation-registry#design.indexProgressByGeneration`](../specs/application/generation-registry.sdp.md?plain=1#L78) + +```ts +.index("by_generation", ["generationId"]) +``` + +[`spec:application.generation-registry#design.typeRegistryReader`](../specs/application/generation-registry.sdp.md?plain=1#L80) + +```ts +type RegistryReader = + { db: GenericDatabaseReader } +``` + +[`spec:application.generation-registry#design.fnGenerationsToWrite`](../specs/application/generation-registry.sdp.md?plain=1#L82) + +```ts +generationsToWrite( + ctx: RegistryReader, + readModel: string, +): Promise +``` + +[`spec:application.generation-registry#design.fnActiveGeneration`](../specs/application/generation-registry.sdp.md?plain=1#L83) + +```ts +activeGeneration( + ctx: RegistryReader, + readModel: string, +): Promise +``` + +
+ +**Other Design entries.** 7 entries pin no declaration: [`indexGenerationsByReadModelUse`](../specs/application/generation-registry.sdp.md?plain=1#L72), [`indexProgressByGenerationUse`](../specs/application/generation-registry.sdp.md?plain=1#L79), [`writableGenerationRole`](../specs/application/generation-registry.sdp.md?plain=1#L81), [`errorNoGenerationToWrite`](../specs/application/generation-registry.sdp.md?plain=1#L84), [`transitionCost`](../specs/application/generation-registry.sdp.md?plain=1#L85), [`limitRegistryReadsPerCommand`](../specs/application/generation-registry.sdp.md?plain=1#L86), [`limitRollbackPeriod`](../specs/application/generation-registry.sdp.md?plain=1#L87). + +### 13. Orders and Inventory, the example domain + +[`spec:application.orders-inventory-example`](../specs/application/orders-inventory-example.sdp.md) · model · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [History projection](#1-history-projection) by dependsOn. + +**Code.** No implementation anchor binds it. + +**References.** 14 units reference this design. A reference says the code answers to the design without claiming to realize it, and it confers no delivery fact and moves no floor. + +- `impl:application.order-allocation-history` at [`example/domain/orderAllocationHistory.ts` line 8](../../example/domain/orderAllocationHistory.ts#L8): the order allocation history's keyOf and project rules. Its role is projection. It belongs to `component:example.parent`, in the application layer and the orders-inventory context. +- `impl:application.orders-inventory-example` at [`example/domain/order.ts` line 10](../../example/domain/order.ts#L10): the order decider. Its role is decider. It belongs to `component:example.orders-domain`, in the domain layer and the orders context. +- `impl:application.orders-inventory-example.inventory-operations` at [`example/convex/inventory/operations.ts` line 2](../../example/convex/inventory/operations.ts#L2): the Inventory sanctioned operations. Its role is service. It belongs to `component:example.inventory-context`, in the adapter layer and the inventory context. +- `impl:application.orders-inventory-example.inventory-schema` at [`example/convex/inventory/schema.ts` line 1](../../example/convex/inventory/schema.ts#L1): the Inventory context schema. Its role is schema. It belongs to `component:example.inventory-context`, in the adapter layer and the inventory context. +- `impl:application.orders-inventory-example.inventory-stock-item-query` at [`example/convex/inventory/queries/stockItem.ts` line 2](../../example/convex/inventory/queries/stockItem.ts#L2): the stock item query. Its role is query. It belongs to `component:example.inventory-context`, in the adapter layer and the inventory context. +- `impl:application.orders-inventory-example.inventory-streams` at [`example/convex/inventory/streams.ts` line 2](../../example/convex/inventory/streams.ts#L2): the stock item stream registration. It belongs to `component:example.inventory-context`, in the adapter layer and the inventory context. +- `impl:application.orders-inventory-example.order-summary` at [`example/convex/orderSummary.ts` line 4](../../example/convex/orderSummary.ts#L4): the order summary read model. Its role is read-model. It belongs to `component:example.parent`, in the application layer and the orders-inventory context. +- `impl:application.orders-inventory-example.orders-operations` at [`example/convex/orders/operations.ts` line 2](../../example/convex/orders/operations.ts#L2): the Orders sanctioned operations. Its role is service. It belongs to `component:example.orders-context`, in the adapter layer and the orders context. +- `impl:application.orders-inventory-example.orders-order-query` at [`example/convex/orders/queries/order.ts` line 2](../../example/convex/orders/queries/order.ts#L2): the order query. Its role is query. It belongs to `component:example.orders-context`, in the adapter layer and the orders context. +- `impl:application.orders-inventory-example.orders-schema` at [`example/convex/orders/schema.ts` line 1](../../example/convex/orders/schema.ts#L1): the Orders context schema. Its role is schema. It belongs to `component:example.orders-context`, in the adapter layer and the orders context. +- `impl:application.orders-inventory-example.orders-streams` at [`example/convex/orders/streams.ts` line 3](../../example/convex/orders/streams.ts#L3): the order stream registration. It belongs to `component:example.orders-context`, in the adapter layer and the orders context. +- `impl:application.orders-inventory-example.stock-item` at [`example/domain/stockItem.ts` line 11](../../example/domain/stockItem.ts#L11): the stock item decider. Its role is decider. It belongs to `component:example.inventory-domain`, in the domain layer and the inventory context. +- `impl:application.parent-use-cases` at [`example/convex/ordering.ts` line 8](../../example/convex/ordering.ts#L8): PlaceOrder and CancelOrder over Orders and then Inventory. Its role is use-case. It belongs to `component:example.parent`, in the application layer and the orders-inventory context. +- `impl:application.parent-use-cases.receiving` at [`example/convex/receiving.ts` line 4](../../example/convex/receiving.ts#L4): ReceiveStock. Its role is use-case. It belongs to `component:example.parent`, in the application layer and the orders-inventory context. + +**Examples.** + +| Example | Tier | Verifier | Title | +|---|---|---|---| +| [`spec:application.orders-inventory-example.cancel-duplicate-answered-from-receipt`](../specs/application/orders-inventory-example.cancel-duplicate-answered-from-receipt.sdp.md) | native backend | bound | A repeated cancel request is answered from its receipt | +| [`spec:application.orders-inventory-example.cancel-placed-order`](../specs/application/orders-inventory-example.cancel-placed-order.sdp.md) | native backend | bound | A placed order is cancelled | +| [`spec:application.orders-inventory-example.cancel-second-context-rejects`](../specs/application/orders-inventory-example.cancel-second-context-rejects.sdp.md) | native backend | bound | Orders cancels, then Inventory refuses the release | +| [`spec:application.orders-inventory-example.cancel-twice-rejected`](../specs/application/orders-inventory-example.cancel-twice-rejected.sdp.md) | native backend | bound | A second cancel of one order is rejected | +| [`spec:application.orders-inventory-example.cancel-unknown-order-rejected`](../specs/application/orders-inventory-example.cancel-unknown-order-rejected.sdp.md) | native backend | bound | A cancel of an order never placed is rejected | +| [`spec:application.orders-inventory-example.cancel-without-grant-refused`](../specs/application/orders-inventory-example.cancel-without-grant-refused.sdp.md) | native backend | bound | A caller without the cancel grant is refused | + +**Pinned declarations.** It pins none. + +### 14. Command declaration and bindings + +[`spec:command.command-declaration`](../specs/command/command-declaration.sdp.md) · contract · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [History projection](#1-history-projection) by dependsOn, [Projection contract](#2-projection-contract) by dependsOn and [The command pipeline](#15-the-command-pipeline) by dependsOn. + +**Code.** No implementation anchor binds it. + +**References.** 2 units reference this design. A reference says the code answers to the design without claiming to realize it, and it confers no delivery fact and moves no floor. + +- `impl:command.command-declaration` at [`src/command/declaration.ts` line 3](../../src/command/declaration.ts#L3): CommandDeclaration, publicCommand and internalCommand. Its role is contract. It belongs to `component:platform.command`, in the application layer and the platform context. +- `impl:command.command-declaration.depot-commands` at [`fixture/convex/depotCommands.ts` line 3](../../fixture/convex/depotCommands.ts#L3): the depot fixture's command declarations. + +**Examples.** + +| Example | Tier | Verifier | Title | +|---|---|---|---| +| [`spec:command.command-declaration.broken-argument-contract-fails-build`](../specs/command/command-declaration.broken-argument-contract-fails-build.sdp.md) | compiled | bound | Break a typed handler's argument contract | +| [`spec:command.command-declaration.renamed-handler-fails-build`](../specs/command/command-declaration.renamed-handler-fails-build.sdp.md) | compiled | bound | Rename a typed handler | + +**Pinned declarations.** 18 Design entries open with a code span. Each pair comes from [`tests/types/pinned-pairs.json`](../../tests/types/pinned-pairs.json) and is the test named by its address in [`tests/types/pinned.test-d.ts`](../../tests/types/pinned.test-d.ts). + +| Design entry | Pair | Compared | Code | Why | +|---|---|---|---|---| +| [`spec:command.command-declaration#design.typeCommandDeclaration`](../specs/command/command-declaration.sdp.md?plain=1#L64) | not compared | nothing | none | leads the code and differs in ctx, in its executor and admission | +| [`spec:command.command-declaration#design.typeSourceRef`](../specs/command/command-declaration.sdp.md?plain=1#L65) | bound | the type | `SourceRef` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | | +| [`spec:command.command-declaration#design.typeWrites`](../specs/command/command-declaration.sdp.md?plain=1#L66) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-declaration#design.typePermissionPolicy`](../specs/command/command-declaration.sdp.md?plain=1#L67) | bound | the type | `PermissionPolicy` in [`src/command/index.ts`](../../src/command/index.ts) | | +| [`spec:command.command-declaration#design.typeExecutor`](../specs/command/command-declaration.sdp.md?plain=1#L68) | not compared | nothing | none | differs only in ctx | +| [`spec:command.command-declaration#design.typeExecutorResult`](../specs/command/command-declaration.sdp.md?plain=1#L69) | bound | the type | `ExecutorResult` in [`src/command/index.ts`](../../src/command/index.ts) | | +| [`spec:command.command-declaration#design.typeReadModelBinding`](../specs/command/command-declaration.sdp.md?plain=1#L70) | ahead | the type | `ReadModelBinding` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | the history view's build binds history projections | +| [`spec:command.command-declaration#design.typeAudit`](../specs/command/command-declaration.sdp.md?plain=1#L71) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-declaration#design.typeDiagnosticSinkField`](../specs/command/command-declaration.sdp.md?plain=1#L72) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-declaration#design.typeAdmissionPolicy`](../specs/command/command-declaration.sdp.md?plain=1#L73) | not compared | nothing | none | differs only in ctx | +| [`spec:command.command-declaration#design.typeBounds`](../specs/command/command-declaration.sdp.md?plain=1#L74) | bound | the type | `Bounds` in [`src/command/index.ts`](../../src/command/index.ts) | | +| [`spec:command.command-declaration#design.typeRetention`](../specs/command/command-declaration.sdp.md?plain=1#L75) | ahead | the type | `Retention` in [`src/command/index.ts`](../../src/command/index.ts) | a receipt's tombstone expiry | +| [`spec:command.command-declaration#design.fnPublicCommand`](../specs/command/command-declaration.sdp.md?plain=1#L76) | not compared | nothing | none | leads the code and differs in ctx, through typeCommandDeclaration | +| [`spec:command.command-declaration#design.fnInternalCommand`](../specs/command/command-declaration.sdp.md?plain=1#L77) | not compared | nothing | none | leads the code and differs in ctx, through typeCommandDeclaration | +| [`spec:command.command-declaration#design.staticExports`](../specs/command/command-declaration.sdp.md?plain=1#L78) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-declaration#design.moduleGrouping`](../specs/command/command-declaration.sdp.md?plain=1#L80) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-declaration#design.canonicalSchema`](../specs/command/command-declaration.sdp.md?plain=1#L81) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-declaration#design.refineResult`](../specs/command/command-declaration.sdp.md?plain=1#L82) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | + +
The 18 declarations as the Spec pins them + +[`spec:command.command-declaration#design.typeCommandDeclaration`](../specs/command/command-declaration.sdp.md?plain=1#L64) + +```ts +type CommandDeclaration = { + name: string; + contractVersion: number; + input: Validator; + refine?: (input: I) => Omit | null; + output: Validator; + permission: PermissionPolicy; + executor: Executor; + writes: readonly SourceRef[]; + readModels?: readonly ReadModelBinding[]; + rejections: readonly string[]; + admission?: AdmissionPolicy; + bounds?: Bounds; + retention?: Retention; + irreversible?: boolean; + audit?: { kind: "security" | "business" }; + diagnosticSink?: DiagnosticSink; +} +``` + +[`spec:command.command-declaration#design.typeSourceRef`](../specs/command/command-declaration.sdp.md?plain=1#L65) + +```ts +type SourceRef = { contextId: string; streamType: string } +``` + +[`spec:command.command-declaration#design.typeWrites`](../specs/command/command-declaration.sdp.md?plain=1#L66) + +```ts +writes +``` + +[`spec:command.command-declaration#design.typePermissionPolicy`](../specs/command/command-declaration.sdp.md?plain=1#L67) + +```ts +type PermissionPolicy = { + permission: string; + subjectFrom?: (input: I) => SubjectRef; +} +``` + +[`spec:command.command-declaration#design.typeExecutor`](../specs/command/command-declaration.sdp.md?plain=1#L68) + +```ts +type Executor = ( + ctx: MutationCtx, + call: { tenantId: string; actor: Actor; operation: OperationRef; input: I }, +) => Promise> +``` + +[`spec:command.command-declaration#design.typeExecutorResult`](../specs/command/command-declaration.sdp.md?plain=1#L69) + +```ts +type ExecutorResult = CommittedOutcome & { streams: StreamDto[] } +``` + +[`spec:command.command-declaration#design.typeReadModelBinding`](../specs/command/command-declaration.sdp.md?plain=1#L70) + +```ts +type ReadModelBinding = { + readModel: ReadModel | HistoryProjection; + source: SourceRef; +} +``` + +[`spec:command.command-declaration#design.typeAudit`](../specs/command/command-declaration.sdp.md?plain=1#L71) + +```ts +audit?: { kind: "security" | "business" } +``` + +[`spec:command.command-declaration#design.typeDiagnosticSinkField`](../specs/command/command-declaration.sdp.md?plain=1#L72) + +```ts +diagnosticSink?: DiagnosticSink +``` + +[`spec:command.command-declaration#design.typeAdmissionPolicy`](../specs/command/command-declaration.sdp.md?plain=1#L73) + +```ts +type AdmissionPolicy = ( + ctx: MutationCtx, + call: PipelineCall, +) => Promise< + | { admitted: true } + | { admitted: false; code: "rateLimited" | "capacity"; retryAfterMs?: number } +> +``` + +[`spec:command.command-declaration#design.typeBounds`](../specs/command/command-declaration.sdp.md?plain=1#L74) + +```ts +type Bounds = { maxItems?: number; maxBytes?: number } +``` + +[`spec:command.command-declaration#design.typeRetention`](../specs/command/command-declaration.sdp.md?plain=1#L75) + +```ts +type Retention = { window: number; afterExpiry: "delete" | "tombstone" } +``` + +[`spec:command.command-declaration#design.fnPublicCommand`](../specs/command/command-declaration.sdp.md?plain=1#L76) + +```ts +publicCommand( + decl: CommandDeclaration, + serviceIssuers: ReadonlySet = new Set(), +): RegisteredMutation< + "public", + { tenantId: string; requestKey?: string; correlationId?: string; input: I }, + CommandResponse +> +``` + +[`spec:command.command-declaration#design.fnInternalCommand`](../specs/command/command-declaration.sdp.md?plain=1#L77) + +```ts +internalCommand( + decl: CommandDeclaration, +): RegisteredMutation< + "internal", + { + tenantId: string; + namespace: CallerNamespace; + actor: Actor; + requestKey: string; + correlationId?: string; + causedBy?: CausedBy; + input: I; + }, + CommandResponse +> +``` + +[`spec:command.command-declaration#design.staticExports`](../specs/command/command-declaration.sdp.md?plain=1#L78) + +```ts +export const placeOrder = publicCommand(placeOrderDeclaration) +``` + +[`spec:command.command-declaration#design.moduleGrouping`](../specs/command/command-declaration.sdp.md?plain=1#L80) + +```ts +class OrdersModule { + static readonly commands = [ + placeOrderDeclaration, + cancelOrderDeclaration, + ] as const; +} +``` + +[`spec:command.command-declaration#design.canonicalSchema`](../specs/command/command-declaration.sdp.md?plain=1#L81) + +```ts +input +``` + +[`spec:command.command-declaration#design.refineResult`](../specs/command/command-declaration.sdp.md?plain=1#L82) + +```ts +refine +``` + +
+ +**Other Design entries.** 1 entry pins no declaration: [`generationTrigger`](../specs/command/command-declaration.sdp.md?plain=1#L79). + +### 15. The command pipeline + +[`spec:command.command-pipeline`](../specs/command/command-pipeline.sdp.md) · workflow · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [History projection](#1-history-projection) by dependsOn, [Projection contract](#2-projection-contract) by dependsOn, [Write pause](#11-write-pause) by dependsOn and [The event envelope](#16-the-event-envelope) by dependsOn. + +**Code.** No implementation anchor binds it. + +**References.** 2 units reference this design. A reference says the code answers to the design without claiming to realize it, and it confers no delivery fact and moves no floor. + +- `impl:application.read-models` at [`src/read-model/write.ts` line 3](../../src/read-model/write.ts#L3): writeReadModels, step 9's read-model half. Its role is service. It belongs to `component:platform.read-model`, in the application layer and the platform context. +- `impl:command.command-pipeline` at [`src/command/pipeline.ts` line 3](../../src/command/pipeline.ts#L3): runPipeline and the command response. Its role is service. It belongs to `component:platform.command`, in the application layer and the platform context. + +**Examples.** + +| Example | Tier | Verifier | Title | +|---|---|---|---| +| [`spec:command.command-pipeline.failure-after-journal-append`](../specs/command/command-pipeline.failure-after-journal-append.sdp.md) | native backend | bound | Inject a failure after the journal append | +| [`spec:command.command-pipeline.failure-after-state-write`](../specs/command/command-pipeline.failure-after-state-write.sdp.md) | native backend | bound | Inject a failure after the state write | +| [`spec:command.command-pipeline.failure-before-receipt`](../specs/command/command-pipeline.failure-before-receipt.sdp.md) | native backend | bound | Inject a failure before the receipt insert | + +**Pinned declarations.** 18 Design entries open with a code span. Each pair comes from [`tests/types/pinned-pairs.json`](../../tests/types/pinned-pairs.json) and is the test named by its address in [`tests/types/pinned.test-d.ts`](../../tests/types/pinned.test-d.ts). + +| Design entry | Pair | Compared | Code | Why | +|---|---|---|---|---| +| [`spec:command.command-pipeline#design.fnPublicEntry`](../specs/command/command-pipeline.sdp.md?plain=1#L111) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-pipeline#design.fnInternalEntry`](../specs/command/command-pipeline.sdp.md?plain=1#L112) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-pipeline#design.fnRunPipeline`](../specs/command/command-pipeline.sdp.md?plain=1#L113) | not compared | nothing | none | leads the code and differs in ctx, through typeCommandDeclaration | +| [`spec:command.command-pipeline#design.typePipelineCall`](../specs/command/command-pipeline.sdp.md?plain=1#L114) | bound | the type | `PipelineCall` in [`src/command/index.ts`](../../src/command/index.ts) | | +| [`spec:command.command-pipeline#design.typeCommandResponse`](../specs/command/command-pipeline.sdp.md?plain=1#L115) | bound | the type | `CommandResponse` in [`src/command/index.ts`](../../src/command/index.ts) | | +| [`spec:command.command-pipeline#design.affectedDerivation`](../specs/command/command-pipeline.sdp.md?plain=1#L117) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-pipeline#design.validatorCommandResponse`](../specs/command/command-pipeline.sdp.md?plain=1#L118) | bound | the type | `commandResponseValidator` in [`src/command/index.ts`](../../src/command/index.ts) | | +| [`spec:command.command-pipeline#design.operationRef`](../specs/command/command-pipeline.sdp.md?plain=1#L119) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-pipeline#design.ctxAuthUse`](../specs/command/command-pipeline.sdp.md?plain=1#L120) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-pipeline#design.gateRead`](../specs/command/command-pipeline.sdp.md?plain=1#L121) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-pipeline#design.contextCall`](../specs/command/command-pipeline.sdp.md?plain=1#L123) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-pipeline#design.writesCheck`](../specs/command/command-pipeline.sdp.md?plain=1#L127) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-pipeline#design.fnWriteReadModels`](../specs/command/command-pipeline.sdp.md?plain=1#L128) | ahead | the type | `writeReadModels` in [`src/read-model/index.ts`](../../src/read-model/index.ts) | through typeReadModelBinding | +| [`spec:command.command-pipeline#design.auditWrite`](../specs/command/command-pipeline.sdp.md?plain=1#L130) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-pipeline#design.diagnosticEmit`](../specs/command/command-pipeline.sdp.md?plain=1#L131) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-pipeline#design.fnRelayFailure`](../specs/command/command-pipeline.sdp.md?plain=1#L132) | not compared | nothing | none | leads the code and differs in ctx, through typeCommandDeclaration | +| [`spec:command.command-pipeline#design.admissionHook`](../specs/command/command-pipeline.sdp.md?plain=1#L134) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:command.command-pipeline#design.subTransactionLimits`](../specs/command/command-pipeline.sdp.md?plain=1#L135) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | + +
The 18 declarations as the Spec pins them + +[`spec:command.command-pipeline#design.fnPublicEntry`](../specs/command/command-pipeline.sdp.md?plain=1#L111) + +```ts +export const placeOrder = mutation({ + args: { + tenantId: v.string(), + requestKey: v.optional(v.string()), + correlationId: v.optional(v.string()), + input: placeOrderInputValidator, + }, + returns: commandResponseValidator(placeOrderDeclaration.output), + handler, +}) +``` + +[`spec:command.command-pipeline#design.fnInternalEntry`](../specs/command/command-pipeline.sdp.md?plain=1#L112) + +```ts +export const placeOrderInternal = internalMutation({ + args: { + tenantId: v.string(), + namespace: callerNamespaceValidator, + actor: actorValidator, + requestKey: v.string(), + correlationId: v.optional(v.string()), + causedBy: v.optional(causedByValidator), + input: placeOrderInputValidator, + }, + returns: commandResponseValidator(placeOrderDeclaration.output), + handler, +}) +``` + +[`spec:command.command-pipeline#design.fnRunPipeline`](../specs/command/command-pipeline.sdp.md?plain=1#L113) + +```ts +runPipeline( + ctx: MutationCtx, + decl: CommandDeclaration, + call: PipelineCall, +): Promise> +``` + +[`spec:command.command-pipeline#design.typePipelineCall`](../specs/command/command-pipeline.sdp.md?plain=1#L114) + +```ts +type PipelineCall = { + tenantId: string; + namespace: CallerNamespace; + actor: Actor; + requestKey?: string; + correlationId?: string; + causedBy?: CausedBy; + input: I; +} +``` + +[`spec:command.command-pipeline#design.typeCommandResponse`](../specs/command/command-pipeline.sdp.md?plain=1#L115) + +```ts +type CommandResponse = { + kind: "applied" | "businessFailure"; + operationId: string; + affected: AffectedRef[]; + versions: StreamVersion[]; +} & ({ replayed: false; result: R } | { replayed: true; result: null }) +``` + +[`spec:command.command-pipeline#design.affectedDerivation`](../specs/command/command-pipeline.sdp.md?plain=1#L117) + +```ts +affected = versions.map(({ contextId, streamType, streamId }) => ({ + contextId, + streamType, + streamId, +})) +``` + +[`spec:command.command-pipeline#design.validatorCommandResponse`](../specs/command/command-pipeline.sdp.md?plain=1#L118) + +```ts +const commandResponseValidator = ( + output: Validator, +) => + v.union( + v.object({ + kind: outcomeKindValidator, + result: output, + operationId: v.string(), + affected: v.array(affectedRefValidator), + versions: v.array(streamVersionValidator), + replayed: v.literal(false), + }), + v.object({ + kind: outcomeKindValidator, + result: v.null(), + operationId: v.string(), + affected: v.array(affectedRefValidator), + versions: v.array(streamVersionValidator), + replayed: v.literal(true), + }), + ) +``` + +[`spec:command.command-pipeline#design.operationRef`](../specs/command/command-pipeline.sdp.md?plain=1#L119) + +```ts +operation = { + operationId: crypto.randomUUID(), + causedBy: call.causedBy ?? { kind: "command", commandType: decl.name }, + ...(call.correlationId === undefined + ? {} + : { correlationId: call.correlationId }), +} +``` + +[`spec:command.command-pipeline#design.ctxAuthUse`](../specs/command/command-pipeline.sdp.md?plain=1#L120) + +```ts +ctx.auth.getUserIdentity() +``` + +[`spec:command.command-pipeline#design.gateRead`](../specs/command/command-pipeline.sdp.md?plain=1#L121) + +```ts +assertWritable +``` + +[`spec:command.command-pipeline#design.contextCall`](../specs/command/command-pipeline.sdp.md?plain=1#L123) + +```ts +await ctx.runMutation(components.inventory.operations.allocate, { + tenantId, + actor, + operation, + input: { orderId, lines }, + facts, +}) +``` + +[`spec:command.command-pipeline#design.writesCheck`](../specs/command/command-pipeline.sdp.md?plain=1#L127) + +```ts +decl.writes +``` + +[`spec:command.command-pipeline#design.fnWriteReadModels`](../specs/command/command-pipeline.sdp.md?plain=1#L128) + +```ts +writeReadModels( + ctx: RegistryReader & RowWriter, + commandType: string, + bindings: readonly ReadModelBinding[], + input: { tenantId: string; streams: readonly StreamDto[] }, +): Promise +``` + +[`spec:command.command-pipeline#design.auditWrite`](../specs/command/command-pipeline.sdp.md?plain=1#L130) + +```ts +writeAudit +``` + +[`spec:command.command-pipeline#design.diagnosticEmit`](../specs/command/command-pipeline.sdp.md?plain=1#L131) + +```ts +emitDiagnostic +``` + +[`spec:command.command-pipeline#design.fnRelayFailure`](../specs/command/command-pipeline.sdp.md?plain=1#L132) + +```ts +relayFailure( + decl: CommandDeclaration, + call: FailedCall, + error: unknown, +): never +``` + +[`spec:command.command-pipeline#design.admissionHook`](../specs/command/command-pipeline.sdp.md?plain=1#L134) + +```ts +admission?: ( + ctx: MutationCtx, + call: PipelineCall, +) => Promise< + | { admitted: true } + | { + admitted: false; + code: "rateLimited" | "capacity"; + retryAfterMs?: number; + } +> +``` + +[`spec:command.command-pipeline#design.subTransactionLimits`](../specs/command/command-pipeline.sdp.md?plain=1#L135) + +```ts +ctx.runMutation +``` + +
+ +**Other Design entries.** 17 entries pin no declaration: [`transactionBoundary`](../specs/command/command-pipeline.sdp.md?plain=1#L109), [`convexSurface`](../specs/command/command-pipeline.sdp.md?plain=1#L110), [`typeAffectedRef`](../specs/command/command-pipeline.sdp.md?plain=1#L116), [`factsCapture`](../specs/command/command-pipeline.sdp.md?plain=1#L122), [`contextCallIsolation`](../specs/command/command-pipeline.sdp.md?plain=1#L124), [`rejectionRelay`](../specs/command/command-pipeline.sdp.md?plain=1#L125), [`readModelWrites`](../specs/command/command-pipeline.sdp.md?plain=1#L126), [`helperGenerics`](../specs/command/command-pipeline.sdp.md?plain=1#L129), [`receiptWrites`](../specs/command/command-pipeline.sdp.md?plain=1#L133), [`limitMutationTimeout`](../specs/command/command-pipeline.sdp.md?plain=1#L136), [`limitTransaction`](../specs/command/command-pipeline.sdp.md?plain=1#L137), [`limitContextCalls`](../specs/command/command-pipeline.sdp.md?plain=1#L138), [`limitTooLarge`](../specs/command/command-pipeline.sdp.md?plain=1#L139), [`fnUtf8Length`](../specs/command/command-pipeline.sdp.md?plain=1#L140), [`limitCallText`](../specs/command/command-pipeline.sdp.md?plain=1#L141), [`errorPath`](../specs/command/command-pipeline.sdp.md?plain=1#L142), [`injectionPointOrder`](../specs/command/command-pipeline.sdp.md?plain=1#L143). + +### 16. The event envelope + +[`spec:context.event-envelope`](../specs/context/event-envelope.sdp.md) · contract · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [The journal](#17-the-journal) by refines. + +**Code.** + +- `impl:context.event-envelope` at [`src/context/envelope.ts` line 2](../../src/context/envelope.ts#L2): the envelope types and validators. Its role is contract. It belongs to `component:platform.context`, in the adapter layer and the platform context. + +**Examples and verifiers.** None is bound to it. + +**Pinned declarations.** 10 Design entries open with a code span. Each pair comes from [`tests/types/pinned-pairs.json`](../../tests/types/pinned-pairs.json) and is the test named by its address in [`tests/types/pinned.test-d.ts`](../../tests/types/pinned.test-d.ts). + +| Design entry | Pair | Compared | Code | Why | +|---|---|---|---|---| +| [`spec:context.event-envelope#design.typeCausedBy`](../specs/context/event-envelope.sdp.md?plain=1#L57) | bound | the type | `CausedBy` in [`src/context/index.ts`](../../src/context/index.ts) | | +| [`spec:context.event-envelope#design.validatorCausedBy`](../specs/context/event-envelope.sdp.md?plain=1#L58) | bound | the type | `causedByValidator` in [`src/context/index.ts`](../../src/context/index.ts) | | +| [`spec:context.event-envelope#design.typeOperationRef`](../specs/context/event-envelope.sdp.md?plain=1#L60) | bound | the type | `OperationRef` in [`src/context/index.ts`](../../src/context/index.ts) | | +| [`spec:context.event-envelope#design.validatorOperationRef`](../specs/context/event-envelope.sdp.md?plain=1#L61) | bound | the type | `operationRefValidator` in [`src/context/index.ts`](../../src/context/index.ts) | | +| [`spec:context.event-envelope#design.typeEventEnvelope`](../specs/context/event-envelope.sdp.md?plain=1#L62) | bound | the type | `EventEnvelope` in [`src/context/index.ts`](../../src/context/index.ts) | | +| [`spec:context.event-envelope#design.validatorEventEnvelope`](../specs/context/event-envelope.sdp.md?plain=1#L63) | bound | the type | `eventEnvelopeValidator` in [`src/context/index.ts`](../../src/context/index.ts) | | +| [`spec:context.event-envelope#design.typeEnvelopeInput`](../specs/context/event-envelope.sdp.md?plain=1#L64) | bound | the type | `EnvelopeInput` in [`src/context/index.ts`](../../src/context/index.ts) | | +| [`spec:context.event-envelope#design.eventIdGeneration`](../specs/context/event-envelope.sdp.md?plain=1#L65) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.event-envelope#design.recordedAtSource`](../specs/context/event-envelope.sdp.md?plain=1#L66) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.event-envelope#design.payloadValidation`](../specs/context/event-envelope.sdp.md?plain=1#L72) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | + +
The 10 declarations as the Spec pins them + +[`spec:context.event-envelope#design.typeCausedBy`](../specs/context/event-envelope.sdp.md?plain=1#L57) + +```ts +type CausedBy = + | { kind: "command"; commandType: string } + | { kind: "event"; tenantId: string; contextId: string; eventId: string } + | { kind: "migration"; migrationName: string } +``` + +[`spec:context.event-envelope#design.validatorCausedBy`](../specs/context/event-envelope.sdp.md?plain=1#L58) + +```ts +const causedByValidator = v.union( + v.object({ kind: v.literal("command"), commandType: v.string() }), + v.object({ + kind: v.literal("event"), + tenantId: v.string(), + contextId: v.string(), + eventId: v.string(), + }), + v.object({ kind: v.literal("migration"), migrationName: v.string() }), +) +``` + +[`spec:context.event-envelope#design.typeOperationRef`](../specs/context/event-envelope.sdp.md?plain=1#L60) + +```ts +type OperationRef = { + operationId: string; + correlationId?: string; + causedBy: CausedBy; +} +``` + +[`spec:context.event-envelope#design.validatorOperationRef`](../specs/context/event-envelope.sdp.md?plain=1#L61) + +```ts +const operationRefValidator = v.object({ + operationId: v.string(), + correlationId: v.optional(v.string()), + causedBy: causedByValidator, +}) +``` + +[`spec:context.event-envelope#design.typeEventEnvelope`](../specs/context/event-envelope.sdp.md?plain=1#L62) + +```ts +type EventEnvelope

= { + eventId: string; + tenantId: string; + contextId: string; + streamType: string; + streamId: string; + streamVersion: number; + eventType: string; + eventSchemaVersion: number; + operationId: string; + correlationId?: string; + causedBy: CausedBy; + actor: Actor; + recordedAt: number; + occurredAt?: number; + payload: P; +} +``` + +[`spec:context.event-envelope#design.validatorEventEnvelope`](../specs/context/event-envelope.sdp.md?plain=1#L63) + +```ts +const eventEnvelopeValidator = v.object({ + eventId: v.string(), + tenantId: v.string(), + contextId: v.string(), + streamType: v.string(), + streamId: v.string(), + streamVersion: v.number(), + eventType: v.string(), + eventSchemaVersion: v.number(), + operationId: v.string(), + correlationId: v.optional(v.string()), + causedBy: causedByValidator, + actor: actorValidator, + recordedAt: v.number(), + occurredAt: v.optional(v.number()), + payload: v.any(), +}) +``` + +[`spec:context.event-envelope#design.typeEnvelopeInput`](../specs/context/event-envelope.sdp.md?plain=1#L64) + +```ts +type EnvelopeInput = { + tenantId: string; + contextId: string; + streamType: string; + streamId: string; + operation: OperationRef; + actor: Actor; + recordedAt: number; +} +``` + +[`spec:context.event-envelope#design.eventIdGeneration`](../specs/context/event-envelope.sdp.md?plain=1#L65) + +```ts +crypto.randomUUID() +``` + +[`spec:context.event-envelope#design.recordedAtSource`](../specs/context/event-envelope.sdp.md?plain=1#L66) + +```ts +Date.now() +``` + +[`spec:context.event-envelope#design.payloadValidation`](../specs/context/event-envelope.sdp.md?plain=1#L72) + +```ts +registration.eventValidators[event.eventType] +``` + +

+ +**Other Design entries.** 6 entries pin no declaration: [`causedByProducers`](../specs/context/event-envelope.sdp.md?plain=1#L59), [`occurredAtSource`](../specs/context/event-envelope.sdp.md?plain=1#L67), [`limitPayloadStructure`](../specs/context/event-envelope.sdp.md?plain=1#L68), [`envelopeTextBounds`](../specs/context/event-envelope.sdp.md?plain=1#L69), [`limitEnvelopeBytes`](../specs/context/event-envelope.sdp.md?plain=1#L70), [`limitEventBytes`](../specs/context/event-envelope.sdp.md?plain=1#L71). + +### 17. The journal + +[`spec:context.journal`](../specs/context/journal.sdp.md) · behavior · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [Write pause](#11-write-pause) by dependsOn. + +**Code.** No implementation anchor binds it. + +**References.** 1 unit references this design. A reference says the code answers to the design without claiming to realize it, and it confers no delivery fact and moves no floor. + +- `impl:context.journal` at [`src/context/journal.ts` line 2](../../src/context/journal.ts#L2): createJournal, load and append. Its role is event-store. It belongs to `component:platform.context`, in the adapter layer and the platform context. + +**Examples.** + +| Example | Tier | Verifier | Title | +|---|---|---|---| +| [`spec:context.journal.rebuild-from-baseline`](../specs/context/journal.rebuild-from-baseline.sdp.md) | native backend | none bound | Rebuild a stream that has a baseline event | + +**Pinned declarations.** 18 Design entries open with a code span. Each pair comes from [`tests/types/pinned-pairs.json`](../../tests/types/pinned-pairs.json) and is the test named by its address in [`tests/types/pinned.test-d.ts`](../../tests/types/pinned.test-d.ts). + +| Design entry | Pair | Compared | Code | Why | +|---|---|---|---|---| +| [`spec:context.journal#design.fnCreateJournal`](../specs/context/journal.sdp.md?plain=1#L89) | bound | the type | `createJournal` in [`src/context/index.ts`](../../src/context/index.ts) | | +| [`spec:context.journal#design.typeJournal`](../specs/context/journal.sdp.md?plain=1#L90) | bound | the type | `Journal` in [`src/context/index.ts`](../../src/context/index.ts) | | +| [`spec:context.journal#design.typeStreamRegistration`](../specs/context/journal.sdp.md?plain=1#L91) | ahead | the type | `StreamRegistration` in [`src/context/index.ts`](../../src/context/index.ts) | baselines and migration, and the deferred streamParts for the derived mapping | +| [`spec:context.journal#design.typeBaselineMigration`](../specs/context/journal.sdp.md?plain=1#L92) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.journal#design.typeStreamMeta`](../specs/context/journal.sdp.md?plain=1#L94) | bound | the type | `StreamMeta` in [`src/context/index.ts`](../../src/context/index.ts) | | +| [`spec:context.journal#design.typeLoadedStream`](../specs/context/journal.sdp.md?plain=1#L95) | ahead | the type | `LoadedStream` in [`src/context/index.ts`](../../src/context/index.ts) | the deferred streamParts | +| [`spec:context.journal#design.fnLoad`](../specs/context/journal.sdp.md?plain=1#L96) | ahead | the type, its ctx parameter erased | `load` in [`src/context/index.ts`](../../src/context/index.ts) | through typeStreamRegistration and typeLoadedStream, its ctx erased | +| [`spec:context.journal#design.fnAppend`](../specs/context/journal.sdp.md?plain=1#L97) | ahead | the type, its ctx parameter erased | `append` in [`src/context/index.ts`](../../src/context/index.ts) | through typeLoadedStream, its ctx erased | +| [`spec:context.journal#design.appendCheck`](../specs/context/journal.sdp.md?plain=1#L98) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.journal#design.limitPayloadBytes`](../specs/context/journal.sdp.md?plain=1#L100) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.journal#design.fnRebuild`](../specs/context/journal.sdp.md?plain=1#L102) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.journal#design.baselineEvent`](../specs/context/journal.sdp.md?plain=1#L105) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.journal#design.fnMigrateToSchemaVersion`](../specs/context/journal.sdp.md?plain=1#L106) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.journal#design.fnDefineMaintenance`](../specs/context/journal.sdp.md?plain=1#L109) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.journal#design.fnWriteBaseline`](../specs/context/journal.sdp.md?plain=1#L110) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.journal#design.writeBaselineBuilder`](../specs/context/journal.sdp.md?plain=1#L111) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.journal#design.driverNotMigrationsComponent`](../specs/context/journal.sdp.md?plain=1#L122) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.journal#design.crossStreamRelation`](../specs/context/journal.sdp.md?plain=1#L125) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | + +
The 18 declarations as the Spec pins them + +[`spec:context.journal#design.fnCreateJournal`](../specs/context/journal.sdp.md?plain=1#L89) + +```ts +createJournal(config: { + contextId: string; + history: "rebuildable" | "auditOnly"; +}): Journal +``` + +[`spec:context.journal#design.typeJournal`](../specs/context/journal.sdp.md?plain=1#L90) + +```ts +type Journal = { + readonly contextId: string; + readonly history: "rebuildable" | "auditOnly"; +} +``` + +[`spec:context.journal#design.typeStreamRegistration`](../specs/context/journal.sdp.md?plain=1#L91) + +```ts +type StreamRegistration = { + decider: Decider; + mapping: StateDocumentMapping; + stateSchemaVersion: number; + eventValidators: Record>; + dto: Validator; + toDto: (state: S, meta: StreamMeta) => Value; + baselineMigrations: readonly BaselineMigration[]; +} +``` + +[`spec:context.journal#design.typeBaselineMigration`](../specs/context/journal.sdp.md?plain=1#L92) + +```ts +type BaselineMigration = { + name: string; + fromSchemaVersion: number; + toSchemaVersion: number; + readsHistory: boolean; + migrate: (saved: unknown, history: readonly EventEnvelope[]) => S; +} +``` + +[`spec:context.journal#design.typeStreamMeta`](../specs/context/journal.sdp.md?plain=1#L94) + +```ts +type StreamMeta = { + tenantId: string; + contextId: string; + streamType: string; + streamId: string; + streamVersion: number; + stateSchemaVersion: number; + deletedAt?: number; + baselineVersion?: number; +} +``` + +[`spec:context.journal#design.typeLoadedStream`](../specs/context/journal.sdp.md?plain=1#L95) + +```ts +type LoadedStream = { + state: S; + version: number; + exists: boolean; + meta: StreamMeta; + rowId?: GenericId<"streams">; + parts?: Record; +} +``` + +[`spec:context.journal#design.fnLoad`](../specs/context/journal.sdp.md?plain=1#L96) + +```ts +load( + ctx: QueryCtx | MutationCtx, + journal: Journal, + registration: StreamRegistration, + tenantId: string, + streamId: string, +): Promise> +``` + +[`spec:context.journal#design.fnAppend`](../specs/context/journal.sdp.md?plain=1#L97) + +```ts +append( + ctx: MutationCtx, + journal: Journal, + loaded: LoadedStream, + envelope: EnvelopeInput, + events: readonly E[], + expectedVersion: number, +): Promise +``` + +[`spec:context.journal#design.appendCheck`](../specs/context/journal.sdp.md?plain=1#L98) + +```ts +append +``` + +[`spec:context.journal#design.limitPayloadBytes`](../specs/context/journal.sdp.md?plain=1#L100) + +```ts +append +``` + +[`spec:context.journal#design.fnRebuild`](../specs/context/journal.sdp.md?plain=1#L102) + +```ts +rebuildStream( + ctx: QueryCtx, + journal: Journal, + registration: StreamRegistration, + tenantId: string, + streamId: string, + maxEvents?: number, +): Promise< + | { + kind: "folded"; + state: S; + version: number; + fromBaseline: number | null; + eventsApplied: number; + } + | { + kind: "refused"; + reason: "tailTooLong" | "awaitingBaseline" | "budgetExhausted"; + tailLength: number; + maxFoldEvents: number; + savedSchemaVersion: number; + currentSchemaVersion: number; + } +> +``` + +[`spec:context.journal#design.baselineEvent`](../specs/context/journal.sdp.md?plain=1#L105) + +```ts +eventType: "baseline" +``` + +[`spec:context.journal#design.fnMigrateToSchemaVersion`](../specs/context/journal.sdp.md?plain=1#L106) + +```ts +migrateToSchemaVersion( + ctx: MutationCtx, + journal: Journal, + registration: StreamRegistration, + loaded: LoadedStream, + envelope: EnvelopeInput, + toSchemaVersion: number, +): Promise> +``` + +[`spec:context.journal#design.fnDefineMaintenance`](../specs/context/journal.sdp.md?plain=1#L109) + +```ts +defineMaintenance( + journal: Journal, + registrations: readonly StreamRegistration[], +): { + writeBaseline: RegisteredMutation< + "public", + WriteBaselineArgs, + WriteBaselineResult + >; +} +``` + +[`spec:context.journal#design.fnWriteBaseline`](../specs/context/journal.sdp.md?plain=1#L110) + +```ts +export const writeBaseline = mutation({ + args: { + tenantId: v.string(), + streamType: v.string(), + migrationName: v.string(), + actor: actorValidator, + operationId: v.string(), + correlationId: v.optional(v.string()), + afterStreamId: v.optional(v.string()), + maxStreams: v.optional(v.number()), + }, + returns: v.object({ + written: v.number(), + skipped: v.number(), + eventsRead: v.number(), + nextAfterStreamId: v.union(v.string(), v.null()), + }), + handler, +}) +``` + +[`spec:context.journal#design.writeBaselineBuilder`](../specs/context/journal.sdp.md?plain=1#L111) + +```ts +writeBaseline +``` + +[`spec:context.journal#design.driverNotMigrationsComponent`](../specs/context/journal.sdp.md?plain=1#L122), as the Spec writes it, which Prettier does not parse + +```ts +@convex-dev/migrations +``` + +[`spec:context.journal#design.crossStreamRelation`](../specs/context/journal.sdp.md?plain=1#L125) + +```ts +operationId +``` + +
+ +**Other Design entries.** 21 entries pin no declaration: [`transactionBoundary`](../specs/context/journal.sdp.md?plain=1#L88), [`migratedStateSource`](../specs/context/journal.sdp.md?plain=1#L93), [`appendWrites`](../specs/context/journal.sdp.md?plain=1#L99), [`limitEnvelopeCheck`](../specs/context/journal.sdp.md?plain=1#L101), [`rebuildReads`](../specs/context/journal.sdp.md?plain=1#L103), [`limitFoldEvents`](../specs/context/journal.sdp.md?plain=1#L104), [`migrateToSchemaVersionCallers`](../specs/context/journal.sdp.md?plain=1#L107), [`baselineRow`](../specs/context/journal.sdp.md?plain=1#L108), [`writeBaselineSequence`](../specs/context/journal.sdp.md?plain=1#L112), [`writeBaselineOperationRef`](../specs/context/journal.sdp.md?plain=1#L113), [`writeBaselineIdempotence`](../specs/context/journal.sdp.md?plain=1#L114), [`limitBaselineBatch`](../specs/context/journal.sdp.md?plain=1#L115), [`baselineDriver`](../specs/context/journal.sdp.md?plain=1#L116), [`limitDriverDeferral`](../specs/context/journal.sdp.md?plain=1#L117), [`sweepVersionBump`](../specs/context/journal.sdp.md?plain=1#L118), [`migrationWindow`](../specs/context/journal.sdp.md?plain=1#L119), [`codeOlderThanData`](../specs/context/journal.sdp.md?plain=1#L120), [`driverTenancy`](../specs/context/journal.sdp.md?plain=1#L121), [`limitRebuildRead`](../specs/context/journal.sdp.md?plain=1#L123), [`noGlobalOrder`](../specs/context/journal.sdp.md?plain=1#L124), [`schemaMigration`](../specs/context/journal.sdp.md?plain=1#L126). + +### 18. The context queries + +[`spec:context.queries`](../specs/context/queries.sdp.md) · contract · stated defined · floor ready · next rung ready, which its floor holds. + +Inside the Pack it rests on [The event envelope](#16-the-event-envelope) by dependsOn and [The journal](#17-the-journal) by dependsOn. + +**Code.** No implementation anchor binds it. + +**References.** 1 unit references this design. A reference says the code answers to the design without claiming to realize it, and it confers no delivery fact and moves no floor. + +- `impl:context.queries` at [`src/context/queries.ts` line 3](../../src/context/queries.ts#L3): defineGet, defineList and boundedPage. Its role is query. It belongs to `component:platform.context`, in the adapter layer and the platform context. + +**Examples and verifiers.** None is bound to it. + +**Pinned declarations.** 18 Design entries open with a code span. Each pair comes from [`tests/types/pinned-pairs.json`](../../tests/types/pinned-pairs.json) and is the test named by its address in [`tests/types/pinned.test-d.ts`](../../tests/types/pinned.test-d.ts). + +| Design entry | Pair | Compared | Code | Why | +|---|---|---|---|---| +| [`spec:context.queries#design.fnDefineGet`](../specs/context/queries.sdp.md?plain=1#L73) | ahead | the type | `defineGet` in [`src/context/index.ts`](../../src/context/index.ts) | through typeStreamRegistration | +| [`spec:context.queries#design.fnDefineList`](../specs/context/queries.sdp.md?plain=1#L74) | ahead | the type | `defineList` in [`src/context/index.ts`](../../src/context/index.ts) | through typeStreamRegistration | +| [`spec:context.queries#design.typeQueryArgs`](../specs/context/queries.sdp.md?plain=1#L75) | bound | the type | `GetArgs` in [`src/context/index.ts`](../../src/context/index.ts) | | +| [`spec:context.queries#design.fnDefineOthers`](../specs/context/queries.sdp.md?plain=1#L76) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.queries#design.typePageLimit`](../specs/context/queries.sdp.md?plain=1#L79) | bound | the type | `PageLimit` in [`src/context/index.ts`](../../src/context/index.ts) | | +| [`spec:context.queries#design.fnBoundedPage`](../specs/context/queries.sdp.md?plain=1#L80) | bound | the type | `boundedPage` in [`src/context/index.ts`](../../src/context/index.ts) | | +| [`spec:context.queries#design.fnHistory`](../specs/context/queries.sdp.md?plain=1#L81) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.queries#design.componentPagination`](../specs/context/queries.sdp.md?plain=1#L82) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.queries#design.fnRebuild`](../specs/context/queries.sdp.md?plain=1#L84) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.queries#design.typeRebuildResult`](../specs/context/queries.sdp.md?plain=1#L85) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.queries#design.validatorRebuildResult`](../specs/context/queries.sdp.md?plain=1#L86) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.queries#design.fnByOperation`](../specs/context/queries.sdp.md?plain=1#L87) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.queries#design.paginationShape`](../specs/context/queries.sdp.md?plain=1#L88) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.queries#design.ordersExample`](../specs/context/queries.sdp.md?plain=1#L89) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.queries#design.parentPassThrough`](../specs/context/queries.sdp.md?plain=1#L90) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.queries#design.parentGet`](../specs/context/queries.sdp.md?plain=1#L91) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.queries#design.dtoMapping`](../specs/context/queries.sdp.md?plain=1#L92) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | +| [`spec:context.queries#design.limitListPage`](../specs/context/queries.sdp.md?plain=1#L95) | no pair | | | `tests/types/pinned-pairs.json` lists no pair for it | + +
The 18 declarations as the Spec pins them + +[`spec:context.queries#design.fnDefineGet`](../specs/context/queries.sdp.md?plain=1#L73) + +```ts +defineGet( + journal: Journal, + registration: StreamRegistration, +): RegisteredQuery<"public", GetArgs, Promise> +``` + +[`spec:context.queries#design.fnDefineList`](../specs/context/queries.sdp.md?plain=1#L74) + +```ts +defineList( + journal: Journal, + registration: StreamRegistration, +): RegisteredQuery<"public", ListArgs, Promise>> +``` + +[`spec:context.queries#design.typeQueryArgs`](../specs/context/queries.sdp.md?plain=1#L75) + +```ts +type GetArgs = { tenantId: string; streamId: string } +``` + +[`spec:context.queries#design.fnDefineOthers`](../specs/context/queries.sdp.md?plain=1#L76) + +```ts +defineHistory(journal, registration) +``` + +[`spec:context.queries#design.typePageLimit`](../specs/context/queries.sdp.md?plain=1#L79) + +```ts +type PageLimit = { items: number; bytes: number } +``` + +[`spec:context.queries#design.fnBoundedPage`](../specs/context/queries.sdp.md?plain=1#L80) + +```ts +boundedPage( + opts: PaginationOptions, + limit: PageLimit, +): PaginationOptions +``` + +[`spec:context.queries#design.fnHistory`](../specs/context/queries.sdp.md?plain=1#L81) + +```ts +export const history = query({ + args: { + tenantId: v.string(), + streamId: v.string(), + fromVersion: v.optional(v.number()), + paginationOpts: paginationOptsValidator, + }, + returns: paginationResultValidator(eventEnvelopeValidator), + handler, +}) +``` + +[`spec:context.queries#design.componentPagination`](../specs/context/queries.sdp.md?plain=1#L82) + +```ts +paginator +``` + +[`spec:context.queries#design.fnRebuild`](../specs/context/queries.sdp.md?plain=1#L84) + +```ts +export const rebuild = query({ + args: { + tenantId: v.string(), + streamId: v.string(), + maxEvents: v.optional(v.number()), + }, + returns: rebuildResultValidator, + handler, +}) +``` + +[`spec:context.queries#design.typeRebuildResult`](../specs/context/queries.sdp.md?plain=1#L85) + +```ts +type RebuildResult = + | { + kind: "folded"; + state: Value; + version: number; + fromBaseline: number | null; + eventsApplied: number; + savedVersion: number; + equal: boolean; + } + | { + kind: "refused"; + reason: "tailTooLong" | "awaitingBaseline" | "budgetExhausted"; + tailLength: number; + maxFoldEvents: number; + savedVersion: number; + savedSchemaVersion: number; + currentSchemaVersion: number; + } +``` + +[`spec:context.queries#design.validatorRebuildResult`](../specs/context/queries.sdp.md?plain=1#L86) + +```ts +const rebuildResultValidator = v.union( + v.object({ + kind: v.literal("folded"), + state: v.any(), + version: v.number(), + fromBaseline: v.union(v.number(), v.null()), + eventsApplied: v.number(), + savedVersion: v.number(), + equal: v.boolean(), + }), + v.object({ + kind: v.literal("refused"), + reason: v.union( + v.literal("tailTooLong"), + v.literal("awaitingBaseline"), + v.literal("budgetExhausted"), + ), + tailLength: v.number(), + maxFoldEvents: v.number(), + savedVersion: v.number(), + savedSchemaVersion: v.number(), + currentSchemaVersion: v.number(), + }), +) +``` + +[`spec:context.queries#design.fnByOperation`](../specs/context/queries.sdp.md?plain=1#L87) + +```ts +export const byOperation = query({ + args: { tenantId: v.string(), operationId: v.string() }, + returns: v.array(eventEnvelopeValidator), + handler, +}) +``` + +[`spec:context.queries#design.paginationShape`](../specs/context/queries.sdp.md?plain=1#L88) + +```ts +paginationOptsValidator +``` + +[`spec:context.queries#design.ordersExample`](../specs/context/queries.sdp.md?plain=1#L89) + +```ts +export const get = defineGet(journal, orderStream) +``` + +[`spec:context.queries#design.parentPassThrough`](../specs/context/queries.sdp.md?plain=1#L90) + +```ts +export const listOrders = query({ + args: { tenantId: v.string(), paginationOpts: paginationOptsValidator }, + returns: paginationResultValidator(orderDtoValidator), + handler: async (ctx, args): Promise> => { + await authorizeQuery(ctx, { + name: "listOrders", + tenantId: args.tenantId, + permission: "orders.read", + }); + return (await ctx.runQuery( + components.orders.queries.order.list, + args, + )) as PaginationResult; + }, +}) +``` + +[`spec:context.queries#design.parentGet`](../specs/context/queries.sdp.md?plain=1#L91) + +```ts +export const getOrder = query({ + args: { tenantId: v.string(), orderId: v.string() }, + returns: v.union(orderDtoValidator, v.null()), + handler: async (ctx, { tenantId, orderId }): Promise => { + await authorizeQuery(ctx, { + name: "getOrder", + tenantId, + permission: "orders.read", + }); + return (await ctx.runQuery(components.orders.queries.order.get, { + tenantId, + streamId: orderId, + })) as OrderDto | null; + }, +}) +``` + +[`spec:context.queries#design.dtoMapping`](../specs/context/queries.sdp.md?plain=1#L92) + +```ts +registration.toDto(state, meta) +``` + +[`spec:context.queries#design.limitListPage`](../specs/context/queries.sdp.md?plain=1#L95), as the Spec writes it, which Prettier does not parse + +```ts +limitListPage(budgetBytes) = max(1, min(100, floor(4 MiB / budgetBytes))) +``` + +
+ +**Other Design entries.** 11 entries pin no declaration: [`fnGet`](../specs/context/queries.sdp.md?plain=1#L77), [`fnList`](../specs/context/queries.sdp.md?plain=1#L78), [`pagingByCursor`](../specs/context/queries.sdp.md?plain=1#L83), [`tenantNarrowing`](../specs/context/queries.sdp.md?plain=1#L93), [`reactivityAssumption`](../specs/context/queries.sdp.md?plain=1#L94), [`limitListBytes`](../specs/context/queries.sdp.md?plain=1#L96), [`limitHistoryPage`](../specs/context/queries.sdp.md?plain=1#L97), [`limitHistoryBytes`](../specs/context/queries.sdp.md?plain=1#L98), [`limitRebuildRead`](../specs/context/queries.sdp.md?plain=1#L99), [`observedFullLastPage`](../specs/context/queries.sdp.md?plain=1#L100), [`registeredFunctions`](../specs/context/queries.sdp.md?plain=1#L101). + +### 19. Rebuild online by default, pause writes only for history-dependent views + +[`spec:decisions.d09-rebuild-online-by-default`](../specs/decisions/d09-rebuild-online-by-default.sdp.md) · decision · stated defined · floor ready · next rung ready, which its floor holds. + +**Code.** No implementation anchor binds it. + +**Examples and verifiers.** None is bound to it. + +**Pinned declarations.** It pins none. + +## Pairs without a member's declaration + +1 pair in `tests/types/pinned-pairs.json` names no pinned declaration of a member. + +| Address | Pair | Compared | Code | Why | +|---|---|---|---|---| +| [`spec:application.aggregate-projection#design.tableProjectionMarkers`](../specs/application/aggregate-projection.sdp.md?plain=1#L56) | not compared | nothing | none | The pin is outside pack:history-view and is not emitted; the aggregate form's table, outside the Pack, while readModelTables still holds it | diff --git a/design/reviews/consensus-ledger.json b/design/reviews/consensus-ledger.json index fd3419d..f2c7896 100644 --- a/design/reviews/consensus-ledger.json +++ b/design/reviews/consensus-ledger.json @@ -504,7 +504,8 @@ "severity": "major", "files": [ "specs/application/rebuild.sdp.md", - "specs/application/write-pause.sdp.md" + "specs/application/write-pause.sdp.md", + "specs/application/history-rebuild.sdp.md" ], "issue": "rebuild's write-pause flow says \"the operator closes the maintenance gate for the read model's scope before `startGeneration`\", but the gate can only let a background writer through by its generation: write-pause `closeGate` takes an optional `generationId`, `assertWritable(ctx, scopes, allowGeneration)` throws unless \"the closed entry's `generationId` equals `allowGeneration`\", and rebuild step2 returns without writing \"when a closed scope covers this read model and the gate was not closed for this generation\". A gate closed before the generation row exists cannot name it, so the paused backfill's own batches refuse themselves and the L2-6 examples cannot reach `verified`.", "fix": "Reverse or fuse the order: `startGeneration` with `pauseRequired: true` inserts the generation row and calls the close in the same mutation (or patches the already-closed scope's `generationId` to the new row), and the rebuild flow reads \"startGeneration closes the gate for the read model's scope and records its own generation on the closed entry\". Update the write-pause Contract bullet and the L2-6 examples' first verification bullet to match.", @@ -1616,8 +1617,10 @@ "specs/command/command-pipeline.sdp.md", "specs/context/persistence-adapter.sdp.md", "specs/application/orders-inventory-example.sdp.md", - "specs/application/rebuild.write-pause-rebuild-interrupt.sdp.md", - "specs/application/rebuild.write-pause-rebuild-abort.sdp.md" + "specs/application/history-rebuild.write-pause-rebuild-interrupt.sdp.md", + "specs/application/history-rebuild.write-pause-rebuild-abort.sdp.md", + "specs/application/history-projection.sdp.md", + "specs/application/history-rebuild.sdp.md" ], "issue": "The only projection forms are `Projection⟨D, Row⟩.project(tenantId, dto, versions)` from one current DTO and `AggregateProjection.contribute(dto)`. The write pause exists for 'read models that depend on history across several streams', and the contract says a cross-context row 'holds one StreamVersion per context', yet no form takes history, a prior row, or two DTOs. On the live path step 9 calls `applyProjection` once per `streams` entry with that entry's DTO only, and the parent never receives the appended events (`StreamDto` is dto, version, appended, created). On the backfill the pause path reuses the online batch steps, one `list` page of current DTOs. So a history-dependent view cannot be written on either path, the cross-context read model the D8 table allows cannot be produced, and the L2-6 examples exercise a per-entity projection under a gate (they assert only that `PlaceOrder` is refused), which passes trivially; the example domain defines no history-dependent view at all, so the first experiment cannot run L2-6 as the doc's row reads.", "fix": "Either add the missing forms under E-9: `HistoryProjection⟨Row⟩ { project(tenantId, prior: Row | null, events: EventEnvelope[], versions) }` fed on the live path by the events the operation appended (add `events: EventEnvelope[]` to `StreamDto`, bounded by the 16 KiB payload bound times `appended`) and on the backfill, under the pause, by `history` pages per enumerated stream; and `CompositeProjection` taking the DTOs of every `streams` entry of one command for a cross-context row, with the backfill composing from one `list` DTO plus bounded `get` reads of the other context. Then name one history-dependent view in orders-inventory-example (for example a per-stock-item allocation history across orders) and make both L2-6 examples build it. Or, narrow the claim explicitly: state in rebuild and projection-contract that Layer 2 pins only per-entity and aggregate projections, that the write-pause path is designed for cross-context current-state rows needing a consistent cut, record the history-projection input shape as a [non-blocking] E-9 open question deferred to Layer 6's trigger, and retitle the L2-6 examples to the view they actually build.", @@ -1633,7 +1636,8 @@ "specs/application/write-pause.sdp.md", "specs/application/rebuild.sdp.md", "specs/command/command-pipeline.sdp.md", - "specs/command/command-declaration.sdp.md" + "specs/command/command-declaration.sdp.md", + "specs/application/history-rebuild.sdp.md" ], "issue": "The pause's scope for a rebuild is `read-model:`, and a use case is in scope only when 'a read model it maintains is closed'. A consistent cut needs every writer of the view's source streams held, but a command that writes a source stream without maintaining the paused view (for example a `ReceiveStock` command that touches Inventory and maintains only the stock summary, while the paused history view depends on Inventory events) passes `assertWritable`, changes the source, and the flow's claim that 'Backfill under the pause enumerates a consistent cut because no source changes' is false. The declaration carries no `contexts`/`sources` field, so a source-based scope cannot even be derived from it as pinned.", "fix": "Key the closed entry by the paused view's sources: `closed[].sources: { contextId; streamType }[]` taken from each `ReadModelBinding.source` of the read model, and have `scopesOfUseCase` also emit `source::` for every stream type the command writes. That requires the declaration to name what it writes: add `writes: { contextId: string; streamType: string }[]` to `CommandDeclaration` under E-38 (the registration check can assert it against the executor's `streams` entries at test time). Until then the cheap safe default is for `startGeneration` with `pauseRequired` to close the tenant scope (or `all` for an all-tenants generation) rather than the read-model scope, and say so in rebuild's flow and write-pause's Contract.", @@ -1839,7 +1843,8 @@ "severity": "minor", "files": [ "specs/application/projection-contract.sdp.md", - "specs/application/generation-registry.sdp.md" + "specs/application/generation-registry.sdp.md", + "specs/application/aggregate-projection.sdp.md" ], "issue": "`AggregateProjection⟨D⟩` and `applyAggregate` are pinned and the pipeline's step 9 and the rebuild's step 4 call them, but the aggregate row itself is not: which field holds the sum, what `key` and `sourceVersions` mean for a row that derives from N entities (the row conventions say `sourceVersions` holds 'one per source stream', unbounded for an aggregate), and how `ApplyMode` and `ApplyResult` apply when the marker rather than the row's version decides. D9 names counts and sums explicitly, so an implementer of the transactional profile has to invent the row.", "fix": "Add an `aggregateRowConventions` bullet: `tenantId`, `generation`, `key` = `aggregateKeyOf`, `projectionVersion`, `value: number`, `entityCount: number`, and no `sourceVersions` (the markers carry them); state that `applyAggregate` ignores `mode` except that `backfill` never creates a marker the live path already wrote, returns `updated` or `unchanged`, and that the marker's `sourceVersions` are compared like a row's so a stale DTO never adjusts the sum.", @@ -1930,7 +1935,9 @@ "specs/application/projection-contract.sdp.md", "specs/application/rebuild.sdp.md", "specs/application/orders-inventory-example.sdp.md", - "specs/application/rebuild.write-pause-rebuild-interrupt.sdp.md" + "specs/application/history-rebuild.write-pause-rebuild-interrupt.sdp.md", + "specs/application/history-projection.sdp.md", + "specs/application/history-rebuild.sdp.md" ], "issue": "A `HistoryProjection` folds events of several streams into one row, and the two paths feed it in different orders. On the live path step 9 calls `applyHistoryProjection` per `streams` entry in the executor's order, command after command, so an order's row sees `OrderPlaced`, its `StockAllocated` events, later `OrderCancelled`, then `AllocationReleased`. On the backfill the rebuild pages one stream's whole history at a time (rebuild flow 'each history batch takes one stream from the enumeration'), so the same row sees every Orders event first and the stock events after, or the reverse. The contract asks only that `project` be 'deterministic in the prior row and the events' with 'version order within each stream'. It never says the fold must give the same row whatever the interleaving across streams, and D2 gives no order across streams to appeal to. A natural projection, where `OrderPlaced` creates the row and a stock event updates an existing one, backfills to a different row than it wrote live, against the contract's own value line that 'a rebuild cannot disagree with a live update'. The doc sends a consumer whose result depends on order to Layer 6's ordered consumer; Layer 2 does not say its history view must not be one. Deletion makes it worse: `null` from `project` deletes the row, and with it the `sourceVersions` that are the fold's only memory. A later event of the same key from another stream, such as `AllocationReleased` after `OrderCancelled` returned `null`, meets no prior row and recreates one, on either path.", "fix": "Add to projection-contract under E-9: a history projection's fold is independent of the order in which different streams' events arrive; only the order within one stream is given (D2), and a view whose meaning depends on cross-stream order waits for the ordered consumer of D18. Add to `determinismCheck` a test that folds one key's events in two interleavings and compares the rows, and have the L2-6 example compare a backfilled row with the live-written one. Rule on deletion: either a history row is never deleted once written and carries its own ended state, or the highest folded version per stream moves off the row into a marker, like `projectionMarkers`, that outlives it. State in orders-inventory-example that the order allocation history fold is written to hold under any interleaving, including a stock event arriving before `OrderPlaced`.", @@ -1947,7 +1954,8 @@ "files": [ "specs/application/rebuild.sdp.md", "specs/application/generation-registry.sdp.md", - "specs/application/orders-inventory-example.sdp.md" + "specs/application/orders-inventory-example.sdp.md", + "specs/application/history-rebuild.sdp.md" ], "issue": "Two parts of the paused rebuild cannot be built as pinned. First, a history view has several sources, the order stream type and the stock item stream type in the example, but `batchCursorValidator` is `{ tenantId, pageCursor, streamId?, eventCursor? }` with no field for the source being enumerated, and `fnHistoryPageCall` names one component query. `historyStep3` says 'it calls `list` with `numItems` 1 from the page cursor to take the next stream' without saying of which context and stream type, so the batch cannot tell where it is among the bindings. Second, `historyVerify` checks one thing per stream: it reads the stream's last event, 'finds that event's row by `keyOf`', and counts a miss when the row is absent or records a lower version. `keyOf` may return `null`, and for many streams the last event's key is `null`: a stock item that was received and never allocated, and every stream whose last event is a `baseline` after a meaning migration. The Spec does not say what verify does then. Read as a miss, the repair refolds the stream, changes nothing, and the next pass counts the same miss, so the generation never reaches `verified` and, since 'writes reopen only after verification', the sources stay closed until an operator aborts. Read as a skip, the stream is unverified. Either way the check proves only that the last event of a stream was folded, not the earlier ones that belong to other keys.", "fix": "Add the source to the cursor, an index into the read model's source list, and iterate tenant, then source, then stream, then event page; say so in `tenantIteration` and `historyStep3`. Replace `historyVerify` with a second pass of the same history batches that counts the events `applyHistoryProjection` did not drop: under the pause nothing changes, so a covered generation folds zero events and a miss is repaired by the pass that found it. The state moves to `verified` when a full pass folds none. That needs no row lookup by key and handles a `null` key by construction. State the cost, one more read of every source's history under the pause.", @@ -1963,8 +1971,8 @@ "severity": "major", "files": [ "specs/application/rebuild.sdp.md", - "specs/application/projection-contract.sdp.md", - "specs/application/write-pause.sdp.md" + "specs/application/write-pause.sdp.md", + "specs/application/history-projection.sdp.md" ], "issue": "`rollbackGeneration` takes any read model: it sets the retired generation to `verifying`, 'live commands write it again from that commit', and a verify pass repairs it. For a history view nothing closes the sources: write-pause says no function other than `startGeneration` closes a source scope for a rebuild. So after a rollback the next live command folds its new events onto a retired row that never saw the events between the switch and the rollback, and `applyHistoryProjection` raises that row's `sourceVersions` to the new event's version. The missed events are now 'not above the recorded version' and are dropped by every later fold, and `historyVerify`, which only compares the last event's version, passes. The rolled-back generation becomes active with holes in it and no check says so. projection-contract's claim that 'the live path never meets a non-active generation of a history view because its building generation exists only while its source scopes are closed' is false on this path, and rebuild's own `rebuildClass` bullet says why it matters: 'a live fold onto a row backfill has not reached cannot be told apart from a complete one'.", "fix": "Rule one way under E-8. Either `rollbackGeneration` refuses a `pauseRequired` generation with a plain error, and rolling a history view back means a new paused generation at the old projection version; or it closes every source scope of the view in the same mutation, naming the reopened generation, runs its catch-up pass under the pause, and the scopes reopen only after the switch, exactly as a first build does. Correct the projection-contract sentence to match, and add the rollback of the history view, or its refusal, to the L2-6 examples.", @@ -2110,7 +2118,8 @@ "specs/command/command-pipeline.sdp.md", "specs/application/projection-contract.sdp.md", "specs/application/first-experiment.sdp.md", - "specs/application/orders-inventory-example.sdp.md" + "specs/application/orders-inventory-example.sdp.md", + "specs/application/history-projection.sdp.md" ], "issue": "Step 9 calls `applyHistoryProjection` 'for each `streams` entry', and that helper reads and replaces the row once per key per call. `PlaceOrder` at 100 lines returns 101 entries that all key to the same order, so the order allocation history row is read and replaced 101 times in one mutation, about 200 operations and up to 3 MiB written for one row. The cost is recorded nowhere: `limitHistoryFold` counts rows per entry, E-45's four read-model rows per command is silent on repeated writes, and `limitExpectedDocuments` omits the view. The row itself, 100 lines plus 101 `sourceVersions` entries, is over the default `rowBudgetBytes` of 16 KiB and the table bullet declares no budget. And the Specs do not say whether the measured `PlaceOrder` binds the view: it 'exists in the experiment for Sc L2-6, not as an essential read', yet a binding on the declaration is written by every command.", "fix": "Have step 9 gather the events of every matching entry of one command per history projection and call `applyHistoryProjection` once, so each key costs one read and one write; say so in `limitHistoryFold`. Declare `rowBudgetBytes` for `orderAllocationHistory`, 64 KiB at the provisional maximum. State in first-experiment whether the Sc L2-3 runs measure `PlaceOrder` with the view bound, and add its row to `limitExpectedDocuments` if they do.", @@ -2125,7 +2134,8 @@ "lens": "architecture", "severity": "minor", "files": [ - "specs/application/rebuild.sdp.md" + "specs/application/rebuild.sdp.md", + "specs/application/history-rebuild.sdp.md" ], "issue": "Under the pause 'each history batch takes one stream from the enumeration' and reads one `history` page of it, so a stream with two events costs a whole scheduled mutation, a `list` call of one item and a `history` call. The pause closes the view's sources for every tenant of the deployment, because a generation spans all tenants, so the outage of those writers lasts as long as the number of source streams times one batch. `limitHistoryBatch` bounds events per batch at 128 by default and the batch uses a fraction of it for short streams. The duration is an unrecorded cost of the mechanism, and a cheaper batch passes the same scenario.", "fix": "Let a history batch take streams from the enumeration until its event bound is spent, with the stream and event cursor saved as now, and state the pause's expected duration as a risk on rebuild: source streams and events divided by the batch bound, times one mutation. Point at the registry's per-tenant alternative as the way to shorten the outage per tenant.", @@ -2397,7 +2407,8 @@ "specs/command/command-declaration.sdp.md", "specs/effects/retry-ownership.sdp.md", "specs/effects/external-effects.sdp.md", - "specs/obligations/obligation-module.sdp.md" + "specs/obligations/obligation-module.sdp.md", + "specs/application/history-rebuild.sdp.md" ], "issue": "Rubric item 2 fails in six places where a bullet's citation names only doc tokens and the doc does not say what the bullet says. The content is a registered extension elsewhere in five of them. (a) projection-contract: 'a new version is a new generation, never an in-place rewrite (D8, D9)'; rebuild marks the same sentence '[extension] ... (E-8, D8)'. D8 says only 'named, versioned and deterministic'. (b) rebuild: 'A refused write under the pause is a transient refusal, retryable and never a stored rejection (D4, D9)'; D4 covers 'rate or capacity', and outcome-boundary's E-35 question says 'the doc does not say how a refused writer is answered'. (c) parent-use-cases: 'the use case either carries it as the use case's business failure ... or throws a rejection so nothing commits; the choice is declared per use case (D4)'; that is E-44, whose own question says it overrides D4's 'belongs in history'. (d) command-declaration: 'A command is bound by two export lines ... and no other binding exists (D12, D6)'; that is E-7. (e) external-effects: 'Every reconcile is a claimed attempt ... (D14, Sc L3-6)'; that is E-53. (f) retry-ownership has no open question and no E-number, yet rules that Workpool 'is constructed with `retryActionsByDefault: false` and each enqueue passes `retry: false`', that passed ownership records '`retryOwner: \"workpool\"`' and that Workpool attempts are 'read from the pool's `onComplete` result and `status`', all cited '(D15, S12)'. D15 says only that Workpool 'runs with its own retries off, or ownership passes to it deliberately and its attempts show the same way'. obligation-module's `deferred` bullet says 'the Workpool binding' is 'written by the build that first needs them', so retry-ownership also pre-writes what its parent defers, against decision method rule 4.", "fix": "Add the E-number to (a) to (e) and start each bullet with [extension], matching the Spec that owns the extension: E-8, E-35, E-44, E-7, E-53. For (f), on `spec:effects.retry-ownership`: either reduce the two Workpool rules to D15's sentence and move the option names to a `deferred:` bullet, or mark them [extension] under a new E-number of Package E's range with an open question and a README row, and make obligation-module's `deferred` bullet say what is and is not deferred.", @@ -2586,8 +2597,8 @@ "lens": "completeness", "severity": "major", "files": [ - "specs/application/projection-contract.sdp.md", - "specs/application/generation-registry.sdp.md" + "specs/application/generation-registry.sdp.md", + "specs/application/aggregate-projection.sdp.md" ], "issue": "The round-2 fix r2-completeness-aggregate-row-shape-unpinned pinned `applyAggregate` in three steps that are wrong when an entity moves from one aggregate key to another, which is the case the Spec's own sample table is built for: `orderTotals` is 'an aggregate of order totals per status, where `key` is the status'. aggregateStep2 computes 'the difference is `contribution` minus the marker's recorded contribution'. aggregateStep3 'reads the aggregate row by `by_key` with `aggregateKeyOf(tenantId, dto)`, inserts it with `value` equal to the difference and `entityCount` 1 when missing, else adds the difference to `value` and adjusts `entityCount` by one for a new marker and by minus one for a removed one'. When an order goes from placed to cancelled its marker exists with `aggregateKey` placed and the same contribution, so the difference is zero: the cancelled row gains nothing and no count, or is inserted with `value` 0 and `entityCount` 1, and the placed row keeps the order's total and count for ever. The marker stores `aggregateKey`, and no step reads it. Two smaller holes sit in the same steps: a missing aggregate row is inserted with `entityCount` 1 even when the contribution is `null`, and the input's `entityKey` has no stated source, since `AggregateProjection` has no `entityKeyOf` and step 9 is said to call the helper 'with the entry's DTO and version'.", "fix": "In `spec:application.projection-contract` split aggregateStep3 on the marker's `aggregateKey`: when it differs from `aggregateKeyOf(tenantId, dto)`, read the old key's row, subtract the recorded contribution and one from `entityCount`, then add the whole new contribution and one to the new key's row; when it is the same, add the difference as now; never insert a row for a `null` contribution. State the cost, a second row read and write on a key change, and count it against E-45's four rows per command. Pin `entityKey` as the entry's stream identity, `streamType` and `streamId` from `dto.version`, in `fnApplyAggregate` and in the pipeline's step 9. Add a key change to `determinismCheck`.", @@ -2810,12 +2821,12 @@ "specs/context/journal.sdp.md", "specs/context/tables.sdp.md", "specs/application/parent-use-cases.sdp.md", - "specs/application/projection-contract.sdp.md", "specs/application/generation-registry.sdp.md", "specs/application/write-pause.sdp.md", "specs/application/first-experiment.sdp.md", "specs/kernel/domain-kernel.sdp.md", - "specs/obligations/obligation-module.sdp.md" + "specs/obligations/obligation-module.sdp.md", + "specs/application/history-projection.sdp.md" ], "issue": "Small things in Layers 0 to 2 that are named and not pinned, each a short guess for the implementer. (1) `Bounds = { maxItems?: number; maxBytes?: number }` is checked 'when the input's items exceed `bounds.maxItems`', and nothing says which part of an input is its items; the L2-3 maximum example needs it to mean `lines`. (2) The pipeline lists parse as step 1 and authenticate as step 2, and parent-use-cases says the bound is 'checked by the pipeline's step 1 before authentication', but `runPipeline` takes a `PipelineCall` whose `actor` is required and the public helper 'establishes the actor ... and calls `runPipeline`', so for the public entry authentication runs first. (3) outcome-boundary says 'a code outside both is a technical failure by definition', while `normalizeThrown(error, commandType)` rethrows any bare kernel `Rejection` as a rejection and never sees the declaration's `rejections`; `irreversible` likewise 'requires' a tombstone with no check named. (4) receipt-table says 'a declaration may set a finite horizon' for `tombstoneHorizon`, and `Retention = { window; afterExpiry }` has no field for it. (5) `limitRequestKey` says a longer key 'is rejection `invalidInput` at step 1'; Step 1 and `errorCodeInvalidInput` do not mention it. (6) `canonicalJson` is defined for JSON values only; a Convex input may hold `v.int64()` or `v.bytes()`. (7) `get` returns `null` for a deleted subject 'unless the DTO policy shows deleted subjects', and no field declares that policy; `list` takes `includeDeleted` and its handler text never uses it. (8) `Journal` is the return type of `createJournal` with no members; `streamParts.streamVersion` has no rule for what is written to it or who reads it. (9) `carryOrReject(outcome, policy)` throws 'through the outcome boundary's `reject`', which requires `commandType`, and takes its `message` from nowhere. (10) `applyHistoryProjection` takes `mode: ApplyMode` and no step reads it; the `events` it is fed include `baseline` events, and the contract does not say `keyOf` must return `null` for them. (11) Validators and types used once and never given: `generationDoc`, `gateDoc`, `AuditRecordInput`, `DiagnosticSink`, `consoleSink`, and in Layer 3 `callInputValidator`, `obligationViewValidator`, `dispatchResultValidator`. (12) One bound has two names, `limitGrantsRead` in actor-and-scope and `limitGrantsPerPrincipal` in tenancy-and-authority, and the handler registry has two shapes, `HandlerRegistry` mapping to a `FunctionReference` in obligation-module while `createStep3` reads `entry.maxAttempts` from the wrapper's `HandlerEntry`. (13) `convexTypeImport` says a value import of `convex/values` 'is a type error'; `verbatimModuleSyntax` rejects only a type imported without `import type`, so `import { v }` compiles and only the dependency check catches it. (14) `limitExpectedDocuments` lists the reads of one `PlaceOrder` and leaves out the grants read of step 4. (15) The experiment's examples rely on 'the production `auth.config.ts`' and no Spec names its provider or issuer.", "fix": "One line each in the owning Spec: add `itemsOf?: (input: I) => number` to `Bounds` in `spec:command.command-declaration`, or say `maxItems` counts the elements of the input's one top-level array; state in the pipeline that the public entry authenticates before step 1's bound and refinement checks, and correct the parent-use-cases sentence; pass `decl.rejections` to `normalizeThrown` and rethrow an unlisted code as a plain error, or drop the 'technical failure by definition' sentence, and name the registration check as the place `irreversible` is checked; add `tombstoneHorizon?: number` to `Retention`; add the key-length check to Step 1; define `canonicalJson` over `convexToJson(value)`; add `showDeleted?: boolean` to `StreamRegistration` or drop the clause, and say `list` filters deleted rows after the page is read unless `includeDeleted`; give `Journal` its two fields and `streamParts.streamVersion` its rule or remove the field; let `carryOrReject` throw the bare kernel `Rejection` and let `normalizeThrown` add the command type; say `mode` is ignored by `applyHistoryProjection` and that `keyOf` returns `null` for `baseline`; define the listed validators beside their one use; keep one name for the grants bound and one registry type; reword `convexTypeImport`; add the grants read to `limitExpectedDocuments`; name the experiment's auth provider in `spec:application.first-experiment`.", @@ -3109,10 +3120,10 @@ "lens": "sdp", "severity": "major", "files": [ - "specs/application/projection-contract.sdp.md", "specs/command/command-pipeline.sdp.md", "specs/application/rebuild.sdp.md", - "specs/application/generation-registry.sdp.md" + "specs/application/generation-registry.sdp.md", + "specs/application/aggregate-projection.sdp.md" ], "issue": "`applyAggregate` takes `input: { tenantId: string; entityKey: string; dto: D; versions: StreamVersion[]; mode: ApplyMode; generations: number[] }` and reads the marker 'by `by_entity` with the tenant, read model, generation and `entityKey`'. No Spec says where `entityKey` comes from. `AggregateProjection⟨D⟩` has `aggregateKeyOf` and `contribute` and no entity key function; the pipeline's step 9 says only that it calls `applyAggregate` 'with the entry's DTO and version'; the rebuild's `step4` says 'for each item it calls `applyProjection` or `applyAggregate` in backfill mode'. The term `entityKey` appears in the corpus only in that signature and in the marker table. The marker exists so that 'live commands and backfill never count one entity twice', which holds only if the live caller and the backfill caller, pinned in two different Specs, derive the same key; a stream ID alone is also not unique when an aggregate is bound to two stream types.", "fix": "In `spec:application.projection-contract` add `entityKeyOf(tenantId: string, dto: D): string` to `AggregateProjection⟨D⟩` and have `applyAggregate` compute it, dropping `entityKey` from its input; or state once that `entityKey` is `contextId + \":\" + streamType + \":\" + streamId` of `dto.version`. Name the rule in the pipeline's step 9 and the rebuild's `step4`.", @@ -3144,7 +3155,8 @@ "lens": "sdp", "severity": "minor", "files": [ - "specs/application/rebuild.sdp.md" + "specs/application/rebuild.sdp.md", + "specs/application/history-rebuild.sdp.md" ], "issue": "The batch size of a history view is computed two ways in one Spec. `fnStartGeneration` computes the row's `batchSize` as `min(args.batchSize ?? 100, floor(4 MiB / (2 * rowBudgetBytes)))` for every read model, which is 100 at the default 16 KiB row budget. `limitHistoryBatch` says `min(500, floor(4 MiB / (2 * rowBudgetBytes)))` events per history batch, '128 at the default 16 KiB row budget', and that 'the count is stored on the generation row as its `batchSize`'. The stored number for a history view at defaults is 100 by the first bullet and 128 by the second, and the cap is 100 or 500.", "fix": "In `spec:application.rebuild` make `fnStartGeneration` state both defaults: `args.batchSize ?? 100` entities for a `Projection` or `AggregateProjection`, `args.batchSize ?? 500` events for a `HistoryProjection`, each capped by `floor(4 MiB / (2 * rowBudgetBytes))`; or lower `limitHistoryBatch` to the same 100.", @@ -3430,7 +3442,8 @@ "specs/application/projection-contract.sdp.md", "specs/command/receipt-table.sdp.md", "specs/obligations/local-reaction-wrapper.sdp.md", - "specs/effects/claim-call-settle.sdp.md" + "specs/effects/claim-call-settle.sdp.md", + "specs/application/history-projection.sdp.md" ], "issue": "Every stated `dependsOn` is used by its Spec's text, so that half of rubric 16 holds. The other direction of PLAN 3.3, 'Contract: ... `dependsOn` the contracts it composes', has gaps where a contract takes its types from another contract and carries no edge, so a query for what depends on a contract misses its consumers. persistence-adapter takes `StreamRegistration`, `load`, `append` and `migrateToSchemaVersion` from `spec:context.journal` and depends on the decider contract, the mapping and the tables only. tables uses `causedByValidator` and `actorValidator` 'of `spec:context.event-envelope`' with no edge to it. queries uses `tenantScopeValidator` and `authorize` of `spec:command.actor-and-scope` and `eventEnvelopeValidator`, with no `dependsOn` at all. projection-contract takes `EventEnvelope` from the envelope and the generations from the registry, with no `dependsOn`. receipt-table imports the kernel's `affectedRefValidator` and `streamVersionValidator`. local-reaction-wrapper and claim-call-settle call `authorize` and store `Authority` of `spec:command.actor-and-scope`. None of these edges would point from a lower layer to a higher one.", "fix": "Add the edges: persistence-adapter to `spec:context.journal`; tables to `spec:context.event-envelope`; queries to `spec:command.actor-and-scope` and `spec:context.event-envelope`; projection-contract to `spec:context.event-envelope` and `spec:application.generation-registry`; receipt-table to `spec:kernel.outcome-model`; local-reaction-wrapper and claim-call-settle to `spec:command.actor-and-scope`. Check for a cycle before adding journal to adapter, since the journal names the adapter only in prose.", @@ -3531,9 +3544,9 @@ "lens": "sdp", "severity": "minor", "files": [ - "specs/application/projection-contract.sdp.md", "specs/command/command-pipeline.sdp.md", - "specs/context/persistence-adapter.sdp.md" + "specs/context/persistence-adapter.sdp.md", + "specs/application/history-projection.sdp.md" ], "issue": "Lead 2, read for this lens: three small mismatches around the newest form. First, `applyHistoryProjection` takes `input: { tenantId: string; events: EventEnvelope[]; mode: ApplyMode; generations: number[] }` and its description never reads `mode`; for `applyAggregate` the contract says '`mode` is accepted for symmetry with `applyProjection` and read by nothing', for the history helper it says nothing, while callers pass 'backfill mode' and the live modes. Second, the pipeline's step 9 calls it 'with the entry's `events` and version', and the helper has no version argument; it reads `streamVersion` off each envelope. Third, the adapter's `streams` entry carries every envelope the call wrote, 'baselines included', and the `history` query returns baseline events 'like any other', so `HistoryProjection.keyOf` receives events of the reserved type `baseline`, whose payload is a whole migrated state of up to 512 KiB, on both paths; the contract speaks only of domain events and does not say that a baseline is skipped or what it means to a view whose fold reads history.", "fix": "In `spec:application.projection-contract` state that `applyHistoryProjection` ignores `mode` (or drop the field), that it reads versions from the envelopes, and that it drops events of type `baseline` before `keyOf`, with one sentence on why a meaning migration of a source needs a new generation of the view. In `spec:command.command-pipeline` step 9 write 'with the entry's `events`'.", @@ -3962,8 +3975,8 @@ "lens": "convex", "severity": "minor", "files": [ - "specs/application/projection-contract.sdp.md", - "specs/application/read-models.sdp.md" + "specs/application/read-models.sdp.md", + "specs/application/aggregate-projection.sdp.md" ], "issue": "An `AggregateProjection` keeps one row per aggregate key and `applyAggregate` reads and rewrites it in every command that contributes ('adds the difference to `value` and adjusts `entityCount`'); the pinned shape `orderTotals` has one row per order status per tenant. Every concurrent command that contributes to the same key therefore reads and writes the same document, and under optimistic concurrency all but one re-run: 'A mutation `updateCounter` always updates the same document ... If this mutation is called many times per second, many of its executions will conflict with each other', with the remediation 'Design your data model such that it doesn't require making many writes to the same document' (docs.convex.dev/error#1, read 2026-10-01). The marker rule makes the sum correct; it does not remove the contention, and the cost is recorded nowhere: projection-contract's limits count reads and writes per command, E-45 counts rows, and neither says that an aggregate with few keys serializes its writers. Convex's own answer is listed on the same page, the Sharded Counter component, and the components page names the Aggregate component, which 'can internally denormalize data'. The first experiment builds no aggregate, so nothing fails today.", "fix": "In `spec:application.projection-contract` add a risk to the aggregate form under E-9: commands that contribute to one aggregate key serialize on its row, so an aggregate whose key has few values is for low write rates, and name the two native options, the Aggregate and Sharded Counter components, as what the build weighs first when a count or sum sits on a hot key, with the marker rule kept for rebuild equality. Add the same sentence to read-models where it offers 'a read model updated in the command' for a summary.", diff --git a/design/specs/advanced/coalesced-recomputation.sdp.md b/design/specs/advanced/coalesced-recomputation.sdp.md index 6868740..28ef0c3 100644 --- a/design/specs/advanced/coalesced-recomputation.sdp.md +++ b/design/specs/advanced/coalesced-recomputation.sdp.md @@ -26,7 +26,7 @@ A command marks a scoped view key dirty in its own transaction, and several chan ### Open questions -- [blocking] The dirty-mark table, the recompute scheduling and the staleness field are deferred to the build on this trigger (D18, Decision method rule 4) +- [blocking #recomputationStalenessRecords] The dirty-mark table, the recompute scheduling and the staleness field are deferred to the build on this trigger (D18, Decision method rule 4) ## Behavior diff --git a/design/specs/advanced/cross-stream-online-rebuild.sdp.md b/design/specs/advanced/cross-stream-online-rebuild.sdp.md index bb09cec..465259c 100644 --- a/design/specs/advanced/cross-stream-online-rebuild.sdp.md +++ b/design/specs/advanced/cross-stream-online-rebuild.sdp.md @@ -26,7 +26,7 @@ The rebuild class is declared first: per entity, bounded current state, or histo ### Open questions -- [blocking] The dependency protocol or the consistent source cut, and the online cutover for this class, are deferred to the build on this trigger (D18, Decision method rule 4) +- [blocking #crossStreamRebuildCut] The dependency protocol or the consistent source cut, and the online cutover for this class, are deferred to the build on this trigger (D18, Decision method rule 4) ## Behavior diff --git a/design/specs/advanced/ordered-consumer.sdp.md b/design/specs/advanced/ordered-consumer.sdp.md index be2a076..01a4a9c 100644 --- a/design/specs/advanced/ordered-consumer.sdp.md +++ b/design/specs/advanced/ordered-consumer.sdp.md @@ -26,7 +26,7 @@ Order is per tenant, consumer, generation and partition, with the smallest parti ### Open questions -- [blocking] The sequence allocation, the consumer position record, the worker's batch bound and the skip authorization are deferred to the build on this trigger (D18, Decision method rule 4) +- [blocking #orderedConsumerPosition] The sequence allocation, the consumer position record, the worker's batch bound and the skip authorization are deferred to the build on this trigger (D18, Decision method rule 4) ## Behavior diff --git a/design/specs/advanced/trigger-table.sdp.md b/design/specs/advanced/trigger-table.sdp.md index e35156e..4917c6c 100644 --- a/design/specs/advanced/trigger-table.sdp.md +++ b/design/specs/advanced/trigger-table.sdp.md @@ -19,7 +19,7 @@ Twelve capabilities each wait for a named trigger. Activating one records the ac ### Open questions -- [blocking] No trigger has fired; each capability's design is deferred until its activation record exists (D18, Decision method rule 4) +- [blocking #capabilityActivationTrigger] No trigger has fired; each capability's design is deferred until its activation record exists (D18, Decision method rule 4) ## Rule diff --git a/design/specs/agents/agent-runs.sdp.md b/design/specs/agents/agent-runs.sdp.md index fdd0c31..424fbf3 100644 --- a/design/specs/agents/agent-runs.sdp.md +++ b/design/specs/agents/agent-runs.sdp.md @@ -33,8 +33,8 @@ Agent reasoning produces a structured proposal. A deterministic policy checks it ### Open questions -- [blocking] The trigger has not fired; the run, proposal and attempt tables, the proposal schema, the policy engine's inputs and the provider adapters are deferred to the build (D17, Decision method rule 4) -- [non-blocking] OQ2 dependency: whether a rejected agent proposal must be recorded as a fact decides whether the generic internal dispatcher of D7 is needed at the agent boundary; the design carries it as a conditional (OQ2, D7) +- [blocking #agentProposalAttempts] The trigger has not fired; the run, proposal and attempt tables, the proposal schema, the policy engine's inputs and the provider adapters are deferred to the build (D17, Decision method rule 4) +- [non-blocking #agentProposalRejectionRecord] OQ2 dependency: whether a rejected agent proposal must be recorded as a fact decides whether the generic internal dispatcher of D7 is needed at the agent boundary; the design carries it as a conditional (OQ2, D7) ## Behavior diff --git a/design/specs/agents/budget-accounting.sdp.md b/design/specs/agents/budget-accounting.sdp.md index 8fa97ce..f1338a8 100644 --- a/design/specs/agents/budget-accounting.sdp.md +++ b/design/specs/agents/budget-accounting.sdp.md @@ -20,7 +20,7 @@ A paid call reserves budget before dispatch and settles once from evidence. Conc ### Open questions -- [blocking] The reservation and usage records, the slot allocator and the reconciliation operation are deferred to the build on the agent trigger (D17, Decision method rule 4) +- [blocking #agentBudgetAccounting] The reservation and usage records, the slot allocator and the reconciliation operation are deferred to the build on the agent trigger (D17, Decision method rule 4) ## Rule diff --git a/design/specs/application.pack.sdp.md b/design/specs/application.pack.sdp.md index 0cdf186..8de820f 100644 --- a/design/specs/application.pack.sdp.md +++ b/design/specs/application.pack.sdp.md @@ -11,10 +11,16 @@ specs: - spec:application.read-models.query-refused-without-grant - spec:application.read-models.list-pages-by-cursor - spec:application.projection-contract + - spec:application.history-projection + - spec:application.aggregate-projection - spec:application.rebuild - spec:application.rebuild.online-rebuild-interrupt-resume - - spec:application.rebuild.write-pause-rebuild-interrupt - - spec:application.rebuild.write-pause-rebuild-abort + - spec:application.history-rebuild + - spec:application.history-rebuild.write-pause-rebuild-interrupt + - spec:application.history-rebuild.write-pause-rebuild-abort + - spec:application.history-rebuild.write-pause-rebuild-version-change + - spec:application.history-rebuild.write-pause-rebuild-installation + - spec:application.history-rebuild.write-pause-rebuild-source-order-mismatch - spec:application.generation-registry - spec:application.generation-registry.first-rebuild-installs-read-model - spec:application.generation-registry.command-without-generation-fails @@ -55,6 +61,6 @@ modelRefs: --- # Application -Package D. Layer 2: parent use cases, read models and the projection contract, online rebuild with the generation registry and the write pause, restore, baseline operations, the cost-target constraints and the first experiment, with the Layer 2 scenarios and the all-layers audit scenario. +Package D. Layer 2: parent use cases, read models and the projection contract, online rebuild with the generation registry and the write pause, and the rebuild of a history view under that pause, restore, baseline operations, the cost-target constraints and the first experiment, with the Layer 2 scenarios and the all-layers audit scenario. -Membership order follows the plan's inventory with each parent's examples directly after it: parent use cases, read models, the projection contract, rebuild with its registry and gate, restore, the first experiment with the example domain, baseline operations, then the eight constraints. Forty-eight Specs: nine application Specs, one operations Spec, eight constraints and thirty examples. Three examples are siblings the plan's table did not list, `second-context-throws`, `committed-state-read-through-query` and `write-pause-rebuild-abort`, added because each doc row enumerates two cases. Six examples verify no doc row: `first-activation-makes-read-model-writable` and `command-without-generation-fails` verify the generation registry's first rebuild and the failure of a command before it under E-8, `query-refused-before-disclosure` and `query-refused-without-grant` verify Law 5 for a parent query, `list-pages-by-cursor` verifies a parent list under E-24, and `restore-door-without-deploy` shows on the pinned release that the restore door's environment switch needs no deploy, under E-42. Six more verify no doc row: the `cancel-` examples of the example domain verify `CancelOrder` under E-46, the second lifecycle command, its rejections, its receipt and its grant. Three more verify no doc row: the stock item ID examples of the first experiment verify the bound on an order line's stock item ID under E-46. +Membership order follows the plan's inventory with each parent's examples directly after it: parent use cases, read models, the projection contract with the history and aggregate projections, rebuild and the history rebuild with its registry and gate, restore, the first experiment with the example domain, baseline operations, then the eight constraints. Fifty-four Specs: twelve application Specs, one operations Spec, eight constraints and thirty-three examples. Three examples are siblings the plan's table did not list, `second-context-throws`, `committed-state-read-through-query` and `write-pause-rebuild-abort`, added because each doc row enumerates two cases. Six examples verify no doc row: `first-activation-makes-read-model-writable` and `command-without-generation-fails` verify the generation registry's first rebuild and the failure of a command before it under E-8, `query-refused-before-disclosure` and `query-refused-without-grant` verify Law 5 for a parent query, `list-pages-by-cursor` verifies a parent list under E-24, and `restore-door-without-deploy` shows on the pinned release that the restore door's environment switch needs no deploy, under E-42. Six more verify no doc row: the `cancel-` examples of the example domain verify `CancelOrder` under E-46, the second lifecycle command, its rejections, its receipt and its grant. Three more verify no doc row: the stock item ID examples of the first experiment verify the bound on an order line's stock item ID under E-46. Three more verify no doc row: `write-pause-rebuild-version-change`, `write-pause-rebuild-installation` and `write-pause-rebuild-source-order-mismatch` verify the history rebuild's handoff for a changed projection version and for an installation, and its compatibility check and recovery for reordered sources, under E-8. diff --git a/design/specs/application/aggregate-projection.sdp.md b/design/specs/application/aggregate-projection.sdp.md new file mode 100644 index 0000000..3ef48ed --- /dev/null +++ b/design/specs/application/aggregate-projection.sdp.md @@ -0,0 +1,72 @@ +--- +id: spec:application.aggregate-projection +kind: contract +altitude: story +readiness: scoped +relations: + refines: spec:application.read-models + dependsOn: + - spec:application.projection-contract + - spec:application.generation-registry + - spec:command.command-declaration + - spec:command.command-pipeline + decidedBy: + - spec:decisions.d02-context-owns-state-and-journal + - spec:decisions.d09-rebuild-online-by-default + constrainedBy: + - spec:laws.law11-tenant-scope-named + - spec:facts.f01-serializable-mutations-under-occ + - spec:facts.f13-transactions-have-limits +--- +# Aggregate projection + +Story · Detail: deferred until a read model needs an aggregate row · Traces: D2, D8, D9, Law 11, F1, F13, E-7, E-8, E-9, E-45. + +An aggregate projection is the form of a read model whose row sums a number over every entity under one key, the counts and sums of D9, such as order totals per status. The contract pins the `AggregateProjection` shape; the aggregate row, which carries `value` and `entityCount` and no `sourceVersions`; the per-entity marker that carries the source versions, so that live commands and the backfill never count one entity twice; `applyAggregate` in four steps; and `orderTotals` as the shape of an aggregate table. No read model of the example domain takes this form, so its detail stops at the signature and the steps until a read model needs an aggregate row. `spec:command.command-declaration` and `spec:command.command-pipeline` bind and write the per-entity and history forms only, `spec:application.generation-registry` holds no marker, and the three `extends` entries below say what each of them takes when that trigger comes. + +## Intent + +- outcome: Pin the aggregate form of a projection, which sums one contribution per entity into one row per key and never counts an entity twice (D9, E-9) + +### Open questions + +- [blocking #aggregateReadModelRebuild] Deferred until a read model needs an aggregate row: OD-075 keeps this form a signature to which no read model of the example domain is bound, `orderTotals` is the shape of an aggregate table and not a read model, and the rebuild has no path for an aggregate; the first read model that needs a count or a sum over its entities is the trigger, and with it this contract takes its rebuild path, its tests and the owner's confirmation of the form under E-9 (OD-075, E-9) + +## Contract + +- [extension] An aggregate row derives from every entity under its key, so it carries no `sourceVersions`; it carries `value`, the sum of the recorded contributions, and `entityCount`, the number of markers, and the per-entity marker carries the `sourceVersions` the row conventions would otherwise hold (E-9, D9) +- [extension] An aggregate row carries no comparable version, so a client that needs its own write reflected compares its entity's row, and the sum's freshness is the subscription's re-run when the command commits (E-45, D8, E-9) +- An aggregate projection contributes a number per entity to an aggregate row and keeps a per-entity marker so live commands and backfill never count one entity twice (D9) +- An aggregate read model keeps one marker per entity per generation with the contribution last recorded and the source versions it was computed from, and every aggregate write reads and writes the marker in the same mutation as the aggregate row; the marker's `sourceVersions` are compared like a row's, so a DTO older than the marker never adjusts the sum (D9, E-8, E-9) +- [extension] `applyAggregate` ignores the mode: the marker's `sourceVersions` decide, a marker that is not older than the input makes the write `skipped-newer`, and otherwise the marker's recorded contribution leaves the row of the marker's `aggregateKey` and the new contribution joins the row of `aggregateKeyOf`, one row moved by the difference when the key is the same and two rows when it changed, a row whose `entityCount` reaches 0 being deleted, and a `null` contribution joins no row and removes the marker; a live update on an entity backfill has not reached is therefore not partial, because `contribute` sees the whole DTO, which is why the missing-row rule of the per-entity path does not apply (E-9, D9, F1) +- [extension] An aggregate's entity is its source stream: `applyAggregate` derives the marker's `entityKey` from the input's one `StreamVersion` as `JSON.stringify([contextId, streamType, streamId])`, so the live path and the backfill, which pass `entry.version` and `dto.version`, name one marker, and equal stream IDs in two contexts or two stream types name two (E-9, D9, D2) + +## Design + +The shapes are TypeScript in the parent. `StreamVersion` is the kernel's shape; `ApplyMode`, `ApplyResult`, `WritableGeneration` and the `by_key` index are those of `spec:application.projection-contract`; the marker is a row of the `projectionMarkers` table, read through its `by_entity` index, both pinned below; the marker table is tenant data and leads with the tenant; `MutationCtx` is Convex's mutation context. + +- typeAggregateProjection: `interface AggregateProjection { name: string; version: number; table: string; rowBudgetBytes?: number; aggregateKeyOf(tenantId: string, dto: D): string; contribute(tenantId: string, dto: D): number | null }` where `null` means the entity no longer contributes, such as a deleted subject, and `rowBudgetBytes` bounds the aggregate row and the marker alike (E-9, D9, F13) +- fnApplyAggregate: `applyAggregate(ctx: MutationCtx, projection: AggregateProjection, input: { tenantId: string; dto: D; versions: StreamVersion[]; mode: ApplyMode; generations: readonly WritableGeneration[] }): Promise` answering, per generation in order, one result for each aggregate row it writes, the old key's row before the new key's on a key change, `inserted` for a row it created, `updated` for a row whose `value` or `entityCount` changed and `deleted` for a row whose `entityCount` reached 0, or one `skipped-newer` when the marker was not older and one `unchanged` when no row moved; `input.versions` holds the one `StreamVersion` of the entity's source stream, and another length throws a plain error; `mode` is accepted for symmetry with `applyProjection` and read by nothing (E-9, D9) +- aggregateStep1: for each generation it derives `entityKey` from `input.versions[0]` as the contract states, reads the marker by `by_entity` with the tenant, read model, generation and `entityKey`, and computes `contribution` as `contribute(tenantId, dto)` and, when that is not `null`, `aggregateKey` as `aggregateKeyOf(tenantId, dto)` (E-9, D9) +- aggregateStep2: when a marker exists and its `sourceVersions` are not older than `input.versions` for the same stream, the result is `skipped-newer` and nothing is written (E-9, D9, F1) +- aggregateStep3: when a marker exists and `contribution` is `null` or `aggregateKey` differs from the marker's, it reads the row of the marker's `aggregateKey` by `by_key`, throwing a plain error when it is missing, a defect, subtracts the marker's contribution from `value` and 1 from `entityCount`, and deletes the row when `entityCount` reaches 0 and replaces it otherwise; when `contribution` is not `null` and the marker is missing or its key differs, it reads the row of `aggregateKey`, inserts it with `value` equal to `contribution` and `entityCount` 1 when missing, and otherwise adds `contribution` to `value` and 1 to `entityCount`; when the marker exists with the same key and `contribution` is not `null`, it adds `contribution` minus the marker's to that row's `value`, leaves `entityCount`, and writes no row when the difference is 0; when the marker is missing and `contribution` is `null`, it reads and writes no row (E-9, D9) +- aggregateStep4: then it inserts or replaces the marker with `entityKey`, `contribution`, `aggregateKey` and `input.versions`, or deletes it when `contribution` is `null`, all in the same mutation; a key change reads and writes two aggregate rows and the marker in each generation, and its two rows count against `limitReadModelWritesPerCommand` of `spec:application.read-models` (E-9, D9, E-8, E-45) +- validatorAggregateRowConventions: `const aggregateRowConventions = { tenantId: v.string(), generation: v.number(), key: v.string(), projectionVersion: v.number(), value: v.number(), entityCount: v.number() }` spread into every aggregate read-model `defineTable`, with the same `by_key` index, where `key` is `aggregateKeyOf(tenantId, dto)` (E-9, Law 11, D9) +- tableOrderTotals: `orderTotals: defineTable({ ...aggregateRowConventions }).index("by_key", ["tenantId", "generation", "key"])` an aggregate of order totals per status, where `key` is the status, `value` the sum of the orders' totals and `entityCount` the number of orders in it, given as the shape of an aggregate table and not as a read model the first experiment builds (E-9, D9) +- tableProjectionMarkers: `projectionMarkers: defineTable({ tenantId: v.string(), readModel: v.string(), generation: v.number(), entityKey: v.string(), aggregateKey: v.string(), contribution: v.number(), sourceVersions: v.array(streamVersionValidator) })` (E-8, D9, Law 11) +- indexMarkersByEntity: `.index("by_entity", ["tenantId", "readModel", "generation", "entityKey"])` (E-8) +- indexMarkersByEntityUse: point read by `applyAggregate` before every aggregate write, with the `entityKey` it derives from the entity's source stream, `JSON.stringify([contextId, streamType, streamId])` of the input's stream version, as `spec:application.aggregate-projection` states (D9, E-8, E-9) +- markerVersionCompare: `applyAggregate` treats the marker's `sourceVersions` as a row's: a marker whose versions are not older than the input's makes the write `skipped-newer` and leaves the sum alone, so a stale DTO on either path never adjusts an aggregate twice or backwards (E-9, E-8, D9, F1) +- limitMarkerCount: one marker per entity per generation; a read model with N entities costs N markers per generation (E-8, F13) +- markerRows: marker rows are operational data owned by the parent; they are never written by a context (D2, E-8) +- extendsReadModelBinding: when a read model needs an aggregate row, `spec:command.command-declaration#design.typeReadModelBinding` takes the form beside the other two, `readModel: ReadModel | AggregateProjection | HistoryProjection`, where `readModel` is a per-entity read model, an aggregate projection or a history projection, and a per-entity read model or an aggregate projection is bound to its one source stream type (E-7, E-9, OD-075) +- extendsStepNine: when a read model needs an aggregate row, the step 9 of `spec:command.command-pipeline` calls, for every binding of a per-entity read model, `applyProjection` of `spec:application.projection-contract`, or `applyAggregate` for a binding of that form, for each matching entry with the entry's DTO and version and the list unchanged, and its `readModelWrites` makes one `applyProjection` or `applyAggregate` call per matching `streams` entry per binding of those forms the declaration lists (E-7, E-9, D8, OD-075) +- extendsRegistry: when a read model needs an aggregate row, `projectionMarkers` and its `by_entity` index join `readModelTables` of `spec:application.generation-registry`, so `ReadModelDataModel` holds them, and the registry's reads and transitions stay as they are, because a marker names a generation and never changes one (E-8, E-9, OD-075) +- keyChangeCheck: a test of `applyAggregate` moves an entity from one aggregate key to another and finds the old key's row without its contribution and count and the new key's with them (D9, E-9) +- deferred: the code of `AggregateProjection` and `applyAggregate`, a rebuild path for an aggregate read model and the tests of this contract, until a read model needs an aggregate row (E-9, OD-075) + +## Verification — reviewed + +- A reviewer confirms that `applyAggregate` compares the marker's `sourceVersions` before it touches `value`, that it reads no mode, that it derives `entityKey` from the input's stream version, that a key change moves the contribution and the count from the marker's key's row to the new key's row, and that a `null` contribution removes the marker and its recorded contribution and creates no row. +- A reviewer confirms that the marker table's index leads with the tenant, that `applyAggregate` reads it before every aggregate write, and that it compares the marker's `sourceVersions` before adjusting the sum. +- A reviewer confirms that the pipeline's step 9 and the rebuild's batches are the only callers of `applyAggregate`. diff --git a/design/specs/application/first-experiment.sdp.md b/design/specs/application/first-experiment.sdp.md index 9ea4446..2cdbe2b 100644 --- a/design/specs/application/first-experiment.sdp.md +++ b/design/specs/application/first-experiment.sdp.md @@ -18,7 +18,12 @@ relations: - spec:application.parent-use-cases - spec:application.read-models - spec:application.rebuild + - spec:application.history-rebuild - spec:application.restore + - spec:application.generation-registry + - spec:application.history-projection + - spec:command.actor-and-scope + - spec:platform.native-harness constrainedBy: - spec:constraints.one-commit-per-successful-command - spec:constraints.zero-core-projection-jobs @@ -56,12 +61,12 @@ The six cost targets are the constraint Specs this workflow is constrained by. T ### Open questions -- [non-blocking] OQ1: a trivial context with no invariants of its own stays a component; the owner answered OQ1 on this experiment's local measurement of component call cost, and a measured workload that misses an agreed target, or a hosted usage reading above an agreed share, reopens it (OQ1, Probe 3) -- [non-blocking] Extension E-46: the doc asks for one more lifecycle command and one essential summary without naming them; the option taken here is `CancelOrder`, which releases the allocation, and the order summary read model (E-46, First experiment) -- [non-blocking] Extension E-47: the doc lists what to count but not how; the option taken here is a measurement record per run captured from the disposable backend's function execution log, documents and bytes from the `usageStats` of the command's top-level completion record, and the calls per context from a pure test of the use case's executor, because the log holds no record of a component call inside a mutation, with any count the backend does not expose recorded as a gap; the owner has not ruled (E-47, First experiment) -- [non-blocking] Extension E-47, the documents a command reads: the design expected one `PlaceOrder` of N lines to read about N + 6 documents, and the pinned backend counted 3N + 5; the measurement run decides where the other 2N reads come from (E-47, F13) -- [non-blocking] Extensions E-8, E-13, E-15 and E-37, as the experiment's setup takes them: stock exists only through `ReceiveStock`, a run seeds its grants and makes the order summary writable with admin access before its first command, the production composition names no service issuer, its `auth.config.ts` reads its issuer from environment variables, and it is the Convex app whose project directory is `example/`; these are options taken here and the owner has not ruled (E-8, E-13, E-15, E-37, E-46) -- [non-blocking] Extension E-17, the rows the experiment requires: the doc says the experiment passes when every Layer 0, 1 and 2 scenario passes, and its one all-layer row, Sc ALL-1, applies to every installed layer; the option taken here requires the twenty-three rows the doc's table gives Layer 0, 1 or 2 and Sc ALL-1, each at the tier the acceptance contract names for it, so that the native proof of the audit and the diagnostic is part of the experiment; the owner confirmed both (E-17, First experiment, Sc ALL-1) +- [non-blocking #contextIsolationCost] OQ1: a trivial context with no invariants of its own stays a component; the owner answered OQ1 on this experiment's local measurement of component call cost, and a measured workload that misses an agreed target, or a hosted usage reading above an agreed share, reopens it (OQ1, Probe 3) +- [non-blocking #lifecycleCommandSummary] Extension E-46: the doc asks for one more lifecycle command and one essential summary without naming them; the option taken here is `CancelOrder`, which releases the allocation, and the order summary read model (E-46, First experiment) +- [non-blocking #commandMeasurementRecord] Extension E-47: the doc lists what to count but not how; the option taken here is a measurement record per run captured from the disposable backend's function execution log, documents and bytes from the `usageStats` of the command's top-level completion record, and the calls per context from a pure test of the use case's executor, because the log holds no record of a component call inside a mutation, with any count the backend does not expose recorded as a gap; the owner has not ruled (E-47, First experiment) +- [non-blocking #commandDocumentReads] Extension E-47, the documents a command reads: the design expected one `PlaceOrder` of N lines to read about N + 6 documents, and the pinned backend counted 3N + 5; the measurement run decides where the other 2N reads come from (E-47, F13) +- [non-blocking #compositionInitialGrants] Extensions E-8, E-13, E-15 and E-37, as the experiment's setup takes them: stock exists only through `ReceiveStock`, a run seeds its grants and makes the order summary writable with admin access before its first command, the production composition names no service issuer, its `auth.config.ts` reads its issuer from environment variables, and it is the Convex app whose project directory is `example/`; these are options taken here and the owner has not ruled (E-8, E-13, E-15, E-37, E-46) +- [non-blocking #requiredAcceptanceScenarios] Extension E-17, the rows the experiment requires: the doc says the experiment passes when every Layer 0, 1 and 2 scenario passes, and its one all-layer row, Sc ALL-1, applies to every installed layer; the option taken here requires the twenty-three rows the doc's table gives Layer 0, 1 or 2 and Sc ALL-1, each at the tier the acceptance contract names for it, so that the native proof of the audit and the diagnostic is part of the experiment; the owner confirmed both (E-17, First experiment, Sc ALL-1) ## Workflow diff --git a/design/specs/application/generation-registry.sdp.md b/design/specs/application/generation-registry.sdp.md index 8fb4bc7..716682b 100644 --- a/design/specs/application/generation-registry.sdp.md +++ b/design/specs/application/generation-registry.sdp.md @@ -14,6 +14,9 @@ relations: - spec:laws.law11-tenant-scope-named - spec:facts.f17-migrations-fits-generation-backfill - spec:laws.law06-technical-failure-never-a-rejection + dependsOn: + - spec:application.projection-contract + - spec:command.actor-and-scope --- # Generation registry @@ -23,7 +26,7 @@ The registry is two parent tables. The `generations` table records every generat ## Intent -- outcome: Pin the generations table and its states, the progress row, every transition with who may make it, the per-entity marker document, and the reads a command and a query make against them (D9, E-8) +- outcome: Pin the generations table and its states, the progress row, every transition with who may make it, and the reads a command and a query make against them (D9, E-8) - value: One table answers which generation is active, which is being built and which is still inside its rollback period, so live update, backfill, queries and operators agree without a second store (D9) - risk: A registry read sits in every command that maintains a read model, at most two indexed reads per read model; every transition of a generation changes a row those reads cover, so each transition re-runs the commands in flight on that read model once (D9, F1, Sc L2-3) - risk: A projection version stays in the code for as long as a generation that records it is written or may be rolled back to (E-9, D9) @@ -31,12 +34,12 @@ The registry is two parent tables. The `generations` table records every generat ### Open questions -- [non-blocking] Extension E-8: the doc names a generation registry, a marker and a gate without their shapes, and says to keep the old generation for a rollback period without saying whether live commands keep writing it; the option taken here is the `generations` and `projectionMarkers` tables below with seven states, a rollback period of 7 days, one generation being built per read model at a time, and a retired generation that is kept and not written, so that a rollback reopens it as `verifying`, one verify pass repairs what the writes since its retirement changed, and the ordinary switch makes it active again; a rollback is therefore a verify pass and then a switch of one write, never one write by itself (E-8, D9, Sc L2-5) -- [non-blocking] Extension E-8, reach of a generation: the doc says nothing about tenants and rebuild; one generation per read model spans every tenant, the registry has no tenant dimension, one row is active for all tenants and the batches iterate the tenant list of `spec:command.actor-and-scope`; the registry is operational data of the deployment, so its indexes lead with the read model name and not with `tenantId`, a confirmed deviation from Law 11's index rule of the same kind the obligation module records under E-57, while every table that holds tenant data, the read-model rows and the markers among them, leads with the tenant; a rebuild for one tenant is not offered (E-8, D9, D11, Law 11, E-57) -- [non-blocking] Extension E-42, as it touches the registry: a restore brings back a generation row and its progress row at their checkpoint and drops the scheduled batch that was driving them, so the row's fence at inventory time is the restore's proof that the generation is still stranded, and `resumeChain` bumping it is the proof that it is not (E-42, F12) -- [non-blocking] Extension E-8, the first generation: the doc says how a generation is rebuilt and not how a read model's first generation comes to exist; the option taken here is that a read model is installed through its first rebuild, generation 1 starting as `building` like any other, so a read model added to a deployment that has history serves that history from its first switch and no transition claims coverage it did not build; a command that finds no generation to write fails as a technical failure and a query that finds no active generation fails the same way (E-8, D9, D19) -- [non-blocking] Extension E-40, the checkpoint: the cursor, the batch size and the counts live on the progress row and not on the generation row, because a batch that wrote its checkpoint on the row every command reads would re-run every command in flight on the read model once per batch; the progress row is the one checkpoint of every batch whatever drives it (E-40, F1) -- [non-blocking] Extension E-49, as it touches the registry: on a deployment whose grants predate the tenant list the list is empty while grants exist, and a generation started then visits no tenant; the option taken here is that `startGeneration` refuses then, with a message naming `fillTenants`, the operator entry of `spec:application.rebuild` that fills the list from the grants, and refuses while that fill runs (E-49, E-8, Law 11) +- [non-blocking #generationRollbackPolicy] Extension E-8: the doc names a generation registry, a marker and a gate without their shapes, and says to keep the old generation for a rollback period without saying whether live commands keep writing it; the option taken here is the `generations` table below with seven states, a rollback period of 7 days, one generation being built per read model at a time, and a retired generation that is kept and not written, so that a rollback reopens it as `verifying`, one verify pass repairs what the writes since its retirement changed, and the ordinary switch makes it active again; a rollback is therefore a verify pass and then a switch of one write, never one write by itself (E-8, D9, Sc L2-5) +- [non-blocking #generationTenantReach] Extension E-8, reach of a generation: the doc says nothing about tenants and rebuild; one generation per read model spans every tenant, the registry has no tenant dimension, one row is active for all tenants and the batches iterate the tenant list of `spec:command.actor-and-scope`; the registry is operational data of the deployment, so its indexes lead with the read model name and not with `tenantId`, a confirmed deviation from Law 11's index rule of the same kind the obligation module records under E-57, while every table that holds tenant data, the read-model rows among them, leads with the tenant; a rebuild for one tenant is not offered (E-8, D9, D11, Law 11, E-57) +- [non-blocking #restoreGenerationFence] Extension E-42, as it touches the registry: a restore brings back a generation row and its progress row at their checkpoint and drops the scheduled batch that was driving them, so the row's fence at inventory time is the restore's proof that the generation is still stranded, and `resumeChain` bumping it is the proof that it is not (E-42, F12) +- [non-blocking #readModelInstallation] Extension E-8, the first generation: the doc says how a generation is rebuilt and not how a read model's first generation comes to exist; the option taken here is that a read model is installed through its first rebuild, generation 1 starting as `building` like any other, so a read model added to a deployment that has history serves that history from its first switch and no transition claims coverage it did not build; a command that finds no generation to write fails as a technical failure and a query that finds no active generation fails the same way (E-8, D9, D19) +- [non-blocking #rebuildProgressRow] Extension E-40, the checkpoint: the cursor, the batch size and the counts live on the progress row and not on the generation row, because a batch that wrote its checkpoint on the row every command reads would re-run every command in flight on the read model once per batch; the progress row is the one checkpoint of every batch whatever drives it (E-40, F1) +- [non-blocking #tenantListGenerationStart] Extension E-49, as it touches the registry: on a deployment whose grants predate the tenant list the list is empty while grants exist, and a generation started then visits no tenant; the option taken here is that `startGeneration` refuses then, with a message naming `fillTenants`, the operator entry of `spec:application.rebuild` that fills the list from the grants, and refuses while that fill runs (E-49, E-8, Law 11) ## Contract @@ -51,17 +54,16 @@ The registry is two parent tables. The `generations` table records every generat - [extension] Each generation is written with the projection whose version its row records, on the live path and by its batches alike, so the rows of one generation are one truth of one projection version (E-9, E-8, D8) - A query reads the active generation and nothing else; a query over a read model that has no active generation throws a plain error that names the read model, a technical failure, and never answers an empty page (D9, E-8, Law 6) - [extension] The generation row carries the state, the projection version, the fence, the fence the last interrupt set and who started and last changed it, and nothing a batch writes on its healthy path except the state that the batch ending a pass sets; the progress row carries the cursor, the batch size and the counts, the batch size set by `startGeneration` and never changed after, so every pass of the generation, a resumed one and its purge included, runs at it, a purge batch capped further by `limitPurgeBatch` of `spec:application.rebuild`, and a batch writes it in the same mutation as its rows, so the progress row is the checkpoint (E-40, D9, F1) -- [extension] A generation spans every tenant of the deployment and the registry row names none; the tenant is named on every read-model row and marker, on every context call a batch makes, and in the batch cursor, so no read or write of tenant data runs with an absent tenant as a wildcard, and no generation can be switched for one tenant alone (E-8, D11, Law 11) -- An aggregate read model keeps one marker per entity per generation with the contribution last recorded and the source versions it was computed from, and every aggregate write reads and writes the marker in the same mutation as the aggregate row; the marker's `sourceVersions` are compared like a row's, so a DTO older than the marker never adjusts the sum (D9, E-8, E-9) +- [extension] A generation spans every tenant of the deployment and the registry row names none; the tenant is named on every read-model row, on every context call a batch makes, and in the batch cursor, so no read or write of tenant data runs with an absent tenant as a wildcard, and no generation can be switched for one tenant alone (E-8, D11, Law 11) - [extension] A generation whose progress row is in the pass `backfill`, `verify` or `purge` has at most one scheduled batch driving it, identified by the row's fence; a restore brings the rows back at their checkpoint and drops that batch with every pending scheduled function, so the generation is stranded until `resumeChain` bumps the fence and reschedules from the saved cursor, which the restore procedure's pending-work inventory and accepted branch drive for every such generation in `building` or `verifying`, and which the operator drives with `resumeGeneration` for a purge the restore stranded, because a purge belongs to a `retired` or `aborted` generation the inventory does not read (E-42, E-8, F12, D19) - The registry is read, never incremented, in the command path; it is not an application-wide counter (D2, E-8) -- Registry, progress and marker rows are operational data owned by the parent; they are never written by a context (D2, E-8) +- Registry and progress rows are operational data owned by the parent; they are never written by a context (D2, E-8) - [extension] A command whose declared read model has no generation to write fails as a technical failure, a plain error that names the read model, so a command never commits without its read model (E-8, D8, Law 6) - [extension] The operator a generation row records is the string the operator entry was given, as `spec:command.actor-and-scope` rules it: `startedBy` from `startGeneration` and `changedBy` from the last operator entry that changed the row (E-48, D19) ## Design -Tables live in the parent's `schema.ts`, spread from the library's `readModelTables`. `streamVersionValidator` is the kernel's validator for `StreamVersion`. The registry has no `tenantId` because every generation spans all tenants; the marker table is tenant data and leads with the tenant. Every refusal below is a plain `Error` with the message given. +Tables live in the parent's `schema.ts`, spread from the library's `readModelTables`. `streamVersionValidator` is the kernel's validator for `StreamVersion`. The registry has no `tenantId` because every generation spans all tenants. Every refusal below is a plain `Error` with the message given. - validatorGenerationState: `const generationStateValidator = v.union(v.literal("building"), v.literal("verifying"), v.literal("verified"), v.literal("active"), v.literal("retired"), v.literal("aborted"), v.literal("purged"))` (E-8, D9) - tableGenerations: `generations: defineTable({ readModel: v.string(), generation: v.number(), projectionVersion: v.number(), state: generationStateValidator, pauseRequired: v.boolean(), fence: v.number(), startedAt: v.number(), startedBy: v.string(), changedAt: v.number(), changedBy: v.string(), switchedAt: v.optional(v.number()), retiredAt: v.optional(v.number()), retireAfter: v.optional(v.number()), interruptedFence: v.optional(v.number()) })` where `changedAt` and `changedBy` are the time and the operator of the last operator entry that changed the row, which no batch writes, `switchedAt` is when the generation last became active, `retiredAt` and `retireAfter` are set while it is `retired` and absent otherwise, and `interruptedFence` is the fence `interruptGeneration` last set, absent before the first interrupt, so it equals `fence` only while the interrupt made the last bump, and every other entry that adds to the fence leaves it as it is (E-8, E-48, D9, F13) @@ -75,24 +77,18 @@ Tables live in the parent's `schema.ts`, spread from the library's `readModelTab - tableGenerationProgress: `generationProgress: defineTable({ generationId: v.id("generations"), pass: progressPassValidator, batchSize: v.number(), cursor: v.union(batchCursorValidator, v.null()), batchesDone: v.number(), rowsWritten: v.number(), rowsSkipped: v.number(), misses: v.number(), rowsPurged: v.number(), lastError: v.optional(v.string()), updatedAt: v.number() })` one row per generation, inserted by `startGeneration` in the mutation that inserts the generation row; `cursor` is the position in the pass that `pass` names and `null` before that pass has read its first tenant; `batchesDone` counts every batch of the generation that reached its checkpoint, the batch that ends a pass included and a parked batch not, `rowsWritten` the rows a backfill batch inserted, updated or deleted, `rowsSkipped` the items a backfill or verify batch left as they were, `misses` the rows the current verify pass inserted, updated or deleted, and `rowsPurged` the rows a purge deleted; every write of the row sets `updatedAt` to the writing mutation's `Date.now()` (E-40, D9, F13) - indexProgressByGeneration: `.index("by_generation", ["generationId"])` (E-40) - indexProgressByGenerationUse: the one read a batch and an operator entry other than `interruptGeneration` make to find a generation's progress row; no command and no query over a read model reads the table (E-40, F1) -- tableProjectionMarkers: `projectionMarkers: defineTable({ tenantId: v.string(), readModel: v.string(), generation: v.number(), entityKey: v.string(), aggregateKey: v.string(), contribution: v.number(), sourceVersions: v.array(streamVersionValidator) })` (E-8, D9, Law 11) -- indexMarkersByEntity: `.index("by_entity", ["tenantId", "readModel", "generation", "entityKey"])` (E-8) -- indexMarkersByEntityUse: point read by `applyAggregate` before every aggregate write, with the `entityKey` it derives from the entity's source stream, `JSON.stringify([contextId, streamType, streamId])` of the input's stream version, as `spec:application.projection-contract` states (D9, E-8, E-9) -- typeRegistryReader: `type RegistryReader = { db: GenericDatabaseReader }` and `type RegistryWriter = { db: GenericDatabaseWriter }`, the part of a ctx the registry's helpers use, where `ReadModelDataModel` is the data model of `readModelTables`, the schema fragment that holds `generations`, `generationProgress`, `projectionMarkers` and the tenant fill's `tenantFill` row of `spec:application.rebuild`; every helper that takes them is generic over the composition's data model, because a database reader is not covariant in its data model (E-8) -- typeWritableGeneration: `type WritableGeneration = { role: "active" | "building" | "verifying"; generation: number; projectionVersion: number }` where the role is `active` for the state `active`, `building` for the state `building`, and `verifying` for the states `verifying` and `verified` (E-8, E-9, D9) +- typeRegistryReader: `type RegistryReader = { db: GenericDatabaseReader }` and `type RegistryWriter = { db: GenericDatabaseWriter }`, the part of a ctx the registry's helpers use, where `ReadModelDataModel` is the data model of `readModelTables`, the schema fragment that holds `generations`, `generationProgress` and the tenant fill's `tenantFill` row of `spec:application.rebuild`; every helper that takes them is generic over the composition's data model, because a database reader is not covariant in its data model (E-8) +- writableGenerationRole: each `WritableGeneration` that `generationsToWrite` returns, the shape `spec:application.projection-contract#design.typeWritableGeneration` pins, takes its role from its generation's state: the role is `active` for the state `active`, `building` for the state `building`, and `verifying` for the states `verifying` and `verified` (E-8, E-9, D9) - fnGenerationsToWrite: `generationsToWrite(ctx: RegistryReader, readModel: string): Promise` making at most two indexed reads, the ranges of `by_read_model_state` from `active` to `building` and from `verified` to `verifying`, throwing `"Read model " + readModel + " has " + a + " active generations and " + b + " in building, verifying or verified"` when they hold two active generations or two in-flight ones, and returning one entry per generation a command writes, the active generation first when there is one, then the in-flight one, and an empty list when the read model has neither; the answer is the same for every tenant (E-8, D9) - fnActiveGeneration: `activeGeneration(ctx: RegistryReader, readModel: string): Promise` making one indexed read and answering `undefined` when the read model has no active generation, which a query over the read model then turns into the plain error `"The read model " + readModel + " has no active generation"`; the answer is the same for every tenant, and the query narrows the rows it reads by its own `tenantId` argument (E-8, D9, Law 11) - errorNoGenerationToWrite: the pipeline's step 9 throws `commandType + " writes the read model " + readModel + ", which has no generation to write"` when `generationsToWrite` answers an empty list, which is the case before `startGeneration` and after the only generation was aborted (E-8, D8, Law 6) -- markerVersionCompare: `applyAggregate` treats the marker's `sourceVersions` as a row's: a marker whose versions are not older than the input's makes the write `skipped-newer` and leaves the sum alone, so a stale DTO on either path never adjusts an aggregate twice or backwards (E-9, E-8, D9, F1) - transitionCost: a transition patches one generation row, or two in the one write of a switch that retires an active generation, a single mutation, and every command in flight on the read model that read the registry before it re-runs once; a batch writes a generation row only in the mutation that ends its pass, so a build costs the commands one such re-run at the end of the backfill and one at the end of the verify pass, plus one per operator entry, `startGeneration` and `switchGeneration` among them (E-40, F1, D9) - limitRegistryReadsPerCommand: at most 2 indexed reads per read model per command, because the pipeline's step 9 calls `generationsToWrite` once per read model name however many bindings name it, and one per query (E-8, F13) - limitRollbackPeriod: 7 days by default, 604,800,000 ms; `switchGeneration` takes an override between 0 and 2,592,000,000 ms, and `rollbackGeneration` is refused once `retireAfter` has passed because the rows may be purged (E-8, D9) -- limitMarkerCount: one marker per entity per generation; a read model with N entities costs N markers per generation (E-8, F13) ## Verification — reviewed - A reviewer confirms that every transition in the rebuild workflow is one of the transitions listed here, that no function sets a state outside them, and that no function inserts a generation in a state other than `building`. - A reviewer confirms that `generationsToWrite` and `activeGeneration` are the only readers of the registry in the command and query paths, that neither writes it, and that neither reads `generationProgress`. - A reviewer confirms that no batch patches a generation row except in the mutation that ends its pass. -- A reviewer confirms that the marker table's index leads with the tenant, that `applyAggregate` reads it before every aggregate write, and that it compares the marker's `sourceVersions` before adjusting the sum. - A reviewer confirms that no field, index or function of the registry carries a tenant, that every batch cursor names one, and that the restore's inventory reads `by_read_model_state` for the three in-flight states and nothing else. diff --git a/design/specs/application/history-projection.sdp.md b/design/specs/application/history-projection.sdp.md new file mode 100644 index 0000000..54ce6a4 --- /dev/null +++ b/design/specs/application/history-projection.sdp.md @@ -0,0 +1,75 @@ +--- +id: spec:application.history-projection +kind: contract +altitude: story +readiness: defined +relations: + refines: spec:application.read-models + dependsOn: + - spec:application.projection-contract + - spec:context.event-envelope + decidedBy: + - spec:decisions.d02-context-owns-state-and-journal + - spec:decisions.d04-four-outcomes + - spec:decisions.d05-rebuildable-history-with-baselines + - spec:decisions.d08-read-models-in-command + - spec:decisions.d09-rebuild-online-by-default + - spec:decisions.d18-everything-else-waits-for-trigger + - spec:decisions.d19-operations-travel-with-capability + constrainedBy: + - spec:laws.law06-technical-failure-never-a-rejection + - spec:laws.law10-replay-never-runs-commands-or-effects + - spec:facts.f01-serializable-mutations-under-occ + - spec:facts.f13-transactions-have-limits +--- +# History projection + +Story · Detail: full · Traces: D2, D4, D5, D8, D9, D18, D19, Law 6, Law 10, F1, F13, Sc L2-6, E-7, E-9, E-25, E-40, E-46. + +A history projection is the form of a read model whose row depends on history across several streams: it folds events into the prior row, where a per-entity projection builds a row from one DTO. The contract pins the `HistoryProjection` shape and `applyHistoryProjection`, the one helper that folds and writes such a row, shared by the command pipeline's step 9 and the rebuild's history batches; a fold that gives one row whatever the order across streams and however the events are split between calls; a row that no fold deletes, which keeps its subject's ended state; baselines dropped before `keyOf`; the `sourceVersions` that keep an event from being folded twice; and the order allocation history of the example domain, with its table, its rules and the bound on what one call reads and writes. + +## Intent + +- outcome: Pin the history form of a projection, which folds the events of several streams into one row that the live path and the backfill write alike (E-9, D8, D9) + +### Open questions + +- [non-blocking #historyFoldOrder] Extension E-9, the history fold: D2 orders events only within a stream, and the two paths feed one history row in different orders, the live path command by command and the backfill stream by stream; the option taken here is a fold that gives one row whatever the order across streams and however its events are split between calls, a row no fold deletes, which keeps its subject's ended state, and baselines dropped before `keyOf`, so a view whose meaning needs the order across streams waits for the ordered consumer of D18 (E-9, D2, D9, D18, E-25) +- [non-blocking #historyProjectionCost] The cost of the binding on the live path, one row read and one write of up to `rowBudgetBytes` per command and the measure of the row, is taken by the first experiment's latency cell with the order allocation history bound, for `PlaceOrder` and for `CancelOrder` of an order of the maximum lines, before the view is declared essential anywhere (E-9, E-47, F13) +- [non-blocking #historyFoldPureTest] Extension E-9, the check of the history fold: the order independence the Contract requires is checked for the order allocation history alone; the option taken here is that every history projection of an application carries a pure test that folds a sample of one key's events in every interleaving of its streams and every split into calls, and a library helper that runs that test is added with a second history view (E-9, D2, D18) +- [non-blocking #historySubjectErasure] A history row keeps a deleted subject's history and every rebuild writes it again from the journal; whether a history view falls under the retention and deletion policy of D19, and what its rebuild does after an erasure, is answered by that policy (D19, D9, E-9) + +## Contract + +- [extension] A history projection folds events into a row: `keyOf` maps each event to the row it belongs to or to none, and `project` folds the events of one key, each stream's in stream-version order and the streams interleaved in any order, into the prior row; it is deterministic in the prior row and the events, reads nothing else, and is the one form a read model that depends on history across several streams may take, which is why such a read model rebuilds only under the write pause (E-9, D8, D9, Law 10) +- [extension] A history projection's fold gives the same row whatever the order in which the events of different streams arrive and however they are split between calls: from one prior row, the events of one key folded in one call or in several, with the streams interleaved in any way that keeps each stream's events in version order, give equal rows; D2 orders events only within a stream, and the live path feeds a row command by command while the backfill feeds it stream by stream, so a view whose meaning depends on the order across streams is not a history projection and waits for the ordered consumer of D18 (E-9, D2, D8, D18) +- [extension] `project` never answers `null`: a history row, once written, keeps its subject's ended state, such as a cancelled order, and leaves its table only when its generation is purged, so its `sourceVersions`, the fold's one memory, outlive the subject's end, and an event that arrives after it, such as `AllocationReleased` after `OrderCancelled`, folds into the row and never makes a new one (E-9, D9) +- [extension] `applyHistoryProjection` drops every event of the reserved type `baseline` before `keyOf`, on both paths: a baseline holds a stream's migrated state and no fact a view folds, and a migration that changes what a source's earlier events mean to the view is a new projection version and so a new generation, never a rewrite of rows in place (E-9, E-25, D9) +- [extension] `project` reads every `eventSchemaVersion` of every event type it folds that its sources' journals hold, because the projection drops baselines and so no baseline retires an older shape for it; a stream type's `evolve` may retire an upcaster, and a history projection keeps its own handling of each older shape while any event of that version is stored (E-9, E-25, D2, D5) +- [extension] A history row's `sourceVersions` hold, per source stream, the highest event version folded into it; `applyHistoryProjection` drops every event whose version is not above the recorded version for its stream and skips the write when none remain, so a live update and a backfill page never fold one event twice, and the live path never meets a non-active generation of a history view, because its generation in flight exists only while its source scopes are closed and `rollbackGeneration` of `spec:application.rebuild` refuses to reopen a retired one (E-9, D9, F1) +- [extension] A history projection is bound once per source stream type on the command declaration; the pipeline's step 9 feeds it, once per command, the events of every `streams` entry its bindings match, and the backfill feeds it each source's history, so a history row has one projection and several bindings; a `Projection` takes one entry's DTO, so a per-entity read model derives each row from one source stream (E-9, E-7, D8) +- [extension] Every command declaration whose `writes` names a source of a history projection binds the projection for each such source, whether or not its events carry a key the view folds, so every event of a source stream reaches `keyOf` on the live path and a row's recorded version for a stream means that every event of that stream with the row's key up to it is folded, on the live path as under the pause; a declaration that writes a source without the binding is a defect, which a pure test over the application's declarations finds (E-9, E-7, D8) +- [extension] A history projection declares one version, so no generation of a view is written by two versions at once: the command pipeline's step 9 fails as a technical failure on a generation that records another version, which `applyHistoryProjection` refuses, and on a view with no generation to write; so a command whose declaration binds the view fails as a technical failure only when it passes step 7 while the view has no writable generation of the deployed version, which happens after a deploy that changes the version or adds the view's first binding, until `startGeneration` closes the view's sources, and after the abort of a rebuild under a deployed version other than the active generation's, or of an installation whose bindings are still deployed, until compatible code is deployed or a replacement generation's start closes the sources again (E-9, D8, D9, Law 6) +- [extension] The release procedure for four cases, a deploy that changes the version, a deploy that adds the view's first binding, the abort of a rebuild and the abort of an installation, is the handoff of `spec:application.history-rebuild#design.historyHandoff` and the protected abort of `spec:application.history-rebuild#design.historyAbortHandoff`, which hold `all` closed through every window above: under them a caller sees `writePaused`, a transient refusal, while `all` is closed and, for a writer of a source, while the source scopes are closed, until the switch or the protected abort, and never a technical failure; technical failures remain only for an operator who skips them, only in the windows above, and stay technical failures, never turned into transient refusals; the abort of a rebuild whose deployed version is the active generation's opens no window, so after it the operator reopens `all`, after the run's acceptance when the abort is part of a restore's recovery, and commands write as before, with no deploy (E-9, E-8, D9, Law 6) +- [extension] What a caller sees through the release procedure, window by window: ordinary answers before `all` closes; `writePaused` for every command that reaches step 7 while `all` is closed, with grant changes through `insertGrant` and `revokeGrant` refused alike and every rebuild batch, the new generation's first included, and every tenant-fill batch parked; `writePaused` only for the writers of the view's sources while the generation's source scopes stay closed through backfill and verification; and ordinary answers after `switchGeneration`, or after the protected abort once it reopens `all`, at once for a rebuild whose deployed version is the active generation's and otherwise after the compatible deploy, and inside a restore's recovery not before the run is accepted; a refusal before step 7 and a duplicate answered from its receipt keep their answers throughout, and a first installation's parent queries have no active generation to read until the switch (E-9, E-8, D9, D4) +- [extension] A history projection's version also fixes what its generations' saved cursors mean: a change of the fold or of a source's `list` or `history` contract is a new version, never a change under the same one, because the history rebuild's compatibility check compares the version and the order and identity of the sources and cannot see a change of meaning beneath them, and code moved without such a change keeps the batches' function references and what each saved cursor names (E-9, E-24, D9, D19) + +## Design + +The shapes are TypeScript in the parent. `EventEnvelope` is the envelope `spec:context.event-envelope` pins, which the adapter's `streams` entry carries on the live path and the context's `history` query on the backfill. `ApplyMode`, `ApplyResult`, `WritableGeneration`, `projectionOf`, `rowConventions` and the `by_key` index are those of `spec:application.projection-contract`, which this form shares with the per-entity form, and `MutationCtx` is Convex's mutation context. The example domain's rules read `HistoryEvent`, the part of `EventEnvelope` they need, and live in `example/domain/orderAllocationHistory.ts`. + +- typeHistoryProjection: `interface HistoryProjection { name: string; version: number; table: string; rowBudgetBytes?: number; keyOf(tenantId: string, event: EventEnvelope): string | null; project(tenantId: string, prior: Row | null, events: EventEnvelope[]): Row }` where `EventEnvelope` is the envelope of `spec:context.event-envelope`, `null` from `keyOf` means the event is not this view's, `prior` is `null` before the key's first event, `events` are the events of one key not yet folded into `prior`, each stream's in version order and the streams interleaved in any order, `project` answers a row and never `null`, and `rowBudgetBytes` is 16 KiB when absent; a history row is a bounded summary of history, never a list that grows with it, so `rowBudgetBytes` binds it like any row (E-9, D8, D9, F13) +- fnApplyHistoryProjection: `applyHistoryProjection(ctx: MutationCtx, projection: HistoryProjection, input: { tenantId: string; events: EventEnvelope[]; mode: ApplyMode; generations: readonly WritableGeneration[] }): Promise<{ key: string; results: ApplyResult[] }[]>` checking first each generation given: one whose `projectionVersion` is not the projection's `version` makes it throw the plain `Error` of `projectionOf`, a technical failure, before it drops or groups any event or reads a row, so a command whose events fold nothing, such as `ReceiveStock`, fails alike on a mismatch; then dropping the events of type `baseline`, grouping the rest by `keyOf` in the order given and leaving out those whose key is `null`, and per key and generation reading the row by `by_key`, dropping every event whose `streamVersion` is not above the row's recorded version for the event's stream, named by the envelope's `tenantId`, `contextId`, `streamType` and `streamId`, answering `skipped-newer` when none remain, and otherwise writing `project(tenantId, prior, remaining)` with `sourceVersions` raised to the highest version folded per stream and sorted by `contextId`, then `streamType`, then `streamId`, strings compared by UTF-16 code units, so the row is one value whatever the order the streams arrived in, and answering `inserted` or `updated`, never `deleted`, in the same mutation; `mode` is read by nothing (E-9, D8, D9, F1, Law 6) +- tableOrderAllocationHistory: `orderAllocationHistory: defineTable({ ...rowConventions, orderId: v.string(), placedAt: v.optional(v.number()), cancelledAt: v.optional(v.number()), allocations: v.array(v.object({ stockItemId: v.string(), quantity: v.number(), allocatedAt: v.number(), releasedAt: v.optional(v.number()) })) }).index("by_key", ["tenantId", "generation", "key"])` the history-dependent cross-stream view of the example domain, the `HistoryProjection` named `orderAllocationHistory` over the table of that name, keyed by order ID and folded from `OrderPlaced` and `OrderCancelled` in Orders and `StockAllocated` and `AllocationReleased` in Inventory, with one entry of `allocations` per stock item the order allocated on and `sourceVersions` holding the order stream and every stock stream folded, so a row holds at most 100 entries and 101 versions; `getConvexSize` measures the largest such row, at 256-byte tenant and order IDs and 100 stock item IDs of 64 bytes, every allocation released, at 55,803 bytes, so the view declares `rowBudgetBytes` 65,536, the ceiling; it is bound once per source stream type, rebuilds only under the write pause, and Sc L2-6 builds it (E-9, E-46, D9, F13, Sc L2-6) +- typeHistoryEvent: `type HistoryEvent = { streamId: string; eventType: string; eventSchemaVersion: number; recordedAt: number; occurredAt?: number; payload: unknown }`, the part of an event envelope the order allocation history's rules read, so that every `EventEnvelope` of `spec:context.event-envelope` is one and `keyOf` and `project` of the view take it where `HistoryProjection` hands them envelopes (E-9) +- typeOrderAllocation: `type OrderAllocation = { stockItemId: string; quantity: number; allocatedAt: number; releasedAt?: number }`, the quantity one stock item allocated to the order, when, and when it was released (E-9) +- typeOrderAllocationHistoryFields: `type OrderAllocationHistoryFields = { orderId: string; placedAt?: number; cancelledAt?: number; allocations: OrderAllocation[] }` where `type OrderAllocation = { stockItemId: string; quantity: number; allocatedAt: number; releasedAt?: number }`, the projected fields of the order allocation history, in `example/domain/orderAllocationHistory.ts` (E-9, E-46, Sc L2-6) +- fnOrderAllocationHistoryKeyOf: `orderAllocationHistoryKeyOf(tenantId: string, event: HistoryEvent): string | null` in `example/domain/orderAllocationHistory.ts`, answering the stream ID of an `OrderPlaced` or an `OrderCancelled`, the payload's `orderId` of a `StockAllocated` or an `AllocationReleased`, and `null` for every other event, where `type HistoryEvent = { streamId: string; eventType: string; eventSchemaVersion: number; recordedAt: number; occurredAt?: number; payload: unknown }` is the part of `EventEnvelope` the rules read, so the example's pure domain code imports no context library (E-9, E-46, Sc L2-6) +- fnProjectOrderAllocationHistory: `projectOrderAllocationHistory(tenantId: string, prior: OrderAllocationHistoryFields | null, events: readonly HistoryEvent[]): OrderAllocationHistoryFields` in the same file, folding the events one by one into `prior`, or, when `prior` is `null`, into `{ orderId, allocations: [] }` with the key of the first event that has one, and throwing the plain `Error` `"The order allocation history needs a prior row or an event of an order"` when there is neither; the view's `HistoryProjection` holds this function and `orderAllocationHistoryKeyOf` as its `project` and `keyOf` (E-9, E-46, Sc L2-6) +- orderAllocationHistoryFold: each field is written by the events of one stream, each event's time being its `occurredAt`, or its `recordedAt` when absent: `OrderPlaced` sets `placedAt` and `OrderCancelled` sets `cancelledAt`; `StockAllocated` on a stock item sets that stock item's entry of `allocations` to the payload's `quantity` and the time as `allocatedAt`, with no `releasedAt`; `AllocationReleased` sets `releasedAt` on that entry and changes nothing when the order has none on that stock item; every other event changes nothing; `allocations` is kept sorted by stock item ID, compared by UTF-16 code units; so the fold gives one row whatever the order across streams, a stock event before `OrderPlaced` included, and a pure test folds one order's events in every interleaving of its three streams and every split into calls and finds one row (E-9, D2, Sc L2-6) +- limitHistoryFold: on the live path step 9 calls `applyHistoryProjection` once per history read model per command, so it reads and writes at most one row per distinct key among the command's events in each generation, each at most `rowBudgetBytes`, and a `PlaceOrder` of 100 lines, whose 101 `streams` entries key to one order, reads and writes the order allocation history's one row once; on the backfill it takes one `history` page of at most the progress row's `batchSize` events, as `spec:application.history-rebuild#design.limitHistoryBatch` bounds it, so one batch reads at most that many events plus as many rows (F13, E-9, E-40) + +## Verification — reviewed + +- A reviewer confirms that every `HistoryProjection` is rebuilt with `pauseRequired` on its generations, that `applyHistoryProjection` drops baselines before `keyOf` and already-folded events by the row's `sourceVersions` before it calls `project`, that no history projection's `project` answers `null`, and that the L2-6 examples assert the fold against the journal. +- A reviewer confirms that a pure test builds the largest order allocation history row, 256-byte tenant and order IDs, 100 allocations on stock item IDs at the stock item ID bound, every allocation released and 101 source versions, measures it with `getConvexSize` and asserts the measure equals the number `spec:application.history-projection#design.tableOrderAllocationHistory` states and sits under the view's `rowBudgetBytes`, so a change of the stock item ID bound, the line count or the row's fields moves that number on purpose. diff --git a/design/specs/application/history-rebuild.sdp.md b/design/specs/application/history-rebuild.sdp.md new file mode 100644 index 0000000..61a31d6 --- /dev/null +++ b/design/specs/application/history-rebuild.sdp.md @@ -0,0 +1,91 @@ +--- +id: spec:application.history-rebuild +kind: workflow +altitude: feature +readiness: defined +relations: + refines: spec:application.read-models + dependsOn: + - spec:application.rebuild + - spec:application.history-projection + - spec:application.write-pause + - spec:application.generation-registry + - spec:context.queries + - spec:context.event-envelope + constrainedBy: + - spec:laws.law11-tenant-scope-named + - spec:facts.f01-serializable-mutations-under-occ + - spec:facts.f13-transactions-have-limits + - spec:facts.f19-nested-calls-share-time-budgets + decidedBy: + - spec:decisions.d09-rebuild-online-by-default + - spec:decisions.d04-four-outcomes + - spec:decisions.d19-operations-travel-with-capability +--- +# Rebuild of a history view under a write pause + +Layer 2 · Detail: full · Traces: D2, D4, D9, D10, D18, D19, Law 11, F1, F13, F19, Sc L2-6, E-8, E-9, E-24, E-38, E-40, E-48, E-49. + +For a read model that depends on history across several streams the rebuild runs under a write pause, because it needs a consistent cut; the scope closed is the view's sources, the context and stream type pairs its bindings name, so every use case that writes one of them and the journal's baseline driver read the gate and are refused, background writers are fenced by that read and by the generation's fence, and an operator can resume or abort. Such a view is a `HistoryProjection` and its backfill pages each source stream's history under the pause. The rebuild closes the gate for its own scope in the mutation that registers its generation. + +This workflow is the paused path of `spec:application.rebuild`, which holds the online path and what the two paths share: the operator entries, the batch chain with its fence and its progress row, the switch, the abort, the purge and the tenant fill. The history form and its fold are `spec:application.history-projection`, the gate and its source scopes are `spec:application.write-pause`, the cursor's fields are pinned by `spec:application.generation-registry`, and each source is read through its `list` and `history` queries of `spec:context.queries`. + +## Intent + +- problem: A cross-stream history view is rebuilt without a consistent cut (Sc L2-6) +- outcome: A history-dependent view rebuilds under a write pause with safe resume and abort, and writes reopen only after verification (D9, Sc L2-6) +- risk: A paused rebuild closes its view's sources in every tenant until the switch or the abort; its backfill and its verify pass each take one batch per source stream, one more for each further `history` page of a stream longer than the batch size, and one per tenant and source to end that source's enumeration, so the pause lasts about twice that count times one batch; a batch that took streams until its event bound was spent, or a generation of one tenant, which the registry does not offer, would shorten it (D9, E-8, E-40, D18) + +### Open questions + +- [non-blocking #historyBatchShape] Extension E-40, the shape of a history batch: a batch takes one stream, so a pass costs at least one scheduled mutation per source stream and the pause lasts about twice the stream count times one batch; the alternative is a batch that takes streams from one `list` page until its event budget is spent, one `history` call per stream, with the byte bound counting one baseline per stream; the first paused rebuild on the native backend measures one batch, and the shape is chosen with that number (E-40, E-9, F13, F19, D9) +- [non-blocking #verifyPassUnderPause] The verify pass under the write pause: a history view's generation is built while no source changes, so its verify pass repeats the backfill's batches and, by `spec:application.history-rebuild#design.historyVerify`, finds nothing over a covered generation, which doubles the pause; the option kept here is the pass, as D9's "Verify coverage, then switch" and Sc L2-6's "writes reopen only after verification" read, and whether a paused generation is `verified` when its backfill ends is the owner's (D9, Sc L2-6) +- [non-blocking #historyBatchWriteThroughput] Whether a history view's batch chain, at up to 2 MiB of rows per batch scheduled with `runAfter(0)`, is held by the deployment class's write throughput limit, `spec:facts.f13-transactions-have-limits#design.limitWriteThroughput`, which the first paused rebuild on the native backend and one hosted run show (F13, E-40) +- [non-blocking #historyMigrationBeforePause] Must every source's migration that reads history finish before `startGeneration` closes its source scopes, with abort, migration completion and a fresh generation as the recovery if it did not? If enumeration must instead remain possible during that migration, which context query provides it without constructing a DTO of the wrong meaning? +- [non-blocking #abortSwitchAtomicity] Should the native-backend abort example inject failure in the second source's audit insert, assert unchanged generation, progress, gate entries and audit records, then retry after removing the fault and assert one successful abort with both sources reopened? The same atomicity check should cover the switch that reopens them. +- [non-blocking #historyTenantPauseIsolation] Should the native-backend example use two tenants with equal local order and stock item IDs, check their folds separately, and refuse a command that writes a paused source without a history binding in both tenants, while a command writing only an unrelated source remains allowed? Include a retry of resume after its response is lost so the same saved event page cannot change the fold twice. + +## Workflow + +- rule: Under the write pause every writer in the scope reads the maintenance gate, background writers are fenced or drained, and an operator can resume or abort (D9) +- rule: A refused write under the pause is a transient refusal, retryable and never a stored rejection (D4, D9) +- rule: [extension] Under the pause a history row's recorded version for a stream means every event of that stream with the row's key up to that version is folded: a stream's events reach the fold in version order, from its first event on the backfill and the verify pass and after its last folded version on the live path, a generation in flight is written by its own batches alone while its sources are closed, and no retired generation is reopened; so a verify pass that repeats the backfill and folds no event finds the generation covering every source (E-8, E-9, D9, F1) +- rule: [extension] A rebuild that needs a consistent cut closes the sources of the paused view, one `source:` scope per context and stream type its bindings name, so every writer of those streams is held whether or not it maintains the view; the closed entries name the generation, and `abortGeneration` and the switch's resume reopen every entry that names it (E-8, E-38, D9, Sc L2-6) +- rule: `startGeneration` closes one entry per source of a paused view and records its own generation on each in the mutation that inserts the generation row, so every entry that names a generation names a row that exists (D9, E-8) +- rule: [extension] A source scope closes only for a generation `startGeneration` registers: `rollbackGeneration` of `spec:application.rebuild` refuses a retired generation with `pauseRequired`, because no function would close its sources while live commands fold onto rows that missed the events since its retirement, so a history view returns to earlier rules only through a new generation, whose start closes its sources again (E-8, D9, Sc L2-6) +- For a history-dependent read model the generation row records `pauseRequired`, and `startGeneration` closes the maintenance gate for every source scope of the read model's bindings in the mutation that inserts the row, recording its own generation on each closed entry, so the gate is closed before the first batch is scheduled and this generation's batches are the one background writer allowed through (D9, E-8, E-38) +- While the gate is closed for the sources, every use case whose declaration writes one of them and the journal's baseline driver refuse with `writePaused` before any write, whether or not they maintain the view; a batch scheduled before the close reads the closed gate when it starts and refuses, a batch running when the close commits loses the optimistic-concurrency race on the gate document and re-runs into the refusal, and a batch of a superseded chain refuses because its generation fence is stale; nothing needs to drain (D9, D4, E-8, E-38, F1) +- Backfill under the pause enumerates a consistent cut because no source changes: each history batch takes the next stream of its cursor's source in its tenant, reads one `history` page of it from the saved event cursor, folds the page into the building generation through `applyHistoryProjection` in backfill mode, and saves the source, the stream and the event cursor on the progress row, moving to the next stream when the page reports `isDone`, to the next source when that source's enumeration in the tenant ends, and to the next tenant after the last source; an interrupted batch resumes from its cursor with the gate still closed, and the gate never reopens by itself (D9, E-9, Sc L2-6) +- Verification runs under the pause: the verify pass repeats the backfill's batches from the first tenant and source and counts as a miss every row they insert or update, which over a covered generation is none, so a miss is repaired by the batch that found it and the generation is `verified` when the pass ends; `switchGeneration` resumes every closed entry that names the generation in the mutation that makes it active, so writes reopen only after verification (D9, E-9, Sc L2-6) + +## Design + +Every entry below extends `spec:application.rebuild`: a history batch keeps that workflow's `step1` to `step3` and checkpoints as its `step6` does, and `startGeneration`, `switchGeneration` and `abortGeneration` are its operator entries. + +- typeHistorySource: `type HistorySource = { source: SourceRef; list: FunctionReference<"query", "internal", ListArgs, PaginationResult<{ version: StreamVersion }>>; history: FunctionReference<"query", "internal", { tenantId: string; streamId: string; fromVersion?: number; paginationOpts: PaginationOptions }, PaginationResult> }` one source stream type of a history view, with the references to its `list` and `history` queries of `spec:context.queries` in its context component, such as `components.inventory.queries.stockItem.list` and `components.inventory.queries.stockItem.history` (E-8, E-9, D10) +- typeHistoryRebuildTarget: `type HistoryRebuildTarget = { readModel: HistoryProjection; sources: readonly [HistorySource, ...HistorySource[]] }` one history view the deployment can rebuild under the write pause, with every context and stream type pair its bindings name, in the order a pass takes them; the order allocation history's lists the stock item source first and the order source second (E-8, E-9, E-38, D9) +- fnHistoryPageCall: `ctx.runQuery(source.history, { tenantId, streamId, paginationOpts: { numItems: batchSize, cursor: eventCursor } })` with `source` the entry of the target's `sources` the cursor names and `batchSize` the progress row's, the one `history` call a history batch makes beside its one `list` call of `numItems` 1, returning a `paginator` page of envelopes in version order, capped by the query's own item and byte bounds, that the batch folds through `applyHistoryProjection` (D2, D10, E-9, E-24) +- historyCursor: a history view's batch cursor adds `sourceIndex`, the position in the target's `sources`, so a pass goes tenant by tenant in tenant-ID order, source by source in `sources` order within a tenant, stream by stream in the source's `list` order and page by page in version order within a stream, and reads the stream a cursor names from that source's `history`, so two sources with equal local stream IDs never meet; a pass of a history view costs at least one batch per tenant and source, the batch whose `list` page holds no stream (E-8, E-40, E-49, Law 11) +- historyStart: for a `HistoryRebuildTarget`, `startGeneration` takes the version the projection declares and refuses a given `projectionVersion` that differs with the message of `projectionOf`, sets `pauseRequired: true`, and, after inserting the generation row and the progress row and before scheduling, calls the write pause's `closeScope` once per entry of `sources`, in order, with `"source:" + contextId + ":" + streamType`, the reason `"rebuild of " + name + " generation " + n`, the operator and the new generation's ID, so a scope already closed or a full gate refuses the whole start and no batch is scheduled (D9, E-8, E-38, E-48) +- historySteps: for a history view a backfill or verify batch keeps steps 1 to 3 and replaces steps 4 to 6: with `source` the entry of `sources` at the cursor's `sourceIndex`, 0 when the cursor is `null`, when the cursor names no stream it calls `source.list` with `numItems` 1, `includeDeleted` and the page cursor; a page with no DTO reads no history: when it is not done the batch checkpoints at its `continueCursor`, and when it is done that source ends in the tenant and the batch checkpoints at the next source of the same tenant with a `null` page cursor, or after the last source at `{ tenantId: next, pageCursor: null, sourceIndex: 0 }` for `next = nextTenant(ctx, tenantId)`, the pass being done when `next` is `null`; a page with a DTO names the stream, its `version.streamId`, with a `null` event cursor and the page's `continueCursor` as the page cursor; for the stream the cursor or the page names, the batch then calls `source.history` as `fnHistoryPageCall` states, calls `applyHistoryProjection(ctx, target.readModel, { tenantId, events: page, mode: "backfill", generations: [thisGeneration] })`, adds one to `rowsWritten`, or to `misses` in a verify batch, for each `inserted` or `updated` and one to `rowsSkipped` for each `skipped-newer`, and checkpoints with the stream and the page's `continueCursor` as the event cursor, or with neither when the page reports `isDone`; every checkpoint patches the progress row and schedules the next batch as step 6 does for a pass that is not done, and the batch whose pass is done moves the generation and the progress row as step 6 and `verifySteps` state (D9, E-9, E-24, E-40, E-49) +- historyVerify: a verify batch of a history view runs `historySteps` from the `null` cursor the end of the backfill set, with `state` `verifying` and `pass` `verify`; under the pause no source changes, so over a generation the backfill covered every key answers `skipped-newer`, an event of type `baseline` or whose key is `null` is never a miss, and a row the pass finds missing or behind is repaired by the batch that read the events; the generation is `verified` when the pass ends, as for every read model, and the pass reads every source's history once more under the pause; because the write rule drops every event at or below a row's recorded version before `project`, the pass writes a missing row and brings a lagging row up to the last event of each source stream with its key, and never folds a covered row again, so it finds a missing or lagging row and not a fold that disagrees with itself (D9, E-8, E-9, Sc L2-6) +- historyHandoff: [extension] the operator's sequence for a deploy that changes a history projection's version or adds its first binding: beforehand the sources' migrations that read history and the tenant fill have ended, other generations' passes have ended or been interrupted with their IDs kept, the gate has room for `all` beside every source scope of the view, because `startGeneration` refuses a full gate and `all` is never reopened to make room, and a compatible deploy back to the active version, or to declarations without the new bindings, is ready; the closing step, in which `closeGate` closes `all` and `getGate` confirms it; the deploy; `startGeneration` under the new code while `all` stays closed, and `getGenerations` and `getGate` confirm the generation and every source scope that names it; `getGenerations` shows the generation's first batch parked on `all`, its progress row's `lastError` naming the scope; `resumeGate` reopens the `all` entry the closing step closed and no other entry; `resumeGeneration` resumes the parked chain, because reopening `all` resumes no chain, and `switchGeneration` follows once the generation is `verified`; and `resumeGeneration` resumes each other chain the sequence interrupted or parked; a failed deploy or start keeps `all` closed until compatible code is deployed or the start succeeds; the sequence closes `all` in its closing step alone, and a replacement generation after `historyAbortHandoff`, which holds `all` closed already, enters it after that step; an abort of the generation while the sequence holds `all` closed is `historyAbortHandoff`, which continues with it closed (E-8, E-9, D9, D19) +- historyAbortHandoff: [extension] the protected abort, the operator's sequence for the abort of a history generation, a rebuild's or an installation's: when `all` is open, `closeGate` closes it and `getGate` confirms it, as in the closing step of `historyHandoff`, and when the operator already holds it closed, as `historyHandoff` and the restore's recovery of `spec:application.restore#design.finishCompatibility` do, `getGate` confirms it and the abort continues with it closed, so no step closes a scope the operator holds; then `abortGeneration`, which reopens the generation's source scopes while `all` stays closed; `all` stays closed after the abort until the deployed code is compatible with what the abort left: at once for a rebuild whose deployed projection version is the active generation's, with no deploy; for a rebuild under another version, once the prepared compatible deploy brings the deployed version back to the active generation's; for an installation, once a compatible deploy removes the view's bindings from the declarations and withholds its parent queries, its table, target and entries kept until `purgeGeneration` has purged the aborted rows; and, for an abort inside the restore's recovery, also until `finishRestoreRun` accepts the run, as `finishCompatibility` states; then `resumeGate` reopens the `all` entry, commands write as before, and `resumeGeneration` resumes each other chain the sequence interrupted or parked, and `fillTenants` a tenant fill it parked, as in `historyHandoff`, because a parked batch schedules nothing and reopening `all` resumes none; the other exit from a rebuild under another version or an installation is a replacement generation, a fresh rebuild under the desired code or a new installation, which enters `historyHandoff` after its closing step, with the deploy when the desired code is not the deployed code, since `all` is already closed by this sequence and confirmed, and which `startGeneration` accepts because the aborted generation is no longer `building`, `verifying` or `verified`; a compatible deploy is not an older commit by default, because newer stream data can make older code invalid (E-8, E-9, D9, D19) +- historyCompatibility: [extension] a generation with `pauseRequired` is compatible with the deployed code when its read model's target is a `HistoryRebuildTarget` whose distinct `sources` give, in order, the same `source:` scope keys as the closed entries of the stored gate document that name the generation, read in their array order and never sorted, and when the projection's declared `version` equals the generation row's `projectionVersion`; a missing gate document, a missing target, or a difference of length, order, identity or version is incompatible; the check reads the gate document itself, never through `gateAllows`, whose restore door reads none, and adds no field to the generation row; its version half is one rule with the throw of `applyHistoryProjection`, which refuses a generation of another version first, before it reads any event or row, a technical failure on the live path, and since that throw ends a folding batch's chain with no `lastError` and never reaches a batch whose page is empty, which calls no write helper, `resumeChain`, `switchGeneration` or the restore's `finishRestoreRun`, this check, made first, parks the batch with a `lastError` and refuses the other three (E-8, E-9, D9, D19) +- historyMismatch: [extension] a backfill or verify batch of a history view makes the check of `historyCompatibility` after step 1's state and fence checks and before step 2, so before it reads the cursor, the tenant list or a source; on a mismatch it keeps the cursor, the counts, the rows, the generation's state and the gate as they are, patches the progress row's `lastError` to one line naming the generation and what differs and `updatedAt`, and schedules nothing, so the source scopes stay closed and the pause holds; `resumeChain`, for a backfill or verify pass, and `switchGeneration` make the check before they change the fence, the state or a gate entry and throw a plain `Error` naming the difference on a mismatch; `abortGeneration`, `purgeGeneration` and the resume of a purge make no check, because an abort or a switch removes the entries the check reads and a purge reads only the read model's table; the recovery is compatible code and `resumeGeneration`, or `historyAbortHandoff` and a fresh generation, which starts from a `null` cursor, never a translated `sourceIndex` or rows kept from the incompatible pass (E-8, E-9, D9) +- limitHistoryBatch: [extension] a history batch reads at most the progress row's `batchSize` events, which `startGeneration` sets through `batchSizeFor` like every read model's and nothing changes after, so the one formula of `limitBatchSize` gives 100 events at the default 16 KiB row budget and 32 at the 64 KiB ceiling, the order allocation history's; one batch reads one stream row of at most the stream's `budgetBytes` through its one `list` call, those events through its one `history` call, each at most `limitEventBytes` of `spec:context.event-envelope` and the 61 bytes of a stored document's system fields, with the page's baselines held to about 9 MiB by the query's byte bound, and at most one history row per event, at most 2 MiB, and writes at most 2 MiB, under the ceilings with the 1 second timeout in mind (E-40, E-9, E-24, F13) + +## Verification — reviewed + +- A reviewer confirms that `startGeneration` closes one entry per source of a paused view and nothing else, and that the write-pause examples assert a refusal of a source writer while closed, the fold of the history view against the journal, and a reopen only after verification. +- A reviewer confirms that a history target's `sources` name exactly the context and stream type pairs the view's bindings name, so `startGeneration` closes every writer of its sources, and that `rollbackGeneration` refuses every generation with `pauseRequired`. +- A reviewer confirms that the test of `spec:application.history-rebuild.write-pause-rebuild-version-change`, at the native backend tier, deploys a changed projection version through `historyHandoff` and finds every command refused `writePaused` while `all` is closed, only the writers of the view's sources refused while the source scopes stay closed, and ordinary answers after the switch, with no technical failure at any step. +- A reviewer confirms that the test of `spec:application.history-rebuild.write-pause-rebuild-installation`, at the native backend tier, deploys the view's first binding through `historyHandoff` and finds the same three windows, with no command failing for a missing generation. +- A reviewer confirms that a test at the native backend tier makes `startGeneration` fail inside `historyHandoff` and finds `all` still closed, no generation row inserted and every command refused `writePaused` until compatible code is deployed or the start succeeds. +- A reviewer confirms that a test at the native backend tier finds the new generation's first batch parked on `all` with its `lastError` before the operator reopens `all`, and that reopening `all` schedules nothing until `resumeGeneration` resumes the chain. +- A reviewer confirms that a test at the native backend tier aborts a rebuild whose deployed version is the active generation's through `historyAbortHandoff` and finds ordinary answers once `all` is reopened, with no deploy. +- A reviewer confirms that a test at the native backend tier aborts a rebuild of a changed version through `historyAbortHandoff` and finds the active generation unchanged and every command refused `writePaused`, never failed, until the compatible deploy and the reopen of `all`, or, for a replacement generation entering `historyHandoff` after its closing step, through the windows of a version change. +- A reviewer confirms that a test at the native backend tier aborts an installation through `historyAbortHandoff`, deploys declarations without the view's bindings and finds ordinary answers after `all` is reopened, with the aborted rows purged by `purgeGeneration`. +- A reviewer confirms that a test at the native backend tier makes the operator audit insert fail on a reopen of the handoff, the `resumeGate` of `all` and the `resumeScope` of an abort or a switch, and finds the gate, the generation and its progress row as they were and the scope still closed. +- A reviewer confirms that a pure test compares a target's ordered source scope keys with gate entries mixed with unrelated closed entries, and finds a reorder, an added source, a removed source and a missing entry incompatible and the same list compatible. +- A reviewer confirms that a test at the native backend tier deploys between an interruption and its resume a reordered, added or removed source or a changed projection version, and finds the batch parked with a `lastError` naming the difference, the test running `backfillBatch` through its admin access with the generation's fence, standing for a batch scheduled before the deploy, since no batch carries the live fence after `interruptGeneration`, `resumeGeneration` refused, the cursor, the rows and the gate unchanged, and a deploy between `verified` and the switch refused by `switchGeneration`; the test of `spec:application.history-rebuild.write-pause-rebuild-source-order-mismatch` is the reordered source, with the protected abort of `historyAbortHandoff` and a fresh generation after it. diff --git a/design/specs/application/history-rebuild.write-pause-rebuild-abort.sdp.md b/design/specs/application/history-rebuild.write-pause-rebuild-abort.sdp.md new file mode 100644 index 0000000..c9e70a8 --- /dev/null +++ b/design/specs/application/history-rebuild.write-pause-rebuild-abort.sdp.md @@ -0,0 +1,40 @@ +--- +id: spec:application.history-rebuild.write-pause-rebuild-abort +kind: example +altitude: story +readiness: defined +relations: + refines: spec:application.history-rebuild + verifies: spec:application.history-rebuild +--- +# Rebuild a cross-stream history view under a write pause; interrupt, then abort + +Sc L2-6 · native backend tier · the abort case, the doc's second enumerated case of the row. + +## Intent + +- outcome: Resume or abort safely; writes reopen only after verification. (Sc L2-6) + +```gwt +Given a {viewKind: "cross-stream history"} read model with an active generation +And an order summary installed before the first order is placed, with {orderSummary: "an active generation"} +And a sample of {orders: 40} orders on {stockItems: 20} stock items, split evenly between {tenants: 2} tenants, read in batches of {batchSize: 32} events +And a new generation registered as {mode: "building under a write pause"} +And live commands {liveCommands: "are refused while paused"} +And the scope of the tenant the enumeration reaches second closed before the start, so the backfill {hold: "parks at the second tenant"} +When the backfill is interrupted while it is parked at the second tenant and then {resumption: "aborted"} +Then no row of the new generation holds data older than its source stream version +And the new generation {coverage: "is discarded and read by no query"} +And no command or external effect ran during the rebuild +And the active generation {cutover: "stays the old one"} +And writes {writes: "reopen after the abort"} +``` + +## Verification — executable + +- Runs in the native backend tier; every test owns its disposable backend. +- Before it places the first order, the test installs the order summary, which `PlaceOrder` and `CancelOrder` bind, through `startGeneration`, its batches and `switchGeneration`, so that its commands have a generation to write. +- The view is the order allocation history of `spec:application.orders-inventory-example`, a `HistoryProjection` bound to the order stream type and the stock item stream type. The sample holds 40 orders on 20 stock items, one stream each, split evenly between two tenants, and the view's batch size is 32 events. +- Before `startGeneration` the test closes, through `closeGate`, the scope of the tenant whose ID sorts second, which the enumeration reaches second, so the backfill runs every batch of the first tenant and parks at the second, whose scope each batch reads, and cannot end before the interrupt; it then starts a generation with `pauseRequired`, asserts that the gate holds that tenant's scope and one closed `source:` entry for each of the two stream types, each naming this generation, waits until `getGenerations` shows the backfill parked at the second tenant, the progress row's cursor naming it and its `lastError` naming its scope, interrupts with `interruptGeneration` while it is parked, and asserts that the pass is still `backfill`; then, as `spec:application.history-rebuild#design.historyAbortHandoff` requires while `all` is open, it closes `all` through `closeGate`, confirms it with `getGate` and calls `abortGeneration`. +- The test asserts that the generation is `aborted`, that the active generation is unchanged and that `all` is still closed; since the deployed projection's version is the active generation's, it then reopens the `all` entry it closed and the tenant's scope through `resumeGate`, and asserts that `PlaceOrder` succeeds again in both tenants, that both source entries are gone from the gate, and that no query returns a row of the aborted generation. +- The test runs `purgeGeneration` and asserts that its batches run until the generation is `purged` and that every batch deleted at most `limitPurgeBatch` rows. diff --git a/design/specs/application/history-rebuild.write-pause-rebuild-installation.sdp.md b/design/specs/application/history-rebuild.write-pause-rebuild-installation.sdp.md new file mode 100644 index 0000000..e1740eb --- /dev/null +++ b/design/specs/application/history-rebuild.write-pause-rebuild-installation.sdp.md @@ -0,0 +1,40 @@ +--- +id: spec:application.history-rebuild.write-pause-rebuild-installation +kind: example +altitude: story +readiness: defined +relations: + refines: spec:application.history-rebuild + verifies: spec:application.history-rebuild +--- +# Install a history view through the handoff + +E-8 · native backend tier · no acceptance row; the example verifies the handoff of `spec:application.history-rebuild#design.historyHandoff` for a deploy that adds a history view's first binding, an extension under E-8 and E-9. + +## Intent + +- outcome: A deploy that adds a history view's first binding leaves every caller an ordinary answer or the transient refusal `writePaused`, never a failure for a missing generation, and the view's first generation is verified, switched and written by the commands after it. (E-8, E-9, D9, D19) + +```gwt +Given a deployed composition whose command declarations bind no {viewKind: "cross-stream history"} read model +And {generationRows: 0} generation rows of the view +And an order summary installed before the deploy, with {orderSummary: "an active generation"} +And {orders: 3} orders placed on stock items received before the deploy +And a composition ready to deploy that binds the view on every declaration that writes its sources +And a gate with {closedEntries: 0} closed entries +When the operator runs the handoff: closes all, deploys, starts a generation, waits for its first batch to park, reopens all, resumes the parked chain and switches the verified generation +Then every command that reaches step 7 while all is closed is {whileAllClosed: "refused with writePaused"} +And {missingGeneration: 0} commands fail for a missing generation +And generation {generation: 1} of the view is {state: "active"} +And the view holds {rows: 3} rows for the orders placed before it +And a command after the switch {afterSwitch: "writes its order's row in generation 1"} +``` + +## Verification — executable + +- Runs in the native backend tier; every test owns its disposable backend. +- The test first deploys, through `deployTemporary`, a copy of the composition whose command declarations bind no history projection and whose rebuild targets hold no history target, installs the order summary, which `PlaceOrder` and `CancelOrder` bind, through `startGeneration`, its batches and `switchGeneration`, so that its commands have a generation to write, places three orders on stock items it received, and asserts that `generations` holds no row of the order allocation history. +- In this order the test closes `all` through `closeGate` and confirms it with `getGate`, deploys the composition that binds the order allocation history to the order stream type and the stock item stream type, sends `PlaceOrder` and asserts the refusal `writePaused` with `data.kind` `"transient"` and nothing stored, runs `startGeneration` for the view, confirms through `getGenerations` and `getGate` generation 1 with `pauseRequired` and one closed `source:` entry for each of the two stream types naming it, waits until `getGenerations` shows the first batch parked on `all`, the progress row's `lastError` naming `all` and its `batchesDone` 0, and only then reopens the `all` entry it closed through `resumeGate`. +- The test asserts that reopening `all` scheduled nothing, the progress row still showing the parked batch, runs `resumeGeneration`, waits until generation 1 is `verified`, and asserts that `PlaceOrder` sent while the source scopes are closed is refused with `writePaused`. +- The test switches the generation and asserts that the gate holds no closed entry, that generation 1 is `active`, and that the view holds one row for each of the three orders, equal to the fold of its order's events and its stock items' allocation events read from the two journals through `history`. +- The test sends `PlaceOrder` for a fourth order, asserts that it succeeds and that generation 1 holds that order's row, and asserts that no answer it received at any step was a technical failure. diff --git a/design/specs/application/rebuild.write-pause-rebuild-interrupt.sdp.md b/design/specs/application/history-rebuild.write-pause-rebuild-interrupt.sdp.md similarity index 51% rename from design/specs/application/rebuild.write-pause-rebuild-interrupt.sdp.md rename to design/specs/application/history-rebuild.write-pause-rebuild-interrupt.sdp.md index c83b4f1..7ad7328 100644 --- a/design/specs/application/rebuild.write-pause-rebuild-interrupt.sdp.md +++ b/design/specs/application/history-rebuild.write-pause-rebuild-interrupt.sdp.md @@ -1,11 +1,11 @@ --- -id: spec:application.rebuild.write-pause-rebuild-interrupt +id: spec:application.history-rebuild.write-pause-rebuild-interrupt kind: example altitude: story readiness: defined relations: - refines: spec:application.rebuild - verifies: spec:application.rebuild + refines: spec:application.history-rebuild + verifies: spec:application.history-rebuild --- # Rebuild a cross-stream history view under a write pause; interrupt, then resume @@ -17,9 +17,12 @@ Sc L2-6 · native backend tier · the resume case. ```gwt Given a {viewKind: "cross-stream history"} read model with an active generation +And an order summary installed before the first order is placed, with {orderSummary: "an active generation"} +And a sample of {orders: 40} orders on {stockItems: 20} stock items, split evenly between {tenants: 2} tenants, read in batches of {batchSize: 32} events And a new generation registered as {mode: "building under a write pause"} And live commands {liveCommands: "are refused while paused"} -When the backfill is interrupted after {batches: 2} batches and then {resumption: "resumed"} +And the scope of the tenant the enumeration reaches second closed before the start, so the backfill {hold: "parks at the second tenant"} +When the backfill is interrupted while it is parked at the second tenant and then {resumption: "resumed"} Then no row of the new generation holds data older than its source stream version And the new generation {coverage: "covers every stream enumerated at cutover"} And no command or external effect ran during the rebuild @@ -30,10 +33,11 @@ And writes {writes: "reopen only after verification"} ## Verification — executable - Runs in the native backend tier; every test owns its disposable backend. -- The view is the order allocation history of `spec:application.orders-inventory-example`, a `HistoryProjection` bound to the order stream type and the stock item stream type, whose rebuild target lists the stock item source first, so the backfill folds every stock event of an order before its `OrderPlaced`. -- The test starts a generation with `pauseRequired` and asserts that the gate holds one closed `source:` entry for each of the two stream types, each naming this generation. -- The test asserts that `PlaceOrder` and `CancelOrder` are refused with `writePaused` because their declarations' `writes` name a closed source, that the error's `data.kind` is `"transient"`, and that nothing is stored. -- The test interrupts after the second history batch, asserts the gate is still closed, resumes with `resumeGeneration`, and asserts the generation reaches `verified` with zero misses. +- Before it places the first order, the test installs the order summary, which `PlaceOrder` and `CancelOrder` bind, through `startGeneration`, its batches and `switchGeneration`, so that its commands have a generation to write. +- The view is the order allocation history of `spec:application.orders-inventory-example`, a `HistoryProjection` bound to the order stream type and the stock item stream type, whose rebuild target lists the stock item source first, so the backfill folds every stock event of an order before its `OrderPlaced`. The sample holds 40 orders on 20 stock items, one stream each, split evenly between two tenants, and the view's batch size is 32 events. +- Before `startGeneration` the test closes, through `closeGate`, the scope of the tenant whose ID sorts second, which the enumeration reaches second, so the backfill runs every batch of the first tenant and parks at the second, whose scope each batch reads, and cannot end before the interrupt; it then starts a generation with `pauseRequired` and asserts that the gate holds that tenant's scope and one closed `source:` entry for each of the two stream types, each naming this generation. +- The test asserts that `PlaceOrder` and `CancelOrder` sent in the first tenant are refused with `writePaused` because their declarations' `writes` name a closed source, that the error's `data.kind` is `"transient"`, and that nothing is stored. +- The test waits until `getGenerations` shows the backfill parked at the second tenant, the progress row's cursor naming it and its `lastError` naming its scope, interrupts with `interruptGeneration` while it is parked, and asserts that the pass is still `backfill`; it then reopens the tenant's scope through `resumeGate`, asserts that both source entries are still closed, resumes with `resumeGeneration`, and asserts the generation reaches `verified` with zero misses. - The test asserts that every row of the new generation equals the fold of its order's events and its stock items' allocation events read from the two journals through `history`, and that its `sourceVersions` hold, per stream, the version of that stream's last event with the order's key; the sample holds a cancelled order whose stock items were released after the cancel, a stock item received and never allocated, and a stock item that allocated to a second order after the first. - The test asserts that each row of the new generation equals, beside `generation`, the row the old generation holds for the same order, which live commands wrote. - The test asserts that a `PlaceOrder` sent once the generation is `verified` and before the switch is still refused, then switches, which resumes both source scopes, and asserts that the same command now succeeds and that the active generation is the new one. diff --git a/design/specs/application/history-rebuild.write-pause-rebuild-source-order-mismatch.sdp.md b/design/specs/application/history-rebuild.write-pause-rebuild-source-order-mismatch.sdp.md new file mode 100644 index 0000000..d0dc562 --- /dev/null +++ b/design/specs/application/history-rebuild.write-pause-rebuild-source-order-mismatch.sdp.md @@ -0,0 +1,43 @@ +--- +id: spec:application.history-rebuild.write-pause-rebuild-source-order-mismatch +kind: example +altitude: story +readiness: defined +relations: + refines: spec:application.history-rebuild + verifies: spec:application.history-rebuild +--- +# Reorder a history view's sources while its generation is interrupted + +E-8 · native backend tier · no acceptance row; the example verifies the check of `spec:application.history-rebuild#design.historyCompatibility` and its recovery for a deploy that reorders a history view's sources while a generation of it is interrupted, an extension under E-8 and E-9. + +## Intent + +- outcome: A generation whose sources a deploy reordered parks with a `lastError` naming the difference and is never resumed, and the protected abort and a fresh generation bring the view back. (E-8, E-9, D9, D19) + +```gwt +Given a {viewKind: "cross-stream history"} read model with an active generation +And an order summary installed before the first order is placed, with {orderSummary: "an active generation"} +And a sample of {orders: 40} orders on {stockItems: 20} stock items, split evenly between {tenants: 2} tenants, read in batches of {batchSize: 32} events +And a new generation building under a write pause over sources in the order {startedOrder: "stock item, then order"} +And the scope of the tenant the enumeration reaches second closed before the start, so the backfill {hold: "parks at the second tenant"} before the deploy +And its backfill interrupted while it is parked at the second tenant +And a deploy, at the same projection version, that lists the sources in the order {deployedOrder: "order, then stock item"} +When a batch carrying the generation's fence runs under the deployed code +Then the batch {batch: "parks with a lastError naming the source order"} +And the cursor, the rows and the gate {unchanged: "stay as the interruption left them"} +And resumeGeneration {resume: "is refused naming the difference"} +And the abort under all {abort: "succeeds and reopens both sources"} +And a fresh generation {fresh: "is verified and switched"} +``` + +## Verification — executable + +- Runs in the native backend tier; every test owns its disposable backend. +- Before it places the first order, the test installs the order summary, which `PlaceOrder` and `CancelOrder` bind, through `startGeneration`, its batches and `switchGeneration`, so that its commands have a generation to write. +- The view is the order allocation history, a `HistoryProjection` bound to the order stream type and the stock item stream type, at one projection version in both compositions; the composition the generation starts under lists the stock item source first in its rebuild target, and the copy the test deploys through `deployTemporary` lists the order source first. The sample holds 40 orders on 20 stock items, one stream each, split evenly between two tenants, and the view's batch size is 32 events. +- Before `startGeneration` the test closes, through `closeGate`, the scope of the tenant whose ID sorts second, which the enumeration reaches second, so the backfill runs every batch of the first tenant and parks at the second, whose scope each batch reads, and cannot end before the interrupt and the deploy; it then starts a generation with `pauseRequired`, asserts that the gate holds that tenant's scope and one closed `source:` entry for each of the two stream types, each naming this generation, the stock item entry first, waits until `getGenerations` shows the backfill parked at the second tenant, the progress row's cursor naming it and its `lastError` naming its scope, interrupts with `interruptGeneration` and deploys the copy while it is parked, and asserts that the pass is still `backfill`. +- The test runs `backfillBatch` through its admin access with the generation's ID and its fence, standing for a batch scheduled before the deploy, since no batch carries the live fence after `interruptGeneration`, and asserts that the progress row's `lastError` names the generation and the two source orders, that the batch scheduled nothing, and that the cursor, the counts, the rows, the generation's state and the gate are as they were before it. +- The test asserts that `resumeGeneration` is refused with a plain `Error` naming the difference and that the fence, the progress row and the gate are unchanged. +- In this order the test closes `all` through `closeGate` and confirms it with `getGate`, as `spec:application.history-rebuild#design.historyAbortHandoff` requires while `all` is open, runs `abortGeneration`, asserts that the generation is `aborted`, that the active generation is unchanged and that both source entries are gone while `all` stays closed, and, since the deployed projection's version is the active generation's, reopens the `all` entry it closed and the tenant's scope through `resumeGate` and asserts that `PlaceOrder` succeeds. +- The test runs `startGeneration` again, asserts that the gate holds the two source entries naming the fresh generation, the order entry first, waits until it is `verified`, switches it, and asserts that every row of it equals the fold of its order's events and its stock items' allocation events read from the two journals through `history`. diff --git a/design/specs/application/history-rebuild.write-pause-rebuild-version-change.sdp.md b/design/specs/application/history-rebuild.write-pause-rebuild-version-change.sdp.md new file mode 100644 index 0000000..3d9e81c --- /dev/null +++ b/design/specs/application/history-rebuild.write-pause-rebuild-version-change.sdp.md @@ -0,0 +1,40 @@ +--- +id: spec:application.history-rebuild.write-pause-rebuild-version-change +kind: example +altitude: story +readiness: defined +relations: + refines: spec:application.history-rebuild + verifies: spec:application.history-rebuild +--- +# Deploy a changed history projection version through the handoff + +E-8 · native backend tier · no acceptance row; the example verifies the handoff of `spec:application.history-rebuild#design.historyHandoff` for a deploy that changes a history projection's version, an extension under E-8 and E-9. + +## Intent + +- outcome: A deploy that changes a history projection's version leaves every caller an ordinary answer or the transient refusal `writePaused`, never a technical failure, and the generation of the deployed version is verified and switched. (E-8, E-9, D9, D19) + +```gwt +Given a {viewKind: "cross-stream history"} read model whose active generation records projection version {activeVersion: 1} +And an order summary installed before the first order is placed, with {orderSummary: "an active generation"} +And a composition ready to deploy that declares the view at projection version {deployedVersion: 2} with the same sources +And a gate with {closedEntries: 0} closed entries +When the operator runs the handoff: closes all, deploys, starts a generation, waits for its first batch to park, reopens all, resumes the parked chain and switches the verified generation +Then every command that reaches step 7 while all is closed is {whileAllClosed: "refused with writePaused"} +And every writer of the view's sources is {whileSourcesClosed: "refused with writePaused"} while the source scopes are closed +And {technicalFailures: 0} commands end in a technical failure +And the active generation records projection version {switchedVersion: 2} +And writes {writes: "reopen after the switch"} +``` + +## Verification — executable + +- Runs in the native backend tier; every test owns its disposable backend. +- Before it places the first order, the test installs the order summary, which `PlaceOrder` and `CancelOrder` bind, through `startGeneration`, its batches and `switchGeneration`, so that its commands have a generation to write. +- The view is the order allocation history, a `HistoryProjection` bound to the order stream type and the stock item stream type, at projection version 1 with an active generation; the deployed composition is a copy, deployed through `deployTemporary`, that declares version 2 with the same rebuild target. +- In this order the test closes `all` through `closeGate` and confirms it with `getGate`, deploys the copy, runs `startGeneration` for the view, confirms through `getGenerations` and `getGate` the new generation with `pauseRequired` and one closed `source:` entry for each of the two stream types naming it, waits until `getGenerations` shows the first batch parked on `all`, the progress row's `lastError` naming `all` and its `batchesDone` 0, and only then reopens the `all` entry it closed through `resumeGate`. +- The test asserts that `PlaceOrder`, `CancelOrder` and `ReceiveStock` sent while `all` is closed, before the deploy and after it, are refused with `writePaused`, that the error's `data.kind` is `"transient"`, and that nothing is stored. +- The test asserts that reopening `all` scheduled nothing, the progress row still showing the parked batch, runs `resumeGeneration`, and waits until the generation is `verified`. +- The test asserts that `PlaceOrder` sent while the source scopes are closed is refused with `writePaused`, switches the generation, and asserts that the gate holds no closed entry, that `PlaceOrder` succeeds and that the active generation records projection version 2. +- The test asserts that no answer it received at any step was a technical failure. diff --git a/design/specs/application/orders-inventory-example.sdp.md b/design/specs/application/orders-inventory-example.sdp.md index fc111e4..eddf19f 100644 --- a/design/specs/application/orders-inventory-example.sdp.md +++ b/design/specs/application/orders-inventory-example.sdp.md @@ -14,6 +14,9 @@ relations: constrainedBy: - spec:laws.law01-sanctioned-writes-only - spec:laws.law09-no-invariant-on-late-read-model + dependsOn: + - spec:context.batch-shaped-api + - spec:application.history-projection --- # Orders and Inventory, the example domain @@ -30,7 +33,7 @@ The second lifecycle command is `CancelOrder`. The stock item keeps two totals a ### Open questions -- [non-blocking] Extension E-46: the second lifecycle command, the essential summary, the command that creates stock, the unit price an order line carries, the bound on an order line's stock item ID, the rejection reasons, the stream budgets of the two stream types, the order of the two context calls and the history-dependent view Sc L2-6 builds are choices the doc leaves open; two of them are decided, that the stock item keeps totals only and `release` takes its quantities from the cancelled order's own lines, and that the order allocation history view stays in the experiment; the owner may substitute the others, and the experiment's terms move with them (E-46, E-2, E-9, First experiment) +- [non-blocking #orderInventoryCommandChoices] Extension E-46: the second lifecycle command, the essential summary, the command that creates stock, the unit price an order line carries, the bound on an order line's stock item ID, the rejection reasons, the stream budgets of the two stream types, the order of the two context calls and the history-dependent view Sc L2-6 builds are choices the doc leaves open; two of them are decided, that the stock item keeps totals only and `release` takes its quantities from the cancelled order's own lines, and that the order allocation history view stays in the experiment; the owner may substitute the others, and the experiment's terms move with them (E-46, E-2, E-9, First experiment) ## Model @@ -48,10 +51,10 @@ The second lifecycle command is `CancelOrder`. The stock item keeps two totals a - **place operation** — The Orders context's list-shaped operation `place({ orders: [{ orderId, lines }] })`, planning one order stream per order at expected version 0, so a second place of the same order ID is answered `entityExists` before its decider runs (D10, E-46, Sc L1-4). - **cancel operation** — The Orders context's list-shaped operation `cancel({ orders: [{ orderId }] })`, planning one `cancel` stream command per order with no expected version, so the order's own state decides it, and declaring `maxStreams` 32 as the place operation does; it returns `{ orders: [{ orderId, lines }] }`, the lines of each cancelled order taken from its DTO (D10, E-46). - **receive operation** — The Inventory context's list-shaped operation `receive({ items })`, planning one stock item stream per distinct stock item, which creates the stream of a stock item that has none and adds the quantity to the quantity on hand; it is the only way a stock item comes to exist (Law 1, D10, E-46). -- **PlaceOrder** — The composed use case that records an order in Orders and allocates every line in Inventory in one mutation, with one receipt and one outcome, calling Orders first because it creates the order, so a UI double submit is answered `entityExists` before Inventory decides, and Inventory second; it rejects when stock is short, and an order with no line, or with a line whose stock item ID is longer than the stock item ID bound, is refused `invalidInput` at step 1 of the command pipeline, before any read and before either context is called (D10, D4, E-39, E-46, Sc L1-4). -- **ReceiveStock** — The command that receives stock, one call to the Inventory context's receive operation with one receipt and one outcome; every stock item of the experiment, a test's and a measurement's included, is created through it, and an item whose stock item ID is longer than the stock item ID bound is refused `invalidInput` at step 1 of the command pipeline, before any read and before the Inventory context is called (Law 1, D10, E-46). +- **PlaceOrder** — The composed use case that records an order in Orders and allocates every line in Inventory in one mutation, with one receipt and one outcome, calling Orders first because it creates the order, so a UI double submit is answered `entityExists` before Inventory decides, and Inventory second; it rejects when stock is short, and an order with no line, or with a line whose stock item ID is longer than the stock item ID bound, is refused `invalidInput` at step 1 of the command pipeline, before any read and before either context is called; its declaration binds the order summary from the Orders `order` and the order allocation history from the Orders `order` and from the Inventory `stockItem` (D10, D4, E-9, E-39, E-46, Sc L1-4). +- **ReceiveStock** — The command that receives stock, one call to the Inventory context's receive operation with one receipt and one outcome; every stock item of the experiment, a test's and a measurement's included, is created through it, and an item whose stock item ID is longer than the stock item ID bound is refused `invalidInput` at step 1 of the command pipeline, before any read and before the Inventory context is called; its declaration writes the Inventory `stockItem` and binds the order allocation history from it, though `StockReceived` carries no key the view folds (Law 1, D10, E-9, E-46). - **CancelOrder** — The second lifecycle command: the composed use case `cancelOrder`, input `{ orderId: string }`, that calls the Orders context's cancel operation once with the one order and then the Inventory context's release operation once with the order ID and the stock item and quantity of each line Orders returned for that order, in one mutation with one receipt and one outcome, and answers `{ orderId: string; released: { stockItemId: string; quantity: number }[] }`, the release operation's lines; Orders comes first because the order's state refuses a cancel before Inventory decides, and the quantities are read from what Orders returned in this mutation, never from a read model (D10, D4, Law 9, E-39, E-46). -- **CancelOrder declaration** — `name: "CancelOrder"`, `contractVersion: 1`, `permission: { permission: "orders.cancel" }`, `writes` the Orders `order` and the Inventory `stockItem`, `readModels` the order summary from the Orders `order`, `rejections: ["orderNotFound", "orderAlreadyCancelled", "insufficientAllocation"]` and no `bounds`, because the input holds no list, the streams the command writes are its one order and the stock items of that order's lines, at most the 100 `PlaceOrder` admitted, and the order ID is bounded at 256 bytes as every stream ID is (D6, D10, E-38, E-46). +- **CancelOrder declaration** — `name: "CancelOrder"`, `contractVersion: 1`, `permission: { permission: "orders.cancel" }`, `writes` the Orders `order` and the Inventory `stockItem`, `readModels` the order summary from the Orders `order` and the order allocation history from the Orders `order` and from the Inventory `stockItem`, `rejections: ["orderNotFound", "orderAlreadyCancelled", "insufficientAllocation"]` and no `bounds`, because the input holds no list, the streams the command writes are its one order and the stock items of that order's lines, at most the 100 `PlaceOrder` admitted, and the order ID is bounded at 256 bytes as every stream ID is (D6, D10, E-38, E-46). - **permissions** — `orders.place`, which `PlaceOrder` requires, `orders.cancel`, which `CancelOrder` requires, `inventory.receive`, which `ReceiveStock` requires, and the read permissions `orders.read`, which the parent queries over orders and the order summary require, and `inventory.read` (D11, E-37, E-46). - **OrderPlaced** — The Orders event recorded by `PlaceOrder`, carrying the accepted lines and the total, `{ lines, total }`, bounded by the events-stay-small constraint (First experiment, D2). - **OrderCancelled** — The Orders event recorded by `CancelOrder` on a placed order, with the empty payload `{}` and the time in `occurredAt`; folding it sets the order's status to `cancelled` and keeps its lines, total and `placedAt` (E-46, D2). diff --git a/design/specs/application/parent-use-cases.sdp.md b/design/specs/application/parent-use-cases.sdp.md index c9b3d5a..af6b892 100644 --- a/design/specs/application/parent-use-cases.sdp.md +++ b/design/specs/application/parent-use-cases.sdp.md @@ -9,6 +9,8 @@ relations: - spec:context.context-component - spec:context.batch-shaped-api - spec:command.command-pipeline + - spec:command.command-declaration + - spec:context.persistence-adapter constrainedBy: - spec:laws.law01-sanctioned-writes-only - spec:laws.law02-state-and-events-commit-together @@ -51,9 +53,9 @@ The boundary sits at the component API. The use case sees DTOs and stream versio ### Open questions -- [non-blocking] Extension E-41: the doc says a too-large operation is rejected or becomes a separate import command with honest partial progress; it does not say what the import command records. The option taken here: the input bound is the declaration's `bounds.maxItems` of `spec:command.command-declaration` under E-38, checked by the pipeline's step 1 and by no body, the rejection is the outcome boundary's reserved `operationTooLarge`, and this extension owns only the import command, a caller-driven sequence of bounded batch commands that record progress on an `imports` row (E-41, E-38, D10) -- [non-blocking] Extension E-44: the doc says keeping a refused request as a fact is a business policy chosen per command, and the executor signature is pinned by `spec:command.command-declaration`; what neither says is how a use case declares what a context's business failure means to it. The option taken here is a `BusinessFailurePolicy` per context call, carry or reject, applied inside the executor body through the pinned helper `carryOrReject` right after the context call, because the executor is a plain function and the declaration carries no slot for it; `reject` rolls back the events the context recorded as its business failure, which D4 says belong in history, and the doc's per-command choice is read as the use case's because D10 makes the use case the command, so the owner confirms that a context's business failure may be overridden by its caller; the throw shape, the kernel's bare `Rejection` that a context and a use case's own code both throw and the pipeline's rethrow normalizes, is taken here, under E-5 for the boundary's rethrow, and the owner has not ruled on it (E-44, E-5, D4, D7, D10) -- [non-blocking] Extension E-39: the doc fixes no order for a use case's context calls; the option taken here is one fixed order the executor states, the creating context first, with no failure handling of a later call, because the calls before it roll back with the mutation; the owner has not ruled (E-39, D1, D10) +- [non-blocking #importCommandProgress] Extension E-41: the doc says a too-large operation is rejected or becomes a separate import command with honest partial progress; it does not say what the import command records. The option taken here: the input bound is the declaration's `bounds.maxItems` of `spec:command.command-declaration` under E-38, checked by the pipeline's step 1 and by no body, the rejection is the outcome boundary's reserved `operationTooLarge`, and this extension owns only the import command, a caller-driven sequence of bounded batch commands that record progress on an `imports` row (E-41, E-38, D10) +- [non-blocking #contextBusinessFailurePolicy] Extension E-44: the doc says keeping a refused request as a fact is a business policy chosen per command, and the executor signature is pinned by `spec:command.command-declaration`; what neither says is how a use case declares what a context's business failure means to it. The option taken here is a `BusinessFailurePolicy` per context call, carry or reject, applied inside the executor body through the pinned helper `carryOrReject` right after the context call, because the executor is a plain function and the declaration carries no slot for it; `reject` rolls back the events the context recorded as its business failure, which D4 says belong in history, and the doc's per-command choice is read as the use case's because D10 makes the use case the command, so the owner confirms that a context's business failure may be overridden by its caller; the throw shape, the kernel's bare `Rejection` that a context and a use case's own code both throw and the pipeline's rethrow normalizes, is taken here, under E-5 for the boundary's rethrow, and the owner has not ruled on it (E-44, E-5, D4, D7, D10) +- [non-blocking #useCaseContextOrder] Extension E-39: the doc fixes no order for a use case's context calls; the option taken here is one fixed order the executor states, the creating context first, with no failure handling of a later call, because the calls before it roll back with the mutation; the owner has not ruled (E-39, D1, D10) ## Behavior diff --git a/design/specs/application/projection-contract.sdp.md b/design/specs/application/projection-contract.sdp.md index def3f02..0bfda2d 100644 --- a/design/specs/application/projection-contract.sdp.md +++ b/design/specs/application/projection-contract.sdp.md @@ -19,7 +19,7 @@ relations: Story · Detail: full · Traces: D8, D9, Law 6, Law 10, Law 11, F1, F13, E-8, E-9. -A projection is the named, versioned, deterministic function that turns a context's DTO at a stream version into a read-model row, and `applyProjection` is the one helper that writes that row, shared by the use case's live update and the rebuild's backfill. The contract pins the projection's shape, the declaration of a per-entity read model, which names its table and row budget and holds its projection, the row conventions every read model follows, the write rule that decides between insert, update and skip, the aggregate form with its per-entity marker, and the history form that folds the events of several streams into one row. Everything here is an extension of D8 and D9: the doc fixes the properties, this contract fixes the signature. +A projection is the named, versioned, deterministic function that turns a context's DTO at a stream version into a read-model row, and `applyProjection` is the one helper that writes that row, shared by the use case's live update and the rebuild's backfill. The contract pins the projection's shape, the declaration of a per-entity read model, which names its table and row budget and holds its projection, the row conventions every read model follows, and the write rule that decides between insert, update and skip. The aggregate form with its per-entity marker and the history form that folds the events of several streams into one row have Specs of their own, `spec:application.aggregate-projection` and `spec:application.history-projection`. Everything here is an extension of D8 and D9: the doc fixes the properties, this contract fixes the signature. ## Intent @@ -30,12 +30,11 @@ A projection is the named, versioned, deterministic function that turns a contex ### Open questions -- [non-blocking] Extension E-9: the doc says a projection is named, versioned and deterministic and that live update and rebuild share its logic; it does not give the signature or the row shape, for counts and sums it names the marker but not the aggregate row, and for the read models that depend on history across several streams, the ones the write pause exists for, it names no input. The option taken here: a per-entity read model declared as a `ReadModel` with `name`, `table`, `rowBudgetBytes` and its `projections`, a list of `Projection` with the latest first, each holding `version`, `keyOf` and `project` alone, rows with `tenantId`, `generation`, `key`, `projectionVersion` and `sourceVersions`, one `applyProjection` helper with a mode per caller that writes each generation by its role, active, building or verifying, with the projection whose version the generation records, for an `AggregateProjection` an aggregate row with `value` and `entityCount` and no `sourceVersions`, because the markers carry them, written by `applyAggregate` under the marker's version compare with the mode ignored, and for a history-dependent view a `HistoryProjection` that folds events into the prior row, fed on the live path by the events the operation appended, which the adapter's `streams` entry carries, and on the backfill by the context's `history` pages under the write pause, with `sourceVersions` recording the highest version folded per stream so that an event is never folded twice; a row that derives from more than one source stream type is a history row, its projection bound once per source stream type, and a small cross-context view of current state is D8's parent query over bounded component reads; the owner confirms the three forms and has not ruled on the split of a per-entity read model into `ReadModel` and `Projection`, on the write by generation role or on the row measured against its budget at every write (E-9, E-23, E-24, D8, D9) -- [non-blocking] Extension E-9, two projection versions: the doc says live update and rebuild share one projection and does not say how a command writes two generations built by two versions; the option taken here is that a read model declares its projection versions as a list, the latest first, and `applyProjection` writes each generation with the projection whose version the generation row records, failing as a technical failure when the read model declares none of that version, so the rows of one generation are one truth of one version; the cost is that an earlier version stays in the code while a generation that records it is written or can be rolled back to, and a projection changed without a version change is still undetected (E-9, D8, D9) -- [non-blocking] The rule that a row with a newer version is never overwritten is read as the backfill path's rule only, so a live write to the active generation always lands, as `step3` has it; the owner has not confirmed that reading (D9, E-9) -- [non-blocking] Extension E-9, the history fold: D2 orders events only within a stream, and the two paths feed one history row in different orders, the live path command by command and the backfill stream by stream; the option taken here is a fold that gives one row whatever the order across streams and however its events are split between calls, a row no fold deletes, which keeps its subject's ended state, and baselines dropped before `keyOf`, so a view whose meaning needs the order across streams waits for the ordered consumer of D18 (E-9, D2, D9, D18, E-25) -- [non-blocking] Extension E-9, a composite row of current state: D8 lets a small cross-context view be a read model updated in the command, and no form here writes one row from the DTOs of two sources, because `project` builds a whole row from one DTO and a backfill enumerates one source; the option taken here is D8's other default, a parent query over bounded component reads, until a view needs such a row, which then needs a form whose `project` takes the DTO of every matching entry of one command and whose backfill reads the other context in the same mutation, online (E-9, D8, E-45) -- [non-blocking] Extension E-9, the history form's one version: a `HistoryProjection` declares one version, so a deploy that changes it leaves the active generation, which records the earlier version, unwritable, and every command whose events feed the view fails as a technical failure until a generation of the new version is switched; the option taken here is that one version, because a view rebuilt under the pause is never written by two versions at once, and a list of versions like the `projections` of a `ReadModel` is the form a view takes when its sources must stay writable across such a deploy (E-9, D8, D9, Law 6) +- [non-blocking #projectionRowContract] Extension E-9: the doc says a projection is named, versioned and deterministic and that live update and rebuild share its logic; it does not give the signature or the row shape, for counts and sums it names the marker but not the aggregate row, and for the read models that depend on history across several streams, the ones the write pause exists for, it names no input. The option taken here: a per-entity read model declared as a `ReadModel` with `name`, `table`, `rowBudgetBytes` and its `projections`, a list of `Projection` with the latest first, each holding `version`, `keyOf` and `project` alone, rows with `tenantId`, `generation`, `key`, `projectionVersion` and `sourceVersions`, one `applyProjection` helper with a mode per caller that writes each generation by its role, active, building or verifying, with the projection whose version the generation records, for the aggregate form the row and the writer `spec:application.aggregate-projection` pins, and for a history-dependent view a `HistoryProjection` that folds events into the prior row, fed on the live path by the events the operation appended, which the adapter's `streams` entry carries, and on the backfill by the context's `history` pages under the write pause, with `sourceVersions` recording the highest version folded per stream so that an event is never folded twice; a row that derives from more than one source stream type is a history row, its projection bound once per source stream type, and a small cross-context view of current state is D8's parent query over bounded component reads; the owner confirms the three forms and has not ruled on the split of a per-entity read model into `ReadModel` and `Projection`, on the write by generation role or on the row measured against its budget at every write (E-9, E-23, E-24, D8, D9) +- [non-blocking #projectionGenerationVersions] Extension E-9, two projection versions: the doc says live update and rebuild share one projection and does not say how a command writes two generations built by two versions; the option taken here is that a read model declares its projection versions as a list, the latest first, and `applyProjection` writes each generation with the projection whose version the generation row records, failing as a technical failure when the read model declares none of that version, so the rows of one generation are one truth of one version; the cost is that an earlier version stays in the code while a generation that records it is written or can be rolled back to, and a projection changed without a version change is still undetected (E-9, D8, D9) +- [non-blocking #backfillVersionOverwrite] The rule that a row with a newer version is never overwritten is read as the backfill path's rule only, so a live write to the active generation always lands, as `step3` has it; the owner has not confirmed that reading (D9, E-9) +- [non-blocking #compositeCurrentStateRow] Extension E-9, a composite row of current state: D8 lets a small cross-context view be a read model updated in the command, and no form here writes one row from the DTOs of two sources, because `project` builds a whole row from one DTO and a backfill enumerates one source; the option taken here is D8's other default, a parent query over bounded component reads, until a view needs such a row, which then needs a form whose `project` takes the DTO of every matching entry of one command and whose backfill reads the other context in the same mutation, online (E-9, D8, E-45) +- [non-blocking #historySourceVersionTenant] Extension E-9, the cost of `sourceVersions`: a history row's entries repeat the row's tenant ID once per source stream, about three quarters of the largest order allocation history row by `spec:application.history-projection#design.tableOrderAllocationHistory`; whether a row's `sourceVersions` may omit `tenantId`, equal to the row's on every entry, and the write rule compare three identity fields, is a choice the owner may take with the pinned conventions (E-9, Law 11, F13) ## Contract @@ -46,7 +45,6 @@ A projection is the named, versioned, deterministic function that turns a contex - The DTO a projection receives is the same `toDto(state, meta)` on both paths, the adapter's `streams` entry on the live path and the context's `list` page on the backfill, and it carries the stream's `StreamVersion` as `version`, so no adapter between two DTO shapes exists (D8, D9, E-9, E-24) - Every read-model row carries `tenantId`, `generation`, `key` and `projectionVersion`, a per-entity or history row carries `sourceVersions` as well, and every read-model table has the `by_key` index leading with the tenant (E-9, Law 11, D9) - `sourceVersions` holds one `StreamVersion` per source stream the row derives from; a per-entity row holds one, and a history row one per stream whose events it folded (E-9, D2, D8) -- [extension] An aggregate row derives from every entity under its key, so it carries no `sourceVersions`; it carries `value`, the sum of the recorded contributions, and `entityCount`, the number of markers, and the per-entity marker carries the `sourceVersions` the row conventions would otherwise hold (E-9, D9) - [extension] A read model declares its name, the parent table that holds its rows, `rowBudgetBytes`, the largest row it writes, 16 KiB by default and at most `limitRowSize`, and its projections, one per version it can still write, the latest first, with distinct versions and at least one; every version writes rows the one table's validator accepts, and a change of shape the table cannot hold beside the earlier one is a new read model; a projection is the versioned rules alone, so they are tested without a table; the rebuild derives its batch bound from the declared budget the way the context derives its page from the stream budget (E-9, E-40, F13) - On the backfill path a row is written only if it is missing or its `sourceVersions` are older than the input's for the same stream, and a row with a newer version is never overwritten; the rule is the backfill path's only, and a live write to the active generation always lands (D9) - In the active generation a live write always lands: a missing row is inserted and an existing row is replaced, in the live-created mode and the live-updated mode alike, because `project` sees the whole DTO and cannot build a partial row, so a subject whose row was deleted gets its row back from the next command that writes it (D9, E-9) @@ -54,15 +52,6 @@ A projection is the named, versioned, deterministic function that turns a contex - [extension] Every write of a read-model row measures the whole row, the conventions included, with `getConvexSize` and throws a plain error, a technical failure that rolls the command back, when it measures above the read model's `rowBudgetBytes`; a declared budget above the ceiling of `limitRowSize` throws the same way (E-9, F13, Law 6) - In the backfill mode the one generation the batch names is written whatever its role: the row is inserted if missing, updated if older, and, when the DTO projects to `null`, deleted if its `sourceVersions` are older than the input's and left as `skipped-newer` otherwise, which is how a verify pass makes absence right as well as presence (D9, E-8, E-9) - A projection may return `null` for a DTO that has no row, such as a deleted subject; `applyProjection` then deletes the row if present, in the backfill mode only when the row is older than the input, and does nothing when it is absent, and nothing records the deletion, because a backfill batch reads the source in the same mutation that writes the row, so a batch that read the subject before its deletion committed re-runs under optimistic concurrency and reads the deleted subject, as the rebuild's `occRace` states (E-9, D2, F1) -- An aggregate projection contributes a number per entity to an aggregate row and keeps a per-entity marker so live commands and backfill never count one entity twice (D9) -- [extension] `applyAggregate` ignores the mode: the marker's `sourceVersions` decide, a marker that is not older than the input makes the write `skipped-newer`, and otherwise the marker's recorded contribution leaves the row of the marker's `aggregateKey` and the new contribution joins the row of `aggregateKeyOf`, one row moved by the difference when the key is the same and two rows when it changed, a row whose `entityCount` reaches 0 being deleted, and a `null` contribution joins no row and removes the marker; a live update on an entity backfill has not reached is therefore not partial, because `contribute` sees the whole DTO, which is why the missing-row rule of the per-entity path does not apply (E-9, D9, F1) -- [extension] An aggregate's entity is its source stream: `applyAggregate` derives the marker's `entityKey` from the input's one `StreamVersion` as `JSON.stringify([contextId, streamType, streamId])`, so the live path and the backfill, which pass `entry.version` and `dto.version`, name one marker, and equal stream IDs in two contexts or two stream types name two (E-9, D9, D2) -- [extension] A history projection folds events into a row: `keyOf` maps each event to the row it belongs to or to none, and `project` folds the events of one key, each stream's in stream-version order and the streams interleaved in any order, into the prior row; it is deterministic in the prior row and the events, reads nothing else, and is the one form a read model that depends on history across several streams may take, which is why such a read model rebuilds only under the write pause (E-9, D8, D9, Law 10) -- [extension] A history projection's fold gives the same row whatever the order in which the events of different streams arrive and however they are split between calls: from one prior row, the events of one key folded in one call or in several, with the streams interleaved in any way that keeps each stream's events in version order, give equal rows; D2 orders events only within a stream, and the live path feeds a row command by command while the backfill feeds it stream by stream, so a view whose meaning depends on the order across streams is not a history projection and waits for the ordered consumer of D18 (E-9, D2, D8, D18) -- [extension] `project` never answers `null`: a history row, once written, keeps its subject's ended state, such as a cancelled order, and leaves its table only when its generation is purged, so its `sourceVersions`, the fold's one memory, outlive the subject's end, and an event that arrives after it, such as `AllocationReleased` after `OrderCancelled`, folds into the row and never makes a new one (E-9, D9) -- [extension] `applyHistoryProjection` drops every event of the reserved type `baseline` before `keyOf`, on both paths: a baseline holds a stream's migrated state and no fact a view folds, and a migration that changes what a source's earlier events mean to the view is a new projection version and so a new generation, never a rewrite of rows in place (E-9, E-25, D9) -- [extension] A history row's `sourceVersions` hold, per source stream, the highest event version folded into it; `applyHistoryProjection` drops every event whose version is not above the recorded version for its stream and skips the write when none remain, so a live update and a backfill page never fold one event twice, and the live path never meets a non-active generation of a history view, because its generation in flight exists only while its source scopes are closed and `rollbackGeneration` of `spec:application.rebuild` refuses to reopen a retired one (E-9, D9, F1) -- [extension] A history projection is bound once per source stream type on the command declaration; the pipeline's step 9 feeds it, once per command, the events of every `streams` entry its bindings match, and the backfill feeds it each source's history, so a history row has one projection and several bindings; a `Projection` or an `AggregateProjection` takes one entry's DTO, so a read model of those forms derives each row from one source stream (E-9, E-7, D8) - A projection's version changes whenever its output for any input changes; a new version is added at the head of the read model's `projections` and is a new generation, never an in-place rewrite, and an earlier version leaves the list only when no generation in `building`, `verifying`, `verified` or `active` records it and none in `retired` is to be rolled back to (D8, D9) - The write rule reads and writes inside one mutation, so two writers of the same row serialize under optimistic concurrency and neither overwrites newer data (F1, D9) @@ -75,26 +64,11 @@ The shapes are TypeScript in the parent. `StreamVersion` is the kernel's shape w - fnProjectionOf: `projectionOf>(readModel: ReadModel, version: number): Projection` returning the declared projection of that version and throwing the plain `Error` `"Read model " + readModel.name + " declares no projection of version " + version` when there is none, a technical failure on the command path and a refusal of `startGeneration` and `rollbackGeneration` (E-9, D8, Law 6) - typeWritableGeneration: `type WritableGeneration = { role: "active" | "building" | "verifying"; generation: number; projectionVersion: number }`, one generation a command writes, as the registry's `generationsToWrite` returns it; the role `verifying` covers the states `verifying` and `verified` (E-8, E-9, D9) - typeRowWriter: `type RowWriter = { db: GenericDatabaseWriter }`, the part of a ctx `applyProjection` uses, generic over the composition's data model because a database writer is not covariant in its data model; `RowDataModel` is the data model of one table that spreads `rowConventions` and has `by_key`, and the library reads and writes the read model's own table, named by its `table`, under that table's type (E-9) -- typeAggregateProjection: `interface AggregateProjection { name: string; version: number; table: string; rowBudgetBytes?: number; aggregateKeyOf(tenantId: string, dto: D): string; contribute(tenantId: string, dto: D): number | null }` where `null` means the entity no longer contributes, such as a deleted subject, and `rowBudgetBytes` bounds the aggregate row and the marker alike (E-9, D9, F13) -- typeHistoryProjection: `interface HistoryProjection { name: string; version: number; table: string; rowBudgetBytes?: number; keyOf(tenantId: string, event: EventEnvelope): string | null; project(tenantId: string, prior: Row | null, events: EventEnvelope[]): Row }` where `EventEnvelope` is the envelope of `spec:context.event-envelope`, `null` from `keyOf` means the event is not this view's, `prior` is `null` before the key's first event, `events` are the events of one key not yet folded into `prior`, each stream's in version order and the streams interleaved in any order, `project` answers a row and never `null`, and `rowBudgetBytes` is 16 KiB when absent; a history row is a bounded summary of history, never a list that grows with it, so `rowBudgetBytes` binds it like any row (E-9, D8, D9, F13) -- fnApplyHistoryProjection: `applyHistoryProjection(ctx: MutationCtx, projection: HistoryProjection, input: { tenantId: string; events: EventEnvelope[]; mode: ApplyMode; generations: readonly WritableGeneration[] }): Promise<{ key: string; results: ApplyResult[] }[]>` dropping the events of type `baseline`, grouping the rest by `keyOf` in the order given and leaving out those whose key is `null`, and per key and generation reading the row by `by_key`, dropping every event whose `streamVersion` is not above the row's recorded version for the event's stream, named by the envelope's `tenantId`, `contextId`, `streamType` and `streamId`, answering `skipped-newer` when none remain, and otherwise writing `project(tenantId, prior, remaining)` with `sourceVersions` raised to the highest version folded per stream and sorted by `contextId`, then `streamType`, then `streamId`, strings compared by UTF-16 code units, so the row is one value whatever the order the streams arrived in, and answering `inserted` or `updated`, never `deleted`, in the same mutation; `mode` is read by nothing, and a generation given whose `projectionVersion` is not the projection's `version` makes it throw the plain `Error` of `projectionOf` before it reads a row, a technical failure (E-9, D8, D9, F1, Law 6) - typeApplyMode: `type ApplyMode = "live-created" | "live-updated" | "backfill"` (E-9, D9) - typeApplyResult: `type ApplyResult = "inserted" | "updated" | "deleted" | "skipped-missing" | "skipped-newer" | "unchanged"` (E-9, D9) - fnApplyProjection: `applyProjection>(ctx: RowWriter, readModel: ReadModel, input: { tenantId: string; dto: D; versions: StreamVersion[]; mode: ApplyMode; generations: readonly WritableGeneration[] }): Promise` writing one result per generation in the order given; the pipeline's step 9 calls it with `versions: [entry.version]` for a per-entity row and the rebuild's batch with `versions: [dto.version]` (E-9, D8, D9) -- callers: the two callers are the command pipeline's step 9, with the mode from the `streams` entry's `created` flag and the list `generationsToWrite` returned, passed unchanged, and the rebuild's backfill and verify batches in `backfill` mode for the one generation they build or verify; a use case body never calls it, and the same two callers are the only callers of `applyAggregate` and `applyHistoryProjection`; step 9 calls `applyHistoryProjection` once per history read model per command, in the live-updated mode, with the events of every `streams` entry any of its bindings matches in the order of the entries, and a history batch calls it once with its one `history` page (E-9, D8, D9, E-7) -- tableOrderAllocationHistory: `orderAllocationHistory: defineTable({ ...rowConventions, orderId: v.string(), placedAt: v.optional(v.number()), cancelledAt: v.optional(v.number()), allocations: v.array(v.object({ stockItemId: v.string(), quantity: v.number(), allocatedAt: v.number(), releasedAt: v.optional(v.number()) })) }).index("by_key", ["tenantId", "generation", "key"])` the history-dependent cross-stream view of the example domain, the `HistoryProjection` named `orderAllocationHistory` over the table of that name, keyed by order ID and folded from `OrderPlaced` and `OrderCancelled` in Orders and `StockAllocated` and `AllocationReleased` in Inventory, with one entry of `allocations` per stock item the order allocated on and `sourceVersions` holding the order stream and every stock stream folded, so a row holds at most 100 entries and 101 versions; `getConvexSize` measures the largest such row, at 256-byte tenant and order IDs and 100 stock item IDs of 64 bytes, every allocation released, at 55,803 bytes, so the view declares `rowBudgetBytes` 65,536, the ceiling; it is bound once per source stream type, rebuilds only under the write pause, and Sc L2-6 builds it (E-9, E-46, D9, F13, Sc L2-6) -- typeOrderAllocationHistoryFields: `type OrderAllocationHistoryFields = { orderId: string; placedAt?: number; cancelledAt?: number; allocations: OrderAllocation[] }` where `type OrderAllocation = { stockItemId: string; quantity: number; allocatedAt: number; releasedAt?: number }`, the projected fields of the order allocation history, in `example/domain/orderAllocationHistory.ts` (E-9, E-46, Sc L2-6) -- fnOrderAllocationHistoryKeyOf: `orderAllocationHistoryKeyOf(tenantId: string, event: HistoryEvent): string | null` in `example/domain/orderAllocationHistory.ts`, answering the stream ID of an `OrderPlaced` or an `OrderCancelled`, the payload's `orderId` of a `StockAllocated` or an `AllocationReleased`, and `null` for every other event, where `type HistoryEvent = { streamId: string; eventType: string; recordedAt: number; occurredAt?: number; payload: unknown }` is the part of `EventEnvelope` the rules read, so the example's pure domain code imports no context library (E-9, E-46, Sc L2-6) -- fnProjectOrderAllocationHistory: `projectOrderAllocationHistory(tenantId: string, prior: OrderAllocationHistoryFields | null, events: readonly HistoryEvent[]): OrderAllocationHistoryFields` in the same file, folding the events one by one into `prior`, or, when `prior` is `null`, into `{ orderId, allocations: [] }` with the key of the first event that has one, and throwing the plain `Error` `"The order allocation history needs a prior row or an event of an order"` when there is neither; the view's `HistoryProjection` holds this function and `orderAllocationHistoryKeyOf` as its `project` and `keyOf` (E-9, E-46, Sc L2-6) -- orderAllocationHistoryFold: each field is written by the events of one stream, each event's time being its `occurredAt`, or its `recordedAt` when absent: `OrderPlaced` sets `placedAt` and `OrderCancelled` sets `cancelledAt`; `StockAllocated` on a stock item sets that stock item's entry of `allocations` to the payload's `quantity` and the time as `allocatedAt`, with no `releasedAt`; `AllocationReleased` sets `releasedAt` on that entry and changes nothing when the order has none on that stock item; every other event changes nothing; `allocations` is kept sorted by stock item ID, compared by UTF-16 code units; so the fold gives one row whatever the order across streams, a stock event before `OrderPlaced` included, and a pure test folds one order's events in every interleaving of its three streams and every split into calls and finds one row (E-9, D2, Sc L2-6) -- fnApplyAggregate: `applyAggregate(ctx: MutationCtx, projection: AggregateProjection, input: { tenantId: string; dto: D; versions: StreamVersion[]; mode: ApplyMode; generations: readonly WritableGeneration[] }): Promise` answering, per generation in order, one result for each aggregate row it writes, the old key's row before the new key's on a key change, `inserted` for a row it created, `updated` for a row whose `value` or `entityCount` changed and `deleted` for a row whose `entityCount` reached 0, or one `skipped-newer` when the marker was not older and one `unchanged` when no row moved; `input.versions` holds the one `StreamVersion` of the entity's source stream, and another length throws a plain error; `mode` is accepted for symmetry with `applyProjection` and read by nothing (E-9, D9) -- aggregateStep1: for each generation it derives `entityKey` from `input.versions[0]` as the contract states, reads the marker by `by_entity` with the tenant, read model, generation and `entityKey`, and computes `contribution` as `contribute(tenantId, dto)` and, when that is not `null`, `aggregateKey` as `aggregateKeyOf(tenantId, dto)` (E-9, D9) -- aggregateStep2: when a marker exists and its `sourceVersions` are not older than `input.versions` for the same stream, the result is `skipped-newer` and nothing is written (E-9, D9, F1) -- aggregateStep3: when a marker exists and `contribution` is `null` or `aggregateKey` differs from the marker's, it reads the row of the marker's `aggregateKey` by `by_key`, throwing a plain error when it is missing, a defect, subtracts the marker's contribution from `value` and 1 from `entityCount`, and deletes the row when `entityCount` reaches 0 and replaces it otherwise; when `contribution` is not `null` and the marker is missing or its key differs, it reads the row of `aggregateKey`, inserts it with `value` equal to `contribution` and `entityCount` 1 when missing, and otherwise adds `contribution` to `value` and 1 to `entityCount`; when the marker exists with the same key and `contribution` is not `null`, it adds `contribution` minus the marker's to that row's `value`, leaves `entityCount`, and writes no row when the difference is 0; when the marker is missing and `contribution` is `null`, it reads and writes no row (E-9, D9) -- aggregateStep4: then it inserts or replaces the marker with `entityKey`, `contribution`, `aggregateKey` and `input.versions`, or deletes it when `contribution` is `null`, all in the same mutation; a key change reads and writes two aggregate rows and the marker in each generation, and its two rows count against `limitReadModelWritesPerCommand` of `spec:application.read-models` (E-9, D9, E-8, E-45) +- callers: the two callers are the command pipeline's step 9, with the mode from the `streams` entry's `created` flag and the list `generationsToWrite` returned, passed unchanged, and the rebuild's backfill and verify batches in `backfill` mode for the one generation they build or verify; a use case body never calls it, and the same two callers are the only callers of `applyHistoryProjection` and, as `spec:application.aggregate-projection` states, of the aggregate form's writer; step 9 calls `applyHistoryProjection` once per history read model per command, in the live-updated mode, with the events of every `streams` entry any of its bindings matches in the order of the entries, and a history batch calls it once with its one `history` page (E-9, D8, D9, E-7) - validatorRowConventions: `const rowConventions = { tenantId: v.string(), generation: v.number(), key: v.string(), projectionVersion: v.number(), sourceVersions: v.array(streamVersionValidator) }` spread into every per-entity and history read-model `defineTable` (E-9, Law 11) -- validatorAggregateRowConventions: `const aggregateRowConventions = { tenantId: v.string(), generation: v.number(), key: v.string(), projectionVersion: v.number(), value: v.number(), entityCount: v.number() }` spread into every aggregate read-model `defineTable`, with the same `by_key` index, where `key` is `aggregateKeyOf(tenantId, dto)` (E-9, Law 11, D9) -- tableOrderTotals: `orderTotals: defineTable({ ...aggregateRowConventions }).index("by_key", ["tenantId", "generation", "key"])` an aggregate of order totals per status, where `key` is the status, `value` the sum of the orders' totals and `entityCount` the number of orders in it, given as the shape of an aggregate table and not as a read model the first experiment builds (E-9, D9) - validatorOrderSummaryFields: `const orderSummaryFields = { orderId: v.string(), status: v.union(v.literal("placed"), v.literal("cancelled")), lineCount: v.number(), total: v.number(), placedAt: v.number() }` the projected fields of the order summary, which its table spreads and its query's view validator spreads (E-9, E-46, First experiment) - tableOrderSummaries: `orderSummaries: defineTable({ ...rowConventions, ...orderSummaryFields }).index("by_key", ["tenantId", "generation", "key"]).index("by_status", ["tenantId", "generation", "status", "placedAt", "key"])` (E-9, E-46, First experiment) - indexByKeyUse: point read by `applyProjection` before every write and by the entity query; `key` is `keyOf(tenantId, dto)`, the application ID of the entity (E-9, D2) @@ -104,15 +78,12 @@ The shapes are TypeScript in the parent. `StreamVersion` is the kernel's shape w - step3: it decides by the mode and the generation's role: in the two live modes it inserts a missing row and replaces an existing one, except that live-updated in the role `building` answers `skipped-missing` for a missing row; in the backfill mode it inserts when missing and updates when the existing `sourceVersions` are older, else `skipped-newer`, whatever the role (D9, E-8, E-9) - step4: before a write it throws a plain error when the row measures above `rowBudgetBytes` with `getConvexSize`; then it writes with `ctx.db.insert`, `ctx.db.replace` or `ctx.db.delete` in the same mutation, a written row carrying the generation's `projectionVersion`, which is the version of the projection that built it, and returns the results in generation order (E-9, F1, F13) - limitRowSize: a read-model row stays under its read model's `rowBudgetBytes`, which `applyProjection` checks at every write, 16 KiB by default and at most 64 KiB, the library constant `limitRowBudgetBytes`, well under the 1 MiB document limit; the rebuild's batch bound is derived from the declared budget, and a row that would grow without bound with history is a wrong read model for this contract, a history projection included, whose row summarizes history within the same budget (F13, E-9, E-40) -- limitHistoryFold: on the live path step 9 calls `applyHistoryProjection` once per history read model per command, so it reads and writes at most one row per distinct key among the command's events in each generation, each at most `rowBudgetBytes`, and a `PlaceOrder` of 100 lines, whose 101 `streams` entries key to one order, reads and writes the order allocation history's one row once; on the backfill it takes one `history` page of at most the progress row's `batchSize` events, as the rebuild's `limitHistoryBatch` bounds it, so one batch reads at most that many events plus as many rows (F13, E-9, E-40) - limitGenerationsPerWrite: at most 2 generations are written per row per command, the active one and the one in building, verifying or verified if any; a retired generation is kept but never written, so one read model costs at most 2 point reads and 2 writes per command and exactly 1 of each outside a rebuild (E-9, E-8, F13) -- determinismCheck: the acceptance suite folds a sample of DTOs through `project` twice and compares, and compares a backfilled row with the live-written row for the same entity; a pure test folds one order's events through the order allocation history's `project` in every interleaving of its streams and every split into calls and finds one row; a test of `applyAggregate` moves an entity from one aggregate key to another and finds the old key's row without its contribution and count and the new key's with them (D8, D9, Sc L2-5, Sc L2-6) +- determinismCheck: the acceptance suite folds a sample of DTOs through `project` twice and compares, and compares a backfilled row with the live-written row for the same entity; a pure test folds one order's events through the order allocation history's `project` in every interleaving of its streams and every split into calls and finds one row; the aggregate form's test is in `spec:application.aggregate-projection` (D8, D9, Sc L2-5, Sc L2-6) ## Verification — reviewed -- A reviewer confirms that every read model in the corpus and the experiment is declared as a `ReadModel` with its `Projection`, an `AggregateProjection` or a `HistoryProjection` bound on a command declaration, and that the pipeline's step 9 and the rebuild's batches are the only callers of `applyProjection`, `applyAggregate` and `applyHistoryProjection`. -- A reviewer confirms that every `HistoryProjection` is rebuilt with `pauseRequired` on its generations, that `applyHistoryProjection` drops baselines before `keyOf` and already-folded events by the row's `sourceVersions` before it calls `project`, that no history projection's `project` answers `null`, and that the L2-6 examples assert the fold against the journal. +- A reviewer confirms that every read model in the corpus and the experiment is declared as a `ReadModel` with its `Projection` or a `HistoryProjection` bound on a command declaration, and that the pipeline's step 9 and the rebuild's batches are the only callers of `applyProjection` and `applyHistoryProjection`. - A reviewer confirms that no `project` function touches `ctx`, the database, the scheduler or the environment. -- A reviewer confirms that every per-entity or history read-model table spreads `rowConventions`, that every aggregate table spreads `aggregateRowConventions`, and that each carries the `by_key` index. -- A reviewer confirms that `applyAggregate` compares the marker's `sourceVersions` before it touches `value`, that it reads no mode, that it derives `entityKey` from the input's stream version, that a key change moves the contribution and the count from the marker's key's row to the new key's row, and that a `null` contribution removes the marker and its recorded contribution and creates no row. +- A reviewer confirms that every per-entity or history read-model table spreads `rowConventions` and carries the `by_key` index; an aggregate table carries the conventions `spec:application.aggregate-projection` pins for it. - A reviewer confirms that a `convex-test` test on the fixture composition writes one `CreateTwiceSummarizedDocument` into an active generation of `documentTitle` at version 1 and a generation being built at version 2 and finds on each row its own generation's `projectionVersion`, that a backfill batch of each generation over a document whose `title` is the depot's `deletedTitle` leaves version 1's row with that title and no row at version 2, the two versions' one difference, and that a test deletes a row through a `null` projection, `applyProjection` in the backfill mode at version 2 over that document, and finds the next live-updated write of the same document at a higher stream version with a live title, `applyProjection` called with the mode `live-updated` because no fixture command amends a document under `documentTitle`, answering `inserted` in the active generation and in a generation in `verifying`, and `skipped-missing` in one in `building`. diff --git a/design/specs/application/read-models.sdp.md b/design/specs/application/read-models.sdp.md index 3e8f240..b4520c3 100644 --- a/design/specs/application/read-models.sdp.md +++ b/design/specs/application/read-models.sdp.md @@ -46,12 +46,12 @@ This Spec carries the read-need table as rules, the properties of a projection, ### Open questions -- [non-blocking] Probe 5 showed that the built-in `.paginate` treats `maximumRowsRead` as advice once a page has an end cursor and that a cursor it issued is refused by any other query, so a subscription over the active generation threw `InvalidCursor` at a switch; a parent list therefore pages with `paginator` of `convex-helpers`, as a context list does, and a pinned page is bounded on every run by the row cap and by the byte cap as `spec:context.queries` states it, the helper stopping at the row that reaches it and admitting that row whole, and answers `SplitRequired` beyond them (Probe 5, F13, D8, D9) -- [non-blocking] Extension E-24, as it reaches read models: a list over a parent read-model table calls `paginator` of `convex-helpers` with the options of `boundedPage`, a row cap of twice the item cap, with each cursor of the client's pair moved into the active generation by `pageInGeneration`, and returns through `paginationResultValidator` with its rows mapped by `readModelView`, and a client pages every list by an explicit cursor pair with no hook; these are options taken here and the owner has not ruled (E-24, S4, Probe 5, D8) -- [non-blocking] Extension E-6, as it reaches queries: a parent query establishes and authorizes its actor through `authorizeQuery`, and its refusal is the command's rejection shape, `RejectionData` with the query's name in the field `entry` and the codes `unauthenticated` or `forbidden`; the owner confirmed the word, and nothing else of the shape is new (E-6, D7, D11, Law 5) -- [non-blocking] Probe 3 pending: the cost of one component call in latency and function-call quota decides whether one entity's detail stays a component query or hot reads move, which is OQ1 (Probe 3, F4, OQ1) -- [non-blocking] Extension E-45: the doc says a small cross-context view is a parent query over bounded component reads and does not give the bound; the option taken here is at most 8 component reads per parent query and at most 4 read-model rows written per command, both measured by the first experiment (E-45, D8) -- [non-blocking] Extension E-9, as it reaches read models: D8 lets a small cross-context view be a parent query over bounded component reads or a read model updated in the command; the option taken here is the parent query for a view of current state, because no form of `spec:application.projection-contract` writes one row from the DTOs of two sources, and the history projection for a view that folds the events of several streams; a composite row of current state is a form for the view that needs it (E-9, E-45, D8) +- [non-blocking #parentListGenerationSwitch] Probe 5 showed that the built-in `.paginate` treats `maximumRowsRead` as advice once a page has an end cursor and that a cursor it issued is refused by any other query, so a subscription over the active generation threw `InvalidCursor` at a switch; a parent list therefore pages with `paginator` of `convex-helpers`, as a context list does, and a pinned page is bounded on every run by the row cap and by the byte cap as `spec:context.queries` states it, the helper stopping at the row that reaches it and admitting that row whole, and answers `SplitRequired` beyond them (Probe 5, F13, D8, D9) +- [non-blocking #readModelListBounds] Extension E-24, as it reaches read models: a list over a parent read-model table calls `paginator` of `convex-helpers` with the options of `boundedPage`, a row cap of twice the item cap, with each cursor of the client's pair moved into the active generation by `pageInGeneration`, and returns through `paginationResultValidator` with its rows mapped by `readModelView`, and a client pages every list by an explicit cursor pair with no hook; these are options taken here and the owner has not ruled (E-24, S4, Probe 5, D8) +- [non-blocking #parentQueryRefusal] Extension E-6, as it reaches queries: a parent query establishes and authorizes its actor through `authorizeQuery`, and its refusal is the command's rejection shape, `RejectionData` with the query's name in the field `entry` and the codes `unauthenticated` or `forbidden`; the owner confirmed the word, and nothing else of the shape is new (E-6, D7, D11, Law 5) +- [non-blocking #contextQueryCost] Probe 3 pending: the cost of one component call in latency and function-call quota decides whether one entity's detail stays a component query or hot reads move, which is OQ1 (Probe 3, F4, OQ1) +- [non-blocking #parentQueryReadBounds] Extension E-45: the doc says a small cross-context view is a parent query over bounded component reads and does not give the bound; the option taken here is at most 8 component reads per parent query and at most 4 read-model rows written per command, both measured by the first experiment (E-45, D8) +- [non-blocking #crossContextReadForm] Extension E-9, as it reaches read models: D8 lets a small cross-context view be a parent query over bounded component reads or a read model updated in the command; the option taken here is the parent query for a view of current state, because no form of `spec:application.projection-contract` writes one row from the DTOs of two sources, and the history projection for a view that folds the events of several streams; a composite row of current state is a form for the view that needs it (E-9, E-45, D8) ## Behavior @@ -97,20 +97,20 @@ Nothing here schedules, and nothing here reads `ctx.auth` inside a component. Th - convexSurface: one static `query` export per read need in the parent; component queries are reached through `ctx.runQuery(components.orders.queries.order.get, args)`, its siblings `list`, `history` and `rebuild`, and `components.orders.queries.operations.byOperation`, as the queries contract names them; no scheduled function, no cron, no action (D8, F11) - fnEntityDetail: `getOrder` as `spec:context.queries` pins it under `parentGet`: it calls `authorizeQuery` and then `ctx.runQuery(components.orders.queries.order.get, { tenantId, streamId: orderId })` (D8, D11, F15) - fnListOrSummary: `export const listOrderSummaries = query({ args: { tenantId: v.string(), status: v.union(v.literal("placed"), v.literal("cancelled")), paginationOpts: paginationOptsValidator }, returns: paginationResultValidator(orderSummaryView), handler: async (ctx, { tenantId, status, paginationOpts }) => { await authorizeQuery(ctx, { name: "listOrderSummaries", tenantId, permission: "orders.read" }); const generation = await activeGeneration(ctx, "orderSummary"); if (generation === undefined) throw new Error("The read model orderSummary has no active generation"); const result = await paginator(ctx.db, schema).query("orderSummaries").withIndex("by_status", (q) => q.eq("tenantId", tenantId).eq("generation", generation).eq("status", status)).paginate(boundedPage(pageInGeneration(paginationOpts, [tenantId, generation, status], 5), limitReadModelList(orderSummary))); return { ...result, page: result.page.map(readModelView) }; } })` one `paginate` of `paginator` from `convex-helpers/server/pagination` with the composition's `schema`, over the active generation's rows in `placedAt` then `key` order within a status, where `orderSummary` is the read model, whose name is `"orderSummary"` and whose table is `orderSummaries` (D8, D9, Law 5, Law 11, E-24) -- fnPageInGeneration: `pageInGeneration(opts: PaginationOptions, prefix: Value[], indexFields: number): PaginationOptions` returns `opts` with its `cursor` and its `endCursor`, when set, each moved into the generation of `prefix`, the query's equality values in index order, `tenantId`, `generation` and the list's equality fields, the generation second: a cursor of `paginator` is the JSON of a row's index key, which the function reads and writes as `paginator` encodes it, `undefined` as the string `"undefined"` and a string that ends in `undefined` with one leading underscore added, so it compares the equality elements and replaces the generation in the decoded values; it replaces the key's second element, the generation that wrote the row, with the prefix's and drops the last two, `_creationTime` and `_id`, which belong to that generation's document, when the key carries them, which it tells by a length of `indexFields`, the index's own field count, plus two, so that a cursor already moved, which `paginator` returns as a pinned page's `continueCursor`, keeps its length and has only its generation replaced; a cursor whose key's length is neither `indexFields` nor `indexFields` plus two, whose generation is not a number, whose other equality elements differ from the prefix's, or that is not a key, throws a plain `Error` `"The cursor is not from this list"`, after `authorizeQuery` and the read of the active generation and before the list reads a row, so a saved cursor never overrides the query's tenant or status; `null` passes as a start or an end cursor and `undefined` as an end cursor, the validator requiring a start cursor, and `"[]"`, the helper's end-of-list mark, passes as the lawful `endCursor` of the last pinned page and, as a start cursor, reads the list from its first row, which is the helper's behavior and discloses nothing; the read-model library exports the function beside `readModelView` (D9, E-24, Probe 5, Law 11) +- fnPageInGeneration: `pageInGeneration(opts: PaginationOptions, prefix: readonly Value[], indexFields: number): PaginationOptions` returns `opts` with its `cursor` and its `endCursor`, when set, each moved into the generation of `prefix`, the query's equality values in index order, `tenantId`, `generation` and the list's equality fields, the generation second: a cursor of `paginator` is the JSON of a row's index key, which the function reads and writes as `paginator` encodes it, `undefined` as the string `"undefined"` and a string that ends in `undefined` with one leading underscore added, so it compares the equality elements and replaces the generation in the decoded values; it replaces the key's second element, the generation that wrote the row, with the prefix's and drops the last two, `_creationTime` and `_id`, which belong to that generation's document, when the key carries them, which it tells by a length of `indexFields`, the index's own field count, plus two, so that a cursor already moved, which `paginator` returns as a pinned page's `continueCursor`, keeps its length and has only its generation replaced; a cursor whose key's length is neither `indexFields` nor `indexFields` plus two, whose generation is not a number, whose other equality elements differ from the prefix's, or that is not a key, throws a plain `Error` `"The cursor is not from this list"`, after `authorizeQuery` and the read of the active generation and before the list reads a row, so a saved cursor never overrides the query's tenant or status; `null` passes as a start or an end cursor and `undefined` as an end cursor, the validator requiring a start cursor, and `"[]"`, the helper's end-of-list mark, passes as the lawful `endCursor` of the last pinned page and, as a start cursor, reads the list from its first row, which is the helper's behavior and discloses nothing; the read-model library exports the function beside `readModelView` (D9, E-24, Probe 5, Law 11) - indexReadModelList: the index a read-model list reads has `tenantId`, `generation` and the list's equality fields first, then its order field, then `key`, as `by_status` of `tableOrderSummaries` in `spec:application.projection-contract`, so a position within a tenant names at most one row of each generation and a cursor moved into another generation lands between the same rows there (D9, E-9, Law 11) - fnReadModelView: `readModelView(row: Row): Omit` what a read-model query returns for a row: the tenant, the key, the projection version, the source versions and the projected fields; the built-in page's rows carry `_id` and `_creationTime`, which an object validator of the row's fields refuses, and no document ID or generation number leaves the parent in a row; a page's cursors are the position tokens of `fnPageInGeneration`, which carry the generation and, as `paginator` issues them, the row's `_creationTime` and `_id`, and a client passes them back unread, since the list refuses a cursor that is not its own (D8, D2, E-9) - validatorReadModelView: `orderSummaryView = v.object({ tenantId: v.string(), key: v.string(), projectionVersion: v.number(), sourceVersions: v.array(streamVersionValidator), ...orderSummaryFields })` where `orderSummaryFields` are the projected fields `spec:application.projection-contract` declares for `tableOrderSummaries` (E-9, D8) - noActiveGeneration: a query over a read model that has no active generation throws a plain `Error` that names the read model, a technical failure, and never answers an empty page (D9, Law 6) - fnCrossContextView: `export const getOrderWithStock = query({ args: { tenantId: v.string(), orderId: v.string() }, returns: v.union(v.object({ data: orderWithStockDto, sourceVersions: v.array(streamVersionValidator) }), v.null()), handler })` making at most 8 component reads and returning a `VersionedResult` whose `sourceVersions` are the DTOs' stream versions; no read-model row serves it, because no form writes one row from the DTOs of two contexts (D8, E-45, E-9) -- typeVersionedResult: `[extension] type VersionedResult = { data: T; sourceVersions: StreamVersion[] }`, the result of a cross-context view composed from component reads, whose `data` is not a row; every query result carries source versions in one of three ways, a per-entity or history read-model row carries `sourceVersions` by the row conventions so a list page needs no wrapper, an entity DTO carries its stream version from `toDto`, and a composed view is wrapped in this type, so the client can always compare with the versions its command returned; an aggregate row carries no comparable version, so a client that needs its own write reflected compares its entity's row, and the sum's freshness is the subscription's re-run when the command commits (E-45, D8, E-9) -- readModelRowShape: every read-model row carries `tenantId`, `generation`, `key`, `projectionVersion` and the projected fields, a per-entity or history row carries `sourceVersions`, and an aggregate row carries `value` and `entityCount` with the versions on its markers, as the projection contract pins (E-9, D8, D9) +- typeVersionedResult: `[extension] type VersionedResult = { data: T; sourceVersions: StreamVersion[] }`, the result of a cross-context view composed from component reads, whose `data` is not a row; every query result carries source versions in one of three ways, a per-entity or history read-model row carries `sourceVersions` by the row conventions so a list page needs no wrapper, an entity DTO carries its stream version from `toDto`, and a composed view is wrapped in this type, so the client can always compare with the versions its command returned; what a client compares for the aggregate form is in `spec:application.aggregate-projection` (E-45, D8, E-9) +- readModelRowShape: every read-model row carries `tenantId`, `generation`, `key`, `projectionVersion` and the projected fields, and a per-entity or history row carries `sourceVersions`, as the projection contract pins; the row of the aggregate form is pinned by `spec:application.aggregate-projection` (E-9, D8, D9) - indexReadModelByKey: `.index("by_key", ["tenantId", "generation", "key"])` on every read-model table (E-9, Law 11) - indexReadModelByKeyUse: point read of one row by the parent query and by `applyProjection` before an update; the generation comes from the registry's active row, which is one per read model and the same for every tenant, so a query narrows by its own `tenantId` and never by a tenant's generation (D8, D9, E-8, Law 11) - pagination: every list returns through `paginationResultValidator(item)` of `convex/server` and passes its one `paginate` call the options of `boundedPage`, as `spec:context.queries` pins; a list over a parent read-model table calls `paginator` of `convex-helpers` with the composition's schema, as a context list does, over the index of `indexReadModelList`, with the client's cursors moved into the active generation by `pageInGeneration`, relays `pageStatus` and `splitCursor` as the helper returns them, and maps the page's rows through `readModelView`; a list served by a context relays the context's page; a client pages either by an explicit cursor pair; a function runs at most one built-in `.paginate`, because Convex refuses a second with "Convex only supports a single paginated query in each function" (D8, S4, Probe 5, F15, E-24) - limitReadModelList: `limitReadModelList(readModel) = { items: limitListPage(readModel.rowBudgetBytes), bytes: limitListBytes }`, a function of the read model, so 100 items at the default 16 KiB and 64 at the 64 KiB cap; `paginator` honours the item cap on a first run and the row cap and the byte cap on every run, and answers `SplitRequired` with a `splitCursor` for a page that reaches either cap (F13, E-9, Probe 5) - limitComponentReadsPerQuery: `[extension]` 8 component reads per parent query (E-45, D8) -- limitReadModelWritesPerCommand: `[extension]` 4 read-model rows per command, counted as the rows step 9 inserts, replaces or deletes, a row in each generation once, and checked after each call of a write helper, one per `streams` entry for a per-entity read model and one per command for a history read model; an aggregate key change counts the two aggregate rows it writes, and a history row counts once however many entries fed it; above it step 9 throws a plain error and nothing of the command commits (E-45, D8, Sc L2-3) +- limitReadModelWritesPerCommand: `[extension]` 4 read-model rows per command, counted as the rows step 9 inserts, replaces or deletes, a row in each generation once, and checked after each call of a write helper, one per `streams` entry for a per-entity read model and one per command for a history read model; a history row counts once however many entries fed it, and the aggregate form's rows count as `spec:application.aggregate-projection` states; above it step 9 throws a plain error and nothing of the command commits (E-45, D8, Sc L2-3) - readModelVisibility: a subscription re-runs when a row it read changes, including on generation switch, because the query reads the registry row too; after the switch and after a switch back each page shows the rows of the generation then active from the position its cursors name, so a client sees the same orders and never `InvalidCursor` (D8, F7, D9) - noProjectionJob: no scheduled function, cron, action or component maintains a read model on the core path; the only batch writer of read models is the rebuild (D8, D9) diff --git a/design/specs/application/rebuild.sdp.md b/design/specs/application/rebuild.sdp.md index 0a87d74..c64ea43 100644 --- a/design/specs/application/rebuild.sdp.md +++ b/design/specs/application/rebuild.sdp.md @@ -6,11 +6,12 @@ readiness: defined relations: refines: spec:application.read-models dependsOn: - - spec:context.event-envelope - spec:application.projection-contract + - spec:application.history-projection - spec:application.generation-registry - spec:application.write-pause - spec:context.queries + - spec:command.actor-and-scope constrainedBy: - spec:laws.law10-replay-never-runs-commands-or-effects - spec:laws.law11-tenant-scope-named @@ -34,19 +35,18 @@ relations: Layer 2 · Detail: full · Traces: D8, D9, D11, D19, Law 10, Law 11, F1, F7, F8, F9, F12, F13, F17, Probe 6, Sc L2-5, Sc L2-6, E-8, E-9, E-40, E-42, E-48, E-49. -A rebuild builds a new numbered generation of a read model while the old one stays active, and it is also how a read model comes to exist: its first generation is built the same way. For a per-entity read model it runs online: the generation is registered as building, live commands write it as well as the active one, backfill batches fill it from each source at its current stream version writing only where the target is missing or older, one verify pass makes presence and absence right, and one write switches the active generation. Optimistic concurrency orders each batch against live commands, so neither side overwrites newer data. The old generation is kept for a rollback period and no longer written; a rollback reopens it, one verify pass repairs it, and the same switch makes it active again. For a read model that depends on history across several streams the rebuild runs under a write pause, because it needs a consistent cut; the scope closed is the view's sources, the context and stream type pairs its bindings name, so every use case that writes one of them and the journal's baseline driver read the gate and are refused, background writers are fenced by that read and by the generation's fence, and an operator can resume or abort. Such a view is a `HistoryProjection` and its backfill pages each source stream's history under the pause. Rebuild never runs commands or external effects. +A rebuild builds a new numbered generation of a read model while the old one stays active, and it is also how a read model comes to exist: its first generation is built the same way. For a per-entity read model it runs online: the generation is registered as building, live commands write it as well as the active one, backfill batches fill it from each source at its current stream version writing only where the target is missing or older, one verify pass makes presence and absence right, and one write switches the active generation. Optimistic concurrency orders each batch against live commands, so neither side overwrites newer data. The old generation is kept for a rollback period and no longer written; a rollback reopens it, one verify pass repairs it, and the same switch makes it active again. A read model that depends on history across several streams rebuilds under a write pause instead, through the same entries and batch chain, as `spec:application.history-rebuild` states. Rebuild never runs commands or external effects. -The registry, the progress row, the per-entity marker and the gate document are pinned by `spec:application.generation-registry` and `spec:application.write-pause`. The projection and the write rule are `spec:application.projection-contract`. The operator entry and the tenant list are `spec:command.actor-and-scope`. Sources are read through the context's queries, never its tables; a rebuild of a stream from its own events is the journal's concern and not this one. +The registry, the progress row and the gate document are pinned by `spec:application.generation-registry` and `spec:application.write-pause`. The projection and the write rule are `spec:application.projection-contract`. The operator entry and the tenant list are `spec:command.actor-and-scope`. Sources are read through the context's queries, never its tables; a rebuild of a stream from its own events is the journal's concern and not this one. ## Intent - actor: The operator who starts, interrupts, resumes, switches, rolls back, aborts or purges a generation through the operator entries, and the use cases whose live writes keep the generation being built current (D9, D19) -- problem: A per-entity read model is rebuilt while commands run and the rebuild is interrupted and resumed; a live command's incremental update on a row backfill has not written builds a partial one, a backfill batch overwrites newer data, an entity is missed at cutover, a deleted subject keeps its row, or a side effect replays; a cross-stream history view is rebuilt without a consistent cut (Sc L2-5, Sc L2-6) -- outcome: A per-entity read model rebuilds online as a new generation with no stale overwrite, no missing entity, no row of a subject that projects to none and no side effect, with the cutover in one write and a rollback that is one verify pass and then that switch of one write; a history-dependent view rebuilds under a write pause with safe resume and abort, and writes reopen only after verification (D9, Sc L2-5, Sc L2-6) +- problem: A per-entity read model is rebuilt while commands run and the rebuild is interrupted and resumed; a live command's incremental update on a row backfill has not written builds a partial one, a backfill batch overwrites newer data, an entity is missed at cutover, a deleted subject keeps its row, or a side effect replays (Sc L2-5) +- outcome: A per-entity read model rebuilds online as a new generation with no stale overwrite, no missing entity, no row of a subject that projects to none and no side effect, with the cutover in one write and a rollback that is one verify pass and then that switch of one write (D9, Sc L2-5) - value: The common rebuild costs no downtime and no projection job; the write pause is paid only where a consistent cut is needed (D9) - risk: The standing cost is a generation registry read in every command that maintains a read model, a generation on every row, the earlier projection version kept in the code while its generation can be rolled back to, the batch chain, and for the write pause a gate every writer reads (D9) - risk: A pass costs at least one batch per tenant of the tenant list, because a batch names one tenant, so a deployment of many empty tenants pays one scheduled mutation for each (E-49, Law 11) -- risk: A paused rebuild closes its view's sources in every tenant until the switch or the abort; its backfill and its verify pass each take one batch per source stream, one more for each further `history` page of a stream longer than the batch size, and one per tenant and source to end that source's enumeration, so the pause lasts about twice that count times one batch; a batch that took streams until its event bound was spent, or a generation of one tenant, which the registry does not offer, would shorten it (D9, E-8, E-40, D18) - risk: The batch bound is a design choice under the transaction limits and no question of correctness: a batch that races a live command on one row is ordered by optimistic concurrency and the row ends at the source's newest stream version in either commit order, and a batch that reads past the read limit fails as a whole and leaves its checkpoint as it was (F13, F1, F17, Probe 6) - assumption: Mutations are serializable under optimistic concurrency, so a backfill batch and a live command that touch the same row serialize and the loser re-runs (F1) - assumption: A scheduled mutation that loses an optimistic-concurrency race is run again by Convex until it commits; the pinned packages do not show a bound on those runs, the design needs none for correctness, and Probe 6 shows what a stalled batch looks like (F9, F1, Probe 6) @@ -56,11 +56,11 @@ The registry, the progress row, the per-entity marker and the gate document are ### Open questions -- [non-blocking] `@convex-dev/migrations` walks only the tables of the component it is defined in, reaches a context's rows only through a parent internal mutation written by hand against a contract its README does not document and the compiled tier needs a cast for, keeps a status apart from the generation's state, and leaves no error on its row when a batch fails past the read limit, so every batch is driven by the rebuild's own self-scheduled internal mutation and the progress row is the one checkpoint; a hosted deployment and sustained contention are not observed (Probe 6, F1, F17, D9) -- [non-blocking] Extension E-40: the doc binds resumable batching to `@convex-dev/migrations` and gives no batch size; the component iterates a parent table while the sources of a read model live in a context component. The option taken here: a batch is one mutation with one cursor on the progress row and a bound of at most 100 entities, capped by the read model's declared `rowBudgetBytes` so that the parent's point reads and writes stay under 4 MiB, and by the context's byte-derived `list` page bound, which is 16 items at the default 256 KiB stream budget; every batch, the backfill and the verify pass over a context's enumeration and the purge over a parent table, is driven by the platform's own self-scheduled internal mutation, and the rebuild uses the component for nothing, because Probe 6 showed that it walks only the tables of the component it is defined in and drives a context's rows only through a parent internal mutation written by hand against a contract its README does not document; the progress row is the one checkpoint (E-40, E-24, E-9, F17, F13, F1) -- [non-blocking] Extension E-8: the registry states, the rollback period, what a rollback does, the gate document, the scope a paused rebuild closes and the reach of a generation are design choices the doc does not make; they are pinned in the two contracts under this workflow (E-8, E-9, E-38, D9) -- [non-blocking] Extension E-8, what this workflow does not offer: the online path is pinned for a per-entity read model with one source stream type and the paused path for a history view, a `HistoryRebuildTarget` with its sources; an `AggregateProjection` has no rebuild here and gets one when a view needs it, and a row made from the current state of more than one source has no form at all, as `spec:application.projection-contract` records; `rollbackGeneration` refuses a generation that needs a write pause, because nothing would hold its sources while live commands fold onto a row that missed the events since its retirement, so such a view returns to earlier rules only through a new generation under the pause (E-8, E-9, D9, D18) -- [non-blocking] Extension E-49, a deployment whose grants predate the tenant list: the tenant list is written by `insertGrant`, so on such a deployment it holds no row while `grants` holds many, and a pass over an empty list visits no tenant and still ends; the option taken here is that `startGeneration` refuses an empty list beside a grant with a message naming `fillTenants`, and `fillTenants`, an operator entry, walks the grants in bounded batches through a progress row of its own and inserts each missing tenant once; a list that holds a tenant whose first grant came after the table and misses an older one is not detected by that refusal, so the operator runs `fillTenants` on such a deployment before its first `startGeneration` (E-49, E-48, E-40, Law 11) +- [non-blocking #rebuildBatchDriver] `@convex-dev/migrations` walks only the tables of the component it is defined in, reaches a context's rows only through a parent internal mutation written by hand against a contract its README does not document and the compiled tier needs a cast for, keeps a status apart from the generation's state, and leaves no error on its row when a batch fails past the read limit, so every batch is driven by the rebuild's own self-scheduled internal mutation and the progress row is the one checkpoint; a hosted deployment and sustained contention are not observed (Probe 6, F1, F17, D9) +- [non-blocking #rebuildBatchBounds] Extension E-40: the doc binds resumable batching to `@convex-dev/migrations` and gives no batch size; the component iterates a parent table while the sources of a read model live in a context component. The option taken here: a batch is one mutation with one cursor on the progress row and a bound of at most 100 entities, capped by the read model's declared `rowBudgetBytes` so that the parent's point reads and writes stay under 4 MiB, and by the context's byte-derived `list` page bound, which is 16 items at the default 256 KiB stream budget; every batch, the backfill and the verify pass over a context's enumeration and the purge over a parent table, is driven by the platform's own self-scheduled internal mutation, and the rebuild uses the component for nothing, because Probe 6 showed that it walks only the tables of the component it is defined in and drives a context's rows only through a parent internal mutation written by hand against a contract its README does not document; the progress row is the one checkpoint (E-40, E-24, E-9, F17, F13, F1) +- [non-blocking #rebuildGenerationPauseContract] Extension E-8: the registry states, the rollback period, what a rollback does, the gate document, the scope a paused rebuild closes and the reach of a generation are design choices the doc does not make; they are pinned in the two contracts under this workflow (E-8, E-9, E-38, D9) +- [non-blocking #rebuildReadModelForms] Extension E-8, what this workflow does not offer: the online path is pinned for a per-entity read model with one source stream type and the paused path for a history view, a `HistoryRebuildTarget` with its sources, in `spec:application.history-rebuild`; the aggregate form's rebuild is deferred with the form in its own Spec, the aggregate projection, and a row made from the current state of more than one source has no form at all, as `spec:application.projection-contract` records; `rollbackGeneration` refuses a generation that needs a write pause, because nothing would hold its sources while live commands fold onto a row that missed the events since its retirement, so such a view returns to earlier rules only through a new generation under the pause (E-8, E-9, D9, D18) +- [non-blocking #tenantListFill] Extension E-49, a deployment whose grants predate the tenant list: the tenant list is written by `insertGrant`, so on such a deployment it holds no row while `grants` holds many, and a pass over an empty list visits no tenant and still ends; the option taken here is that `startGeneration` refuses an empty list beside a grant with a message naming `fillTenants`, and `fillTenants`, an operator entry, walks the grants in bounded batches through a progress row of its own and inserts each missing tenant once; a list that holds a tenant whose first grant came after the table and misses an older one is not detected by that refusal, so the operator runs `fillTenants` on such a deployment before its first `startGeneration` (E-49, E-48, E-40, Law 11) ## Workflow @@ -74,9 +74,6 @@ The registry, the progress row, the per-entity marker and the gate document are - rule: [extension] The active generation switches in one write: one mutation that patches two generation rows together, the verified one to `active` and the active one to `retired`, or one row when the read model had no active generation, so no command and no query sees one row changed without the other; the old generation is kept for a rollback period (D9, E-8, F1) - rule: [extension] A retired generation is not written; a rollback reopens the retired generation the operator names as `verifying`, one verify pass repairs what the writes since its retirement changed, presence and absence alike, and the ordinary switch makes it active, so a rollback is a verify pass and then a switch of one write, never one write by itself, and the current generation serves until that switch; a generation that needs a write pause is never reopened, and its view returns to earlier rules through a new generation under the pause (E-8, D9, Sc L2-5) - rule: [extension] Each generation is written with the projection whose version its row records, by its batches and by live commands alike, and `startGeneration` and `rollbackGeneration` refuse a version the read model does not declare (E-9, E-8, D8) -- rule: Under the write pause every writer in the scope reads the maintenance gate, background writers are fenced or drained, and an operator can resume or abort (D9) -- rule: A refused write under the pause is a transient refusal, retryable and never a stored rejection (D4, D9) -- rule: [extension] Under the pause a history row's recorded version for a stream means every event of that stream with the row's key up to that version is folded: a stream's events reach the fold in version order, from its first event on the backfill and the verify pass and after its last folded version on the live path, a generation in flight is written by its own batches alone while its sources are closed, and no retired generation is reopened; so a verify pass that repeats the backfill and folds no event finds the generation covering every source (E-8, E-9, D9, F1) - rule: Every batch has a stated bound below the transaction limits, tested against the pinned Convex version; the limits are ceilings, not batch sizes (D19, F13) - rule: [extension] Every function that starts or changes a generation is an operator entry of `spec:command.actor-and-scope`: an internal function that records the stated operator on the generation row and reads no grant (E-48, D11, D19) - rule: [extension] A generation spans every tenant of the deployment: its batches take one tenant at a time from `nextTenant` of the tenant list, in tenant-ID order, and name that tenant on every context call, so no call runs with an absent tenant as a wildcard, and the one registry row per read model answers every tenant's queries (D11, Law 11, E-8, E-49) @@ -97,12 +94,8 @@ The registry, the progress row, the per-entity marker and the gate document are - Each verify batch enumerates the sources again from the progress row's cursor, which the end of the backfill reset to `null`, the first tenant, and calls `applyProjection` in backfill mode for every DTO, so a missing row is inserted, an older one updated, and the row of a DTO that projects to `null` deleted; each such repair is counted as a miss; when the last tenant's last page is done the generation moves to `verified` and no batch is scheduled (D9, E-8, Sc L2-5) - `switchGeneration` sets the verified generation `active` and, when the read model has an active generation, sets that one `retired` with `retiredAt` and `retireAfter`, in one mutation; every query reads the active generation from the registry, so every subscription on the read model re-runs (D9, F7, F1) - [extension] During the rollback period the retired generation's rows are kept and not written; `rollbackGeneration` reopens the retired generation it is given, one without `pauseRequired`, as `verifying` with a new fence and a `null` cursor and schedules a verify batch, live commands write it again from that commit, its one verify pass repairs every row that is missing, older or left behind by a deleted subject, and the ordinary `switchGeneration` makes it active and retires the generation that served meanwhile (E-8, D9, Sc L2-5) -- `abortGeneration` marks a building, verifying or verified generation `aborted`, leaves the active generation untouched, and resumes every closed gate entry that names this generation; an aborted rollback leaves the reopened generation `aborted`, never `retired` again (D9, Sc L2-6, E-8) +- `abortGeneration` marks a building, verifying or verified generation `aborted`, leaves the active generation untouched, and resumes every closed gate entry that names this generation; an aborted rollback leaves the reopened generation `aborted`, never `retired` again; reopening the sources is not a promise that commands can write, because a bound history view whose deployed version is not its active generation's, or that has no other generation, still fails its commands as a technical failure once `all` is open, unless the abort runs under `spec:application.history-rebuild#design.historyAbortHandoff`, which holds `all` closed until compatible code is deployed or a replacement generation's start closes the sources again (D9, Sc L2-6, E-8) - `purgeGeneration` starts the purge of an aborted generation, or of a retired one once `retireAfter` has passed; each purge batch deletes at most the progress row's batch size, capped by `limitPurgeBatch`, of rows of the generation for one tenant by the `by_key` range, stays on that tenant while a full batch came back, moves to the next tenant of the tenant list otherwise, and marks the generation `purged` after the last tenant; nothing starts a purge by itself, and a purge whose chain ended, a restore among the causes, is continued by `resumeGeneration` from its progress row (D19, E-40, E-49) -- For a history-dependent read model the generation row records `pauseRequired`, and `startGeneration` closes the maintenance gate for every source scope of the read model's bindings in the mutation that inserts the row, recording its own generation on each closed entry, so the gate is closed before the first batch is scheduled and this generation's batches are the one background writer allowed through (D9, E-8, E-38) -- While the gate is closed for the sources, every use case whose declaration writes one of them and the journal's baseline driver refuse with `writePaused` before any write, whether or not they maintain the view; a batch scheduled before the close reads the closed gate when it starts and refuses, a batch running when the close commits loses the optimistic-concurrency race on the gate document and re-runs into the refusal, and a batch of a superseded chain refuses because its generation fence is stale; nothing needs to drain (D9, D4, E-8, E-38, F1) -- Backfill under the pause enumerates a consistent cut because no source changes: each history batch takes the next stream of its cursor's source in its tenant, reads one `history` page of it from the saved event cursor, folds the page into the building generation through `applyHistoryProjection` in backfill mode, and saves the source, the stream and the event cursor on the progress row, moving to the next stream when the page reports `isDone`, to the next source when that source's enumeration in the tenant ends, and to the next tenant after the last source; an interrupted batch resumes from its cursor with the gate still closed, and the gate never reopens by itself (D9, E-9, Sc L2-6) -- Verification runs under the pause: the verify pass repeats the backfill's batches from the first tenant and source and counts as a miss every row they insert or update, which over a covered generation is none, so a miss is repaired by the batch that found it and the generation is `verified` when the pass ends; `switchGeneration` resumes every closed entry that names the generation in the mutation that makes it active, so writes reopen only after verification (D9, E-9, Sc L2-6) ## Design @@ -113,10 +106,8 @@ The enumeration is the context's `list` query with `includeDeleted`, the one rea - transactionBoundary: each batch is one scheduled internal mutation that commits its rows, its checkpoint and the scheduling of the next batch together; every operator entry that changes anything is one mutation, and `getGenerations` is one query (D9, F8, F9) - convexSurface: `startGeneration`, `interruptGeneration`, `resumeGeneration`, `switchGeneration`, `rollbackGeneration`, `abortGeneration`, `purgeGeneration` and `fillTenants` as the operator entries, `backfillBatch`, `verifyBatch`, `purgeBatch` and `fillTenantsBatch` as the scheduled batches, all internal mutations, and `getGenerations` as an internal query, in the composition's `rebuild` module; the gate entries are in the write-pause contract (D9, E-8, E-48) - typeRebuildTarget: `type RebuildTarget = { readModel: ReadModel; source: SourceRef; list: FunctionReference<"query", "internal", ListArgs, PaginationResult> }` one per-entity read model the deployment can rebuild, its one source stream type, and the reference to that stream type's `list` query in its context component, such as `components.orders.queries.order.list` (E-8, E-9, D10) -- typeHistorySource: `type HistorySource = { source: SourceRef; list: FunctionReference<"query", "internal", ListArgs, PaginationResult<{ version: StreamVersion }>>; history: FunctionReference<"query", "internal", { tenantId: string; streamId: string; fromVersion?: number; paginationOpts: PaginationOptions }, PaginationResult> }` one source stream type of a history view, with the references to its `list` and `history` queries of `spec:context.queries` in its context component, such as `components.inventory.queries.stockItem.list` and `components.inventory.queries.stockItem.history` (E-8, E-9, D10) -- typeHistoryRebuildTarget: `type HistoryRebuildTarget = { readModel: HistoryProjection; sources: readonly [HistorySource, ...HistorySource[]] }` one history view the deployment can rebuild under the write pause, with every context and stream type pair its bindings name, in the order a pass takes them; the order allocation history's lists the stock item source first and the order source second (E-8, E-9, E-38, D9) - typeRebuildConfig: `type RebuildConfig = { targets: readonly (RebuildTarget | HistoryRebuildTarget)[]; refs: { backfillBatch: BatchRef; verifyBatch: BatchRef; purgeBatch: BatchRef; fillTenantsBatch: FillBatchRef } }` where `type BatchRef = FunctionReference<"mutation", "internal", { generationId: Id<"generations">; fence: number }, null>`, `type FillBatchRef = FunctionReference<"mutation", "internal", { fence: number }, null>` and `refs` are the composition's own registered batches, which a batch schedules by reference; a target that carries `sources` is a history view, and no two targets share a read model name (E-40, E-49, F8) -- fnStartGeneration: `export const startGeneration = internalMutation({ args: { readModel: v.string(), projectionVersion: v.optional(v.number()), batchSize: v.optional(v.number()), operator: v.string() }, returns: v.id("generations"), handler })` in order: `assertOperator`; the target by name, else `"This deployment declares no read model " + readModel`; the version, `projectionVersion` when given and else the version of the first of the read model's `projections`, checked by `projectionOf`; the batch size by `batchSizeFor`; the in-flight check, throwing `"Read model " + name + " already has generation " + n + " in " + state` when a generation is `building`, `verifying` or `verified`; the tenant list check, throwing `"The tenant list is empty while grants exist; fillTenants fills it from the grants table"` when `nextTenant(ctx, null)` answers `null` and `ctx.db.query("grants").first()` answers a row, and `"The tenant list is being filled; startGeneration waits until fillTenants has ended"` when the `tenantFill` row is in the pass `fill`; then it inserts the generation row with the next number, one above the last row of `by_read_model` in descending order and 1 when there is none, `state: "building"`, `pauseRequired` false for a `RebuildTarget` and true for a `HistoryRebuildTarget`, whose start `historyStart` states, `fence: 1`, `startedAt` and `changedAt` at the mutation's `Date.now()` and `startedBy` and `changedBy` the operator, inserts the progress row with `pass: "backfill"`, the batch size, `cursor: null`, every count 0 and `updatedAt` at the same `Date.now()`, schedules `refs.backfillBatch` with fence 1 through `ctx.scheduler.runAfter(0, ...)`, and returns the generation row's ID (D9, E-8, E-9, E-40, E-48, E-49) +- fnStartGeneration: `export const startGeneration = internalMutation({ args: { readModel: v.string(), projectionVersion: v.optional(v.number()), batchSize: v.optional(v.number()), operator: v.string() }, returns: v.id("generations"), handler })` in order: `assertOperator`; the target by name, else `"This deployment declares no read model " + readModel`; the version, `projectionVersion` when given and else the version of the first of the read model's `projections`, checked by `projectionOf`; the batch size by `batchSizeFor`; the in-flight check, throwing `"Read model " + name + " already has generation " + n + " in " + state` when a generation is `building`, `verifying` or `verified`; the tenant list check, throwing `"The tenant list is empty while grants exist; fillTenants fills it from the grants table"` when `nextTenant(ctx, null)` answers `null` and `ctx.db.query("grants").first()` answers a row, and `"The tenant list is being filled; startGeneration waits until fillTenants has ended"` when the `tenantFill` row is in the pass `fill`; then it inserts the generation row with the next number, one above the last row of `by_read_model` in descending order and 1 when there is none, `state: "building"`, `pauseRequired` false for a `RebuildTarget` and true for a `HistoryRebuildTarget`, whose start `spec:application.history-rebuild#design.historyStart` states, `fence: 1`, `startedAt` and `changedAt` at the mutation's `Date.now()` and `startedBy` and `changedBy` the operator, inserts the progress row with `pass: "backfill"`, the batch size, `cursor: null`, every count 0 and `updatedAt` at the same `Date.now()`, schedules `refs.backfillBatch` with fence 1 through `ctx.scheduler.runAfter(0, ...)`, and returns the generation row's ID (D9, E-8, E-9, E-40, E-48, E-49) - fnBatchSizeFor: `batchSizeFor(readModel: ReadModel | HistoryProjection, requested?: number): number` throwing `"The batch size must be a whole number of at least 1, and " + requested + " is not"` unless `requested` is absent or an integer of at least 1, and returning `min(requested ?? rebuildBatchCeiling(readModel), rebuildBatchCeiling(readModel))`, so a batch size of 0, a negative one and a fraction are refused and a larger one is lowered to the ceiling (E-40, F13) - fnBackfillBatch: `export const backfillBatch = internalMutation({ args: { generationId: v.id("generations"), fence: v.number() }, returns: v.null(), handler })` (D9, E-40) - fnVerifyBatch: `export const verifyBatch = internalMutation({ args: { generationId: v.id("generations"), fence: v.number() }, returns: v.null(), handler })` (D9) @@ -126,15 +117,15 @@ The enumeration is the context's `list` query with `includeDeleted`, the one rea - step2: it takes its tenant: the cursor's `tenantId`, or `nextTenant(ctx, null)` when the cursor is `null`; when the tenant list is empty the batch has no tenant, reads the gate as step 3 states, and goes to step 6 with the pass done (E-49, Law 11) - step3: it calls the write pause's `gateAllows(ctx, ["all", "tenant:" + tenantId], generationId)`, with `["all"]` alone when it has no tenant, and when the answer is not allowed it parks: it patches the progress row's `lastError` to `"write paused for " + scopeKey + ": " + reason` and `updatedAt`, schedules nothing and returns, so the generation waits for `resumeGeneration` after the gate reopens; no batch calls `assertWritable`, because a batch parks its generation rather than failing its scheduled mutation with a thrown refusal (E-8, E-42, D9) - step4: it calls `list` once as `fnEnumerateCall` states, with the cursor's `pageCursor` (D10, E-40) -- step5: for each DTO of the page it calls `applyProjection(ctx, target.readModel, { tenantId, dto, versions: [dto.version], mode: "backfill", generations: [thisGeneration] })` for this generation alone, where `thisGeneration` is `{ role, generation: n, projectionVersion }` with the role `typeWritableGeneration` of `spec:application.generation-registry` gives the generation's state, adding one to `rowsWritten` for `inserted`, `updated` or `deleted` and one to `rowsSkipped` for `skipped-newer` or `unchanged` (D9, E-9) +- step5: for each DTO of the page it calls `applyProjection(ctx, target.readModel, { tenantId, dto, versions: [dto.version], mode: "backfill", generations: [thisGeneration] })` for this generation alone, where `thisGeneration` is `{ role, generation: n, projectionVersion }` with the role `spec:application.generation-registry#design.writableGenerationRole` gives the generation's state, adding one to `rowsWritten` for `inserted`, `updated` or `deleted` and one to `rowsSkipped` for `skipped-newer` or `unchanged` (D9, E-9) - step6: it checkpoints: when the page is not done the cursor becomes `{ tenantId, pageCursor: continueCursor }`; when it is done the cursor becomes `{ tenantId: next, pageCursor: null }` for `next = nextTenant(ctx, tenantId)`, and the pass is done when `next` is `null`; a pass that is not done patches the progress row with the cursor, the counts, `batchesDone` plus 1, no `lastError` and `updatedAt`, and schedules `refs.backfillBatch` with the same fence through `ctx.scheduler.runAfter(0, ...)`; a pass that is done patches the generation row to `state: "verifying"` and nothing else on it, patches the progress row to `pass: "verify"`, `cursor: null`, `misses: 0`, the counts, `batchesDone` plus 1 and `updatedAt`, and schedules `refs.verifyBatch` with the same fence (D9, F8, E-40, E-49) - verifySteps: `verifyBatch` runs the same six steps with `state` `verifying` and `pass` `verify`, adding one to `misses` for `inserted`, `updated` or `deleted` and one to `rowsSkipped` otherwise; a pass that is not done schedules `refs.verifyBatch`; a pass that is done patches the generation row to `state: "verified"` and nothing else on it, and the progress row to `pass: "idle"`, `cursor: null`, the counts, `batchesDone` plus 1 and `updatedAt`, and schedules nothing (D9, E-8, Sc L2-5) - purgeSteps: `purgeBatch` runs steps 1 to 3 with `state` `retired` or `aborted` and `pass` `purge`; it then reads `ctx.db.query(readModel.table).withIndex("by_key", (q) => q.eq("tenantId", tenantId).eq("generation", n)).take(Math.min(batchSize, limitPurgeBatch(readModel)))` with `batchSize` the progress row's and `readModel` the generation's read model, whose cap `limitPurgeBatch` declares, deletes each row and adds their number to `rowsPurged`; when it deleted that many rows the cursor stays on the tenant, and otherwise it moves to `nextTenant(ctx, tenantId)`, a purge cursor being `{ tenantId, pageCursor: null }`; a pass that is not done checkpoints, patching the progress row with the cursor, `rowsPurged`, `batchesDone` plus 1, no `lastError` and `updatedAt`, and schedules `refs.purgeBatch`; a pass that is done patches the generation row to `state: "purged"`, no `retiredAt` and no `retireAfter`, and nothing else on it, and the progress row to `pass: "idle"`, `cursor: null`, `rowsPurged`, `batchesDone` plus 1 and `updatedAt` (D19, E-40, E-49, Law 11) - purgeDriver: the purge batch is driven by the same self-scheduled internal mutation as the other batches (E-40, F17, Probe 6) - fnInterruptGeneration: `export const interruptGeneration = internalMutation({ args: { generationId: v.id("generations"), operator: v.string() }, returns: v.null(), handler })` after `assertOperator`, reading the generation row and nothing else, throwing `"Generation " + n + " of " + name + " has no batch to interrupt"` when `state` is `verified`, `active` or `purged`, in which no pass runs; otherwise adding 1 to the fence, setting `interruptedFence` to the new fence, setting `changedAt` and `changedBy`, and scheduling nothing; a `retired` or `aborted` generation runs a pass only during a purge, which its row does not tell, so the interrupt accepts either and the new fence changes nothing when no batch runs; it neither reads nor patches the progress row, so no checkpoint conflicts with it: the chain's pending batch finds `interruptedFence` equal to the fence and records the interruption as step 1 states, the generation stays at its checkpoint, and a parked or stranded chain, which has no pending batch, keeps its progress row as it was (E-40, E-48, D9, Sc L2-5) -- fnResumeChain: `resumeChain(ctx: MutationCtx, config: RebuildConfig, generationId: Id<"generations">, operator: string): Promise` throwing `"Generation " + n + " of " + name + " has no batch to resume"` when `pass` is `idle`; otherwise adding 1 to the fence, setting `changedAt` and `changedBy`, patching the progress row to no `lastError` and `updatedAt` and leaving its `batchSize`, cursor and counts as they are, and scheduling the batch of the pass, `refs.backfillBatch`, `refs.verifyBatch` or `refs.purgeBatch`, with the new fence, all in the caller's mutation; called by `resumeGeneration` and by the restore's `finishRestoreRun` for every generation the backup archive stranded (D9, F8, E-40, E-42) +- fnResumeChain: `resumeChain(ctx: MutationCtx, config: RebuildConfig, generationId: Id<"generations">, operator: string): Promise` throwing `"Generation " + n + " of " + name + " has no batch to resume"` when `pass` is `idle`; otherwise adding 1 to the fence, setting `changedAt` and `changedBy`, patching the progress row to no `lastError` and `updatedAt` and leaving its `batchSize`, cursor and counts as they are, and scheduling the batch of the pass, `refs.backfillBatch`, `refs.verifyBatch` or `refs.purgeBatch`, with the new fence, all in the caller's mutation; called by `resumeGeneration` and by the restore's `finishRestoreRun` for every generation the backup archive stranded; for a backfill or verify pass of a generation with `pauseRequired` it first makes the check of `spec:application.history-rebuild#design.historyCompatibility` and throws on a mismatch before it changes anything (D9, F8, E-40, E-42) - fnResumeGeneration: `export const resumeGeneration = internalMutation({ args: { generationId: v.id("generations"), operator: v.string() }, returns: v.null(), handler })` `assertOperator` and then `resumeChain`; it takes no batch size, so a resumed pass runs at the progress row's `batchSize`, a purge capped by `limitPurgeBatch`; for a generation in any state whose progress row names a pass, so it continues a backfill or verify pass of a generation in `building` or `verifying` and a started purge of a `retired` or `aborted` one, which is how an operator continues a purge a restore stranded (D9, Sc L2-5, E-40, E-48) -- fnSwitchGeneration: `export const switchGeneration = internalMutation({ args: { generationId: v.id("generations"), rollbackPeriodMs: v.optional(v.number()), operator: v.string() }, returns: v.null(), handler })` after `assertOperator`, throwing `"Generation " + n + " of " + name + " is " + state + "; switchGeneration needs verified"` unless the state is `verified`, and `"The rollback period must be between 0 and 2592000000 ms, and " + rollbackPeriodMs + " is not"` unless it is absent or a whole number in that range; then, with `now` the mutation's `Date.now()`, it reads the read model's active generation by `by_read_model_state` and, when there is one, patches it to `state: "retired"`, `retiredAt: now`, `retireAfter: now + (rollbackPeriodMs ?? 604800000)`, `changedAt` and `changedBy`, patches the verified generation to `state: "active"`, `switchedAt: now`, no `retiredAt`, no `retireAfter`, `changedAt` and `changedBy`, and, for a generation with `pauseRequired`, calls the write pause's `resumeScope` for every closed entry that names it (D9, E-8, E-48, F1) +- fnSwitchGeneration: `export const switchGeneration = internalMutation({ args: { generationId: v.id("generations"), rollbackPeriodMs: v.optional(v.number()), operator: v.string() }, returns: v.null(), handler })` after `assertOperator`, throwing `"Generation " + n + " of " + name + " is " + state + "; switchGeneration needs verified"` unless the state is `verified`, and `"The rollback period must be between 0 and 2592000000 ms, and " + rollbackPeriodMs + " is not"` unless it is absent or a whole number in that range, and, for a generation with `pauseRequired`, the error of the check of `spec:application.history-rebuild#design.historyCompatibility` on a mismatch; then, with `now` the mutation's `Date.now()`, it reads the read model's active generation by `by_read_model_state` and, when there is one, patches it to `state: "retired"`, `retiredAt: now`, `retireAfter: now + (rollbackPeriodMs ?? 604800000)`, `changedAt` and `changedBy`, patches the verified generation to `state: "active"`, `switchedAt: now`, no `retiredAt`, no `retireAfter`, `changedAt` and `changedBy`, and, for a generation with `pauseRequired`, calls the write pause's `resumeScope` for every closed entry that names it (D9, E-8, E-48, F1) - fnRollbackGeneration: `export const rollbackGeneration = internalMutation({ args: { generationId: v.id("generations"), operator: v.string() }, returns: v.null(), handler })` the operator names the retired generation to return to, and any retired generation inside its rollback period may be named; after `assertOperator` it throws `"Generation " + n + " of " + name + " is " + state + "; rollbackGeneration needs retired"` unless the state is `retired`, `"Generation " + n + " of " + name + " is past its rollback period"` when the mutation's `Date.now()` is at or after `retireAfter`, `"Generation " + n + " of " + name + " is already being purged"` when `pass` is `purge`, `"Generation " + n + " of " + name + " needs a write pause and cannot be rolled back"` when `pauseRequired` is set, `"The tenant list is being filled; rollbackGeneration waits until fillTenants has ended"` while the `tenantFill` row's `pass` is `fill`, the in-flight error of `startGeneration`, naming that generation's number and state, when another generation is `building`, `verifying` or `verified`, and the error of `projectionOf` when the read model no longer declares the generation's projection version; otherwise it patches the row to `state: "verifying"`, fence plus 1, no `retiredAt`, no `retireAfter`, `changedAt` and `changedBy`, patches the progress row to `pass: "verify"`, `cursor: null`, `misses: 0`, no `lastError` and `updatedAt`, with `batchSize` and the other counts kept, and schedules `refs.verifyBatch` with the new fence; the switch that completes the rollback is `switchGeneration` (D9, E-8, E-9, E-48) - fnAbortGeneration: `export const abortGeneration = internalMutation({ args: { generationId: v.id("generations"), reason: v.string(), operator: v.string() }, returns: v.null(), handler })` after `assertOperator`, throwing `"The reason must be between 1 and 256 bytes"` unless `utf8Length(reason)` is, and `"Generation " + n + " of " + name + " is " + state + "; abortGeneration needs building, verifying or verified"` unless the state is one of the three; otherwise patching the row to `state: "aborted"`, fence plus 1, `changedAt` and `changedBy`, patching the progress row to `pass: "idle"`, `lastError: "aborted: " + reason` and `updatedAt`, with the cursor and the counts kept, and calling the write pause's `resumeScope` for every closed entry that names this generation (D9, Sc L2-6, E-8, E-48) - fnPurgeGeneration: `export const purgeGeneration = internalMutation({ args: { generationId: v.id("generations"), operator: v.string() }, returns: v.null(), handler })` after `assertOperator`, throwing `"Generation " + n + " of " + name + " is " + state + "; purgeGeneration needs retired or aborted"` unless the state is one of the two, `"Generation " + n + " of " + name + " is inside its rollback period"` for a retired generation while the mutation's `Date.now()` is before `retireAfter`, and `"Generation " + n + " of " + name + " is already being purged; resumeGeneration continues it"` when `pass` is `purge`; otherwise adding 1 to the fence, setting `changedAt` and `changedBy`, patching the progress row to `pass: "purge"`, `cursor: null`, no `lastError` and `updatedAt`, and scheduling `refs.purgeBatch` with the new fence (D19, E-40, E-48) @@ -153,13 +144,7 @@ The enumeration is the context's `list` query with `includeDeleted`, the one rea - limitBuildingGenerations: at most one generation in `building`, `verifying` or `verified` per read model at a time, so a command writes at most two generations, the active one and that one (E-8, E-9) - limitRollbackPeriod: `[extension]` 7 days by default, stored as `retireAfter` on the retired row; the retired generation is kept and not written, and a rollback at or after `retireAfter` is refused (E-8, D9) - limitGenerationsListed: 20 generations per `getGenerations` answer, the newest first, a library constant (E-8, F13) -- fnHistoryPageCall: `ctx.runQuery(source.history, { tenantId, streamId, paginationOpts: { numItems: batchSize, cursor: eventCursor } })` with `source` the entry of the target's `sources` the cursor names and `batchSize` the progress row's, the one `history` call a history batch makes beside its one `list` call of `numItems` 1, returning a `paginator` page of envelopes in version order, capped by the query's own item and byte bounds, that the batch folds through `applyHistoryProjection` (D2, D10, E-9, E-24) -- historyCursor: a history view's batch cursor adds `sourceIndex`, the position in the target's `sources`, so a pass goes tenant by tenant in tenant-ID order, source by source in `sources` order within a tenant, stream by stream in the source's `list` order and page by page in version order within a stream, and reads the stream a cursor names from that source's `history`, so two sources with equal local stream IDs never meet; a pass of a history view costs at least one batch per tenant and source, the batch whose `list` page holds no stream (E-8, E-40, E-49, Law 11) -- historyStart: for a `HistoryRebuildTarget`, `startGeneration` takes the version the projection declares and refuses a given `projectionVersion` that differs with the message of `projectionOf`, sets `pauseRequired: true`, and, after inserting the generation row and the progress row and before scheduling, calls the write pause's `closeScope` once per entry of `sources`, in order, with `"source:" + contextId + ":" + streamType`, the reason `"rebuild of " + name + " generation " + n`, the operator and the new generation's ID, so a scope already closed or a full gate refuses the whole start and no batch is scheduled (D9, E-8, E-38, E-48) -- historySteps: for a history view a backfill or verify batch keeps steps 1 to 3 and replaces steps 4 to 6: with `source` the entry of `sources` at the cursor's `sourceIndex`, 0 when the cursor is `null`, when the cursor names no stream it calls `source.list` with `numItems` 1, `includeDeleted` and the page cursor; a page with no DTO reads no history: when it is not done the batch checkpoints at its `continueCursor`, and when it is done that source ends in the tenant and the batch checkpoints at the next source of the same tenant with a `null` page cursor, or after the last source at `{ tenantId: next, pageCursor: null, sourceIndex: 0 }` for `next = nextTenant(ctx, tenantId)`, the pass being done when `next` is `null`; a page with a DTO names the stream, its `version.streamId`, with a `null` event cursor and the page's `continueCursor` as the page cursor; for the stream the cursor or the page names, the batch then calls `source.history` as `fnHistoryPageCall` states, calls `applyHistoryProjection(ctx, target.readModel, { tenantId, events: page, mode: "backfill", generations: [thisGeneration] })`, adds one to `rowsWritten`, or to `misses` in a verify batch, for each `inserted` or `updated` and one to `rowsSkipped` for each `skipped-newer`, and checkpoints with the stream and the page's `continueCursor` as the event cursor, or with neither when the page reports `isDone`; every checkpoint patches the progress row and schedules the next batch as step 6 does for a pass that is not done, and the batch whose pass is done moves the generation and the progress row as step 6 and `verifySteps` state (D9, E-9, E-24, E-40, E-49) -- historyVerify: a verify batch of a history view runs `historySteps` from the `null` cursor the end of the backfill set, with `state` `verifying` and `pass` `verify`; under the pause no source changes, so over a generation the backfill covered every key answers `skipped-newer`, an event of type `baseline` or whose key is `null` is never a miss, and a row the pass finds missing or behind is repaired by the batch that read the events; the generation is `verified` when the pass ends, as for every read model, and the pass reads every source's history once more under the pause (D9, E-8, E-9, Sc L2-6) - rebuildClass: a read model declares its class at declaration time, per entity for the online path, as a `ReadModel` with its projections, and history across streams for the write pause, as a `HistoryProjection`; `pauseRequired` on the generation row records it, and a history view never rebuilds online because a live fold onto a row backfill has not reached cannot be told apart from a complete one (D9, D18, E-9) -- limitHistoryBatch: [extension] a history batch reads at most the progress row's `batchSize` events, which `startGeneration` sets through `batchSizeFor` like every read model's and nothing changes after, so the one formula of `limitBatchSize` gives 100 events at the default 16 KiB row budget and 32 at the 64 KiB ceiling, the order allocation history's; one batch reads one stream row of at most the stream's `budgetBytes` through its one `list` call, those events through its one `history` call, each at most `limitEventBytes` of `spec:context.event-envelope` and the 61 bytes of a stored document's system fields, with the page's baselines held to about 9 MiB by the query's byte bound, and at most one history row per event, at most 2 MiB, and writes at most 2 MiB, under the ceilings with the 1 second timeout in mind (E-40, E-9, E-24, F13) ## Example space @@ -182,9 +167,8 @@ And writes {writes:"never stopped"|"reopen only after verification"|"reopen afte - A reviewer confirms that the version compare in `applyProjection` is the only thing that prevents a stale overwrite and that L2-5's example asserts it against a concurrent live command. - A reviewer confirms that a batch patches the generation row only in the mutation that ends its pass, and that no command or query over a read model reads the progress row. - A reviewer confirms that every entry that changes a generation calls `assertOperator` first and writes the operator on the row, and that no rebuild function reads a grant or `ctx.auth`. -- A reviewer confirms that the gate is read inside every writer's mutation, not cached, that every batch reads it through `gateAllows` and parks its generation instead of throwing, that `startGeneration` closes one entry per source of a paused view and nothing else, and that the write-pause examples assert a refusal of a source writer while closed, the fold of the history view against the journal, and a reopen only after verification. +- A reviewer confirms that the gate is read inside every writer's mutation, not cached, and that every batch reads it through `gateAllows` and parks its generation instead of throwing. - A reviewer confirms that no registry function, batch or query takes a tenant scope for a generation, that every batch takes its tenant from `nextTenant` and names it on its context call, and that L2-5's example seeds more than one tenant and asserts both are covered at cutover. -- A reviewer confirms that a history target's `sources` name exactly the context and stream type pairs the view's bindings name, so `startGeneration` closes every writer of its sources, and that `rollbackGeneration` refuses every generation with `pauseRequired`. - A reviewer confirms that `resumeChain` is the only code path that reschedules a stranded generation and that the restore's `finishRestoreRun` calls it for every generation its inventory lists in `building` or `verifying`. - A reviewer confirms that `fillTenants` and `fillTenantsBatch` write only tenant rows and the `tenantFill` row, read a grant only through `by_principal` to find the next tenant, insert a tenant row only after a point read finds none, and that `startGeneration` refuses an empty tenant list beside a grant and a fill in its pass `fill`, and `fillTenants` a generation in flight. - A reviewer confirms that no function writes a retired generation, that `rollbackGeneration` is the only transition out of `retired` besides purge, and that L2-5's example asserts that the reopened generation is `verified`, holds a row for a subject created since its retirement and none for a subject deleted since, before it is active. diff --git a/design/specs/application/rebuild.write-pause-rebuild-abort.sdp.md b/design/specs/application/rebuild.write-pause-rebuild-abort.sdp.md deleted file mode 100644 index 9e3c692..0000000 --- a/design/specs/application/rebuild.write-pause-rebuild-abort.sdp.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -id: spec:application.rebuild.write-pause-rebuild-abort -kind: example -altitude: story -readiness: defined -relations: - refines: spec:application.rebuild - verifies: spec:application.rebuild ---- -# Rebuild a cross-stream history view under a write pause; interrupt, then abort - -Sc L2-6 · native backend tier · the abort case, the doc's second enumerated case of the row. - -## Intent - -- outcome: Resume or abort safely; writes reopen only after verification. (Sc L2-6) - -```gwt -Given a {viewKind: "cross-stream history"} read model with an active generation -And a new generation registered as {mode: "building under a write pause"} -And live commands {liveCommands: "are refused while paused"} -When the backfill is interrupted after {batches: 2} batches and then {resumption: "aborted"} -Then no row of the new generation holds data older than its source stream version -And the new generation {coverage: "is discarded and read by no query"} -And no command or external effect ran during the rebuild -And the active generation {cutover: "stays the old one"} -And writes {writes: "reopen after the abort"} -``` - -## Verification — executable - -- Runs in the native backend tier; every test owns its disposable backend. -- The view is the order allocation history of `spec:application.orders-inventory-example`, a `HistoryProjection` bound to the order stream type and the stock item stream type. -- The test starts a generation with `pauseRequired`, asserts that the gate holds one closed `source:` entry for each of the two stream types, each naming this generation, interrupts after the second history batch and calls `abortGeneration`. -- The test asserts that the generation is `aborted`, that the active generation is unchanged, that both source entries are gone from the gate, that `PlaceOrder` succeeds again, and that no query returns a row of the aborted generation. -- The test runs `purgeGeneration` and asserts that its batches run until the generation is `purged` and that every batch deleted at most `limitPurgeBatch` rows. diff --git a/design/specs/application/restore.sdp.md b/design/specs/application/restore.sdp.md index 2989bff..8871ad2 100644 --- a/design/specs/application/restore.sdp.md +++ b/design/specs/application/restore.sdp.md @@ -8,12 +8,14 @@ relations: dependsOn: - spec:context.event-envelope - spec:application.write-pause + - spec:application.history-rebuild - spec:application.generation-registry - spec:application.rebuild - spec:application.projection-contract - spec:application.parent-use-cases - spec:context.journal - spec:context.queries + - spec:command.actor-and-scope constrainedBy: - spec:laws.law03-events-only-source-of-state - spec:laws.law08-durable-capability-ships-operations @@ -52,10 +54,11 @@ The parts of restore that concern obligations, external dispatch and provider ev ### Open questions -- [non-blocking] Probe 7 observes local CLI replacement restoring the parent, Orders and Inventory documents with IDs and creation times; fresh replacement leaves scheduler tables empty and replacement in place keeps existing schedules; the complete Layer 2 restore drill, hosted dashboard restore, Workpool and Workflow remain open (Probe 7, F12, D19) -- [non-blocking] Extension E-42: the doc says restore runs with matching code and configuration, that domain, journal and read-model invariants hold, and that restore is tested; it does not list the checks or the record, and it does not say what a restore does to a rebuild that was in flight at the backup. The option taken here: four checks, journal, domain, read model and receipt, each a bounded checkpointed batch, the stream check bounded through the `rebuild` query's `maxEvents` argument so a batch folds at most 500 events in all, each folded stream also reading its own stored baseline event, a fifth step that inventories the closed gate entries, the generations in `building`, `verifying` or `verified` and the `imports` rows in `running` that the backup archive restored, a `restoreRuns` row as the record, an accepted branch that resumes every stranded generation the operator did not abort, the `MAINTENANCE_MODE` environment variable as the door, a failed check keeping the whole deployment closed with no override, and the drill before every release with its cadence reconsidered on the first drill's measured check times; the owner confirmed the checks, the door and both policies (E-42, D19, F12) -- [non-blocking] Extension E-42, the stream check's batch size: `limitCheckBatch` states the read of a batch that folds one stream, and each further stream the same batch folds reads its own stored baseline event of at most `budgetBytes + 4,096 + 61` bytes; the batch size that keeps a batch of several folded streams under the 16 MiB read ceiling is not pinned (E-42, F13) -- [non-blocking] Extension E-42, the enumeration of a stream awaiting a baseline: the journal check enumerates streams through `list`, and `list` refuses a row behind a registered step that reads history with the transient refusal `writePaused`, so the check cannot reach such a stream to inventory it as `awaitingBaseline`; that enumeration needs a read that does not refuse it, designed together with the entry of the baseline-migration sweep (E-42, E-25) +- [non-blocking #restoreBackupCoverage] Probe 7 observes local CLI replacement restoring the parent, Orders and Inventory documents with IDs and creation times; fresh replacement leaves scheduler tables empty and replacement in place keeps existing schedules; the complete Layer 2 restore drill, hosted dashboard restore, Workpool and Workflow remain open (Probe 7, F12, D19) +- [non-blocking #restoreInvariantChecks] Extension E-42: the doc says restore runs with matching code and configuration, that domain, journal and read-model invariants hold, and that restore is tested; it does not list the checks or the record, and it does not say what a restore does to a rebuild that was in flight at the backup. The option taken here: four checks, journal, domain, read model and receipt, each a bounded checkpointed batch, the stream check bounded through the `rebuild` query's `maxEvents` argument so a batch folds at most 500 events in all, each folded stream also reading its own stored baseline event, a fifth step that inventories the closed gate entries, the generations in `building`, `verifying` or `verified` and the `imports` rows in `running` that the backup archive restored, a `restoreRuns` row as the record, an accepted branch that resumes every stranded generation the operator did not abort, the `MAINTENANCE_MODE` environment variable as the door, a failed check keeping the whole deployment closed with no override, and the drill before every release with its cadence reconsidered on the first drill's measured check times; the owner confirmed the checks, the door and both policies (E-42, D19, F12) +- [non-blocking #restoreStreamBatchBounds] Extension E-42, the stream check's batch size: `limitCheckBatch` states the read of a batch that folds one stream, and each further stream the same batch folds reads its own stored baseline event of at most `budgetBytes + 4,096 + 61` bytes; the batch size that keeps a batch of several folded streams under the 16 MiB read ceiling is not pinned (E-42, F13) +- [non-blocking #restoreBaselineEnumeration] Extension E-42, the enumeration of a stream awaiting a baseline: the journal check enumerates streams through `list`, and `list` refuses a row behind a registered step that reads history with the transient refusal `writePaused`, so the check cannot reach such a stream to inventory it as `awaitingBaseline`; that enumeration needs a read that does not refuse it, designed together with the entry of the baseline-migration sweep (E-42, E-25) +- [non-blocking #restoreHistoryFoldVerification] Before a restore accepts an active history generation, how does its bounded check compare the projected fields and source versions against an independent fold without changing the read model, including when the saved code declares a different projection version from the active generation? What history-specific assertion belongs in the restore example? ## Workflow @@ -77,7 +80,7 @@ The parts of restore that concern obligations, external dispatch and provider ev - The operator starts the run record with `startRestoreRun`, which stores the backup ID, the backup time, the commit, the hashes and the operator (E-42, D19) - The journal check runs in batches: for each stream of a `list` page taken with `includeDeleted`, the context's `rebuild` query, called with the events the batch may still fold as `maxEvents`, reports `version`, `savedVersion`, `fromBaseline` and `eventsApplied`, and the check asserts that `eventsApplied` equals `version` minus the baseline version and that `version` equals `savedVersion`, which shows the events since the latest baseline are contiguous and the stream row agrees; a `rebuild` answer refused with `tailTooLong` is recorded as a journal finding naming the stream and its tail length, never retried with a bigger batch, while one refused with `budgetExhausted` is not a finding but the point where this batch stops and the next one resumes; event identity is enforced by the append's indexed read and is not rescanned (E-42, E-24, E-25, D2, D5) - The domain check runs in the same batches from the same `rebuild` call: its `equal` flag shows that folding the events since the latest baseline through `evolve` gives the saved state at the stream version (E-42, D3, Law 3, Sc L0-2) -- The read-model check runs in batches through the rebuild's verify logic in compare mode: for each read model's active generation, every enumerated stream has a row whose source versions equal the current versions and whose fields equal the projection of the current DTO (E-42, D8, D9) +- The read-model check runs in batches through the rebuild's verify logic in compare mode: for each per-entity read model's active generation, every enumerated stream has a row whose source versions equal the current versions and whose fields equal the projection of the current DTO; a history view's row records the last folded event of each source stream with its key, not one stream's DTO, and the check of its active generation is the open question `spec:application.restore#question.restoreHistoryFoldVerification` (E-42, D8, D9) - The receipt check runs in batches over receipts: for each receipt the context's `byOperation` query returns the events of its operation ID, and the check asserts that the stream versions the receipt lists are among them (E-42, D6, D19) - Each check batch writes its cursor, its examined count and its findings on the run record in the same mutation, so an interrupted check resumes from its cursor (E-42, F13) - The pending-work inventory runs after the four checks: it reads the restored gate document and records every closed entry, reads every generation in `building`, `verifying` or `verified` for each read model the parent declares and records it with its state and fence, and pages the `imports` table recording every row in `running`; nothing it records is a finding, because pending work is not a broken invariant (E-42, F12, D19) @@ -88,7 +91,7 @@ The parts of restore that concern obligations, external dispatch and provider ev ## Design -The restore itself is a platform operation outside any mutation. Everything the procedure adds is in the parent: a run record table, one internal mutation per check batch, one for the pending-work inventory, and the door in the writer check. Each check batch is one mutation. The journal and domain checks share one batch function, because both read the context's `rebuild` query, which folds a stream inside the context and compares it with the saved row; that query is per stream and runs in the batch's own transaction, so the batch hands it the events it may still fold and the query refuses before reading when the tail would not fit, which makes one 500-event fold the batch's worst case rather than a fold on top of a nearly full budget. The read-model check reuses the rebuild's verify logic, and the receipt check reads the parent's receipts and the context's `byOperation` query. The pending-work inventory reads the gate document, the registry and the `imports` table and writes what it finds on the run record. The procedure's six mutations are the only mutations in the parent that do not read the write pause's gate: they write only the `restoreRuns` record and, in the accepted branch, the generation rows they resume; they run only while a run is `running`; and the door they hold closed would otherwise refuse them. +The restore itself is a platform operation outside any mutation. Everything the procedure adds is in the parent: a run record table, one internal mutation per check batch, one for the pending-work inventory, and the door in the writer check. Each check batch is one mutation. The journal and domain checks share one batch function, because both read the context's `rebuild` query, which folds a stream inside the context and compares it with the saved row; that query is per stream and runs in the batch's own transaction, so the batch hands it the events it may still fold and the query refuses before reading when the tail would not fit, which makes one 500-event fold the batch's worst case rather than a fold on top of a nearly full budget. The read-model check reuses the rebuild's verify logic, and the receipt check reads the parent's receipts and the context's `byOperation` query. The pending-work inventory reads the gate document, the registry and the `imports` table and writes what it finds on the run record. The procedure's six mutations are among the mutations in the parent that do not read the write pause's gate, with the gate's own entries, the rebuild's `fillTenants` and the rebuild's other operator entries, the inventory `spec:application.write-pause` states: the six write only the `restoreRuns` record and, in the accepted branch, the generation rows they resume; they run only while a run is `running`; and the door they hold closed would otherwise refuse them. - transactionBoundary: none for the import of the backup archive; each check batch is one internal mutation that reads through one component query and writes only the run record; `finishRestoreRun` is one mutation that marks the run and reschedules the stranded generations together, so acceptance and the rescheduling commit or fail as one (D19, F8, F13) - convexSurface: `startRestoreRun`, `streamCheckBatch`, `readModelCheckBatch`, `receiptCheckBatch`, `pendingWorkBatch`, `finishRestoreRun` as internal mutations and `getRestoreRun` as an internal query, all in the parent (E-42, D19) @@ -103,6 +106,7 @@ The restore itself is a platform operation outside any mutation. Everything the - fnReceiptCheckBatch: `export const receiptCheckBatch = internalMutation({ args: { runId: v.id("restoreRuns"), scope: checkScopeValidator }, returns: v.boolean(), handler })` reading 25 receipts by tenant from the receipt cursor and calling `ctx.runQuery(components.orders.queries.operations.byOperation, { tenantId, operationId })` per receipt for each context the receipt's versions name (E-42, D6, D19) - fnPendingWorkBatch: `export const pendingWorkBatch = internalMutation({ args: { runId: v.id("restoreRuns") }, returns: v.boolean(), handler })` which on its first call, when the inventory's cursor is empty, reads the gate document and appends one `closedScope` item per closed entry, then reads `by_read_model_state` for each read model the parent's declarations name and each of `building`, `verifying` and `verified` and appends one `generation` item per row with its state and fence, and on every call pages the `imports` table with `.paginate` in creation order, `limitPendingWorkBatch` rows at a time, appending one `import` item per row in `running`; it saves the page cursor on its `checks` entry and returns whether the paging is done (E-42, F12, D19, E-41) - fnFinishRestoreRun: `export const finishRestoreRun = internalMutation({ args: { runId: v.id("restoreRuns"), operator: v.string() }, returns: v.null(), handler })` an operator entry of `spec:command.actor-and-scope`, which calls `assertOperator` first, writes the stated operator on the run as `finishedBy` whether it marks the run failed or accepted, and passes it to `resumeChain`, marking the run failed when any check has a finding; otherwise refusing with a plain error while `MAINTENANCE_MODE` is still `restore` or the inventory has not finished, and then, for every `generation` item whose row is still in `building` or `verifying` at the inventoried fence, calling the rebuild's `resumeChain` in this mutation, which bumps the fence and schedules the next batch from the saved cursor, writing `resumed` on the item, `aborted` on one whose row is `aborted`, `kept` on a `closedScope` item still closed, `recorded` on an `import` item, and marking the run accepted with the finish time (E-42, E-48, D19, F8, F12, Sc L2-8) +- finishCompatibility: [extension] `resumeChain` refuses a history generation that is not compatible with the deployed code, as the history rebuild's compatibility check states, and `finishRestoreRun` does not catch the refusal, so one incompatible generation refuses the whole finish: the run stays unaccepted and no fence, schedule or item resolution commits; before clearing `MAINTENANCE_MODE` for a finish that resumes a history generation the operator closes `all` with `closeGate` and confirms it with `getGate`, so a refused finish leaves callers refused with `writePaused`; the operator then deploys compatible code and calls `finishRestoreRun` again, or, with `all` still closed, aborts the incompatible generation through `spec:application.history-rebuild#design.historyAbortHandoff`, which continues with `all` closed and keeps it closed until the run is accepted, and calls it again, the item then answered `aborted`; `all` is reopened only after the run is accepted and the deployed bindings and projection versions are compatible, and each chain parked under it is then resumed with `resumeGeneration` (E-42, E-8, E-9, D19) - doorExemption: `startRestoreRun`, `streamCheckBatch`, `readModelCheckBatch`, `receiptCheckBatch`, `pendingWorkBatch` and `finishRestoreRun` do not read the gate through `assertWritable` or `gateAllows` and write only `restoreRuns`, except that `finishRestoreRun`'s accepted branch also patches the generation rows it resumes and their progress rows; `startRestoreRun` refuses when `MAINTENANCE_MODE` is not `restore`, and each later mutation refuses when its run is not `running`, so no operational row is written outside a restore and no business table is written by them at all (E-42, D19) - checkOrder: journal, domain, read model, receipt, then the inventory; a later check does not start while an earlier one has findings, because a broken journal makes the fold check meaningless, and the inventory runs last because it is not a check and its items never fail the run (E-42, D3) - gapRecord: the run record's `backupTakenAt` is the gap start; the procedure writes no synthetic receipt and invents no event for the gap (D19, Existing systems) @@ -126,8 +130,9 @@ And a generation stranded in the backup archive is {stranded:"resumed by the acc ## Verification — reviewed - A reviewer confirms that no check batch calls a context operation, schedules anything or writes anything but the run record, and that the only scheduling in the procedure is `finishRestoreRun`'s rescheduling of inventoried generations in its accepted branch. -- A reviewer confirms that the six restore mutations are the only functions in the parent that skip the gate read, that each writes only `restoreRuns` apart from the generation rows `finishRestoreRun` resumes, and that each refuses outside a running restore. +- A reviewer confirms that the gate's own entries, the rebuild's `fillTenants`, which reads the restore door alone and is refused while it is closed, the rebuild's other operator entries, `startGeneration`, `interruptGeneration`, `resumeGeneration`, `switchGeneration`, `rollbackGeneration`, `abortGeneration` and `purgeGeneration`, which change only the generation, progress and gate rows they name while the batches they schedule read the gate and park, and the six restore mutations are the only mutations in the parent that skip the gate read, that each restore mutation writes only `restoreRuns` apart from the generation rows `finishRestoreRun` resumes, and that each restore mutation refuses outside a running restore. - A reviewer confirms that the door is read by the writer check before the gate document and that the procedure clears it only in the accepted branch, before `finishRestoreRun` reschedules anything. - A reviewer confirms that `streamCheckBatch` passes `maxEvents` on every `rebuild` call, that a `budgetExhausted` refusal is never recorded as a finding, and that the read of a batch that folds one stream, one full fold with its stored baseline event plus the page read twice, is stated below the ceiling, with each further folded stream's own stored baseline event named beside it. - A reviewer confirms that every generation the inventory lists is `resumed` or `aborted` on an accepted run record and that no closed gate entry on an accepted run names a generation that is not `building`, `verifying` or `verified` again. +- A reviewer confirms that a test restores two resumable history generations of which the second is incompatible with the deployed code, and finds `finishRestoreRun` refused, the run unaccepted, no fence or schedule changed and `all` and the source scopes closed, and the finish accepted after compatible code or the abort handoff. - A reviewer confirms that the recovery plan entry named in the first flow exists for every backup the drill uses. diff --git a/design/specs/application/write-pause.sdp.md b/design/specs/application/write-pause.sdp.md index 3e2f81a..42110d0 100644 --- a/design/specs/application/write-pause.sdp.md +++ b/design/specs/application/write-pause.sdp.md @@ -15,12 +15,15 @@ relations: - spec:facts.f11-components-have-no-ctx-auth - spec:facts.f12-backups-exclude-pending-scheduled-functions - spec:facts.f09-scheduled-mutation-and-action-retry-semantics + dependsOn: + - spec:command.actor-and-scope + - spec:operations.baseline-operations --- # Write pause Story · Detail: full · Traces: D4, D9, D19, Law 6, F1, F9, F12, F13, Sc L2-6, Sc L2-8, E-8, E-42, E-43, E-48. -The write pause is the maintenance gate every writer in a scope reads inside its own mutation. It exists for the rebuild of a read model that depends on history across several streams, where a consistent cut is needed, and the restore procedure holds the same door closed for every scope through the environment, because the backup archive replaces the gate document, until its checks pass. A closed gate turns every write in scope into a transient refusal, retryable and never stored; a background batch scheduled before the close reads the closed gate when it starts, and a batch of a superseded chain is refused by its generation's fence. The rebuild closes the gate for its own scope in the mutation that registers its generation, an operator entry closes a tenant or the whole deployment, and operator entries resume and abort. Nothing reopens the gate by itself. Optimistic concurrency makes the close a consistent cut, so the gate has one closed state and no drain. +The write pause is the maintenance gate every writer in a scope reads inside its own mutation. It exists for the rebuild of a read model that depends on history across several streams, where a consistent cut is needed, and the restore procedure holds the same door closed for every scope through the environment, because the backup archive replaces the gate document, until its checks pass. A closed gate turns every write in scope into a transient refusal, retryable and never stored; a background batch scheduled before the close reads the closed gate when it starts, and a batch of a superseded chain is refused by its generation's fence. An operator entry closes a tenant or the whole deployment, and operator entries resume and abort. Nothing reopens the gate by itself. Optimistic concurrency makes the close a consistent cut, so the gate has one closed state and no drain. ## Intent @@ -33,24 +36,23 @@ The write pause is the maintenance gate every writer in a scope reads inside its ### Open questions -- [non-blocking] Extension E-8: the doc names a maintenance gate every writer reads, fencing or draining of background writers, and operator resume and abort; the option taken here is one gate document per deployment holding the closed scopes, each in one `closed` state, with no fence of its own and no drain interval, because optimistic concurrency serializes every writer's gate read against the close and the generation row's fence already fences a rebuild's batches, and a refusal answered with `writePaused`, the transient code the outcome boundary reserves for this contract under its E-35, which this contract confirms; the scope a paused rebuild closes is the paused view's sources, one `source:` entry per context and stream type its bindings name, because a consistent cut must hold every writer of those streams and a use case that writes a source stream without maintaining the view would otherwise pass the gate, which is why the command declaration names what it writes under E-38 and the pipeline proves the list at its step 9; an operator entry closes only a tenant or `all`, the coarser and always safe scopes, and no scope names a read model, because nothing would close it (E-8, E-38, D9, D4, F1) +- [non-blocking #writePauseSourceScopes] Extension E-8: the doc names a maintenance gate every writer reads, fencing or draining of background writers, and operator resume and abort; the option taken here is one gate document per deployment holding the closed scopes, each in one `closed` state, with no fence of its own and no drain interval, because optimistic concurrency serializes every writer's gate read against the close and the generation row's fence already fences a rebuild's batches, and a refusal answered with `writePaused`, the transient code the outcome boundary reserves for this contract under its E-35, which this contract confirms; the scope a paused rebuild closes is the paused view's sources, one `source:` entry per context and stream type its bindings name, because a consistent cut must hold every writer of those streams and a use case that writes a source stream without maintaining the view would otherwise pass the gate, which is why the command declaration names what it writes under E-38 and the pipeline proves the list at its step 9; an operator entry closes only a tenant or `all`, the coarser and always safe scopes, and no scope names a read model, because nothing would close it (E-8, E-38, D9, D4, F1) +- [non-blocking #generationGateExemption] Extension E-8, the generation exemption: a batch reads the gate with the scopes `all` and its tenant, and no entry of either scope ever names a generation, so the exemption `gateAllows` makes for a given generation is reached by no reader; either the exemption goes, or a history batch reads the gate with its source scopes as well and passes them by it (E-8, D9) +- [non-blocking #writePauseOperatingInstructions] Should its operating instructions distinguish a stopped pass from a verified generation awaiting switch, using `getGenerations` and `getGate`, and measure the closed interval through the successful switch or abort? Keep any acceptable pause length for the owner's decision rather than deriving a timeout from a batch estimate. ## Contract -- There is one gate document per deployment, read through `assertWritable`, which throws, by the command pipeline at its step 7, by the grant helpers `insertGrant` and `revokeGrant` of `spec:command.actor-and-scope` before they write, and by the receipt sweep `sweep` of `spec:command.receipt-table`, which `sweepNext` runs, after its argument check and before it reads a receipt, and by every background batch at the start of its mutation through `gateAllows`, which answers and lets the batch park, never cached across mutations; the background readers are the rebuild's backfill, verify and purge batches, its tenant fill's `fillTenantsBatch`, and the journal's baseline driver of `spec:context.journal`, which writes a source stream and reschedules itself when refused; the only mutations that skip the read are the gate's own entries, the rebuild's `fillTenants`, which reads the restore door alone and is refused while it is closed, and the restore procedure's `startRestoreRun`, its check batches, its pending-work inventory and `finishRestoreRun`, which write only the `restoreRuns` record and the generation rows the accepted branch resumes, with their progress rows, while the restore door is closed, as `spec:application.restore` states; every command that reaches step 7 reads the gate; a duplicate answered from its receipt does not, because its answer writes nothing (D9, D6, E-8, E-25, E-37, E-42) +- There is one gate document per deployment, read through `assertWritable`, which throws, by the command pipeline at its step 7, by the grant helpers `insertGrant` and `revokeGrant` of `spec:command.actor-and-scope` before they write, and by the receipt sweep `sweep` of `spec:command.receipt-table`, which `sweepNext` runs, after its argument check and before it reads a receipt, and by every background batch at the start of its mutation through `gateAllows`, which answers and lets the batch park, never cached across mutations; the background readers are the rebuild's backfill, verify and purge batches, its tenant fill's `fillTenantsBatch`, and the journal's baseline driver of `spec:context.journal`, which writes a source stream and reschedules itself when refused; the only mutations that skip the read are the gate's own entries, the rebuild's `fillTenants`, which reads the restore door alone and is refused while it is closed, the rebuild's other operator entries, `startGeneration`, `interruptGeneration`, `resumeGeneration`, `switchGeneration`, `rollbackGeneration`, `abortGeneration` and `purgeGeneration`, which change only the generation, progress and gate rows they name while the batches they schedule read the gate and park, and the restore procedure's `startRestoreRun`, its check batches, its pending-work inventory and `finishRestoreRun`, which write only the `restoreRuns` record and the generation rows the accepted branch resumes, with their progress rows, while the restore door is closed, as `spec:application.restore` states; every command that reaches step 7 reads the gate; a duplicate answered from its receipt does not, because its answer writes nothing (D9, D6, E-8, E-25, E-37, E-42) - A scope is `all`, one tenant, or one source stream type of one context; the gate holds the set of closed scopes, each with a reason, the generation it serves if any, and the operator who closed it (E-8, E-48, D9) - A writer in a closed scope refuses before any write with the transient refusal `writePaused`, whose message carries the scope and the reason; nothing is stored, no operation ID is minted, and the same request key succeeds once the scope is open (D9, D4, Law 6) - A use case is in scope when `all` is closed, when its tenant is closed, or when a stream type its declaration's `writes` names is closed (E-8, E-38, D9) - The restore door is read before the gate document: while the deployment's `MAINTENANCE_MODE` environment variable is exactly `restore`, every gate read answers closed with the scope `all` and the reason `restore` and reads no document; any other value, and an unset variable, leave the answer to the gate document (D19, F12, Sc L2-8, E-42) - A deployment with no gate document is open: the gate read never writes, and the first close inserts the document (E-8, D9) -- [extension] A rebuild that needs a consistent cut closes the sources of the paused view, one `source:` scope per context and stream type its bindings name, so every writer of those streams is held whether or not it maintains the view; the closed entries name the generation, and `abortGeneration` and the switch's resume reopen every entry that names it (E-8, E-38, D9, Sc L2-6) - [extension] The rebuild a scope was closed for is the one background writer allowed through it: a gate read that states a generation passes a closed entry only when the entry names a generation and names that one, so an entry with no generation admits no one, and a gate read that states no generation passes no closed entry (E-8, D9) - Background writers are fenced, not drained: a batch scheduled before the close reads the closed gate when it starts and refuses, a batch running when the close commits loses the optimistic-concurrency race on the gate document and re-runs into the refusal, a batch that committed first wrote before the cut, and a batch of a superseded chain checkpoints nothing and schedules nothing because the generation row's fence differs from its argument, writing at most the interruption on the progress row while the row's `interruptedFence` equals its fence (D9, E-8, F1) - The gate has one `closed` state per scope and no draining state, drain interval or confirmation step, because the serialization above already gives the consistent cut and a second state would only add an operator duty (E-8, F1) -- `startGeneration` closes one entry per source of a paused view and records its own generation on each in the mutation that inserts the generation row, so every entry that names a generation names a row that exists (D9, E-8) - [extension] The operator entries are `closeGate` and `resumeGate` for the scope `all` and for a tenant scope, and `getGate` and `getGateAudit` to read; each follows the operator entry contract of `spec:command.actor-and-scope`, and a source scope is closed and reopened only by the rebuild (E-8, E-48, D19) - Abort marks the generation aborted and resumes its scopes in one mutation (D9, Sc L2-6) -- [extension] A source scope closes only for a generation `startGeneration` registers: `rollbackGeneration` of `spec:application.rebuild` refuses a retired generation with `pauseRequired`, because no function would close its sources while live commands fold onto rows that missed the events since its retirement, so a history view returns to earlier rules only through a new generation, whose start closes its sources again (E-8, D9, Sc L2-6) - The restore procedure cannot use the gate document, because the backup archive replaces it; the restore door keeps every writer out through the environment, which survives a restore because backups exclude environment variables; the restore procedure's own mutations, which write only `restoreRuns` and the generation rows the accepted branch resumes with their progress rows, are the one exemption, and that workflow states it (D19, F12, Sc L2-8, E-42) - [extension] A closed entry the backup archive brings back is as closed as one written live: an entry that names a generation stays closed while the restore's pending-work step resumes that generation, and reopens only when that generation is switched or an operator aborts it, and a tenant or `all` entry stays closed until an operator resumes it, so the gate never reopens by itself through a restore either (E-8, E-42, F12, D19) - [extension] Gate changes are treated as mandatory audit: every close and every resume inserts one record through `writeOperatorAudit` of `spec:operations.baseline-operations` inside the same mutation and fails closed, so a gate change whose audit insert fails changes nothing: the gate document is as it was and no record exists; the doc fails closed only for mandatory business or security audit and does not name gate changes among them (E-8, E-43, D19) @@ -61,9 +63,9 @@ The write pause is the maintenance gate every writer in a scope reads inside its The gate is one document so that a command pays one point read. Scopes are strings so that the same document serves tenants, source stream types and the whole deployment without a table per kind. The refusal is a `ConvexError` whose data is the outcome boundary's `TransientData`. The table is `gateTables`, a schema fragment the library exports, and `GateDataModel` is its data model; every composition that registers a command spreads it into its schema, because step 7 reads it. The library exports the four operator entries as registered internal functions, built with `internalMutationGeneric` and `internalQueryGeneric` of `convex/server`, and a composition's `gate` module re-exports them, so they are reached as `internal.gate.closeGate`, `internal.gate.resumeGate`, `internal.gate.getGate` and `internal.gate.getGateAudit`. Every refusal of an operator entry is a plain `Error` with the message given. - validatorClosedEntry: `const closedEntryValidator = v.object({ scopeKey: v.string(), reason: v.string(), generationId: v.optional(v.id("generations")), changedAt: v.number(), changedBy: v.string() })` where `changedBy` is the stated operator and `changedAt` the closing mutation's `Date.now()` (E-8, E-48) -- tableMaintenanceGates: `maintenanceGates: defineTable({ key: v.literal("gates"), closed: v.array(closedEntryValidator), updatedAt: v.number() }).index("by_key", ["key"])` holding at most one document, whose `closed` lists the entries in the order they were closed (E-8, D9) +- tableMaintenanceGates: `maintenanceGates: defineTable({ key: v.literal("gates"), closed: v.array(closedEntryValidator), updatedAt: v.number() }).index("by_key", ["key"])` holding at most one document, whose `closed` lists the entries in the order they were closed, an order `closeScope` and `resumeScope` keep, because a history generation's compatibility check reads its source entries in that order (E-8, D9) - indexGatesByKeyUse: the one point read every writer makes, `.withIndex("by_key", (q) => q.eq("key", "gates")).unique()`; an absent document reads as no closed scope (E-8, D9) -- typeScopeKey: `type ScopeKey = "all" | string` where a tenant scope is `"tenant:" + tenantId` and a source scope is `"source:" + contextId + ":" + streamType`, built only by `scopesOfUseCase`, `startGeneration`, the rebuild's batches, the journal's baseline driver, the receipt sweep `sweep` of `spec:command.receipt-table`, through `tenantScope`, and the operator entries (E-8, E-38, D11) +- typeScopeKey: `type ScopeKey = "all" | (string & {})` where a tenant scope is `"tenant:" + tenantId` and a source scope is `"source:" + contextId + ":" + streamType`, built only by `scopesOfUseCase`, `startGeneration`, the rebuild's batches, the journal's baseline driver, the receipt sweep `sweep` of `spec:command.receipt-table`, through `tenantScope`, and the operator entries (E-8, E-38, D11) - fnGateAllows: `gateAllows(ctx: { db: GenericDatabaseReader }, scopes: readonly ScopeKey[], allowGeneration?: Id<"generations">): Promise<{ allowed: true } | { allowed: false; scopeKey: ScopeKey; reason: string }>` the one gate read: it first reads `process.env.MAINTENANCE_MODE` and answers `{ allowed: false, scopeKey: "all", reason: "restore" }` without reading the document when the value is `restore`; otherwise it reads the gate document once and answers not allowed with the `scopeKey` and `reason` of the first entry of `closed`, in array order, whose `scopeKey` is one of `scopes` and which is not exempt, and allowed when there is none; an entry is exempt only when `allowGeneration` is given, the entry has a `generationId`, and the two are equal; it writes nothing (E-8, E-42, D9, F9) - fnAssertWritable: `assertWritable(ctx: { db: GenericDatabaseReader }, scopes: readonly ScopeKey[]): Promise` calling `gateAllows` with no generation and, when the answer is not allowed, throwing through the boundary's `refuseTransient({ code: "writePaused", message: "write paused for " + scopeKey + ": " + reason })`; the command pipeline's step 7 calls it before it mints the operation, and `insertGrant` and `revokeGrant` of `spec:command.actor-and-scope` call it with `scopesOfUseCase(grant.tenantId, [])` before their first read, and the receipt sweep `sweep` of `spec:command.receipt-table` calls it with `["all", tenantScope(tenantId)]`, no source scope because it writes no source stream, after its argument check and before its first receipt read, so `sweepNext` reads it through `sweep` after its tenant-list read; nothing else does (E-8, D9, E-35, E-37) - errorCodeWritePaused: `writePaused`, the transient code the outcome boundary reserves; the refusal's data is exactly `{ kind: "transient", code: "writePaused", message }` with no `retryAfterMs`, where `message` is `"write paused for all: restore"` under the restore door and `"write paused for tenant:" + tenantId + ": " + reason` for a closed tenant; a transient refusal, never a stored rejection, so no receipt, event, read-model row or audit record exists for the refused command (D4, Law 6, E-35) @@ -83,7 +85,7 @@ The gate is one document so that a command pays one point read. Scopes are strin ## Verification — reviewed -- A reviewer confirms that `assertWritable` is called by the command pipeline's step 7 for every command that reaches it, by the two grant helpers before they write and by the receipt sweep before it reads a receipt, that `gateAllows` is called by every background batch before its first write and that the batch parks rather than throws on a refusal, that the two are one read because `assertWritable` is `gateAllows` plus the throw, that the gate's entries and the restore procedure's six mutations are the only exemptions, and that no function caches the gate across mutations. +- A reviewer confirms that `assertWritable` is called by the command pipeline's step 7 for every command that reaches it, by the two grant helpers before they write and by the receipt sweep before it reads a receipt, that `gateAllows` is called by every background batch before its first write and that the batch parks rather than throws on a refusal, that the two are one read because `assertWritable` is `gateAllows` plus the throw, that the gate's own entries, the rebuild's `fillTenants`, which reads the restore door alone and is refused while it is closed, the rebuild's other operator entries, `startGeneration`, `interruptGeneration`, `resumeGeneration`, `switchGeneration`, `rollbackGeneration`, `abortGeneration` and `purgeGeneration`, which change only the generation, progress and gate rows they name while the batches they schedule read the gate and park, and the restore procedure's six mutations are the only exemptions, and that no function caches the gate across mutations. - A reviewer confirms that `gateAllows` reads the environment before the document, that an entry with no generation is never exempt, and that a gate read with no generation passes no closed entry. - A reviewer confirms that no function other than `startGeneration` closes a source scope, so every closed entry that names a generation names a row that exists, and that the journal's baseline driver reads the gate with the source scope of the stream type it migrates before every batch. - A reviewer confirms that `closeScope` and `resumeScope` are the only writers of the gate document, that each inserts its audit record in the same mutation with no catch around it, and that a test whose audit table refuses the record finds the gate unchanged. diff --git a/design/specs/command/actor-and-scope.sdp.md b/design/specs/command/actor-and-scope.sdp.md index c106d49..6665553 100644 --- a/design/specs/command/actor-and-scope.sdp.md +++ b/design/specs/command/actor-and-scope.sdp.md @@ -28,11 +28,11 @@ This contract pins the shapes the parent establishes and passes into components: ### Open questions -- [non-blocking] Extension E-6: the doc names the actor kinds, the envelope's actor field and the worker's two modes; the field names, the namespace union, the grant fields and the `authorize` signature are the design's, and the grant's `permission` string vocabulary is per product; the worker's two modes are one `Authority` shape, `{ actor, scope, mode }` with the literals `recheckDelegator` and `serviceAuthority`, stored on the obligation record and never restated, where the delegation travels on the actor's own `onBehalfOf` and `delegationRef`; and the doc says a derived command carries a server namespace without naming it, so this design assigns `worker` to every command a reaction issues, which is the namespace Sc L1-7 refuses to a client and the one D6 lists among the receipted callers, and reserves `system` for the platform's own maintenance chains and operator operations that run a receipted command, such as a restore or a migration driver, so a `system` receipt is never a reaction's; the owner confirms both (E-6, E-54, D6, D11, D13, Sc L1-7) -- [non-blocking] Extension E-6: the tenant scope of an operation, of a command and of a query is the `tenantId` argument alone, so none takes a separate scope argument; this is a provisional reading the owner rules (E-6, Law 11, D11) -- [non-blocking] Extension E-6: the doc says a query authorizes before it discloses and does not say how; here a parent query calls `authorizeQuery`, which establishes and authorizes its actor through the same two helpers a command's entry uses and is generic over the parent's data model; the owner has not ruled (E-6, D11, Law 5) -- [non-blocking] Extension E-48, the operator entry: the doc lists operators among the actors and names operator duties without saying how an operator reaches them; the option taken here is that admin access is the authority of an operation over the whole deployment, so every operator entry is an internal function that only admin access reaches, takes the operator as a required string, reads no grant and records that string on what it changes; a stated operator is what admin access chose to state and proves no identity, D11's one trusted deployment is the boundary, and no scope beyond the tenant is added (E-48, D11, D19) -- [non-blocking] Extension E-49, the tenant list: the doc has every record name its tenant and names no list of tenants, while a batch over the whole deployment needs the next tenant; the option taken here is a `tenants` table in the parent with one row per tenant, written by `insertGrant` in the mutation that inserts a tenant's first grant and never deleted, read one tenant at a time in tenant-ID order; the do-nothing option, reading the distinct tenant IDs off the `grants` index, is not taken because revocation deletes grant rows and a tenant whose grants are all revoked still owns records (E-49, D11, Law 11, E-37) +- [non-blocking #actorCallerNamespaceContract] Extension E-6: the doc names the actor kinds, the envelope's actor field and the worker's two modes; the field names, the namespace union, the grant fields and the `authorize` signature are the design's, and the grant's `permission` string vocabulary is per product; the worker's two modes are one `Authority` shape, `{ actor, scope, mode }` with the literals `recheckDelegator` and `serviceAuthority`, stored on the obligation record and never restated, where the delegation travels on the actor's own `onBehalfOf` and `delegationRef`; and the doc says a derived command carries a server namespace without naming it, so this design assigns `worker` to every command a reaction issues, which is the namespace Sc L1-7 refuses to a client and the one D6 lists among the receipted callers, and reserves `system` for the platform's own maintenance chains and operator operations that run a receipted command, such as a restore or a migration driver, so a `system` receipt is never a reaction's; the owner confirms both (E-6, E-54, D6, D11, D13, Sc L1-7) +- [non-blocking #commandQueryTenantScope] Extension E-6: the tenant scope of an operation, of a command and of a query is the `tenantId` argument alone, so none takes a separate scope argument; this is a provisional reading the owner rules (E-6, Law 11, D11) +- [non-blocking #parentQueryAuthorization] Extension E-6: the doc says a query authorizes before it discloses and does not say how; here a parent query calls `authorizeQuery`, which establishes and authorizes its actor through the same two helpers a command's entry uses and is generic over the parent's data model; the owner has not ruled (E-6, D11, Law 5) +- [non-blocking #statedOperatorEntry] Extension E-48, the operator entry: the doc lists operators among the actors and names operator duties without saying how an operator reaches them; the option taken here is that admin access is the authority of an operation over the whole deployment, so every operator entry is an internal function that only admin access reaches, takes the operator as a required string, reads no grant and records that string on what it changes; a stated operator is what admin access chose to state and proves no identity, D11's one trusted deployment is the boundary, and no scope beyond the tenant is added (E-48, D11, D19) +- [non-blocking #tenantListContract] Extension E-49, the tenant list: the doc has every record name its tenant and names no list of tenants, while a batch over the whole deployment needs the next tenant; the option taken here is a `tenants` table in the parent with one row per tenant, written by `insertGrant` in the mutation that inserts a tenant's first grant and never deleted, read one tenant at a time in tenant-ID order; the do-nothing option, reading the distinct tenant IDs off the `grants` index, is not taken because revocation deletes grant rows and a tenant whose grants are all revoked still owns records (E-49, D11, Law 11, E-37) ## Contract @@ -59,8 +59,8 @@ This contract pins the shapes the parent establishes and passes into components: - typeActorKind: `type ActorKind = "human" | "service" | "agent" | "reviewer" | "operator"` (D11) - typeActorRef: `type ActorRef = { kind: ActorKind; id: string }` (D11, E-6) - typeActor: `type Actor = { kind: ActorKind; id: string; issuer?: string; onBehalfOf?: ActorRef; delegationRef?: string }` where `delegationRef` names the obligation, approval or agent run the delegation rests on (D11, D2, E-6) -- validatorActorKind: `actorKindValidator = v.union(v.literal("human"), v.literal("service"), v.literal("agent"), v.literal("reviewer"), v.literal("operator"))` (D11) -- validatorActor: `actorValidator = v.object({ kind: actorKindValidator, id: v.string(), issuer: v.optional(v.string()), onBehalfOf: v.optional(v.object({ kind: actorKindValidator, id: v.string() })), delegationRef: v.optional(v.string()) })` (D11, D2, E-6) +- validatorActorKind: `const actorKindValidator = v.union(v.literal("human"), v.literal("service"), v.literal("agent"), v.literal("reviewer"), v.literal("operator"))` (D11) +- validatorActor: `const actorValidator = v.object({ kind: actorKindValidator, id: v.string(), issuer: v.optional(v.string()), onBehalfOf: v.optional(v.object({ kind: actorKindValidator, id: v.string() })), delegationRef: v.optional(v.string()) })` (D11, D2, E-6) - typeTenantScope: `type TenantScope = { tenantId: string }` (Law 11) - validatorTenantScope: `tenantScopeValidator = v.object({ tenantId: v.string() })` (Law 11) - typeCallerNamespace: `type CallerNamespace = "public" | "service" | "worker" | "agent" | "system"` (D6, D11, D13, E-6) diff --git a/design/specs/command/command-declaration.sdp.md b/design/specs/command/command-declaration.sdp.md index 3f19d1f..5c3d87f 100644 --- a/design/specs/command/command-declaration.sdp.md +++ b/design/specs/command/command-declaration.sdp.md @@ -9,6 +9,10 @@ relations: - spec:command.command-pipeline - spec:command.outcome-boundary - spec:kernel.outcome-model + - spec:application.projection-contract + - spec:application.history-projection + - spec:context.persistence-adapter + - spec:operations.baseline-operations decidedBy: spec:decisions.d12-one-declaration-per-command constrainedBy: - spec:facts.f13-transactions-have-limits @@ -31,9 +35,9 @@ The canonical schema is the declaration's `input` validator plus an optional ref ### Open questions -- [non-blocking] Extension E-7: the doc says a small composition helper and shared schema values; the helper signatures, the executor signature and the read-model binding below are the design's; the cost is two registered mutations per command instead of one, and the cheaper alternative the design refuted is one public export per command plus one generic internal dispatcher mutation keyed by command type, as D7's conditional dispatcher already does; it lost because trusted server callers would then reach every command through one untyped `internalMutation`, so a renamed handler or a broken argument contract would pass `tsc` at the caller and fail only at runtime, which is the check Sc L2-4 exists to keep before traffic, and because the receipt fingerprint's contract version would be looked up by string rather than typed; the owner may still prefer the dispatcher form once code generation makes the per-command internal entry free (E-7, D12, D7, Sc L2-4, Decision method rule 5) -- [non-blocking] Extension E-7: the pipeline receives the declaration and takes the fingerprint's contract version from it, and the type check at a typed caller is the check of Sc L2-4, so no standing map from command type to declaration exists; the conditional dispatcher of `spec:command.outcome-boundary` is the one thing that would need such a map and brings it with it; this is a provisional reading the owner rules (E-7, D12, D18, OQ2) -- [non-blocking] Extension E-38: the doc names five declaration fields; this design adds `contractVersion` for the fingerprint, `retention` and `irreversible` for receipts, `admission`, `bounds`, `rejections`, `audit`, `diagnosticSink` and `writes`, each serving a cited rule of another Spec, and asks whether the owner accepts them as part of the one declaration; `bounds` is the one input bound in the corpus, `maxItems` and `maxBytes`, checked by the pipeline's step 1 and by no executor body, so `spec:application.parent-use-cases` reads it rather than declaring a bound of its own; `writes` is required on every declaration, because an absent list would be an unchecked one, and `readModels` is optional, absent meaning none, which the owner has not ruled; `writes` names the context and stream type pairs the executor may write, from which the pipeline's step 7 derives the source scopes the write pause closes for a consistent cut and which step 9 proves against the `streams` entries the executor returns, because a use case that writes a source stream without maintaining the paused view would otherwise pass the gate; the declaration still imports nothing from the obligation family, so a command's static reactions are not a field here but the `createObligation` calls of its executor, as `spec:obligations.fan-out-and-chains` rules under E-54 (E-38, E-8, E-54, D6, D9, D10, D19, Thesis) +- [non-blocking #commandEntrySignatures] Extension E-7: the doc says a small composition helper and shared schema values; the helper signatures, the executor signature and the read-model binding below are the design's; the cost is two registered mutations per command instead of one, and the cheaper alternative the design refuted is one public export per command plus one generic internal dispatcher mutation keyed by command type, as D7's conditional dispatcher already does; it lost because trusted server callers would then reach every command through one untyped `internalMutation`, so a renamed handler or a broken argument contract would pass `tsc` at the caller and fail only at runtime, which is the check Sc L2-4 exists to keep before traffic, and because the receipt fingerprint's contract version would be looked up by string rather than typed; the owner may still prefer the dispatcher form once code generation makes the per-command internal entry free (E-7, D12, D7, Sc L2-4, Decision method rule 5) +- [non-blocking #commandDeclarationLookup] Extension E-7: the pipeline receives the declaration and takes the fingerprint's contract version from it, and the type check at a typed caller is the check of Sc L2-4, so no standing map from command type to declaration exists; the conditional dispatcher of `spec:command.outcome-boundary` is the one thing that would need such a map and brings it with it; this is a provisional reading the owner rules (E-7, D12, D18, OQ2) +- [non-blocking #commandDeclarationFields] Extension E-38: the doc names five declaration fields; this design adds `contractVersion` for the fingerprint, `retention` and `irreversible` for receipts, `admission`, `bounds`, `rejections`, `audit`, `diagnosticSink` and `writes`, each serving a cited rule of another Spec, and asks whether the owner accepts them as part of the one declaration; `bounds` is the one input bound in the corpus, `maxItems` and `maxBytes`, checked by the pipeline's step 1 and by no executor body, so `spec:application.parent-use-cases` reads it rather than declaring a bound of its own; `writes` is required on every declaration, because an absent list would be an unchecked one, and `readModels` is optional, absent meaning none, which the owner has not ruled; `writes` names the context and stream type pairs the executor may write, from which the pipeline's step 7 derives the source scopes the write pause closes for a consistent cut and which step 9 proves against the `streams` entries the executor returns, because a use case that writes a source stream without maintaining the paused view would otherwise pass the gate; the declaration still imports nothing from the obligation family, so a command's static reactions are not a field here but the `createObligation` calls of its executor, as `spec:obligations.fan-out-and-chains` rules under E-54 (E-38, E-8, E-54, D6, D9, D10, D19, Thesis) ## Contract @@ -63,7 +67,7 @@ What generation would emit, once two modules show the pattern: for each declarat - typePermissionPolicy: `type PermissionPolicy = { permission: string; subjectFrom?: (input: I) => SubjectRef }` (D11, D12) - typeExecutor: `type Executor = (ctx: MutationCtx, call: { tenantId: string; actor: Actor; operation: OperationRef; input: I }) => Promise>` where `OperationRef` is the one the pipeline minted at step 7, forwarded unchanged to every context call, and the executor's context calls are `ctx.runMutation` on component APIs with the facts it captured (D12, D10, D4, D3, E-26) - typeExecutorResult: `type ExecutorResult = CommittedOutcome & { streams: StreamDto[] }` where `CommittedOutcome` is the kernel's committed half of `Outcome` and `streams` concatenates the `streams` of every context outcome the executor received, the `StreamDto` of `spec:context.persistence-adapter`, so the pipeline's step 9 has every DTO, version and created flag it needs (D12, D4, D8, E-7, E-23) -- typeReadModelBinding: `type ReadModelBinding = { readModel: ReadModel | AggregateProjection | HistoryProjection; source: SourceRef }`, where `readModel` is a per-entity read model, an aggregate projection or a history projection as `spec:application.projection-contract` pins them and `source` selects which `streams` entries feed it; the binding repeats neither the read model's name nor its projection's version, a history projection is bound once per source stream type it folds while a per-entity read model or an aggregate projection is bound to its one source stream type, and the binding has no update function of its own (D8, D12, E-7, E-9) +- typeReadModelBinding: `type ReadModelBinding = { readModel: ReadModel | HistoryProjection; source: SourceRef }`, where `readModel` is a per-entity read model or a history projection as `spec:application.projection-contract` and `spec:application.history-projection` pin them and `source` selects which `streams` entries feed it; the binding repeats neither the read model's name nor its projection's version, a history projection is bound once per source stream type it folds while a per-entity read model is bound to its one source stream type, and the binding has no update function of its own (D8, D12, E-7, E-9) - typeAudit: `audit?: { kind: "security" | "business" }` asks the pipeline's step 10 to write one audit record through `writeAudit` of `spec:operations.baseline-operations`, which fails closed, after an applied command and after a business failure; absent, the default, the command writes none (D19, E-38, E-43) - typeDiagnosticSinkField: `diagnosticSink?: DiagnosticSink`, the sink type of `spec:operations.baseline-operations`, names where the pipeline writes the command's diagnostic lines; absent, the default, it is `consoleSink`; the production composition names none, and the fixture composition's `PlaceOrder` names the sink through which Sc ALL-1 breaks metrics and logging (D19, E-38, E-43) - typeAdmissionPolicy: `type AdmissionPolicy = (ctx: MutationCtx, call: PipelineCall) => Promise<{ admitted: true } | { admitted: false; code: "rateLimited" | "capacity"; retryAfterMs?: number }>`; a consumption it writes commits only with the command, so it counts committed intent (D6, E-32) diff --git a/design/specs/command/command-pipeline.sdp.md b/design/specs/command/command-pipeline.sdp.md index e3bf15b..4a2a22e 100644 --- a/design/specs/command/command-pipeline.sdp.md +++ b/design/specs/command/command-pipeline.sdp.md @@ -10,6 +10,15 @@ relations: - spec:command.idempotency-and-receipts - spec:command.tenancy-and-authority - spec:command.outcome-boundary + - spec:command.actor-and-scope + - spec:application.projection-contract + - spec:application.history-projection + - spec:application.write-pause + - spec:command.receipt-table + - spec:context.persistence-adapter + - spec:context.event-envelope + - spec:kernel.outcome-model + - spec:operations.baseline-operations constrainedBy: - spec:laws.law01-sanctioned-writes-only - spec:laws.law02-state-and-events-commit-together @@ -56,13 +65,13 @@ The pipeline is one code path with two registered entries per command. The publi ### Open questions -- [non-blocking] Extension E-30: the doc names the outcome, the operation ID and the affected stream versions as what a command returns, but not a wire shape; the shape here is a union discriminated by `replayed`, whose executed member carries the declaration's result and whose replayed member carries `result` null because a receipt stores no result, so the entry's returns validator admits a replay as a value; affected IDs are application IDs, never private document IDs (E-30, D2, D6, D8) -- [non-blocking] Extension E-31: the doc says an operation's ID ties together everything one request caused but not how it is minted, and it says a derived command carries causation and an event's `causedBy` may be a triggering event without saying how that cause reaches the pipeline; here the pipeline mints the operation ID once per top-level mutation with `crypto.randomUUID()` before the first context call, a duplicate returns the operation ID stored on the receipt, and the internal entry takes an optional `causedBy` argument that step 7 copies onto the `OperationRef` when present, so a reaction that issues a derived command passes `causedBy` of kind `event` naming its publication event and the envelope's `event` variant has exactly one producer, while an entry called without it mints `causedBy` of kind `command` from the declaration's name; the obligation and attempt behind a derived command are found through its receipt, whose request key is the obligation's effect key, and never through the envelope (E-31, E-54, D2, D13, Vocab: operation) -- [non-blocking] Extension E-32: the doc places rate admission for new intent after the duplicate check and makes a rate or capacity refusal a retryable error, but names no admission mechanism; here admission is an optional per-command policy in the declaration, absent by default, evaluated inside the mutation so that a refusal that throws leaves no consumption behind; the flip side is that a consumption the policy writes commits only with the command, so the policy counts committed intent and cannot bound refused or unauthorized traffic, and a product that must bound refused traffic needs the dispatcher form of D7, where the parent survives the body's rejection (E-32, D4, D6, D7) -- [non-blocking] Extension E-33: the doc lists the callers that need receipts and leaves UI commands to the client guarantee; here a public caller may still supply a request key to opt into a receipt, which Probe 1 showed a caller of `ConvexHttpClient` needs when it retries, and the internal entry always requires one (E-33, D6, Probe 1) -- [non-blocking] Probe 4 showed on the pinned backend that the transaction limits are one budget across the parent and its component sub-transactions, so the per-command bounds are checked against one budget; `ctx.runMutation` accepts a `transactionLimits` option, the design passes none on a context call, and whether a call should reserve room with it for the work after it stays open (Probe 4, F13, S3) -- [non-blocking] Extension E-9: the doc says essential read models are updated inside the command and does not say from what or in what order; here step 9 writes a per-entity read model from the DTO of each matching `streams` entry, never from its events, and a history read model from the events of every matching entry in one call per command, checks `writes` first, then that each bound read model has a generation to write, then the count of the rows it writes against `limitReadModelWritesPerCommand`, each failure a plain error, and the library helpers that step 9 calls are generic over the application's data model; the owner has not ruled on any of it (E-9, E-7, E-8, E-45, D8, D9) -- [non-blocking] Extension E-34: step 1 bounds every text field a caller sets beside the input, the tenant ID, the request key, the correlation ID, the actor's fields and the fields of `causedBy`, in UTF-8 bytes as `limitCallText` lists them, as rejection `invalidInput`, and treats a declaration name above `limitIdLength` as a defect, a plain error, because these fields become fields of the receipt row or of every event's envelope; the doc says events stay small and names no number; the owner has not ruled (E-34, D6, D12, D19) +- [non-blocking #commandOutcomeWireShape] Extension E-30: the doc names the outcome, the operation ID and the affected stream versions as what a command returns, but not a wire shape; the shape here is a union discriminated by `replayed`, whose executed member carries the declaration's result and whose replayed member carries `result` null because a receipt stores no result, so the entry's returns validator admits a replay as a value; affected IDs are application IDs, never private document IDs (E-30, D2, D6, D8) +- [non-blocking #operationIdentityCause] Extension E-31: the doc says an operation's ID ties together everything one request caused but not how it is minted, and it says a derived command carries causation and an event's `causedBy` may be a triggering event without saying how that cause reaches the pipeline; here the pipeline mints the operation ID once per top-level mutation with `crypto.randomUUID()` before the first context call, a duplicate returns the operation ID stored on the receipt, and the internal entry takes an optional `causedBy` argument that step 7 copies onto the `OperationRef` when present, so a reaction that issues a derived command passes `causedBy` of kind `event` naming its publication event and the envelope's `event` variant has exactly one producer, while an entry called without it mints `causedBy` of kind `command` from the declaration's name; the obligation and attempt behind a derived command are found through its receipt, whose request key is the obligation's effect key, and never through the envelope (E-31, E-54, D2, D13, Vocab: operation) +- [non-blocking #commandAdmissionPolicy] Extension E-32: the doc places rate admission for new intent after the duplicate check and makes a rate or capacity refusal a retryable error, but names no admission mechanism; here admission is an optional per-command policy in the declaration, absent by default, evaluated inside the mutation so that a refusal that throws leaves no consumption behind; the flip side is that a consumption the policy writes commits only with the command, so the policy counts committed intent and cannot bound refused or unauthorized traffic, and a product that must bound refused traffic needs the dispatcher form of D7, where the parent survives the body's rejection (E-32, D4, D6, D7) +- [non-blocking #publicRequestKey] Extension E-33: the doc lists the callers that need receipts and leaves UI commands to the client guarantee; here a public caller may still supply a request key to opt into a receipt, which Probe 1 showed a caller of `ConvexHttpClient` needs when it retries, and the internal entry always requires one (E-33, D6, Probe 1) +- [non-blocking #contextCallBudgetReservation] Probe 4 showed on the pinned backend that the transaction limits are one budget across the parent and its component sub-transactions, so the per-command bounds are checked against one budget; `ctx.runMutation` accepts a `transactionLimits` option, the design passes none on a context call, and whether a call should reserve room with it for the work after it stays open (Probe 4, F13, S3) +- [non-blocking #commandReadModelWrites] Extension E-9: the doc says essential read models are updated inside the command and does not say from what or in what order; here step 9 writes a per-entity read model from the DTO of each matching `streams` entry, never from its events, and a history read model from the events of every matching entry in one call per command, checks `writes` first, then that each bound read model has a generation to write, then the count of the rows it writes against `limitReadModelWritesPerCommand`, each failure a plain error, and the library helpers that step 9 calls are generic over the application's data model; the owner has not ruled on any of it (E-9, E-7, E-8, E-45, D8, D9) +- [non-blocking #commandTextBounds] Extension E-34: step 1 bounds every text field a caller sets beside the input, the tenant ID, the request key, the correlation ID, the actor's fields and the fields of `causedBy`, in UTF-8 bytes as `limitCallText` lists them, as rejection `invalidInput`, and treats a declaration name above `limitIdLength` as a defect, a plain error, because these fields become fields of the receipt row or of every event's envelope; the doc says events stay small and names no number; the owner has not ruled (E-34, D6, D12, D19) ## Workflow @@ -86,7 +95,7 @@ The pipeline is one code path with two registered entries per command. The publi - Step 6, admit: for new intent only, the pipeline evaluates the declaration's admission policy; a refusal throws transient `rateLimited` or `capacity`, nothing is stored, and the same intent may be retried (D6, D4, Sc L1-9) - Step 7, gate and mint the operation: the pipeline reads the maintenance gate once through `assertWritable(ctx, scopesOfUseCase(tenantId, decl.writes))` of `spec:application.write-pause`, which refuses with the transient `writePaused` while a scope of this command is closed, the whole deployment, the tenant or a stream type it writes, or the deployment is in restore, and only then mints the whole `OperationRef`, the operation ID, `causedBy` from the call's `causedBy` argument when the internal entry received one and otherwise of kind `command` with the declaration's name, and the caller's optional correlation ID, and passes it to every context call so that events in different streams relate through it (Vocab: operation, D2, D9, D13, E-8, E-26, E-31) - Step 8, execute: the executor captures the outside facts the decision needs, from parent tables or configuration read in this transaction, and calls each context named by the declaration once through `ctx.runMutation` on the component's sanctioned operation, `components..operations.`, with tenant scope, actor, the operation reference, the command's input as lists and the captured facts; each call is an isolated sub-transaction that commits with the parent and rolls back on throw; a rejection thrown inside a context rolls its sub-transaction back and propagates to the boundary's outermost rethrow, and the executor returns the kernel's `CommittedOutcome` together with the `streams` entries every context call returned, never a rejection value (F2, F3, D3, D10, D7, E-23) -- Step 9, update read models: the pipeline first checks that every `streams` entry the executor returned names a context and stream type in the declaration's `writes`, throwing a plain error, a defect and a technical failure, when one does not, so the list the gate of step 7 was derived from is proven on the first native run; then, once for each read model its bindings name, however many bindings name it, it reads the generations to write through `generationsToWrite` and throws a plain error that names the command and the read model when there is none, and, for every binding of a per-entity read model, calls `applyProjection` of `spec:application.projection-contract`, or `applyAggregate` for a binding of that form, for each matching entry with the entry's DTO and version and the list unchanged, in the live-created mode when the entry's `created` flag is set and the live-updated mode otherwise, and, once for each bound history read model, calls `applyHistoryProjection` with the events of every entry any of its bindings matches, in the order of the entries, with no version, the live-updated mode and the list unchanged, in the same transaction; it counts the rows those calls insert, replace or delete, a row in each generation once, a history row once however many entries fed it, and after each call throws a plain error when they pass `limitReadModelWritesPerCommand`, and the count, history rows included, is what the diagnostic record carries as `readModelRows`; the executor writes no read-model table itself (D8, D9, D12, Law 9, E-7, E-8, E-9, E-23, E-45) +- Step 9, update read models: the pipeline first checks that every `streams` entry the executor returned names a context and stream type in the declaration's `writes`, throwing a plain error, a defect and a technical failure, when one does not, so the list the gate of step 7 was derived from is proven on the first native run; then, once for each read model its bindings name, however many bindings name it, it reads the generations to write through `generationsToWrite` and throws a plain error that names the command and the read model when there is none, and, for every binding of a per-entity read model, calls `applyProjection` of `spec:application.projection-contract` for each matching entry with the entry's DTO and version and the list unchanged, in the live-created mode when the entry's `created` flag is set and the live-updated mode otherwise, and, once for each bound history read model, calls `applyHistoryProjection` with the events of every entry any of its bindings matches, in the order of the entries, with no version, the live-updated mode and the list unchanged, in the same transaction; it counts the rows those calls insert, replace or delete, a row in each generation once, a history row once however many entries fed it, and after each call throws a plain error when they pass `limitReadModelWritesPerCommand`, and the count, history rows included, is what the diagnostic record carries as `readModelRows`; the executor writes no read-model table itself (D8, D9, D12, Law 9, E-7, E-8, E-9, E-23, E-45) - Step 10, insert the receipt and the audit record: for a receipted call the pipeline inserts the thin receipt, outcome kind, operation ID, affected IDs derived from the stream versions, and the stream versions, in the same transaction as the lookup of step 5, so two concurrent identical requests serialize under optimistic concurrency and only one effect commits; for a declaration that sets `audit` it then calls `writeAudit` of `spec:operations.baseline-operations` with the record `auditRecordSources` of that Spec builds, whose failure throws and rolls the command back, a refused insert being a plain error and so a technical failure with nothing of the command stored (D6, D19, F1, Sc L1-3, E-43) - Step 11, return: the mutation builds the diagnostic record of the outcome from values it already holds and passes it to `emitDiagnostic` with the declaration's sink, which swallows every error, then returns the outcome kind, the result DTO, the operation ID, the affected IDs, the stream versions and `replayed` false (D4, D8, D2, D19, E-43) - At any step, a throw leaves the mutation uncommitted: the state and journal writes of every context sub-transaction, the read-model writes and the receipt are all rolled back, and nothing is stored as a business outcome (Law 2, D7, F2, Sc L1-2) @@ -114,7 +123,7 @@ The pipeline depends on three contracts it does not own: the context component's - contextCall: `await ctx.runMutation(components.inventory.operations.allocate, { tenantId, actor, operation, input: { orderId, lines }, facts })`, one call per context named by the declaration, lists in and the operation outcome out, the DTO with the versions and one `streams` entry per planned stream with its DTO, version, created flag and the events the call appended; the `args` shape is the one `spec:context.context-component` fixes, `operation` is the `OperationRef` of `spec:context.event-envelope` minted at step 7, and the return is the `OperationOutcome` of `spec:context.persistence-adapter` (D10, F2, E-22, E-23, E-26) - contextCallIsolation: each component call is a sub-transaction isolated from the other calls; a throw inside it rolls that sub-transaction back, and uncaught it rolls the parent back, so the parent never observes a half-applied context (F2, F3, S2) - rejectionRelay: a `ConvexError` carrying the kernel's `Rejection` thrown inside a component reaches the pipeline with its data intact under F14, which Probe 2 showed on the pinned backend; `normalizeThrown(error, decl.name, decl.rejections)` at the handler's outermost level rethrows it as the boundary's `RejectionData` when its code is a platform code or one the declaration's `rejections` names, and as a plain error otherwise; the fallback for a refuted F14, contexts returning the rejection as a value, is not needed (F14, Probe 2, D7) -- readModelWrites: parent-owned read-model tables written by the pipeline at step 9, one `applyProjection` or `applyAggregate` call per matching `streams` entry per binding of those forms the declaration lists and one `applyHistoryProjection` call per bound history read model with the events of every entry its bindings match, with the list `generationsToWrite` returns, each generation with its role; the executor holds the DTOs and the appended events only long enough to return them and writes no read-model table, so the declaration's list is the one wiring fact for read models (D8, D9, D12, E-7, E-9) +- readModelWrites: parent-owned read-model tables written by the pipeline at step 9, one `applyProjection` call per matching `streams` entry per binding of a per-entity read model the declaration lists and one `applyHistoryProjection` call per bound history read model with the events of every entry its bindings match, with the list `generationsToWrite` returns, each generation with its role; the executor holds the DTOs and the appended events only long enough to return them and writes no read-model table, so the declaration's list is the one wiring fact for read models (D8, D9, D12, E-7, E-9) - writesCheck: `decl.writes` is proven at step 9 against the `streams` entries the executor returned, in memory and with no extra read; an entry outside the list throws a plain error, so a declaration whose `writes` understates its executor fails its first native run rather than passing a closed source scope at step 7 (E-8, E-38, Law 6) - fnWriteReadModels: `writeReadModels(ctx: RegistryReader & RowWriter, commandType: string, bindings: readonly ReadModelBinding[], input: { tenantId: string; streams: readonly StreamDto[] }): Promise`, the read-model half of step 9 and the one function of the read-model library the pipeline calls, returning the number of read-model rows it inserted, replaced or deleted, a row in each generation once and a history row once however many entries fed it, the count it holds against `limitReadModelWritesPerCommand`; the pipeline passes its own ctx, which at run time is the parent's and holds the registry and the read model's table (D8, E-7, E-9) - helperGenerics: [extension] every library helper that takes the parent's own ctx or reads one of the parent's tables, `writeReadModels`, `activeGeneration` and `authorizeQuery` among them, is generic over the parent's data model, because a database reader is not covariant in its data model under the repository's compiler settings, so a helper typed with the library's tables alone refuses the parent's ctx (E-7, E-9, D11) diff --git a/design/specs/command/idempotency-and-receipts.sdp.md b/design/specs/command/idempotency-and-receipts.sdp.md index c970e03..1fbf367 100644 --- a/design/specs/command/idempotency-and-receipts.sdp.md +++ b/design/specs/command/idempotency-and-receipts.sdp.md @@ -5,7 +5,10 @@ altitude: feature readiness: defined relations: refines: spec:command.command-pipeline - dependsOn: spec:command.tenancy-and-authority + dependsOn: + - spec:application.write-pause + - spec:command.tenancy-and-authority + - spec:context.queries constrainedBy: - spec:laws.law04-server-scoped-idempotency-key - spec:laws.law05-authorization-before-execution-and-disclosure @@ -39,11 +42,11 @@ This Spec owns the rules: where receipts are required, how the key and the finge ### Open questions -- [non-blocking] Probe 1 showed on the pinned backend that a closed client leaves its mutation executed once or not at all, that a backend restart leaves it executed once, and that `ConvexHttpClient` does not retry, so a caller's retry is a second execution; whether the public entry requires a request key from callers of the HTTP client, or leaves it optional, is the owner's ruling; the design takes it optional on the public entry and required on the internal entry, a provisional reading the owner rules (Probe 1, D6, E-33) -- [non-blocking] Extension E-33: a public caller may supply a request key to opt into a receipt, and the internal entry always requires one; the doc lists the callers that need receipts and does not say whether a UI caller may opt in (E-33, D6) -- [non-blocking] Extension E-34: the doc says expiry is explicit and names no number; here the default retry window is 7 days per command, expiry is checked lazily at lookup so the core path needs no job, and a bounded per-tenant sweep deletes expired receipts in batches of at most 1,000; the tenant ID and the request key are bounded to 256 bytes of UTF-8, a declaration name above that bound is a defect and a plain error, and a receipt is bounded to 1,000 affected refs; the owner has not ruled (E-34, D6, D19, F13) -- [non-blocking] Extension E-36: the doc says the fingerprint covers business input and its contract version; here it is the SHA-256 digest, computed with the Convex runtime's `crypto.subtle.digest`, over the canonical JSON of the business input with object keys sorted, followed by the contract version; the outside facts the executor captures travel beside the input and are excluded, so a retry that captures a fresher fact is not a conflict, which the decider contract cites; a change to the canonical form is a contract version bump (E-36, D6, D3) -- [non-blocking] Extension E-39: the doc says a client-generated entity ID plus a uniqueness check gives one entity on a UI double submit and does not say what the second submit is told; here the second submit is answered with the rejection `entityExists` naming the existing entity, not with an idempotent success, because no receipt exists on the UI path to replay an outcome from and the client already holds the entity ID it needs to read the result; the owner may prefer an idempotent success, which would need a receipt on every create (E-39, D6, Sc L1-4) +- [non-blocking #httpCallerRequestKey] Probe 1 showed on the pinned backend that a closed client leaves its mutation executed once or not at all, that a backend restart leaves it executed once, and that `ConvexHttpClient` does not retry, so a caller's retry is a second execution; whether the public entry requires a request key from callers of the HTTP client, or leaves it optional, is the owner's ruling; the design takes it optional on the public entry and required on the internal entry, a provisional reading the owner rules (Probe 1, D6, E-33) +- [non-blocking #publicReceiptChoice] Extension E-33: a public caller may supply a request key to opt into a receipt, and the internal entry always requires one; the doc lists the callers that need receipts and does not say whether a UI caller may opt in (E-33, D6) +- [non-blocking #receiptRetentionBounds] Extension E-34: the doc says expiry is explicit and names no number; here the default retry window is 7 days per command, expiry is checked lazily at lookup so the core path needs no job, and a bounded per-tenant sweep deletes expired receipts in batches of at most 1,000; the tenant ID and the request key are bounded to 256 bytes of UTF-8, a declaration name above that bound is a defect and a plain error, and a receipt is bounded to 1,000 affected refs; the owner has not ruled (E-34, D6, D19, F13) +- [non-blocking #businessInputFingerprint] Extension E-36: the doc says the fingerprint covers business input and its contract version; here it is the SHA-256 digest, computed with the Convex runtime's `crypto.subtle.digest`, over the canonical JSON of the business input with object keys sorted, followed by the contract version; the outside facts the executor captures travel beside the input and are excluded, so a retry that captures a fresher fact is not a conflict, which the decider contract cites; a change to the canonical form is a contract version bump (E-36, D6, D3) +- [non-blocking #duplicateSubjectCreation] Extension E-39: the doc says a client-generated entity ID plus a uniqueness check gives one entity on a UI double submit and does not say what the second submit is told; here the second submit is answered with the rejection `entityExists` naming the existing entity, not with an idempotent success, because no receipt exists on the UI path to replay an outcome from and the client already holds the entity ID it needs to read the result; the owner may prefer an idempotent success, which would need a receipt on every create (E-39, D6, Sc L1-4) ## Behavior diff --git a/design/specs/command/outcome-boundary.sdp.md b/design/specs/command/outcome-boundary.sdp.md index 5ee3061..0d428b7 100644 --- a/design/specs/command/outcome-boundary.sdp.md +++ b/design/specs/command/outcome-boundary.sdp.md @@ -35,10 +35,10 @@ The closed code list has two halves. The platform codes below are closed here an ### Open questions -- [non-blocking] Extension E-5: the doc fixes the four outcomes and that a rejection is a structured `ConvexError`, but not the data shape or the code list; the shape and the platform codes below are the design's, and the domain codes are closed per declaration (E-5, D4, D7) -- [non-blocking] Extension E-5: the doc says a query authorizes before it discloses and does not say how a refused query is answered; here a parent query's refusal is the same `RejectionData` a command's is, with the name of the entry that refused in the field `entry` and only the codes `unauthenticated` and `forbidden`, and a use case's own rejection is thrown as the kernel's bare `Rejection`, the way a context throws one, while a context query's transient refusal `writePaused` reaches the caller unchanged, in the transient shape with no `entry`, because it is not a refusal of the caller; the owner confirmed the shared shape, the two codes, the bare use-case rejection and the word entry for the field (E-5, D7, Law 5) -- [non-blocking] Extension E-35: the write pause of D9 refuses writers in its scope while a cross-stream history view rebuilds; the doc does not say how a refused writer is answered; this contract reserves the transient code `writePaused` for Package D so the refusal is retryable and stores nothing, which Package D confirms or replaces (E-35, D9, D4) -- [non-blocking] OQ2: whether the product needs a record of refused commands anywhere, for security audit or agent proposals, decides whether the generic internal dispatcher below exists; until then it is a conditional design and no command uses it (OQ2, D7) +- [non-blocking #rejectionShapeCodes] Extension E-5: the doc fixes the four outcomes and that a rejection is a structured `ConvexError`, but not the data shape or the code list; the shape and the platform codes below are the design's, and the domain codes are closed per declaration (E-5, D4, D7) +- [non-blocking #parentQueryRefusalShape] Extension E-5: the doc says a query authorizes before it discloses and does not say how a refused query is answered; here a parent query's refusal is the same `RejectionData` a command's is, with the name of the entry that refused in the field `entry` and only the codes `unauthenticated` and `forbidden`, and a use case's own rejection is thrown as the kernel's bare `Rejection`, the way a context throws one, while a context query's transient refusal `writePaused` reaches the caller unchanged, in the transient shape with no `entry`, because it is not a refusal of the caller; the owner confirmed the shared shape, the two codes, the bare use-case rejection and the word entry for the field (E-5, D7, Law 5) +- [non-blocking #writePauseTransientRefusal] Extension E-35: the write pause of D9 refuses writers in its scope while a cross-stream history view rebuilds; the doc does not say how a refused writer is answered; this contract reserves the transient code `writePaused` for Package D so the refusal is retryable and stores nothing, which Package D confirms or replaces (E-35, D9, D4) +- [non-blocking #refusedCommandRecord] OQ2: whether the product needs a record of refused commands anywhere, for security audit or agent proposals, decides whether the generic internal dispatcher below exists; until then it is a conditional design and no command uses it (OQ2, D7) ## Contract diff --git a/design/specs/command/receipt-table.sdp.md b/design/specs/command/receipt-table.sdp.md index 4f65bd1..1e1d678 100644 --- a/design/specs/command/receipt-table.sdp.md +++ b/design/specs/command/receipt-table.sdp.md @@ -5,7 +5,11 @@ altitude: story readiness: defined relations: refines: spec:command.idempotency-and-receipts - dependsOn: spec:command.actor-and-scope + dependsOn: + - spec:application.write-pause + - spec:command.actor-and-scope + - spec:context.queries + - spec:kernel.outcome-model constrainedBy: - spec:facts.f13-transactions-have-limits - spec:laws.law11-tenant-scope-named @@ -25,7 +29,7 @@ The `receipts` table lives in the parent, as the doc's ownership table assigns i ### Open questions -- [non-blocking] Extension E-4: the doc fixes what a receipt holds and that expiry is explicit; the concrete fields, the three indexes, the tombstone shape and the defaults are the design's, and the expiry number is a product decision the declaration overrides (E-4, D6) +- [non-blocking #receiptFieldsIndexes] Extension E-4: the doc fixes what a receipt holds and that expiry is explicit; the concrete fields, the three indexes, the tombstone shape and the defaults are the design's, and the expiry number is a product decision the declaration overrides (E-4, D6) ## Contract diff --git a/design/specs/command/tenancy-and-authority.sdp.md b/design/specs/command/tenancy-and-authority.sdp.md index a6aade7..757ec1b 100644 --- a/design/specs/command/tenancy-and-authority.sdp.md +++ b/design/specs/command/tenancy-and-authority.sdp.md @@ -14,6 +14,7 @@ relations: decidedBy: - spec:decisions.d11-tenant-scope-and-authority - spec:decisions.d06-idempotency-client-and-receipts + dependsOn: spec:application.write-pause --- # Tenancy and authority @@ -34,9 +35,9 @@ This Spec owns the rules of tenant scope, actor establishment, authorization and ### Open questions -- [non-blocking] Extension E-37: the doc says the parent authenticates and that grants are authoritative data, but not how an identity becomes an actor or how revocation takes effect; here the public entry and a parent query's `authorizeQuery` map `ctx.auth.getUserIdentity()` to a human actor, or a service actor when the issuer is a configured service issuer, grants live in a parent table by default and are revoked by deleting the row, with no cache, so a revoked grant fails the next command (E-37, D11) -- [non-blocking] Extension E-6: the doc names the actor kinds and the worker's two modes but not the types; the actor, scope, namespace, grant and `Authority` shapes are the design's and are pinned once in `spec:command.actor-and-scope`, which also rules that a derived command's namespace is `worker` (E-6, D11) -- [non-blocking] Extension E-37: the design takes grants as written by plain library helpers, `insertGrant` and `revokeGrant`, over which the application registers its own internal mutations, with a tenant's first grant created with admin access through the composition's own internal grant mutation, and later grants the same way or by whatever command the composition registers; the library registers no grant function and promises no recorded bootstrap, first-admin flow or invitation, which are an adopter's to build on the helpers (E-37, D11, Law 5) +- [non-blocking #actorGrantRevocation] Extension E-37: the doc says the parent authenticates and that grants are authoritative data, but not how an identity becomes an actor or how revocation takes effect; here the public entry and a parent query's `authorizeQuery` map `ctx.auth.getUserIdentity()` to a human actor, or a service actor when the issuer is a configured service issuer, grants live in a parent table by default and are revoked by deleting the row, with no cache, so a revoked grant fails the next command (E-37, D11) +- [non-blocking #actorCallerNamespaceShapes] Extension E-6: the doc names the actor kinds and the worker's two modes but not the types; the actor, scope, namespace, grant and `Authority` shapes are the design's and are pinned once in `spec:command.actor-and-scope`, which also rules that a derived command's namespace is `worker` (E-6, D11) +- [non-blocking #grantCreationEntries] Extension E-37: the design takes grants as written by plain library helpers, `insertGrant` and `revokeGrant`, over which the application registers its own internal mutations, with a tenant's first grant created with admin access through the composition's own internal grant mutation, and later grants the same way or by whatever command the composition registers; the library registers no grant function and promises no recorded bootstrap, first-admin flow or invitation, which are an adopter's to build on the helpers (E-37, D11, Law 5) ## Behavior diff --git a/design/specs/constraints/events-stay-small.sdp.md b/design/specs/constraints/events-stay-small.sdp.md index 38f91f5..2715565 100644 --- a/design/specs/constraints/events-stay-small.sdp.md +++ b/design/specs/constraints/events-stay-small.sdp.md @@ -23,7 +23,7 @@ The doc says events stay small and that personal data sits behind references whe ### Open questions -- [non-blocking] Extension E-12: the payload byte bound is the design's number, 16 KiB, chosen with the envelope bound of 4 KiB so that a fold of 500 events of at most 20 KiB each stays under 9.8 MiB, under the 16 MiB read ceiling with headroom; the owner may set another, and the journal's 500-event fold bound moves with it; the reserved `baseline` event of E-25 is exempt from the 16 KiB bound and bounded by its stream type's `budgetBytes` instead, at most 512 KiB, because its payload is the migrated state of its stream, which the fold reads once as its start and never folds, and the owner confirms the exemption with E-25 (E-12, E-2, E-25, D19, F13) +- [non-blocking #eventPayloadBound] Extension E-12: the payload byte bound is the design's number, 16 KiB, chosen with the envelope bound of 4 KiB so that a fold of 500 events of at most 20 KiB each stays under 9.8 MiB, under the 16 MiB read ceiling with headroom; the owner may set another, and the journal's 500-event fold bound moves with it; the reserved `baseline` event of E-25 is exempt from the 16 KiB bound and bounded by its stream type's `budgetBytes` instead, at most 512 KiB, because its payload is the migrated state of its stream, which the fold reads once as its start and never folds, and the owner confirms the exemption with E-25 (E-12, E-2, E-25, D19, F13) ## Constraints diff --git a/design/specs/context/context-component.sdp.md b/design/specs/context/context-component.sdp.md index 46dd4ea..d347937 100644 --- a/design/specs/context/context-component.sdp.md +++ b/design/specs/context/context-component.sdp.md @@ -8,6 +8,7 @@ relations: dependsOn: - spec:kernel.domain-kernel - spec:command.actor-and-scope + - spec:context.event-envelope decidedBy: - spec:decisions.d02-context-owns-state-and-journal - spec:decisions.d05-rebuildable-history-with-baselines @@ -23,6 +24,7 @@ relations: - spec:facts.f01-serializable-mutations-under-occ - spec:facts.f03-nested-run-mutation-partial-rollback - spec:facts.f13-transactions-have-limits + - spec:facts.f19-nested-calls-share-time-budgets --- # The context component @@ -46,9 +48,9 @@ The component's boundary is the component API. Below it sit the kernel, the jour ### Open questions -- [non-blocking] OQ1: a trivial context with no invariants of its own stays a component; the owner answered OQ1 on the first experiment's local measurement, with the hosted usage reading as the one measurement still to take, and a measured workload that misses an agreed target or a usage reading above an agreed share reopens it (OQ1, D2) -- [non-blocking] Probe 3: the cost of one component call in latency and function-call quota decides where hot reads land; the latency is an order of magnitude read on a local backend, the quota needs a hosted deployment to read, and isolation itself stays (Probe 3, F4, D2, D8) -- [non-blocking] Extension E-22: the doc says the parent calls the context's sanctioned operations and does not say how they are registered; this Spec registers one Convex mutation per sanctioned operation built by `defineOperation` with args `{ tenantId, actor, operation, input, facts }`, fixes the context ID as a code constant equal to the mount name, sets the per-call bounds in documents and in bytes derived from each stream type's budget, names the query and maintenance surface, and adds `convex-helpers` as the component's one dependency because the built-in `.paginate()` does not work in a component; the alternative of one generic `execute` mutation per context with an untyped command was not taken because D12 wants static exports with one canonical schema; the owner confirms (D10, D12, S4, E-22) +- [non-blocking #contextIsolationMeasurement] OQ1: a trivial context with no invariants of its own stays a component; the owner answered OQ1 on the first experiment's local measurement, with the hosted usage reading as the one measurement still to take, and a measured workload that misses an agreed target or a usage reading above an agreed share reopens it (OQ1, D2) +- [non-blocking #contextQueryLatencyQuota] Probe 3: the cost of one component call in latency and function-call quota decides where hot reads land; the latency is an order of magnitude read on a local backend, the quota needs a hosted deployment to read, and isolation itself stays (Probe 3, F4, D2, D8) +- [non-blocking #contextOperationRegistration] Extension E-22: the doc says the parent calls the context's sanctioned operations and does not say how they are registered; this Spec registers one Convex mutation per sanctioned operation built by `defineOperation` with args `{ tenantId, actor, operation, input, facts }`, fixes the context ID as a code constant equal to the mount name, sets the per-call bounds in documents and in bytes derived from each stream type's budget, names the query and maintenance surface, and adds `convex-helpers` as the component's one dependency because the built-in `.paginate()` does not work in a component; the alternative of one generic `execute` mutation per context with an untyped command was not taken because D12 wants static exports with one canonical schema; the owner confirms (D10, D12, S4, E-22) ## Behavior diff --git a/design/specs/context/event-envelope.sdp.md b/design/specs/context/event-envelope.sdp.md index 9a04760..f44fae7 100644 --- a/design/specs/context/event-envelope.sdp.md +++ b/design/specs/context/event-envelope.sdp.md @@ -6,14 +6,13 @@ readiness: defined relations: refines: spec:context.journal dependsOn: - - spec:context.persistence-adapter - spec:command.actor-and-scope - - spec:command.command-pipeline decidedBy: spec:decisions.d02-context-owns-state-and-journal constrainedBy: - spec:facts.f13-transactions-have-limits - spec:facts.f01-serializable-mutations-under-occ - spec:laws.law06-technical-failure-never-a-rejection + - spec:constraints.events-stay-small --- # The event envelope @@ -29,8 +28,8 @@ Every event carries the fifteen fields the doc fixes: a stable application ident ### Open questions -- [non-blocking] Extension E-26: the doc lists the fifteen fields and their meaning; this Spec fixes the validator shapes, the `CausedBy` union with a migration variant for baseline events, the `OperationRef` the parent passes, and the event ID as a version 4 UUID from the runtime's `crypto`; the owner confirms (D2, E-26) -- [non-blocking] Extension E-12: the doc says events stay small and gives no number for the envelope; this Spec bounds the envelope beside the payload at `limitEnvelopeBytes`, 4,096 bytes, and each text field of it that a caller can set at 256 bytes of UTF-8, 512 for an actor's ID, so the largest stored event, `limitEventBytes`, is derived and enforced and never assumed; the owner may set other numbers, and the fold bound of `spec:context.journal` moves with them (E-12, D19, F13) +- [non-blocking #eventEnvelopeFields] Extension E-26: the doc lists the fifteen fields and their meaning; this Spec fixes the validator shapes, the `CausedBy` union with a migration variant for baseline events, the `OperationRef` the parent passes, and the event ID as a version 4 UUID from the runtime's `crypto`; the owner confirms (D2, E-26) +- [non-blocking #envelopeTextBounds] Extension E-12: the doc says events stay small and gives no number for the envelope; this Spec bounds the envelope beside the payload at `limitEnvelopeBytes`, 4,096 bytes, and each text field of it that a caller can set at 256 bytes of UTF-8, 512 for an actor's ID, so the largest stored event, `limitEventBytes`, is derived and enforced and never assumed; the owner may set other numbers, and the fold bound of `spec:context.journal` moves with them (E-12, D19, F13) ## Contract diff --git a/design/specs/context/journal.sdp.md b/design/specs/context/journal.sdp.md index a7d0af9..0637d92 100644 --- a/design/specs/context/journal.sdp.md +++ b/design/specs/context/journal.sdp.md @@ -8,6 +8,7 @@ relations: decidedBy: - spec:decisions.d02-context-owns-state-and-journal - spec:decisions.d05-rebuildable-history-with-baselines + - spec:kernel.state-document-mapping constrainedBy: - spec:laws.law03-events-only-source-of-state - spec:laws.law10-replay-never-runs-commands-or-effects @@ -20,6 +21,10 @@ relations: - spec:facts.f02-component-calls-commit-with-caller - spec:facts.f08-scheduling-commits-with-mutation - spec:facts.f09-scheduled-mutation-and-action-retry-semantics + - spec:constraints.events-stay-small + dependsOn: + - spec:application.write-pause + - spec:context.tables --- # The journal @@ -43,7 +48,7 @@ The journal's boundary is the library API the persistence adapter and the querie ### Open questions -- [non-blocking] Extension E-25: the doc says a migration writes a baseline event holding the migrated state and that rebuild starts from the latest baseline, and it does not say who computes that state, who drives the migration, or what a live command does on a stream the migration has not reached; this Spec fixes the representation, an event of the reserved type `baseline` whose payload holds the state and its schema version and is bounded by the stream's `budgetBytes` instead of the payload bound, a `baselineVersion` on the stream row, a `BaselineMigration` registered on the stream type whose `migrate` function computes the migrated state from the saved state, or, for a step that declares `readsHistory`, from the saved state and the events since the last baseline in the sweep only, the rule that `stateSchemaVersion` counts meaning only and lives on the registration, one helper `migrateToSchemaVersion` that writes the baseline chain, called by the adapter on load for a chain that reads no history and by the `writeBaseline` maintenance mutation for every chain, the code-older-than-data error, the `writeBaseline` mutation built by `defineMaintenance` and declared with the component's public `mutation` builder, the parent-side scheduled chain that drives it by cursor, the batch bound derived from `budgetBytes`, the upcasting rule for old event schema versions, the fold bound of 500 events with the refusals `rebuildStream` returns above it and for a stream awaiting its baseline, the retirement rule for a migration, and the payload byte check in `append`; the option of refusing commands on unmigrated streams with a transient code until the sweep arrives was not taken for a state-only migration, because it turns a deploy into a partial write pause the doc reserves for history-dependent views, and is taken for a history-reading migration, the exception, where it holds only the streams the sweep has not reached and keeps D3 whole on the command path, which the owner confirmed; a consequence the owner should weigh is that every baseline bumps its stream's version, so a reviewed version held across the sweep is answered `staleVersion` and re-reviewed rather than silently accepted, because Sc L1-10 forbids reinterpreting a reviewed version against fresh state and the baseline is a real event of the stream, while a live stream migrated on load never trips it because its expected-version check runs before the chain; and the driver reads the write pause's gate before every batch and reschedules itself when refused, so a sweep never changes a source under a paused rebuild's cut; the owner confirms (D5, D9, S4, Sc L1-10, E-2, E-8, E-12, E-25) +- [non-blocking #baselineMigrationContract] Extension E-25: the doc says a migration writes a baseline event holding the migrated state and that rebuild starts from the latest baseline, and it does not say who computes that state, who drives the migration, or what a live command does on a stream the migration has not reached; this Spec fixes the representation, an event of the reserved type `baseline` whose payload holds the state and its schema version and is bounded by the stream's `budgetBytes` instead of the payload bound, a `baselineVersion` on the stream row, a `BaselineMigration` registered on the stream type whose `migrate` function computes the migrated state from the saved state, or, for a step that declares `readsHistory`, from the saved state and the events since the last baseline in the sweep only, the rule that `stateSchemaVersion` counts meaning only and lives on the registration, one helper `migrateToSchemaVersion` that writes the baseline chain, called by the adapter on load for a chain that reads no history and by the `writeBaseline` maintenance mutation for every chain, the code-older-than-data error, the `writeBaseline` mutation built by `defineMaintenance` and declared with the component's public `mutation` builder, the parent-side scheduled chain that drives it by cursor, the batch bound derived from `budgetBytes`, the upcasting rule for old event schema versions, the fold bound of 500 events with the refusals `rebuildStream` returns above it and for a stream awaiting its baseline, the retirement rule for a migration, and the payload byte check in `append`; the option of refusing commands on unmigrated streams with a transient code until the sweep arrives was not taken for a state-only migration, because it turns a deploy into a partial write pause the doc reserves for history-dependent views, and is taken for a history-reading migration, the exception, where it holds only the streams the sweep has not reached and keeps D3 whole on the command path, which the owner confirmed; a consequence the owner should weigh is that every baseline bumps its stream's version, so a reviewed version held across the sweep is answered `staleVersion` and re-reviewed rather than silently accepted, because Sc L1-10 forbids reinterpreting a reviewed version against fresh state and the baseline is a real event of the stream, while a live stream migrated on load never trips it because its expected-version check runs before the chain; and the driver reads the write pause's gate before every batch and reschedules itself when refused, so a sweep never changes a source under a paused rebuild's cut; the owner confirms (D5, D9, S4, Sc L1-10, E-2, E-8, E-12, E-25) ## Behavior diff --git a/design/specs/context/persistence-adapter.sdp.md b/design/specs/context/persistence-adapter.sdp.md index 6d636ad..7d9dc9c 100644 --- a/design/specs/context/persistence-adapter.sdp.md +++ b/design/specs/context/persistence-adapter.sdp.md @@ -9,6 +9,9 @@ relations: - spec:kernel.decider-contract - spec:kernel.state-document-mapping - spec:context.tables + - spec:context.journal + - spec:command.outcome-boundary + - spec:context.event-envelope decidedBy: - spec:decisions.d03-events-only-source-of-next-state - spec:decisions.d04-four-outcomes @@ -40,7 +43,7 @@ An operation is a list-shaped call over one or more streams of the stream types ### Open questions -- [non-blocking] Extension E-23: the doc gives the sequence and the one-authority rule; this Spec fixes the request and result types, the operation runner with `plan` and `combine`, the aggregation of per-stream outcomes, the per-stream `dto` and `created` flag that the operation returns beside the kernel's outcome so the parent's projections and the backfill take one DTO shape, the semantics of a caller-named expected version including expected version 0 as the create's uniqueness check, the byte-derived bounds on streams and writes per call and the bound on the value a call returns, the on-load baseline migration of step 3a whose rules `spec:context.journal` carries under E-25, and the internal guards that turn a defect into a technical failure; this is a provisional reading the owner has not ruled (D3, D6, E-23, E-25) +- [non-blocking #contextOperationOutcome] Extension E-23: the doc gives the sequence and the one-authority rule; this Spec fixes the request and result types, the operation runner with `plan` and `combine`, the aggregation of per-stream outcomes, the per-stream `dto` and `created` flag that the operation returns beside the kernel's outcome so the parent's projections and the backfill take one DTO shape, the semantics of a caller-named expected version including expected version 0 as the create's uniqueness check, the byte-derived bounds on streams and writes per call and the bound on the value a call returns, the on-load baseline migration of step 3a whose rules `spec:context.journal` carries under E-25, and the internal guards that turn a defect into a technical failure; this is a provisional reading the owner has not ruled (D3, D6, E-23, E-25) ## Contract diff --git a/design/specs/context/queries.sdp.md b/design/specs/context/queries.sdp.md index da85f0d..ae5460a 100644 --- a/design/specs/context/queries.sdp.md +++ b/design/specs/context/queries.sdp.md @@ -13,6 +13,11 @@ relations: - spec:facts.f11-components-have-no-ctx-auth - spec:facts.f15-parent-query-over-component-query-stays-reactive - spec:facts.f13-transactions-have-limits + dependsOn: + - spec:context.journal + - spec:command.actor-and-scope + - spec:command.outcome-boundary + - spec:context.event-envelope --- # The context queries @@ -33,9 +38,9 @@ A context exposes queries that return DTOs. The parent authenticates and authori ### Open questions -- [non-blocking] Probe 5 showed on the pinned backend with `convex-helpers` 0.1.124 that the parent query stays reactive, that pages pinned by their end cursors stay contiguous, that a capped page crosses the boundary as `SplitRequired` with its `splitCursor`, and that the `usePaginatedQuery` hook of `convex-helpers/react` then shows 40 of 45 rows; this design uses no hook across the boundary, and the examples are rerun when either pin moves (Probe 5, F15, S4, D8, E-24) -- [non-blocking] Extension E-24: the doc says component queries are authorized and return DTOs; this Spec fixes one definer per query, the one `tenantId` argument with no separate scope, the `toDto` mapping with the stream version on every DTO, pagination through `paginator`, the item cap with a row cap of twice the item cap and a byte cap passed on every run, the relay of `pageStatus` and `splitCursor` unchanged, paging by an explicit cursor pair with no client hook, the parent query's one `authorizeQuery` call and its refusal as a rejection, the `maxEvents` argument of the `rebuild` query and its `budgetExhausted` refusal for a batching caller, and the fold refusal of the `rebuild` query; this is a provisional reading the owner has not ruled (D8, S4, E-24, E-42) -- [non-blocking] A parent query's refusal carries the name of the entry that refused in the field `entry` of the rejection data; the wire shape stays `RejectionData` of `spec:command.outcome-boundary` unchanged, and the owner confirmed the word (D7, E-24) +- [non-blocking #contextPaginationSplit] Probe 5 showed on the pinned backend with `convex-helpers` 0.1.124 that the parent query stays reactive, that pages pinned by their end cursors stay contiguous, that a capped page crosses the boundary as `SplitRequired` with its `splitCursor`, and that the `usePaginatedQuery` hook of `convex-helpers/react` then shows 40 of 45 rows; this design uses no hook across the boundary, and the examples are rerun when either pin moves (Probe 5, F15, S4, D8, E-24) +- [non-blocking #contextQueryContract] Extension E-24: the doc says component queries are authorized and return DTOs; this Spec fixes one definer per query, the one `tenantId` argument with no separate scope, the `toDto` mapping with the stream version on every DTO, pagination through `paginator`, the item cap with a row cap of twice the item cap and a byte cap passed on every run, the relay of `pageStatus` and `splitCursor` unchanged, paging by an explicit cursor pair with no client hook, the parent query's one `authorizeQuery` call and its refusal as a rejection, the `maxEvents` argument of the `rebuild` query and its `budgetExhausted` refusal for a batching caller, and the fold refusal of the `rebuild` query; this is a provisional reading the owner has not ruled (D8, S4, E-24, E-42) +- [non-blocking #queryRefusalEntry] A parent query's refusal carries the name of the entry that refused in the field `entry` of the rejection data; the wire shape stays `RejectionData` of `spec:command.outcome-boundary` unchanged, and the owner confirmed the word (D7, E-24) ## Contract diff --git a/design/specs/context/tables.sdp.md b/design/specs/context/tables.sdp.md index 6525e7c..d9395c4 100644 --- a/design/specs/context/tables.sdp.md +++ b/design/specs/context/tables.sdp.md @@ -5,7 +5,9 @@ altitude: story readiness: defined relations: refines: spec:context.context-component - dependsOn: spec:kernel.state-document-mapping + dependsOn: + - spec:kernel.state-document-mapping + - spec:context.event-envelope decidedBy: spec:decisions.d02-context-owns-state-and-journal constrainedBy: - spec:facts.f13-transactions-have-limits @@ -26,7 +28,7 @@ Each context component owns a `streams` table and an `events` table, and a conte ### Open questions -- [non-blocking] Extension E-3: the doc names three indexes and the fields of the envelope; this Spec fixes the concrete tables, the operation index, the `streamParts` table for the derived mapping, the deleted-subject marker as `deletedAt` on the stream row and the size budgets; the owner confirms (D2, E-3) +- [non-blocking #contextJournalTables] Extension E-3: the doc names three indexes and the fields of the envelope; this Spec fixes the concrete tables, the operation index, the `streamParts` table for the derived mapping, the deleted-subject marker as `deletedAt` on the stream row and the size budgets; the owner confirms (D2, E-3) ## Contract diff --git a/design/specs/decisions/d02-context-owns-state-and-journal.sdp.md b/design/specs/decisions/d02-context-owns-state-and-journal.sdp.md index 1b7f273..872509c 100644 --- a/design/specs/decisions/d02-context-owns-state-and-journal.sdp.md +++ b/design/specs/decisions/d02-context-owns-state-and-journal.sdp.md @@ -27,8 +27,8 @@ Each context is a component that owns its current state, its event journal and i ### Open questions -- [non-blocking] Probe 3: the latency of one component call is measured and keeps every context a component; its function-call quota is a hosted usage reading, and isolation itself stays (Probe 3, D2, OQ1) -- [non-blocking] OQ1: a trivial context with no invariants of its own stays a component; the owner answered OQ1 on the first experiment's local measurement, and a measured workload that misses an agreed target, or a hosted usage reading above an agreed share, reopens it (OQ1, D2) +- [non-blocking #contextCallLatencyQuota] Probe 3: the latency of one component call is measured and keeps every context a component; its function-call quota is a hosted usage reading, and isolation itself stays (Probe 3, D2, OQ1) +- [non-blocking #contextIsolationChoice] OQ1: a trivial context with no invariants of its own stays a component; the owner answered OQ1 on the first experiment's local measurement, and a measured workload that misses an agreed target, or a hosted usage reading above an agreed share, reopens it (OQ1, D2) ## Decision diff --git a/design/specs/decisions/d03-events-only-source-of-next-state.sdp.md b/design/specs/decisions/d03-events-only-source-of-next-state.sdp.md index 8dec1ef..80d4735 100644 --- a/design/specs/decisions/d03-events-only-source-of-next-state.sdp.md +++ b/design/specs/decisions/d03-events-only-source-of-next-state.sdp.md @@ -23,8 +23,8 @@ Provenance: carried from v0.1; the two gaps at the end are new. Feature · Trace ### Open questions -- [non-blocking] Gap, initial state: the v0.1 interface has none; add `initial()` or let `evolve` accept an empty state so a rebuild from the creation event has a defined start; ruled provisionally as E-1 in the kernel's initial-state decision (D3, E-1) -- [non-blocking] Gap, state across several documents, its size budget: one document per stream stays the default while it fits the size budget, and where it cannot, the document writes are derived from the folded state in one place per context, as the kernel's state-document mapping decision states; the size budget there is a provisional number of E-2 (D3, E-2, OQ3) +- [non-blocking #initialStateChoice] Gap, initial state: the v0.1 interface has none; add `initial()` or let `evolve` accept an empty state so a rebuild from the creation event has a defined start; ruled provisionally as E-1 in the kernel's initial-state decision (D3, E-1) +- [non-blocking #streamStateSizeBudget] Gap, state across several documents, its size budget: one document per stream stays the default while it fits the size budget, and where it cannot, the document writes are derived from the folded state in one place per context, as the kernel's state-document mapping decision states; the size budget there is a provisional number of E-2 (D3, E-2, OQ3) ## Decision diff --git a/design/specs/decisions/d06-idempotency-client-and-receipts.sdp.md b/design/specs/decisions/d06-idempotency-client-and-receipts.sdp.md index d07168b..6448c76 100644 --- a/design/specs/decisions/d06-idempotency-client-and-receipts.sdp.md +++ b/design/specs/decisions/d06-idempotency-client-and-receipts.sdp.md @@ -30,7 +30,7 @@ The Convex React client retries a mutation until it is confirmed, and the backen ### Open questions -- [non-blocking] Probe 1 ran on 2026-10-01: the guarantee holds across a backend restart, a closed client leaves its mutation executed once or not at all, and the HTTP client does not retry; `ConvexHttpClient` is a Convex client the guarantee does not reach, so whether its callers join the callers that need receipts is the owner's ruling (Probe 1, D6) +- [non-blocking #httpCallerReceipts] Probe 1 ran on 2026-10-01: the guarantee holds across a backend restart, a closed client leaves its mutation executed once or not at all, and the HTTP client does not retry; `ConvexHttpClient` is a Convex client the guarantee does not reach, so whether its callers join the callers that need receipts is the owner's ruling (Probe 1, D6) ## Decision diff --git a/design/specs/decisions/d07-rejections-thrown-not-stored.sdp.md b/design/specs/decisions/d07-rejections-thrown-not-stored.sdp.md index d95fe78..9cf5a48 100644 --- a/design/specs/decisions/d07-rejections-thrown-not-stored.sdp.md +++ b/design/specs/decisions/d07-rejections-thrown-not-stored.sdp.md @@ -28,7 +28,7 @@ A rejected command throws a structured `ConvexError`. The whole mutation rolls b ### Open questions -- [non-blocking] OQ2: whether the product needs a record of refused commands anywhere, for security audit or agent proposals, decides where the nested mutation is required and whether the generic internal dispatcher exists (OQ2, D7) +- [non-blocking #refusedCommandHistory] OQ2: whether the product needs a record of refused commands anywhere, for security audit or agent proposals, decides where the nested mutation is required and whether the generic internal dispatcher exists (OQ2, D7) ## Decision diff --git a/design/specs/decisions/d08-read-models-in-command.sdp.md b/design/specs/decisions/d08-read-models-in-command.sdp.md index b35cc76..67f761f 100644 --- a/design/specs/decisions/d08-read-models-in-command.sdp.md +++ b/design/specs/decisions/d08-read-models-in-command.sdp.md @@ -27,7 +27,7 @@ The core path runs zero projection jobs and the simplest read that works wins. O ### Open questions -- [non-blocking] Probe 5 ran on 2026-10-01: a parent query over component queries stays reactive, and pagination crosses the boundary with pages pinned by their end cursors; the client hook lost rows after a capped page split, which slice S2 settles before it builds a list (Probe 5, F15, D8) +- [non-blocking #parentQueryPaginationSplit] Probe 5 ran on 2026-10-01: a parent query over component queries stays reactive, and pagination crosses the boundary with pages pinned by their end cursors; the client hook lost rows after a capped page split, which slice S2 settles before it builds a list (Probe 5, F15, D8) ## Decision diff --git a/design/specs/decisions/d09-rebuild-online-by-default.sdp.md b/design/specs/decisions/d09-rebuild-online-by-default.sdp.md index 72b492d..145a214 100644 --- a/design/specs/decisions/d09-rebuild-online-by-default.sdp.md +++ b/design/specs/decisions/d09-rebuild-online-by-default.sdp.md @@ -27,7 +27,7 @@ Because read models update inside commands, rebuilding a per-entity or current-s ### Open questions -- [non-blocking] Probe 6 ran a backfill batch racing a live command and the migrations component on the native backend, with its results on F17; a hosted deployment and sustained contention are not run (Probe 6, F1, F17, D9) +- [non-blocking #rebuildContentionCoverage] Probe 6 ran a backfill batch racing a live command and the migrations component on the native backend, with its results on F17; a hosted deployment and sustained contention are not run (Probe 6, F1, F17, D9) ## Decision diff --git a/design/specs/decisions/d13-deferred-work-is-an-obligation.sdp.md b/design/specs/decisions/d13-deferred-work-is-an-obligation.sdp.md index 5387f22..6ef96da 100644 --- a/design/specs/decisions/d13-deferred-work-is-an-obligation.sdp.md +++ b/design/specs/decisions/d13-deferred-work-is-an-obligation.sdp.md @@ -32,7 +32,7 @@ An obligation records one promise. It commits in the same transaction as the bus ### Open questions -- [non-blocking] Probe 7 observes the local scheduled reaction, states, scan boundary and both CLI replacement modes; the owner confirmed D13 on these observations; seven-day expiry, hosted dashboard restore, Workpool and Workflow remain unprobed (Probe 7, D13, F12, F16) +- [non-blocking #obligationSchedulerCoverage] Probe 7 observes the local scheduled reaction, states, scan boundary and both CLI replacement modes; the owner confirmed D13 on these observations; seven-day expiry, hosted dashboard restore, Workpool and Workflow remain unprobed (Probe 7, D13, F12, F16) ## Decision diff --git a/design/specs/decisions/d19-operations-travel-with-capability.sdp.md b/design/specs/decisions/d19-operations-travel-with-capability.sdp.md index a4646e8..8e1ecca 100644 --- a/design/specs/decisions/d19-operations-travel-with-capability.sdp.md +++ b/design/specs/decisions/d19-operations-travel-with-capability.sdp.md @@ -28,7 +28,7 @@ Each installed capability carries its own operations: logging that never cancels ### Open questions -- [non-blocking] Probe 7 pending: what a restore leaves of Workpool and Workflow state decides how much of the restore procedure the obligation module must rebuild (Probe 7, F12, D19) +- [non-blocking #restoreObligationCoverage] Probe 7 pending: what a restore leaves of Workpool and Workflow state decides how much of the restore procedure the obligation module must rebuild (Probe 7, F12, D19) ## Decision diff --git a/design/specs/effects/claim-call-settle.sdp.md b/design/specs/effects/claim-call-settle.sdp.md index 93ee895..3920419 100644 --- a/design/specs/effects/claim-call-settle.sdp.md +++ b/design/specs/effects/claim-call-settle.sdp.md @@ -9,6 +9,7 @@ relations: - spec:obligations.record-contract - spec:obligations.lifecycle-transitions - spec:obligations.local-reaction-wrapper + - spec:command.actor-and-scope constrainedBy: - spec:facts.f09-scheduled-mutation-and-action-retry-semantics - spec:facts.f10-action-mutation-calls-are-separate-transactions @@ -38,8 +39,8 @@ The action repeats exactly one thing when it throws: its settle call, within a s ### Open questions -- [non-blocking] Extension E-11: the doc gives the three steps and the three policies but no signatures; this Spec pins them and adds a fourth policy value `none`, which is the declared absence of a safe policy the doc describes (D14, E-11) -- [non-blocking] Extension E-58: the doc gives no lease duration, key derivation, late evidence bound, reconcile shape or handling of a settle call that throws; this design sets the lease to the handler runtime's action timeout plus 5 minutes, derives the key from tenant and effect key, keeps ten late reports, defines a reconcile as a provider query that returns the same report union, and lets the action repeat a thrown settle call three times, with a duplicate rule in the settle, before leaving the attempt to lease expiry (D14, D15, F13, E-58) +- [non-blocking #effectAttemptPolicies] Extension E-11: the doc gives the three steps and the three policies but no signatures; this Spec pins them and adds a fourth policy value `none`, which is the declared absence of a safe policy the doc describes (D14, E-11) +- [non-blocking #effectAttemptLeaseEvidence] Extension E-58: the doc gives no lease duration, key derivation, late evidence bound, reconcile shape or handling of a settle call that throws; this design sets the lease to the handler runtime's action timeout plus 5 minutes, derives the key from tenant and effect key, keeps ten late reports, defines a reconcile as a provider query that returns the same report union, and lets the action repeat a thrown settle call three times, with a duplicate rule in the settle, before leaving the attempt to lease expiry (D14, D15, F13, E-58) ## Contract diff --git a/design/specs/effects/external-effects.sdp.md b/design/specs/effects/external-effects.sdp.md index df66df7..1f1856d 100644 --- a/design/specs/effects/external-effects.sdp.md +++ b/design/specs/effects/external-effects.sdp.md @@ -39,9 +39,9 @@ An external effect is an obligation whose attempt must leave the transaction to ### Open questions -- [non-blocking] Extension E-58: the doc does not give the lease duration, the provider key derivation, the late evidence bound, the shape of a reconcile, or what the action does when its settle call throws; `spec:effects.claim-call-settle` pins them (D14, E-58) -- [non-blocking] Extension E-51: a declined provider report settles the obligation as cancelled with `businessRejection` evidence, by the same reading as the local wrapper (D14, D4, E-51) -- [non-blocking] Decision method rule 4: the doc writes Layer 3 from what the first experiment shows, and this Spec and `spec:effects.claim-call-settle` are written before it runs; the lease durations, the settle repeats and the late-evidence bound of E-58 and the backoff shared under E-51 are provisional defaults until the experiment's measurements and Probe 7 confirm or replace them, and the owner confirms that writing the structure now does not pre-empt the experiment (Decision method rule 4, D14, D15, E-51, E-58) +- [non-blocking #effectAttemptRecoveryContract] Extension E-58: the doc does not give the lease duration, the provider key derivation, the late evidence bound, the shape of a reconcile, or what the action does when its settle call throws; `spec:effects.claim-call-settle` pins them (D14, E-58) +- [non-blocking #providerBusinessRejection] Extension E-51: a declined provider report settles the obligation as cancelled with `businessRejection` evidence, by the same reading as the local wrapper (D14, D4, E-51) +- [non-blocking #effectAttemptDefaultBounds] Decision method rule 4: the doc writes Layer 3 from what the first experiment shows, and this Spec and `spec:effects.claim-call-settle` are written before it runs; the lease durations, the settle repeats and the late-evidence bound of E-58 and the backoff shared under E-51 are provisional defaults until the experiment's measurements and Probe 7 confirm or replace them, and the owner confirms that writing the structure now does not pre-empt the experiment (Decision method rule 4, D14, D15, E-51, E-58) ## Behavior diff --git a/design/specs/facts/f11-components-have-no-ctx-auth.sdp.md b/design/specs/facts/f11-components-have-no-ctx-auth.sdp.md index 9281795..29c3ba8 100644 --- a/design/specs/facts/f11-components-have-no-ctx-auth.sdp.md +++ b/design/specs/facts/f11-components-have-no-ctx-auth.sdp.md @@ -18,7 +18,7 @@ Code inside a component cannot read data that is not explicitly provided to it, ### Open questions -- [non-blocking] Observed against the documentation on 2026-10-01: the components page says `ctx.auth` is not available within a component; on the pinned native backend, release `precompiled-2026-09-28-5c7cb5b` with `convex` 1.46.0, `ctx.auth.getUserIdentity()` inside a component query returned the caller's identity, and `convex-test` 0.0.60 returns null for the same call; the design does not depend on which holds, because D11 passes the actor explicitly and no component function reads `ctx.auth`; whether this fact's wording and status follow the documentation or the backend is the owner's ruling; the design takes the provisional reading that the documentation's sentence stays the fact and that the Specs of the kernel, context and command families state no component function reads `ctx.auth` or `process.env` as the design's rule, which a lint check over `src/context` and every component directory of the fixture composition enforces (F11, D11) +- [non-blocking #contextActorIdentity] Observed against the documentation on 2026-10-01: the components page says `ctx.auth` is not available within a component; on the pinned native backend, release `precompiled-2026-09-28-5c7cb5b` with `convex` 1.46.0, `ctx.auth.getUserIdentity()` inside a component query returned the caller's identity, and `convex-test` 0.0.60 returns null for the same call; the design does not depend on which holds, because D11 passes the actor explicitly and no component function reads `ctx.auth`; whether this fact's wording and status follow the documentation or the backend is the owner's ruling; the design takes the provisional reading that the documentation's sentence stays the fact and that the Specs of the kernel, context and command families state no component function reads `ctx.auth` or `process.env` as the design's rule, which a lint check over `src/context` and every component directory of the fixture composition enforces (F11, D11) ## Constraints diff --git a/design/specs/facts/f12-backups-exclude-pending-scheduled-functions.sdp.md b/design/specs/facts/f12-backups-exclude-pending-scheduled-functions.sdp.md index ff517c9..453f6f8 100644 --- a/design/specs/facts/f12-backups-exclude-pending-scheduled-functions.sdp.md +++ b/design/specs/facts/f12-backups-exclude-pending-scheduled-functions.sdp.md @@ -18,7 +18,7 @@ The backup page excludes scheduled functions, code, configuration and environmen ### Open questions -- [non-blocking] Hosted dashboard restore, file storage, Workpool and Workflow are not run; local CLI import does not settle those paths. CLI replacement in place on a hosted deployment is the hosted example `spec:facts.f12-backups-exclude-pending-scheduled-functions.probe-7-hosted-in-place-import`; dashboard restore has no example. (F12, Probe 7, E-59) +- [non-blocking #hostedRestoreCoverage] Hosted dashboard restore, file storage, Workpool and Workflow are not run; local CLI import does not settle those paths. CLI replacement in place on a hosted deployment is the hosted example `spec:facts.f12-backups-exclude-pending-scheduled-functions.probe-7-hosted-in-place-import`; dashboard restore has no example. (F12, Probe 7, E-59) ## Constraints diff --git a/design/specs/facts/f13-transactions-have-limits.sdp.md b/design/specs/facts/f13-transactions-have-limits.sdp.md index b5d32b9..5fef61c 100644 --- a/design/specs/facts/f13-transactions-have-limits.sdp.md +++ b/design/specs/facts/f13-transactions-have-limits.sdp.md @@ -10,7 +10,7 @@ relations: F13 · Status: documented · Doc status: Documented, S9 · Decisions: D10, D19. -The numbers below were read from the limits page on 2026-09-30 and match the plan's section 9. Every batch, sweeper, fan-out and backfill in the design states its own bound below them, because platform limits are ceilings, not batch sizes. The limits page states no limit on nested `runQuery` or `runMutation` calls per function. The page on writing data, read on 2026-10-01, states that "a mutation or query called by another mutation or query share the overall transaction limits" and documents `ctx.meta.getTransactionMetrics()` and the `transactionLimits` option; it does not name components. Probe 4 ran on a native backend on 2026-10-01, release `precompiled-2026-09-28-5c7cb5b` with `convex` 1.46.0: the 16 MiB read limit and the 16 MiB write limit are each one budget across a nested call and across a component boundary, and a call stack of nine functions commits where ten fails. +The numbers below were read from the limits page on 2026-09-30 and match the plan's section 9. The limits page was read again on 2026-10-03 for three entries, `limitDocument`, `limitWriteThroughput` and `limitIoOperations`. Every batch, sweeper, fan-out and backfill in the design states its own bound below them, because platform limits are ceilings, not batch sizes. The limits page states no limit on nested `runQuery` or `runMutation` calls per function. The page on writing data, read on 2026-10-01, states that "a mutation or query called by another mutation or query share the overall transaction limits" and documents `ctx.meta.getTransactionMetrics()` and the `transactionLimits` option; it does not name components. Probe 4 ran on a native backend on 2026-10-01, release `precompiled-2026-09-28-5c7cb5b` with `convex` 1.46.0: the 16 MiB read limit and the 16 MiB write limit are each one budget across a nested call and across a component boundary, and a call stack of nine functions commits where ten fails. ## Intent @@ -27,11 +27,13 @@ The numbers below were read from the limits page on 2026-09-30 and match the pla The documented numbers bound batches. Probe 4 observes transaction budgets; Probe 7 distinguishes the scheduled-argument page values from the pinned backend's enforced budget. A warning on this release does not remove the documented ceiling from the design. -- limitDocument: 1 MiB per document, 1024 fields, nesting depth 16, 8192 array elements (F13, S9) +- limitDocument: 1 MiB per document and 1024 top-level fields, the system fields included, with nesting depth 16; per value, 1024 fields per object, 8192 elements per array and nesting depth 64 (F13, S9) - limitTransactionRead: 16 MiB read, 32,000 documents scanned and 4,096 index ranges read per transaction (F13, S9) - limitTransactionWrite: 16 MiB written and 16,000 documents written per transaction (F13, S9) +- limitWriteThroughput: the limits page lists a mutation write throughput limit per deployment class, 4 MiB for S16, 8 MiB for S256, 32 MiB for D1024 and 64 MiB for D2048, and states no unit on that row; the native harness paces its writes at 4,194,304 bytes a second; a batch chain that writes near its per-batch bound is measured against it before a bound is raised (F13, S9) - limitFunctionPayload: 16 MiB per function argument set and 16 MiB per return value; a Node action's arguments are capped at 5 MiB (F13, S9) - limitQueryMutationTimeout: 1 second of execution per query or mutation, a limit on the function's own computation and not on the elapsed time of its call tree, because nested calls draw on a separate system-operation budget (F13, S9, F19) +- limitIoOperations: the limits page states 1000 concurrent IO operations per function, a database operation among them, and does not say whether a sequence of awaited operations counts; no probe has reached it (F13, S9) - limitActionTimeout: 30 minutes in the Convex runtime and 10 minutes in the Node runtime (F13, S9) - limitScheduling: the limits page states 1000 scheduled functions per mutation, 4 MiB per scheduled argument set, 16 MiB summed arguments per mutation and 1000000 outstanding functions; the scheduling page states 8 MB total, both read 2026-10-02; the design keeps 4 MiB per call as its ceiling because this release names that limit as a future hard error (F13, S9, Probe 7) - limitSchema: 32 indexes per table, 16 fields per index, 10,000 tables per deployment (F13, S9) diff --git a/design/specs/facts/f16-scheduled-functions-table-shows-failed-runs.sdp.md b/design/specs/facts/f16-scheduled-functions-table-shows-failed-runs.sdp.md index 7c2ee0f..b5e0b6a 100644 --- a/design/specs/facts/f16-scheduled-functions-table-shows-failed-runs.sdp.md +++ b/design/specs/facts/f16-scheduled-functions-table-shows-failed-runs.sdp.md @@ -18,7 +18,7 @@ The scheduled-functions page documents five states and a seven-day completion-re ### Open questions -- [non-blocking] Seven-day retention and expiry, hosted dashboard restore, Workpool and Workflow are not run; the short native retention observation does not establish the documented window. The hosted example `spec:facts.f16-scheduled-functions-table-shows-failed-runs.probe-7-hosted-retention` records the ages at which planted schedules are read or missing, so that runs bracket the window a hosted deployment keeps. (F16, Probe 7, F12, E-59) +- [non-blocking #scheduledFunctionRetentionCoverage] Seven-day retention and expiry, hosted dashboard restore, Workpool and Workflow are not run; the short native retention observation does not establish the documented window. The hosted example `spec:facts.f16-scheduled-functions-table-shows-failed-runs.probe-7-hosted-retention` records the ages at which planted schedules are read or missing, so that runs bracket the window a hosted deployment keeps. (F16, Probe 7, F12, E-59) ## Constraints diff --git a/design/specs/facts/f17-migrations-fits-generation-backfill.sdp.md b/design/specs/facts/f17-migrations-fits-generation-backfill.sdp.md index fa58786..264e66a 100644 --- a/design/specs/facts/f17-migrations-fits-generation-backfill.sdp.md +++ b/design/specs/facts/f17-migrations-fits-generation-backfill.sdp.md @@ -28,7 +28,7 @@ So the component does not fit a generation backfill as the platform needs one: i ### Open questions -- [non-blocking] Hosted quotas, sustained contention, automatic stall detection, backup replacement, old encrypted cursors, fence advancement, parallel callbacks and a rebuild across changing tenants are not run by these examples; the native backend evidence proves only the named boundaries and does not settle those behaviors. (F17, Probe 6) +- [non-blocking #rebuildBackendCoverage] Hosted quotas, sustained contention, automatic stall detection, backup replacement, old encrypted cursors, fence advancement, parallel callbacks and a rebuild across changing tenants are not run by these examples; the native backend evidence proves only the named boundaries and does not settle those behaviors. (F17, Probe 6) ## Constraints diff --git a/design/specs/facts/f18-commit-timestamps.sdp.md b/design/specs/facts/f18-commit-timestamps.sdp.md index 6f786be..cdbfbcb 100644 --- a/design/specs/facts/f18-commit-timestamps.sdp.md +++ b/design/specs/facts/f18-commit-timestamps.sdp.md @@ -18,7 +18,7 @@ The pinned package declares `db.vars.commitTs`, a placeholder that a mutation wr ### Open questions -- [non-blocking] The documented upper bound, `ctx.meta.getSnapshotTs()`, is absent from the pinned package's public declarations; ordering under overlapping transactions and behavior on a hosted deployment are outside the native example; a platform consumer adopts the bound only through a public declaration of a pinned package (F18, Probe 8) +- [non-blocking #commitTimestampBound] The documented upper bound, `ctx.meta.getSnapshotTs()`, is absent from the pinned package's public declarations; ordering under overlapping transactions and behavior on a hosted deployment are outside the native example; a platform consumer adopts the bound only through a public declaration of a pinned package (F18, Probe 8) ## Constraints diff --git a/design/specs/facts/f19-nested-calls-share-time-budgets.sdp.md b/design/specs/facts/f19-nested-calls-share-time-budgets.sdp.md index aad8f87..486bcd7 100644 --- a/design/specs/facts/f19-nested-calls-share-time-budgets.sdp.md +++ b/design/specs/facts/f19-nested-calls-share-time-budgets.sdp.md @@ -18,7 +18,7 @@ A nested or component call runs through an asynchronous system call of the pinne ### Open questions -- [non-blocking] The public package declares no duration for the system-operation budget. The local refusal near fifteen seconds is a local backend's observation, not a hosted limit. The examples exercise mutations; query-only call trees and how execution time is counted across child functions that compute are not measured (F19, Probe 9, Probe 11) +- [non-blocking #nestedCallTimeBudget] The public package declares no duration for the system-operation budget. The local refusal near fifteen seconds is a local backend's observation, not a hosted limit. The examples exercise mutations; query-only call trees and how execution time is counted across child functions that compute are not measured (F19, Probe 9, Probe 11) ## Constraints diff --git a/design/specs/facts/f20-pagination-checks-bytes-after-reading.sdp.md b/design/specs/facts/f20-pagination-checks-bytes-after-reading.sdp.md index eadc3d5..9377506 100644 --- a/design/specs/facts/f20-pagination-checks-bytes-after-reading.sdp.md +++ b/design/specs/facts/f20-pagination-checks-bytes-after-reading.sdp.md @@ -18,7 +18,7 @@ F20 · Status: probed · Doc status: not in the doc's ledger · Decisions: D8, D ### Open questions -- [non-blocking] The bound applies to a plain index stream. Filtered, joined or mapped streams may charge reads whose rows are not retained, and the examples do not establish an overshoot bound for those compositions. A hosted deployment is not measured (F20, Probe 10) +- [non-blocking #paginationReadOvershoot] The bound applies to a plain index stream. Filtered, joined or mapped streams may charge reads whose rows are not retained, and the examples do not establish an overshoot bound for those compositions. A hosted deployment is not measured (F20, Probe 10) ## Constraints diff --git a/design/specs/facts/probe-plan.sdp.md b/design/specs/facts/probe-plan.sdp.md index b7150f1..a8cc7a0 100644 --- a/design/specs/facts/probe-plan.sdp.md +++ b/design/specs/facts/probe-plan.sdp.md @@ -13,6 +13,7 @@ relations: - spec:facts.f15-parent-query-over-component-query-stays-reactive - spec:facts.f16-scheduled-functions-table-shows-failed-runs - spec:facts.f17-migrations-fits-generation-backfill + - spec:platform.native-harness constrainedBy: - spec:facts.f04-nested-calls-cost-more-than-helpers - spec:facts.f05-react-client-retries-until-confirmed @@ -40,12 +41,12 @@ The probes are not the first experiment. The experiment builds Layers 0 to 2 and ### Open questions -- [non-blocking] Probe 1 ran on 2026-10-01: a closed client leaves its mutation executed once or not at all, a backend restart leaves it executed once, and the HTTP client does not retry; whether callers of `ConvexHttpClient` must supply a request key is D6's question for the owner (Probe 1, D6) -- [non-blocking] Probe 3 ran on 2026-10-01 for latency on a local backend; a local backend has no function-call quota, so that half is the hosted example `spec:facts.f04-nested-calls-cost-more-than-helpers.probe-3-hosted-function-calls`, which only a hosted deployment can run (Probe 3, F4, D2, D8, E-59) -- [non-blocking] Probe 5 ran on 2026-10-01: a parent query over a component query stays reactive, pages pinned by their end cursors stay contiguous, and a capped page crosses the boundary as `SplitRequired`; the `convex-helpers/react` hook then lost rows, and slice S2 decides how a context list is capped before it builds one (Probe 5, F15, S4, D8) -- [non-blocking] Probe 6 records native backend evidence on the pins named by F17: a row race, cancel and resume, thrown-batch rollback, restart, table scope and a parent batch returning a context cursor to the migrations driver; hosted quotas and sustained contention are not run. (Probe 6, F17, D9) -- [non-blocking] Probe 7 observes all five scheduler states, local CLI replacement into a fresh backend and in place, stored scheduler IDs, kept reactions, scheduled-argument limits and failed-function scan cost; seven-day expiry, hosted dashboard restore, Workpool and Workflow remain open before the decisions resting on those observations. (Probe 7, F12, F13, F16, D13, D19) -- [non-blocking] Extension E-16: the doc says each probe is one small test on a native backend and that a probed fact is one a native backend test showed, and does not say how a test that settles an unknown states what it expects or what follows when the backend answers otherwise; the option taken here: a probe is written as example Specs that verify its fact, each bound to its native test; an example's bound values are the expectation written before the first run; a first run that shows another value is a finding recorded on the fact, and the example is then bound to what that release does and says so; an example fails when its run did not reach the boundary it is about; a later run on a new release that shows another value is again a finding, and the test is not repaired to keep the old value; a size is recorded and not asserted, while a limit the backend enforces is bound as observed; and what a local backend cannot answer is named on the example and stays open on the fact; the owner confirms (E-16, Probes, Acceptance scenarios) +- [non-blocking #httpClientRequestKey] Probe 1 ran on 2026-10-01: a closed client leaves its mutation executed once or not at all, a backend restart leaves it executed once, and the HTTP client does not retry; whether callers of `ConvexHttpClient` must supply a request key is D6's question for the owner (Probe 1, D6) +- [non-blocking #hostedContextCallQuota] Probe 3 ran on 2026-10-01 for latency on a local backend; a local backend has no function-call quota, so that half is the hosted example `spec:facts.f04-nested-calls-cost-more-than-helpers.probe-3-hosted-function-calls`, which only a hosted deployment can run (Probe 3, F4, D2, D8, E-59) +- [non-blocking #contextListSplit] Probe 5 ran on 2026-10-01: a parent query over a component query stays reactive, pages pinned by their end cursors stay contiguous, and a capped page crosses the boundary as `SplitRequired`; the `convex-helpers/react` hook then lost rows, and slice S2 decides how a context list is capped before it builds one (Probe 5, F15, S4, D8) +- [non-blocking #rebuildNativeCoverage] Probe 6 records native backend evidence on the pins named by F17: a row race, cancel and resume, thrown-batch rollback, restart, table scope and a parent batch returning a context cursor to the migrations driver; hosted quotas and sustained contention are not run. (Probe 6, F17, D9) +- [non-blocking #schedulerRestoreCoverage] Probe 7 observes all five scheduler states, local CLI replacement into a fresh backend and in place, stored scheduler IDs, kept reactions, scheduled-argument limits and failed-function scan cost; seven-day expiry, hosted dashboard restore, Workpool and Workflow remain open before the decisions resting on those observations. (Probe 7, F12, F13, F16, D13, D19) +- [non-blocking #probeExpectationContract] Extension E-16: the doc says each probe is one small test on a native backend and that a probed fact is one a native backend test showed, and does not say how a test that settles an unknown states what it expects or what follows when the backend answers otherwise; the option taken here: a probe is written as example Specs that verify its fact, each bound to its native test; an example's bound values are the expectation written before the first run; a first run that shows another value is a finding recorded on the fact, and the example is then bound to what that release does and says so; an example fails when its run did not reach the boundary it is about; a later run on a new release that shows another value is again a finding, and the test is not repaired to keep the old value; a size is recorded and not asserted, while a limit the backend enforces is bound as observed; and what a local backend cannot answer is named on the example and stays open on the fact; the owner confirms (E-16, Probes, Acceptance scenarios) ## Workflow diff --git a/design/specs/history-view.pack.sdp.md b/design/specs/history-view.pack.sdp.md new file mode 100644 index 0000000..4c433eb --- /dev/null +++ b/design/specs/history-view.pack.sdp.md @@ -0,0 +1,28 @@ +--- +id: pack:history-view +specs: + - spec:application.history-projection + - spec:application.projection-contract + - spec:application.read-models + - spec:application.rebuild + - spec:application.history-rebuild + - spec:application.history-rebuild.write-pause-rebuild-interrupt + - spec:application.history-rebuild.write-pause-rebuild-abort + - spec:application.history-rebuild.write-pause-rebuild-version-change + - spec:application.history-rebuild.write-pause-rebuild-installation + - spec:application.history-rebuild.write-pause-rebuild-source-order-mismatch + - spec:application.write-pause + - spec:application.generation-registry + - spec:application.orders-inventory-example + - spec:command.command-declaration + - spec:command.command-pipeline + - spec:context.event-envelope + - spec:context.journal + - spec:context.queries + - spec:decisions.d09-rebuild-online-by-default +modelRefs: + - spec:platform.vocabulary +--- +# History view + +The Specs a history view rests on, in reading order. A history view is a read model whose row folds the events of several streams, such as the order allocation history of the example domain. The history projection comes first because it pins the form, its fold and that view; the projection contract and the read models follow with the row conventions and the read-model rules every view shares. The rebuild, the history rebuild with its five examples, the write pause and the generation registry say how such a view comes to exist and is rebuilt while the commands that write its sources are refused. The rebuild comes first because it holds the operator entries and the batch chain both paths share; the history rebuild holds the paused path alone and the five examples verify it, so the rebuild's implementation binding claims the online path and the history rebuild shows the paused path designed apart from it. The example domain names the events the view folds, and the command declaration and the command pipeline bind the view to the commands that feed it and write it inside each command. The event envelope, the journal and the queries are where the events come from on the live path and on the backfill, and D9 is the decision that a read model which depends on history across several streams rebuilds under a write pause. diff --git a/design/specs/kernel/decider-contract.sdp.md b/design/specs/kernel/decider-contract.sdp.md index f079e31..ad6c164 100644 --- a/design/specs/kernel/decider-contract.sdp.md +++ b/design/specs/kernel/decider-contract.sdp.md @@ -5,7 +5,9 @@ altitude: story readiness: defined relations: refines: spec:kernel.domain-kernel - dependsOn: spec:kernel.initial-state + dependsOn: + - spec:kernel.initial-state + - spec:kernel.outcome-model decidedBy: spec:decisions.d03-events-only-source-of-next-state constrainedBy: - spec:laws.law03-events-only-source-of-state @@ -26,7 +28,7 @@ The generic parameters are the state `S`, the command `C`, the domain event `E` ### Open questions -- [non-blocking] Extension E-20: the concrete generic parameters, the fields of `DecisionContext`, the `DomainEvent` shape, the at-least-one-event rule and the helper shapes are this design's; the owner confirms them before code names them (D3, E-20) +- [non-blocking #deciderEventContract] Extension E-20: the concrete generic parameters, the fields of `DecisionContext`, the `DomainEvent` shape, the at-least-one-event rule and the helper shapes are this design's; the owner confirms them before code names them (D3, E-20) ## Contract diff --git a/design/specs/kernel/domain-kernel.sdp.md b/design/specs/kernel/domain-kernel.sdp.md index b6b7a59..7d2b936 100644 --- a/design/specs/kernel/domain-kernel.sdp.md +++ b/design/specs/kernel/domain-kernel.sdp.md @@ -34,8 +34,8 @@ Its boundary is the decider contract. Above it sits the context component of Lay ### Open questions -- [non-blocking] Extension E-20: the doc gives `decide(state, command, context)`, `fold(evolve, state, newEvents)` and the purity rule, and nothing about the concrete generic parameters, the fields of the decision context, the shape of a domain event, or the state-machine and invariant helpers; the decider contract fixes them and the owner confirms them before code names them (D3, E-20) -- [non-blocking] Extension E-1: the doc leaves open whether the interface gains `initial()` or `evolve` accepts an empty state; the initial-state decision rules for `initial()` provisionally (D3, E-1) +- [non-blocking #deciderStateContract] Extension E-20: the doc gives `decide(state, command, context)`, `fold(evolve, state, newEvents)` and the purity rule, and nothing about the concrete generic parameters, the fields of the decision context, the shape of a domain event, or the state-machine and invariant helpers; the decider contract fixes them and the owner confirms them before code names them (D3, E-20) +- [non-blocking #initialStateChoice] Extension E-1: the doc leaves open whether the interface gains `initial()` or `evolve` accepts an empty state; the initial-state decision rules for `initial()` provisionally (D3, E-1) ## Behavior diff --git a/design/specs/kernel/initial-state.sdp.md b/design/specs/kernel/initial-state.sdp.md index d37f433..04faa39 100644 --- a/design/specs/kernel/initial-state.sdp.md +++ b/design/specs/kernel/initial-state.sdp.md @@ -21,7 +21,7 @@ The v0.1 interface had no initial state, so a rebuild from the creation event ha ### Open questions -- [non-blocking] Extension E-1: the doc offers `initial()` or an `evolve` that accepts an empty state and rules on neither; this Spec takes `initial()` provisionally and the owner confirms or reverses it (D3, E-1) +- [non-blocking #initialStateChoice] Extension E-1: the doc offers `initial()` or an `evolve` that accepts an empty state and rules on neither; this Spec takes `initial()` provisionally and the owner confirms or reverses it (D3, E-1) ## Decision diff --git a/design/specs/kernel/outcome-model.sdp.md b/design/specs/kernel/outcome-model.sdp.md index eaea3ac..284c89a 100644 --- a/design/specs/kernel/outcome-model.sdp.md +++ b/design/specs/kernel/outcome-model.sdp.md @@ -32,7 +32,7 @@ The kernel defines the shapes and does not define wire codes. The closed error-c ### Open questions -- [non-blocking] Extension E-21: the doc names the four outcomes and their commit behavior but not their TypeScript shape; this Spec fixes `Outcome`, `Rejection`, `CommittedOutcome`, `AffectedRef`, the rule that a committed outcome carries at least one event, and the three rejection codes the adapter reserves, `staleVersion`, `operationTooLarge` and `entityExists`, the last being the adapter's answer to a create planned at expected version 0 whose stream already exists, which is the uniqueness check D6 names for a client-generated entity ID; the owner confirms the shapes or renames the reserved codes before code names them (D4, D6, E-21) +- [non-blocking #outcomeRejectionContract] Extension E-21: the doc names the four outcomes and their commit behavior but not their TypeScript shape; this Spec fixes `Outcome`, `Rejection`, `CommittedOutcome`, `AffectedRef`, the rule that a committed outcome carries at least one event, and the three rejection codes the adapter reserves, `staleVersion`, `operationTooLarge` and `entityExists`, the last being the adapter's answer to a create planned at expected version 0 whose stream already exists, which is the uniqueness check D6 names for a client-generated entity ID; the owner confirms the shapes or renames the reserved codes before code names them (D4, D6, E-21) ## Rule diff --git a/design/specs/kernel/state-document-mapping.sdp.md b/design/specs/kernel/state-document-mapping.sdp.md index 2c998ff..df66956 100644 --- a/design/specs/kernel/state-document-mapping.sdp.md +++ b/design/specs/kernel/state-document-mapping.sdp.md @@ -24,7 +24,7 @@ Folding is simple when a stream's state is one document. When state spans severa ### Open questions -- [non-blocking] Extension E-2: the doc prefers one document per stream within a size budget and names no budget; this Spec sets `budgetBytes` as one number per stream type meaning the whole stream, the single document or the head and every part together, at most 256 KiB for the single mapping and at most 512 KiB over at most 32 parts for the derived mapping, provisionally, lets a stream type declare a smaller `budgetBytes` on its mapping, caps the derived budget at half the document ceiling because a baseline event holds the whole migrated state in one document, and derives every bound on streams per call, list page, backfill batch and baseline batch from that budget so that the 16 MiB read ceiling binds in bytes and not only in documents; the owner confirms the numbers (D3, D5, OQ3, F13, E-2, E-25) +- [non-blocking #streamStateBudget] Extension E-2: the doc prefers one document per stream within a size budget and names no budget; this Spec sets `budgetBytes` as one number per stream type meaning the whole stream, the single document or the head and every part together, at most 256 KiB for the single mapping and at most 512 KiB over at most 32 parts for the derived mapping, provisionally, lets a stream type declare a smaller `budgetBytes` on its mapping, caps the derived budget at half the document ceiling because a baseline event holds the whole migrated state in one document, and derives every bound on streams per call, list page, backfill batch and baseline batch from that budget so that the 16 MiB read ceiling binds in bytes and not only in documents; the owner confirms the numbers (D3, D5, OQ3, F13, E-2, E-25) ## Decision diff --git a/design/specs/obligations/do-nothing-check.sdp.md b/design/specs/obligations/do-nothing-check.sdp.md index 41fc9b9..1913b3c 100644 --- a/design/specs/obligations/do-nothing-check.sdp.md +++ b/design/specs/obligations/do-nothing-check.sdp.md @@ -30,7 +30,7 @@ Before the obligation module is built, the do-nothing option is checked against ### Open questions -- [non-blocking] Probe 7 observes a plain scheduled local reaction and all five states on the pinned native backend; a failed-function scan counts zero documents and bytes but exhausts the 4096-read allowance at 4092 total rows; fresh CLI replacement leaves no scheduler intent, while replacement in place preserves later intent outside restored data; seven-day expiry, hosted dashboard restore, Workpool and Workflow remain open, and the owner confirmed D13 on these observations (Probe 7, F12, F16, F13) +- [non-blocking #schedulerObligationEvidence] Probe 7 observes a plain scheduled local reaction and all five states on the pinned native backend; a failed-function scan counts zero documents and bytes but exhausts the 4096-read allowance at 4092 total rows; fresh CLI replacement leaves no scheduler intent, while replacement in place preserves later intent outside restored data; seven-day expiry, hosted dashboard restore, Workpool and Workflow remain open, and the owner confirmed D13 on these observations (Probe 7, F12, F16, F13) ## Decision diff --git a/design/specs/obligations/fan-out-and-chains.sdp.md b/design/specs/obligations/fan-out-and-chains.sdp.md index 846a8a6..980f3b3 100644 --- a/design/specs/obligations/fan-out-and-chains.sdp.md +++ b/design/specs/obligations/fan-out-and-chains.sdp.md @@ -9,6 +9,8 @@ relations: - spec:obligations.record-contract - spec:command.command-pipeline - spec:command.actor-and-scope + - spec:command.command-declaration + - spec:context.event-envelope constrainedBy: - spec:facts.f13-transactions-have-limits - spec:facts.f08-scheduling-commits-with-mutation @@ -27,7 +29,7 @@ A command with a small static set of reactions creates their obligations in its ### Open questions -- [non-blocking] Extension E-54: the doc bounds neither static fan-out, nor the dynamic fan-out batch, nor the chain depth, and it says where a static subscription's obligations are created without saying where the subscription is declared; this design allows 16 obligations per publishing transaction, 500 derived obligations per fan-out attempt with one child fan-out obligation per further page, and a causation depth of 8, and it reads the static subscription as the `createObligation` calls in the command's executor body, reviewed at build time and counted at creation, rather than a reactions field on the command declaration, because D12 lists no such field and the declaration is Layer 1 machinery that imports nothing from this family; the namespace a derived command carries, `worker`, is E-6's ruling on `spec:command.actor-and-scope`, and the path by which its causation reaches the pipeline, the internal entry's `causedBy` argument, is E-31's on `spec:command.command-pipeline`; the owner confirms the numbers and the reading (D12, D13, F13, Thesis, E-6, E-31, E-54) +- [non-blocking #obligationFanOutBounds] Extension E-54: the doc bounds neither static fan-out, nor the dynamic fan-out batch, nor the chain depth, and it says where a static subscription's obligations are created without saying where the subscription is declared; this design allows 16 obligations per publishing transaction, 500 derived obligations per fan-out attempt with one child fan-out obligation per further page, and a causation depth of 8, and it reads the static subscription as the `createObligation` calls in the command's executor body, reviewed at build time and counted at creation, rather than a reactions field on the command declaration, because D12 lists no such field and the declaration is Layer 1 machinery that imports nothing from this family; the namespace a derived command carries, `worker`, is E-6's ruling on `spec:command.actor-and-scope`, and the path by which its causation reaches the pipeline, the internal entry's `causedBy` argument, is E-31's on `spec:command.command-pipeline`; the owner confirms the numbers and the reading (D12, D13, F13, Thesis, E-6, E-31, E-54) ## Rule diff --git a/design/specs/obligations/lifecycle-transitions.sdp.md b/design/specs/obligations/lifecycle-transitions.sdp.md index f8c9ddc..1720df8 100644 --- a/design/specs/obligations/lifecycle-transitions.sdp.md +++ b/design/specs/obligations/lifecycle-transitions.sdp.md @@ -22,8 +22,8 @@ D13 gives six states and their meaning. It does not list which transitions exist ### Open questions -- [non-blocking] Extension E-10: the doc names the six states but not the allowed transitions or their causes; this Spec lists them and refuses every other transition (D13, E-10) -- [non-blocking] Extension E-51: the doc says a known business rejection settles the business result but not which state it settles to; this design reads a body's business failure, which commits like a success, as succeeded with evidence naming its event, and a thrown rejection, which commits nothing, as cancelled with the settled result as `businessRejection` evidence, because cancelled means the effect will not happen and no call is in flight; the owner may prefer a seventh state (D13, D4, E-51) +- [non-blocking #obligationStateTransitions] Extension E-10: the doc names the six states but not the allowed transitions or their causes; this Spec lists them and refuses every other transition (D13, E-10) +- [non-blocking #obligationBusinessRejection] Extension E-51: the doc says a known business rejection settles the business result but not which state it settles to; this design reads a body's business failure, which commits like a success, as succeeded with evidence naming its event, and a thrown rejection, which commits nothing, as cancelled with the settled result as `businessRejection` evidence, because cancelled means the effect will not happen and no call is in flight; the owner may prefer a seventh state (D13, D4, E-51) ## Rule diff --git a/design/specs/obligations/local-reaction-wrapper.sdp.md b/design/specs/obligations/local-reaction-wrapper.sdp.md index 529aace..7205ca5 100644 --- a/design/specs/obligations/local-reaction-wrapper.sdp.md +++ b/design/specs/obligations/local-reaction-wrapper.sdp.md @@ -9,6 +9,9 @@ relations: - spec:obligations.record-contract - spec:obligations.lifecycle-transitions - spec:command.outcome-boundary + - spec:application.write-pause + - spec:command.actor-and-scope + - spec:obligations.fan-out-and-chains constrainedBy: - spec:facts.f03-nested-run-mutation-partial-rollback - spec:facts.f08-scheduling-commits-with-mutation @@ -31,7 +34,7 @@ The wrapper is the scheduled mutation that carries out one attempt of a local re ### Open questions -- [non-blocking] Extension E-51: the doc does not give the backoff schedule, the default attempt budget, the default deadline or what a maintenance refusal does to the attempt count; this design uses exponential backoff from 1 second with base 2 capped at 1 hour, 8 attempts by default and no deadline by default, all per handler in the registry, and treats a transient `writePaused` or `capacity` refusal as a delay of the same attempt after `retryAfterMs` or 60 seconds; every number is a provisional default until the first experiment (D13, E-51, Decision method rule 4) +- [non-blocking #obligationAttemptBackoff] Extension E-51: the doc does not give the backoff schedule, the default attempt budget, the default deadline or what a maintenance refusal does to the attempt count; this design uses exponential backoff from 1 second with base 2 capped at 1 hour, 8 attempts by default and no deadline by default, all per handler in the registry, and treats a transient `writePaused` or `capacity` refusal as a delay of the same attempt after `retryAfterMs` or 60 seconds; every number is a provisional default until the first experiment (D13, E-51, Decision method rule 4) ## Contract diff --git a/design/specs/obligations/obligation-module.sdp.md b/design/specs/obligations/obligation-module.sdp.md index 5e08552..90de3a3 100644 --- a/design/specs/obligations/obligation-module.sdp.md +++ b/design/specs/obligations/obligation-module.sdp.md @@ -50,10 +50,10 @@ The module lives in the parent as a shared library and two parent tables, becaus ### Open questions -- [non-blocking] Probe 7 observes local scheduler states, short retention, both CLI replacement modes, scheduler-ID validation and the failed-function scan boundary; a stored `v.id("_scheduled_functions")` can validate after fresh replacement while resolving to no row, so it is a typed reference and not proof of a live dispatch; seven-day expiry, hosted restore, Workpool and Workflow remain open, and the owner confirmed the do-nothing check's rejection on these observations (Probe 7, D13, F12, F16) -- [non-blocking] Extension E-50: the doc does not say where the module lives; this design places it in the parent as a library plus the `obligations` and `obligationRepairs` tables with a static handler registry keyed by handler key and version, because the wrapper runs parent-side bodies as nested mutations and the record reuses the parent's actor and scope types; the alternative, a Convex component driven by function handles like Workpool, is not pre-empted (D13, D11, E-50) -- [non-blocking] OQ1 dependency: OQ1 is answered with a component for every context, so a local reaction's body runs as a nested mutation; if a measured workload reopens it, only the bodies change shape (OQ1, Probe 3) -- [non-blocking] Decision method rule 4: the doc says Layers 0 to 2 get full detail, the first experiment runs, and Layer 3 is written from what it showed; this module and its children are written now, at the depth D13 to D15 give, with numeric defaults the doc does not give, the backoff and attempt budget of E-51, the sweeper's cadence and bounds of E-52, the fan-out bounds of E-54, the retention horizons of E-55, the lease and settle repeats of E-58 and the index set of E-57; every such number is a provisional default until the experiment's measurements and Probe 7 confirm or replace it, and the owner confirms that writing the structure now does not pre-empt the experiment (Decision method rule 4, D13, Probe 7, E-51, E-52, E-54, E-55, E-57, E-58) +- [non-blocking #obligationDispatchEvidence] Probe 7 observes local scheduler states, short retention, both CLI replacement modes, scheduler-ID validation and the failed-function scan boundary; a stored `v.id("_scheduled_functions")` can validate after fresh replacement while resolving to no row, so it is a typed reference and not proof of a live dispatch; seven-day expiry, hosted restore, Workpool and Workflow remain open, and the owner confirmed the do-nothing check's rejection on these observations (Probe 7, D13, F12, F16) +- [non-blocking #parentObligationPlacement] Extension E-50: the doc does not say where the module lives; this design places it in the parent as a library plus the `obligations` and `obligationRepairs` tables with a static handler registry keyed by handler key and version, because the wrapper runs parent-side bodies as nested mutations and the record reuses the parent's actor and scope types; the alternative, a Convex component driven by function handles like Workpool, is not pre-empted (D13, D11, E-50) +- [non-blocking #contextReactionIsolation] OQ1 dependency: OQ1 is answered with a component for every context, so a local reaction's body runs as a nested mutation; if a measured workload reopens it, only the bodies change shape (OQ1, Probe 3) +- [non-blocking #obligationDefaultBounds] Decision method rule 4: the doc says Layers 0 to 2 get full detail, the first experiment runs, and Layer 3 is written from what it showed; this module and its children are written now, at the depth D13 to D15 give, with numeric defaults the doc does not give, the backoff and attempt budget of E-51, the sweeper's cadence and bounds of E-52, the fan-out bounds of E-54, the retention horizons of E-55, the lease and settle repeats of E-58 and the index set of E-57; every such number is a provisional default until the experiment's measurements and Probe 7 confirm or replace it, and the owner confirms that writing the structure now does not pre-empt the experiment (Decision method rule 4, D13, Probe 7, E-51, E-52, E-54, E-55, E-57, E-58) ## Behavior diff --git a/design/specs/obligations/operator-operations.sdp.md b/design/specs/obligations/operator-operations.sdp.md index 98d7aaa..bdcda88 100644 --- a/design/specs/obligations/operator-operations.sdp.md +++ b/design/specs/obligations/operator-operations.sdp.md @@ -9,6 +9,7 @@ relations: - spec:obligations.record-contract - spec:obligations.lifecycle-transitions - spec:command.tenancy-and-authority + - spec:effects.claim-call-settle constrainedBy: - spec:laws.law05-authorization-before-execution-and-disclosure - spec:laws.law08-durable-capability-ships-operations @@ -29,7 +30,7 @@ The module ships five scoped operations: inspect, retry, reconcile, cancel and a ### Open questions -- [non-blocking] Extension E-53: the doc names the five operations and the repair record's fields but not their signatures or their transaction shape; this design makes them authorized parent mutations and queries that write the repair in the same transaction, refuses cancel and reconcile while a lease is live, and lets reconcile either mint a reconcile attempt that the claim mutation of `spec:effects.claim-call-settle` dispatches, or accept supplied provider evidence from needs attention (D13, D14, E-53) +- [non-blocking #operatorObligationRepair] Extension E-53: the doc names the five operations and the repair record's fields but not their signatures or their transaction shape; this design makes them authorized parent mutations and queries that write the repair in the same transaction, refuses cancel and reconcile while a lease is live, and lets reconcile either mint a reconcile attempt that the claim mutation of `spec:effects.claim-call-settle` dispatches, or accept supplied provider evidence from needs attention (D13, D14, E-53) ## Contract diff --git a/design/specs/obligations/record-contract.sdp.md b/design/specs/obligations/record-contract.sdp.md index f040a76..3181a57 100644 --- a/design/specs/obligations/record-contract.sdp.md +++ b/design/specs/obligations/record-contract.sdp.md @@ -25,8 +25,8 @@ The `obligations` table is the one record of promised deferred work, and `obliga ### Open questions -- [non-blocking] Extension E-10: the doc lists the fields and the six states but not the concrete table, the index set, the payload bound or the evidence shapes; this Spec pins them, with the inline payload bounded at 64 KiB and completion evidence as a closed union; the doc names four meanings of needs attention, exhausted, uncertain, authority revoked and unsupported version, and this design adds two reasons the doc describes without naming, `recoveryFailing` for the sweeper's escalation when recovery itself keeps failing and `chainBound` for a reaction refused at the chain's depth bound, which the owner confirms (D13, E-10, E-52, E-54) -- [non-blocking] Extension E-57: the sweeper and the retention batch are deployment-wide, so `by_status_next_attempt`, `by_status_lease_expiry` and `by_status_settled_at` lead with `status` rather than `tenantId`; every index a tenant-scoped query reads still leads with `tenantId`, and tenant fairness in the sweeper's scan order stays a policy to add when one tenant can starve another (D13, Law 11, E-57) +- [non-blocking #obligationCompletionEvidence] Extension E-10: the doc lists the fields and the six states but not the concrete table, the index set, the payload bound or the evidence shapes; this Spec pins them, with the inline payload bounded at 64 KiB and completion evidence as a closed union; the doc names four meanings of needs attention, exhausted, uncertain, authority revoked and unsupported version, and this design adds two reasons the doc describes without naming, `recoveryFailing` for the sweeper's escalation when recovery itself keeps failing and `chainBound` for a reaction refused at the chain's depth bound, which the owner confirms (D13, E-10, E-52, E-54) +- [non-blocking #obligationTenantIndexes] Extension E-57: the sweeper and the retention batch are deployment-wide, so `by_status_next_attempt`, `by_status_lease_expiry` and `by_status_settled_at` lead with `status` rather than `tenantId`; every index a tenant-scoped query reads still leads with `tenantId`, and tenant fairness in the sweeper's scan order stays a policy to add when one tenant can starve another (D13, Law 11, E-57) ## Contract diff --git a/design/specs/obligations/retention-and-restore.sdp.md b/design/specs/obligations/retention-and-restore.sdp.md index 194b773..ccc4de0 100644 --- a/design/specs/obligations/retention-and-restore.sdp.md +++ b/design/specs/obligations/retention-and-restore.sdp.md @@ -10,6 +10,7 @@ relations: - spec:obligations.sweeper - spec:effects.claim-call-settle - spec:operations.baseline-operations + - spec:application.restore decidedBy: spec:decisions.d19-operations-travel-with-capability constrainedBy: - spec:facts.f12-backups-exclude-pending-scheduled-functions @@ -30,9 +31,9 @@ Retention and restore are the two operations that make the obligation table earn ### Open questions -- [non-blocking] Extension E-55: the doc gives no retention horizon, compaction rule or batch bound; this design keeps a completed obligation for the longest repetition policy validity in the module and never less than 30 days, compacts `lastError.message` and `lateEvidence` after 7 days into a count, and deletes at most 500 rows per retention run (D19, F13, E-55) -- [non-blocking] Extension E-56: the doc says restore starts with dispatch off but not how the switch works; this design reads a deployment environment variable in the parent's claim mutation only, where a call-mode claim defers its attempt without counting it and a reconcile-mode claim proceeds, sets it before the restore, adds a cursor-driven `rebuildSchedules` operation for a backlog larger than the sweeper's batch, and orders the two switches: `OBLIGATIONS_DISPATCH` goes off before the backup archive is restored and on last, and `rebuildSchedules` refuses to run while `MAINTENANCE_MODE` is `restore`, so it runs only after `spec:application.restore` has accepted the run and reopened writers (D19, F12, E-42, E-56) -- [non-blocking] Probe 7 pending: what a restore leaves of Workpool and Workflow state decides whether the rebuild covers only the module's own dispatches or also passed-ownership work (Probe 7, F12, D19) +- [non-blocking #obligationRetentionBounds] Extension E-55: the doc gives no retention horizon, compaction rule or batch bound; this design keeps a completed obligation for the longest repetition policy validity in the module and never less than 30 days, compacts `lastError.message` and `lateEvidence` after 7 days into a count, and deletes at most 500 rows per retention run (D19, F13, E-55) +- [non-blocking #restoreDispatchSwitch] Extension E-56: the doc says restore starts with dispatch off but not how the switch works; this design reads a deployment environment variable in the parent's claim mutation only, where a call-mode claim defers its attempt without counting it and a reconcile-mode claim proceeds, sets it before the restore, adds a cursor-driven `rebuildSchedules` operation for a backlog larger than the sweeper's batch, and orders the two switches: `OBLIGATIONS_DISPATCH` goes off before the backup archive is restored and on last, and `rebuildSchedules` refuses to run while `MAINTENANCE_MODE` is `restore`, so it runs only after `spec:application.restore` has accepted the run and reopened writers (D19, F12, E-42, E-56) +- [non-blocking #restoreDeferredWorkCoverage] Probe 7 pending: what a restore leaves of Workpool and Workflow state decides whether the rebuild covers only the module's own dispatches or also passed-ownership work (Probe 7, F12, D19) ## Rule diff --git a/design/specs/obligations/sweeper.sdp.md b/design/specs/obligations/sweeper.sdp.md index 699579d..b385a96 100644 --- a/design/specs/obligations/sweeper.sdp.md +++ b/design/specs/obligations/sweeper.sdp.md @@ -8,6 +8,7 @@ relations: dependsOn: - spec:obligations.record-contract - spec:obligations.lifecycle-transitions + - spec:effects.claim-call-settle constrainedBy: - spec:facts.f16-scheduled-functions-table-shows-failed-runs - spec:facts.f13-transactions-have-limits @@ -33,9 +34,9 @@ One run costs one nested mutation per recovered candidate and per expired lease, ### Open questions -- [non-blocking] Extension E-52: the doc says the sweeper is batched and bounded but gives no cadence, batch bound, grace period, rearm bound or time guard; this design runs every 60 seconds, examines at most 100 candidates and 100 expired leases per scan, stops a run after 500 ms of elapsed time by `performance.now()`, waits a grace of 120 seconds past the next attempt time, allows 5 rearms, and isolates each candidate in a nested mutation; the numbers are tested against the pinned Convex version (D13, D19, E-52) -- [non-blocking] Probe 3 pending: the candidate bound of 100 is provisional; it assumes a nested `ctx.runMutation` costs a few milliseconds, which F4 records as unknown, so the bound is reset from the probe's measured cost so that a full scan of 200 nested calls fits inside the elapsed guard, and until then the guard rather than the count is what keeps a run under the mutation timeout (Probe 3, F4, F13, E-52) -- [non-blocking] Extension E-57: the sweeper scans deployment-wide by status; tenant fairness in its order is a policy to add when one tenant can starve another (D13, E-57) +- [non-blocking #obligationSweepBounds] Extension E-52: the doc says the sweeper is batched and bounded but gives no cadence, batch bound, grace period, rearm bound or time guard; this design runs every 60 seconds, examines at most 100 candidates and 100 expired leases per scan, stops a run after 500 ms of elapsed time by `performance.now()`, waits a grace of 120 seconds past the next attempt time, allows 5 rearms, and isolates each candidate in a nested mutation; the numbers are tested against the pinned Convex version (D13, D19, E-52) +- [non-blocking #sweepContextCallCost] Probe 3 pending: the candidate bound of 100 is provisional; it assumes a nested `ctx.runMutation` costs a few milliseconds, which F4 records as unknown, so the bound is reset from the probe's measured cost so that a full scan of 200 nested calls fits inside the elapsed guard, and until then the guard rather than the count is what keeps a run under the mutation timeout (Probe 3, F4, F13, E-52) +- [non-blocking #tenantSweepFairness] Extension E-57: the sweeper scans deployment-wide by status; tenant fairness in its order is a policy to add when one tenant can starve another (D13, E-57) ## Contract diff --git a/design/specs/operations/baseline-operations.sdp.md b/design/specs/operations/baseline-operations.sdp.md index 044d595..0130af6 100644 --- a/design/specs/operations/baseline-operations.sdp.md +++ b/design/specs/operations/baseline-operations.sdp.md @@ -18,6 +18,10 @@ relations: - spec:constraints.bulk-operations-bounded - spec:laws.law05-authorization-before-execution-and-disclosure - spec:facts.f01-serializable-mutations-under-occ + dependsOn: + - spec:command.actor-and-scope + - spec:command.outcome-boundary + - spec:context.event-envelope --- # Baseline operations @@ -39,8 +43,8 @@ Security, bounded resource use, useful errors and tests start in Layer 1, so non ### Open questions -- [non-blocking] Extension E-43: the doc says audit is written inside the transaction and fails closed, that diagnostics hold no raw prompts, credentials or unbounded payloads, and that personal data sits behind references; it does not give the audit record, the diagnostic record or the reference. The option taken here: an `auditRecords` table with indexes by operation, subject and request key, whose every field has a stated source in the command, an `operatorAudit` table for the gate changes an operator entry makes, which name no tenant when the scope is the whole deployment, a bounded diagnostic line for every execution of a command entry's handler, whatever it ends in, written through a sink the declaration may name by a function that never throws, with no duration, because the backend records the time of each execution, and a `personalData` table that events point at by reference. The do-nothing option is Convex's own audit logging, which the pinned `convex` package declares as `audit(body)`; it is not taken as the record, because an entry in a log outside the database is not read by the operator's queries by operation, subject and request key, and whether such an entry proves a commit is not shown by the pinned package; a deployment may add it beside the tables as an export (E-43, D19) -- [non-blocking] The retention and deletion policy's content is the owner's: which historical claims survive deletion and how long receipts, audit records and diagnostics are kept; the design gives the hooks and states no periods (D19) +- [non-blocking #commandAuditDiagnostics] Extension E-43: the doc says audit is written inside the transaction and fails closed, that diagnostics hold no raw prompts, credentials or unbounded payloads, and that personal data sits behind references; it does not give the audit record, the diagnostic record or the reference. The option taken here: an `auditRecords` table with indexes by operation, subject and request key, whose every field has a stated source in the command, an `operatorAudit` table for the gate changes an operator entry makes, which name no tenant when the scope is the whole deployment, a bounded diagnostic line for every execution of a command entry's handler, whatever it ends in, written through a sink the declaration may name by a function that never throws, with no duration, because the backend records the time of each execution, and a `personalData` table that events point at by reference. The do-nothing option is Convex's own audit logging, which the pinned `convex` package declares as `audit(body)`; it is not taken as the record, because an entry in a log outside the database is not read by the operator's queries by operation, subject and request key, and whether such an entry proves a commit is not shown by the pinned package; a deployment may add it beside the tables as an export (E-43, D19) +- [non-blocking #historicalClaimRetention] The retention and deletion policy's content is the owner's: which historical claims survive deletion and how long receipts, audit records and diagnostics are kept; the design gives the hooks and states no periods (D19) ## Rule diff --git a/design/specs/platform/acceptance-contract.sdp.md b/design/specs/platform/acceptance-contract.sdp.md index 53ac1ce..289709e 100644 --- a/design/specs/platform/acceptance-contract.sdp.md +++ b/design/specs/platform/acceptance-contract.sdp.md @@ -21,8 +21,8 @@ The doc's acceptance table is the contract. It lists forty-three scenario rows b ### Open questions -- [non-blocking] Extension E-14: the doc's four tiers, domain, simulator, native and end to end, name no compile-time check, while Sc L2-4's pass condition is a build, type or registration check that fails before any traffic; the option taken here is the compiled tier, a compile with no backend of a typed caller that states the change, which asserts that `tsc` refuses it, run with every type check and recorded like any run, and this corpus reads the doc's end to end tier as the native backend tier on the production composition; the alternative, folding it into the native backend tier as a compile step that precedes the native run, was not taken because a native run needs a backend and the check needs none; the owner confirms (E-14, Sc L2-4, Acceptance scenarios) -- [non-blocking] Extension E-17: the doc says that a link between source and test is not a passing run and that the first experiment passes when every Layer 0, 1 and 2 scenario passes, and it does not say how a pass of the whole is established from runs; the option taken here is the acceptance check: the required scenarios are generated from the graph, one per example of a required row with its tier, composition and verifiers, and read against one run's record, so that a scenario with no verifier, with no result or with a result other than passed keeps the whole from passing; the alternative, a list of scenarios and verifiers kept by hand beside the tests, was not taken because the graph already holds each example's row, tier, composition and verifiers; the owner confirms (E-17, Acceptance scenarios, First experiment) +- [non-blocking #compiledAcceptanceTier] Extension E-14: the doc's four tiers, domain, simulator, native and end to end, name no compile-time check, while Sc L2-4's pass condition is a build, type or registration check that fails before any traffic; the option taken here is the compiled tier, a compile with no backend of a typed caller that states the change, which asserts that `tsc` refuses it, run with every type check and recorded like any run, and this corpus reads the doc's end to end tier as the native backend tier on the production composition; the alternative, folding it into the native backend tier as a compile step that precedes the native run, was not taken because a native run needs a backend and the check needs none; the owner confirms (E-14, Sc L2-4, Acceptance scenarios) +- [non-blocking #scenarioAcceptanceCheck] Extension E-17: the doc says that a link between source and test is not a passing run and that the first experiment passes when every Layer 0, 1 and 2 scenario passes, and it does not say how a pass of the whole is established from runs; the option taken here is the acceptance check: the required scenarios are generated from the graph, one per example of a required row with its tier, composition and verifiers, and read against one run's record, so that a scenario with no verifier, with no result or with a result other than passed keeps the whole from passing; the alternative, a list of scenarios and verifiers kept by hand beside the tests, was not taken because the graph already holds each example's row, tier, composition and verifiers; the owner confirms (E-17, Acceptance scenarios, First experiment) ## Rule diff --git a/design/specs/platform/native-harness.sdp.md b/design/specs/platform/native-harness.sdp.md index 666dc86..acc0b58 100644 --- a/design/specs/platform/native-harness.sdp.md +++ b/design/specs/platform/native-harness.sdp.md @@ -21,10 +21,10 @@ The doc says that native tests prove component and nested-mutation behavior, sch ### Open questions -- [non-blocking] Extension E-13: the doc says native acceptance runs with production authority and a disposable backend per test and does not say how a test obtains a signed identity, which a production issuer cannot mint for a throwaway backend; the option taken here is a fixture issuer: a composition that runs natively reads its issuer, application ID and key set from the deployment's environment variables, the harness sets the key set to a data URI whose private key it holds, and each caller under test is an ordinary client carrying a token the harness signs; the option first taken, the backend's admin key acting as an identity, was dropped because such a caller passes Convex's visibility check for internal functions, which `spec:platform.native-harness.admin-key-acting-as-identity-reaches-internal-function` shows; this reading reaches into production, because running the production composition natively with no changed file means the example application's `auth.config.ts` is a `customJwt` provider whose trust root is whatever the deployment's environment names, and a provider that fetches its keys from an issuer's own endpoint cannot be run on a disposable backend this way; the owner rules on both halves, the test identity and the production provider (E-13, S13, Sc L2-9, D11) -- [non-blocking] Extension E-15: the doc does not say what the fixture app contains, how a fault enters a run, what a test may observe, or what a test controls on a backend that offers no clock and no scheduler switch; the option taken here is the rules below; two of them read the doc for the owner: the production-configuration run of Sc L2-9 is taken as the scenarios routed to the production composition, where the corpus earlier asked for the full Layer 0 to 2 suite on a release build, and evidence records that back a pass claim are kept in the repository; whether the durations Layer 3 pins become configuration, so that a test can outwait them, is left to Layer 3; and because the Convex CLI reads `convex.json` and `package.json` from its working directory, a composition is a name, a project directory and a functions directory, the production composition's project directory is `example/` with a `package.json` that links to the repository's one, and both compositions' generated files are committed; the owner has not ruled on the layout (E-15, Acceptance scenarios, Sc L2-9) -- [non-blocking] Extension E-15: the design takes the fixture action `nonUiCaller:send`, which any authenticated client of the fixture composition may call to run any fixture command's internal entry under any namespace and actor kind, as the fixture's stand-in for trusted server code, kept out of the production composition; it is a provisional reading the owner rules, because within the fixture composition it lets a client do what `spec:command.tenancy-and-authority` refuses a client (E-15, D11, Sc L1-7) -- [non-blocking] Extension E-59: the doc's native tier is a test on a real backend, S13 names a local one, and Sc L2-9 is read as the local observation; the doc does not say whether a native test runs on a hosted deployment, where alone Probe 3 can read function-call quota and where F12 and F16 keep parts that only documentation states; the option taken here: the same harness, run record and compositions run natively on one hosted development deployment in a Convex team and project of their own, selected by process variables under a test configuration of its own, one run at a time, from a workflow whose failure fails no other check, with one development deploy key limited to the actions `deployKeyActions` lists; the alternatives were each developer's own deployment, which still needs one more key for the continuous integration, a preview deployment per run, which on the Free and Starter plans expires by default five days after it is created, before F16's seven-day window, and a fresh development deployment per run, which needs a project key, a credential the docs give total control over a project; the team, the project, the plan and the key are the owner's to provide (E-59, Sc L2-9, F4, F12, F16, Probe 3, Probe 7) +- [non-blocking #nativeCallerIdentity] Extension E-13: the doc says native acceptance runs with production authority and a disposable backend per test and does not say how a test obtains a signed identity, which a production issuer cannot mint for a throwaway backend; the option taken here is a fixture issuer: a composition that runs natively reads its issuer, application ID and key set from the deployment's environment variables, the harness sets the key set to a data URI whose private key it holds, and each caller under test is an ordinary client carrying a token the harness signs; the option first taken, the backend's admin key acting as an identity, was dropped because such a caller passes Convex's visibility check for internal functions, which `spec:platform.native-harness.admin-key-acting-as-identity-reaches-internal-function` shows; this reading reaches into production, because running the production composition natively with no changed file means the example application's `auth.config.ts` is a `customJwt` provider whose trust root is whatever the deployment's environment names, and a provider that fetches its keys from an issuer's own endpoint cannot be run on a disposable backend this way; the owner rules on both halves, the test identity and the production provider (E-13, S13, Sc L2-9, D11) +- [non-blocking #fixtureCompositionContract] Extension E-15: the doc does not say what the fixture app contains, how a fault enters a run, what a test may observe, or what a test controls on a backend that offers no clock and no scheduler switch; the option taken here is the rules below; two of them read the doc for the owner: the production-configuration run of Sc L2-9 is taken as the scenarios routed to the production composition, where the corpus earlier asked for the full Layer 0 to 2 suite on a release build, and evidence records that back a pass claim are kept in the repository; whether the durations Layer 3 pins become configuration, so that a test can outwait them, is left to Layer 3; and because the Convex CLI reads `convex.json` and `package.json` from its working directory, a composition is a name, a project directory and a functions directory, the production composition's project directory is `example/` with a `package.json` that links to the repository's one, and both compositions' generated files are committed; the owner has not ruled on the layout (E-15, Acceptance scenarios, Sc L2-9) +- [non-blocking #fixtureCallerNamespace] Extension E-15: the design takes the fixture action `nonUiCaller:send`, which any authenticated client of the fixture composition may call to run any fixture command's internal entry under any namespace and actor kind, as the fixture's stand-in for trusted server code, kept out of the production composition; it is a provisional reading the owner rules, because within the fixture composition it lets a client do what `spec:command.tenancy-and-authority` refuses a client (E-15, D11, Sc L1-7) +- [non-blocking #hostedNativeTarget] Extension E-59: the doc's native tier is a test on a real backend, S13 names a local one, and Sc L2-9 is read as the local observation; the doc does not say whether a native test runs on a hosted deployment, where alone Probe 3 can read function-call quota and where F12 and F16 keep parts that only documentation states; the option taken here: the same harness, run record and compositions run natively on one hosted development deployment in a Convex team and project of their own, selected by process variables under a test configuration of its own, one run at a time, from a workflow whose failure fails no other check, with one development deploy key limited to the actions `deployKeyActions` lists; the alternatives were each developer's own deployment, which still needs one more key for the continuous integration, a preview deployment per run, which on the Free and Starter plans expires by default five days after it is created, before F16's seven-day window, and a fresh development deployment per run, which needs a project key, a credential the docs give total control over a project; the team, the project, the plan and the key are the owner's to provide (E-59, Sc L2-9, F4, F12, F16, Probe 3, Probe 7) ## Rule diff --git a/design/specs/platform/transactional-domain-platform.sdp.md b/design/specs/platform/transactional-domain-platform.sdp.md index 825c253..d1eba31 100644 --- a/design/specs/platform/transactional-domain-platform.sdp.md +++ b/design/specs/platform/transactional-domain-platform.sdp.md @@ -41,7 +41,7 @@ Citations in this corpus use the tokens the plan fixes: D for a decision, Law fo ### Open questions -- [non-blocking] OQ5 asks whether the decisions move into SDP carriers; this corpus is that move, with ledger facts as constraint Specs, decisions as decision Specs that list the do-nothing option first, and scenarios as example Specs under the component that provides the capability, which is the form the doc's behavior rules with examples take here; the owner confirmed the mapping (OQ5) +- [non-blocking #decisionSpecMapping] OQ5 asks whether the decisions move into SDP carriers; this corpus is that move, with ledger facts as constraint Specs, decisions as decision Specs that list the do-nothing option first, and scenarios as example Specs under the component that provides the capability, which is the form the doc's behavior rules with examples take here; the owner confirmed the mapping (OQ5) ## Behavior diff --git a/design/specs/platform/vocabulary.sdp.md b/design/specs/platform/vocabulary.sdp.md index 939306d..dd124c5 100644 --- a/design/specs/platform/vocabulary.sdp.md +++ b/design/specs/platform/vocabulary.sdp.md @@ -22,8 +22,8 @@ The two clashes are answered: a row that proves an effect happened is completion ### Open questions -- [non-blocking] OQ6, receipt: this design uses receipt for the stored outcome of a command for idempotency, while today's glossary uses receipt for a row that proves an effect happened; the owner answered OQ6 with receipt for the stored outcome of a command and completion evidence for the row that proves an effect happened (OQ6, Vocabulary) -- [non-blocking] OQ6, generation: this design uses generation for a numbered build of a read model, while today's glossary uses it for the attempt number of watched work; the owner answered OQ6 with generation for a numbered build of a read model and attempt for one try at an obligation (OQ6, Vocabulary) +- [non-blocking #receiptCompletionEvidence] OQ6, receipt: this design uses receipt for the stored outcome of a command for idempotency, while today's glossary uses receipt for a row that proves an effect happened; the owner answered OQ6 with receipt for the stored outcome of a command and completion evidence for the row that proves an effect happened (OQ6, Vocabulary) +- [non-blocking #generationObligationAttempt] OQ6, generation: this design uses generation for a numbered build of a read model, while today's glossary uses it for the attempt number of watched work; the owner answered OQ6 with generation for a numbered build of a read model and attempt for one try at an obligation (OQ6, Vocabulary) ## Model diff --git a/design/specs/processes/approvals.sdp.md b/design/specs/processes/approvals.sdp.md index 79d784c..7fa6c09 100644 --- a/design/specs/processes/approvals.sdp.md +++ b/design/specs/processes/approvals.sdp.md @@ -19,7 +19,7 @@ An approval is a record that an authorized person approved one exact proposal: o ### Open questions -- [blocking] The approvals table, its indexes and the approval command's declaration are deferred to the build on the process trigger; the rules here are the promise the build must keep (D16, Decision method rule 4) +- [blocking #approvalCommandDeclaration] The approvals table, its indexes and the approval command's declaration are deferred to the build on the process trigger; the rules here are the promise the build must keep (D16, Decision method rule 4) ## Rule diff --git a/design/specs/processes/start-checkout-example.sdp.md b/design/specs/processes/start-checkout-example.sdp.md index db92ef0..2dea528 100644 --- a/design/specs/processes/start-checkout-example.sdp.md +++ b/design/specs/processes/start-checkout-example.sdp.md @@ -23,8 +23,8 @@ Layer 4 · Detail: deferred until the operation spans time, external systems or ### Open questions -- [blocking] The trigger has not fired; the workflow definition, the process record, the reservation's business expiry and the payment obligation's handler are deferred to the build until the operation spans time, external systems or human decisions (D16, Decision method rule 4) -- [non-blocking] Which of the three late-payment policies applies is a product decision the build records per deployment: reallocate, ask a person, or refund as a separately tracked operation (D16) +- [blocking #checkoutProcessDefinition] The trigger has not fired; the workflow definition, the process record, the reservation's business expiry and the payment obligation's handler are deferred to the build until the operation spans time, external systems or human decisions (D16, Decision method rule 4) +- [non-blocking #latePaymentPolicy] Which of the three late-payment policies applies is a product decision the build records per deployment: reallocate, ask a person, or refund as a separately tracked operation (D16) ## Workflow diff --git a/design/specs/processes/workflow-processes.sdp.md b/design/specs/processes/workflow-processes.sdp.md index e83a5fd..4cee26a 100644 --- a/design/specs/processes/workflow-processes.sdp.md +++ b/design/specs/processes/workflow-processes.sdp.md @@ -34,8 +34,8 @@ A process is a business operation that spans time, external systems or human dec ### Open questions -- [blocking] The trigger has not fired; the process record's table, the step signatures, the wait mechanism for an obligation's result and the deploy procedure are deferred to the build (D16, Decision method rule 4) -- [non-blocking] Whether an old run is kept, migrated or blocked on a given deploy is a per-release choice the build records; the README's determinism rule means keeping requires the old definition to stay registered under its version (D16, S11) +- [blocking #processObligationWait] The trigger has not fired; the process record's table, the step signatures, the wait mechanism for an obligation's result and the deploy procedure are deferred to the build (D16, Decision method rule 4) +- [non-blocking #processDefinitionRetention] Whether an old run is kept, migrated or blocked on a given deploy is a per-release choice the build records; the README's determinism rule means keeping requires the old definition to stay registered under its version (D16, S11) ## Behavior diff --git a/design/tools/check.py b/design/tools/check.py index 55c94f8..50e4a6a 100644 --- a/design/tools/check.py +++ b/design/tools/check.py @@ -11,7 +11,13 @@ The Protocol's CLI is the pinned dependency in the repository's package.json; run `npm ci` once. What `sdp validate` checks is the Protocol's. What this script adds is this project's policy: -citations carry edges, extensions are registered, nobody but the owner states `ready`. +citations carry edges, extensions are registered, nobody but the owner states `ready`, a prose +mention with no relation is listed with its reason, an entry address outside the Specs resolves +through the Protocol's address recipe, and no Specs rest on each other through `dependsOn`, as the +Protocol's cycle recipe reports. It prints the roles, layers and contexts the code anchors state, as +the Protocol's roles-and-layers recipe reads them, and judges none of them. One rule it does judge, +from the graph: no component in the context `platform` uses a component in another context, so the +platform's core never depends on the orders-inventory example (design/PLAN.md section 2.1, the context rule). """ import collections import glob @@ -22,16 +28,59 @@ import subprocess import sys +sys.dont_write_bytecode = True + +from decisions import graph_recipe, question_resolutions, question_source_errors + DESIGN = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) ROOT = os.path.dirname(DESIGN) PACKAGE = os.path.join(ROOT, "node_modules", "@libar-dev", "software-delivery-protocol") SDP = os.environ.get("SDP", os.path.join(PACKAGE, "dist", "cli", "sdp.js")) -PINNED = json.load(open(os.path.join(ROOT, "package.json"), encoding="utf-8"))["devDependencies"]["@libar-dev/software-delivery-protocol"] +with open(os.path.join(ROOT, "package.json"), encoding="utf-8") as package: + PINNED = json.load(package)["devDependencies"]["@libar-dev/software-delivery-protocol"] READINESS_DIVERGENCE = ( 'const rungs = ["idea", "scoped", "defined", "ready"];' " const rank = (r) => (r === undefined ? -1 : rungs.indexOf(r));" " return g.specs().filter((s) => rank(s.derivedReadiness) < rank(s.statedReadiness)).map((s) => s.id)" ) +GRAPH_FACTS = ( + 'return { mentions: report.findings.filter((f) => f.validatorId === "conformance/prose-mentions")' + ".map((f) => [f.subjectId, f.relatedId, f.path]) }" +) +PROSE_MENTIONS = os.path.join(DESIGN, "tools", "prose-mentions.json") +# The context rule of design/PLAN.md section 2.1: a component in the context `platform` has no `uses` edge to a +# component in another context. Each endpoint of a `uses` edge resolves to its component, directly or through +# its `memberOf` edge, before the two contexts are compared. The edges are the declared `uses`, not imports. +PLATFORM_ISOLATION = ( + "const byId = new Map(graph.nodes.map((n) => [n.id, n]));" + " const memberOf = new Map(graph.edges.filter((e) => e.type === \"memberOf\").map((e) => [e.from, e.to]));" + " const componentOf = (id) => (id.startsWith(\"component:\") ? id : memberOf.get(id));" + " const contextOf = (id) => (id === undefined ? undefined : byId.get(id)?.context);" + " const uses = graph.edges.filter((e) => e.type === \"uses\");" + " const crossings = uses.flatMap((e) => { const from = componentOf(e.from); const to = componentOf(e.to);" + " const context = contextOf(to);" + " return contextOf(from) === \"platform\" && context !== undefined && context !== \"platform\"" + " ? [{ from: e.from, to: e.to, fromComponent: from, toComponent: to, toContext: context }] : []; });" + " return { platformComponents: graph.nodes.filter((n) => n.id.startsWith(\"component:\") && n.context === \"platform\").length," + " platformUses: uses.filter((e) => contextOf(componentOf(e.from)) === \"platform\").length, crossings }" +) +# An entry address, spec:#design., #ui. or #question.. A token runs from `spec:` to a +# separator, loses trailing punctuation, and goes whole to the Protocol's address recipe (recipe 25), so a +# malformed key is refused rather than read as its valid prefix. The recipe holds the grammar and the +# lookup; finding the tokens and saying where each stands is this script's. Historical positional +# citations in prose, spec:#, are kept as written and not sent. +MENTION_TOKEN = re.compile(r"spec:[^\s`'\"()\[\]{}<>|,;]+") +ADDRESS_SECTION = re.compile(r"#(design|ui|question)\b") +# The recipe's reasons for an address that does not resolve, as this script words them. +ADDRESS_REASONS = { + "malformed": "is outside the address grammar", + "spec": "names no Spec", + "entry": "names no entry of its Spec", +} +# Where an address outside the Specs lives: the code, the tests, the register, the ledger and the work documents. +# The Protocol checks an address inside a Spec's prose; nothing checks these unless this script does. +ADDRESS_SOURCES = ("src", "fixture", "example", "harness", "tests", "scripts") +ADDRESS_RECORDS = ("decisions/register.json", "reviews/consensus-ledger.json", "STATE.md", "ROADMAP.md", "README.md", "PLAN.md", "SESSIONS.md") TYPED_SECTIONS = {"Behavior", "Workflow", "Rule", "Contract", "Design", "Decision", "Constraints", "Constraint", "Model"} LEDGER_STATUSES = {"open", "fixed", "partially-fixed", "rejected", "owner"} @@ -46,131 +95,269 @@ def run(args): return subprocess.run(["node", SDP] + args, capture_output=True, text=True, cwd=ROOT) -if "--slice" in sys.argv: - wanted = sys.argv[sys.argv.index("--slice") + 1] - for item in json.load(open(os.path.join(DESIGN, "reviews", "consensus-ledger.json"), encoding="utf-8"))["ledger"]: - if item.get("slice") == wanted and item["status"] in ("open", "partially-fixed", "owner"): - print(f"{item['id']} | {item['severity']} | {item['status']} | caught by: {item.get('detector', '?')} | {', '.join(item['files'])}") - sys.exit(0) - - -# 1. The graph: validate and readiness divergence. -out = run(["validate", "."]) -text = out.stdout + out.stderr -graph = re.search(r"(\d+) specs · (\d+) packs · (\d+) anchors → (\d+) nodes · (\d+) edges", text) -verdict = re.search(r"validate: (\d+) errors · (\d+) warnings", text) -if not graph or not verdict: - fail("validate printed no summary line; run `npm ci` at the repository root, then check " + SDP) - print("\n".join(failures)) - sys.exit(1) -n_specs, n_packs, n_anchors, n_nodes, n_edges = (int(x) for x in graph.groups()) -n_errors, n_warnings = (int(x) for x in verdict.groups()) -warnings = [l for l in text.splitlines() if "[warning]" in l] -if n_errors: - fail(f"validate: {n_errors} errors") -if n_warnings: - fail(f"validate: {n_warnings} warnings, first: {(warnings or ['not printed'])[0][:200]}") - -out = run(["q", READINESS_DIVERGENCE, "--root", ".", "--json"]) -try: - diverging = json.loads(out.stdout.strip().splitlines()[-1]) -except (ValueError, IndexError): - diverging = None -if diverging is None: - fail("readiness divergence query did not return JSON") -elif diverging: - fail(f"stated readiness not earned by: {diverging}") - -# 2. Read the Specs. -kinds = collections.Counter() -readiness = collections.Counter() -questions = {} -cited_extensions = set() -edge_gaps = [] -for path in sorted(glob.glob(os.path.join(DESIGN, "specs", "**", "*.sdp.md"), recursive=True)): - if path.endswith(".pack.sdp.md"): - continue - raw = open(path, encoding="utf-8").read() - front, body = raw.split("\n---\n", 1) - sid = re.search(r"^id: (\S+)", front, re.M).group(1) - kind = re.search(r"^kind: (\w+)", front, re.M).group(1) - kinds[kind] += 1 - stated = re.search(r"^readiness: (\w+)", front, re.M).group(1) - readiness[stated] += 1 - if stated == "ready": - fail(f"{sid} states ready; only the owner states it, and STATE.md must record the ruling") - cited_extensions |= set(re.findall(r"\bE-(\d+)\b", raw)) - block = re.search(r"constrainedBy:(.*?)(?=\n [a-zA-Z]+:|\Z)", front, re.S) - block = block.group(1) if block else "" - linked_laws = {int(x) for x in re.findall(r"laws\.law(\d+)", block)} - linked_facts = {int(x) for x in re.findall(r"facts\.f(\d+)", block)} - section, cited_laws, cited_facts = None, set(), set() - for line in body.split("\n"): - heading = re.match(r"^#{2,3} (\S+)", line) - if heading: - section = heading.group(1) +def address_tokens(paths, base=ROOT): + """Each entry address written in the files, as (token, where), in file and line order.""" + found = [] + for path in sorted(paths): + with open(path, encoding="utf-8") as lines: + for number, line in enumerate(lines, 1): + for token in MENTION_TOKEN.findall(line): + token = token.rstrip(".:!?") + if ADDRESS_SECTION.search(token): + found.append((token, f"{os.path.relpath(path, base)}:{number}")) + return found + + +def address_failures(found, rows): + """The tokens recipe 25 does not resolve. `rows` are its answer, one row per token in input order.""" + if len(rows) != len(found): + return [f"the address recipe answered {len(rows)} rows for {len(found)} entry addresses"] + return [f"entry address {token} at {where} {ADDRESS_REASONS.get(row['reason'], 'does not resolve: ' + repr(row['reason']))}" + for (token, where), row in zip(found, rows) if not row["resolves"]] + + +def cycle_failures(answer): + """Each set of Specs recipe 26 finds resting on each other through dependsOn, with its witness cycle.""" + return [f"dependsOn cycle among {', '.join(entry['members'])}: {' -> '.join(entry['cycle'])}" + for entry in answer["cycles"]] + + +def main(): + if "--slice" in sys.argv: + wanted = sys.argv[sys.argv.index("--slice") + 1] + for item in json.load(open(os.path.join(DESIGN, "reviews", "consensus-ledger.json"), encoding="utf-8"))["ledger"]: + if item.get("slice") == wanted and item["status"] in ("open", "partially-fixed", "owner"): + print(f"{item['id']} | {item['severity']} | {item['status']} | caught by: {item.get('detector', '?')} | {', '.join(item['files'])}") + sys.exit(0) + + + # 1. The graph: validate and readiness divergence. + out = run(["validate", "."]) + text = out.stdout + out.stderr + graph = re.search(r"(\d+) specs · (\d+) packs · (\d+) anchors → (\d+) nodes · (\d+) edges", text) + verdict = re.search(r"validate: (\d+) errors · (\d+) warnings", text) + if not graph or not verdict: + fail("validate printed no summary line; run `npm ci` at the repository root, then check " + SDP) + print("\n".join(failures)) + sys.exit(1) + n_specs, n_packs, n_anchors, n_nodes, n_edges = (int(x) for x in graph.groups()) + n_errors, n_warnings = (int(x) for x in verdict.groups()) + warnings = [l for l in text.splitlines() if "[warning]" in l and "conformance/prose-mentions" not in l] + if n_errors: + fail(f"validate: {n_errors} errors") + + out = run(["q", GRAPH_FACTS, "--root", ".", "--json"]) + try: + facts = json.loads(out.stdout) + except ValueError: + facts = None + fail("the graph facts query did not return JSON") + + # 1a. A prose mention with no declared relation is lawful when no typed relation fits; the pair is listed + # with its reason. A new pair fails until it gets a relation or a line, and a listed pair that stopped warning goes. + listed = {(p["from"], p["to"]): p["reason"] for p in json.load(open(PROSE_MENTIONS, encoding="utf-8"))["pairs"]} + warned = {(m[0], m[1]): m[2] for m in facts["mentions"]} if facts else {} + for (source, target), path in sorted(warned.items()): + if (source, target) not in listed: + fail(f"{source} mentions {target} at {path} with no relation: declare the relation that fits, or list the pair with its reason in design/tools/prose-mentions.json") + for source, target in sorted(set(listed) - set(warned)): + fail(f"{source} → {target} is listed in design/tools/prose-mentions.json and no longer warns; remove the line") + for (source, target), reason in sorted(listed.items()): + if not reason.strip(): + fail(f"{source} → {target} is listed in design/tools/prose-mentions.json with no reason") + if n_warnings != len(warned): + fail(f"validate: {n_warnings - len(warned)} warnings other than prose mentions, first: {(warnings or ['not printed'])[0][:200]}") + + # 1b. Entry addresses outside the Specs resolve, through the Protocol's address recipe. + files = [p for top in ADDRESS_SOURCES for ext in ("ts", "tsx", "mjs") for p in glob.glob(os.path.join(ROOT, top, "**", f"*.{ext}"), recursive=True)] + files = [p for p in files if "_generated" not in p and not p.endswith(".test.generated.ts")] + files += [os.path.join(DESIGN, name) for name in ADDRESS_RECORDS] + found = address_tokens(files) + addresses = len(found) + try: + for message in address_failures(found, graph_recipe(25, ROOT, [token for token, _ in found])["rows"]): + fail(message) + except (ValueError, KeyError) as error: + fail(f"the address recipe did not answer: {str(error)[:200]}") + + # 1c. No Spec rests on itself through dependsOn, as the Protocol's cycle recipe reports. + cycles = None + try: + answer = graph_recipe(26, ROOT) + cycles = len(answer["cycles"]) + for message in cycle_failures(answer): + fail(message) + except (ValueError, KeyError) as error: + fail(f"the cycle recipe did not answer: {str(error)[:200]}") + + # 1d. The architecture the code anchors state, as the Protocol's roles-and-layers recipe reads it. + # The values are free (a layer is one of five); this line reports them and judges none. + structure = None + try: + answer = graph_recipe(28, ROOT) + taxonomy = lambda rows: ", ".join(f"{row['value']} {len(row['units'])}" for row in rows) or "none" + totals = answer["totals"] + structure = (f"{totals['codeUnits']} code units; roles: {taxonomy(answer['roles'])}; " + f"layers: {taxonomy(answer['layers'])}; contexts: {taxonomy(answer['contexts'])}; " + f"{totals['references']} references edges from {totals['referencingUnits']} units") + except (ValueError, KeyError) as error: + fail(f"the roles-and-layers recipe did not answer: {str(error)[:200]}") + + # 1e. The context rule, read from the graph: the platform's components use nothing in another context. + isolation = None + out = run(["q", PLATFORM_ISOLATION, "--root", ".", "--json"]) + try: + answer = json.loads(out.stdout) + isolation = (f"platform isolation: {answer['platformComponents']} platform components, " + f"{answer['platformUses']} uses edges from them, {len(answer['crossings'])} into another context") + for crossing in answer["crossings"]: + via = "" if crossing["from"] == crossing["fromComponent"] else f" (through {crossing['fromComponent']})" + fail(f"{crossing['from']}{via} uses {crossing['to']} in the context {crossing['toContext']}: " + "a platform component uses nothing in another context (design/PLAN.md section 2.1)") + except (ValueError, KeyError, TypeError) as error: + fail(f"the platform isolation query did not answer: {str(error)[:200]}") + + out = run(["q", READINESS_DIVERGENCE, "--root", ".", "--json"]) + try: + diverging = json.loads(out.stdout.strip().splitlines()[-1]) + except (ValueError, IndexError): + diverging = None + if diverging is None: + fail("readiness divergence query did not return JSON") + elif diverging: + fail(f"stated readiness not earned by: {diverging}") + + # 2. Read the Specs. + kinds = collections.Counter() + readiness = collections.Counter() + question_graph = graph_recipe(20) + questions = {spec["id"]: [q["question"] for q in spec["questions"]] for spec in question_graph["specs"]} + if question_graph["malformed"]: + fail("open-question recipe reports malformed questions") + for spec in question_graph["specs"]: + for question in spec["questions"]: + if not question.get("key"): + fail(f"{spec['id']} carries an open question without a key") + for spec in question_graph["specs"]: + if spec["totals"]["blocking"] and spec["statedReadiness"] != "scoped": + fail(f"{spec['id']} carries a blocking question and states {spec['statedReadiness']}") + spec_files = {} + n_blocking = question_graph["totals"]["blocking"] + cited_extensions = set() + edge_gaps = [] + for path in sorted(glob.glob(os.path.join(DESIGN, "specs", "**", "*.sdp.md"), recursive=True)): + if path.endswith(".pack.sdp.md"): continue - if section in TYPED_SECTIONS and line.startswith("- "): - cited_laws |= {int(x) for x in re.findall(r"\bLaw (\d+)", line)} - cited_facts |= {int(x) for x in re.findall(r"\bF(\d+)\b", line)} - if section == "Open" and line.startswith("- ["): - questions.setdefault(sid, []).append(line) - if line.startswith("- [blocking]") and stated != "scoped": - fail(f"{sid} carries a blocking question and states {stated}") - family = sid.split(":")[1].split(".")[0] - exempt = kind in ("decision", "example") or family in ("laws", "facts") or sid == "spec:platform.vocabulary" - missing_laws, missing_facts = cited_laws - linked_laws, cited_facts - linked_facts - if not exempt and (missing_laws or missing_facts): - edge_gaps.append(f"{sid}: laws {sorted(missing_laws)} facts {sorted(missing_facts)}") -for gap in edge_gaps: - fail("cited without a constrainedBy edge (rubric 16): " + gap) - -# 3. README.md against the Specs. -readme = open(os.path.join(DESIGN, "README.md"), encoding="utf-8").read() -expected_line = f"{n_specs} specs · {n_packs} packs · {n_anchors} anchors → {n_nodes} nodes · {n_edges} edges (0 errors, 0 warnings)" -if expected_line not in readme: - fail("README's expected validate output is stale; it should read: " + expected_line) -if "validate: 0 errors · 0 warnings" not in readme: - fail("README's expected validate verdict is stale; it should read 0 errors and 0 warnings") -registered = set(re.findall(r"^\| E-(\d+) \|", readme, re.M)) -if cited_extensions - registered: - fail(f"extensions cited in Specs but missing from README's register: E-{sorted(cited_extensions - registered, key=int)}") -if registered - cited_extensions: - fail(f"extensions registered in README but cited by no Spec: E-{sorted(registered - cited_extensions, key=int)}") -n_questions = sum(len(v) for v in questions.values()) -n_blocking = sum(1 for v in questions.values() for line in v if line.startswith("- [blocking]")) -for link in re.findall(r"\]\(([^)#]+)\)", readme): - if not link.startswith("http") and not os.path.exists(os.path.join(DESIGN, link)): - fail(f"README link does not resolve: {link}") - -# 4. The review ledger. -ledger = json.load(open(os.path.join(DESIGN, "reviews", "consensus-ledger.json"), encoding="utf-8"))["ledger"] -by_status = collections.Counter() -for item in ledger: - if item["status"] not in LEDGER_STATUSES: - fail(f"ledger item {item['id']} has status {item['status']!r}; allowed: {sorted(LEDGER_STATUSES)}") - by_status[item["status"]] += 1 - -by_slice = collections.Counter(item.get("slice", "none") for item in ledger if item["status"] == "open") - -digest = hashlib.sha256() -tracked = sorted(glob.glob(os.path.join(DESIGN, "specs", "**", "*.sdp.md"), recursive=True)) -tracked += [os.path.join(DESIGN, name) for name in ("README.md", "PLAN.md", "SESSIONS.md", "reviews/consensus-ledger.json")] -for path in tracked: - digest.update(os.path.relpath(path, DESIGN).encode()) - digest.update(open(path, "rb").read()) - -print(expected_line) -print(f"validate: {n_errors} errors · {n_warnings} warnings; readiness divergence: {diverging}") -print(f"open questions: {len(questions)} Specs, {n_questions} questions, {n_blocking} blocking; extensions registered: {len(registered)}") -print(f"stated readiness: {dict(sorted(readiness.items()))}") -print(f"ledger: {len(ledger)} findings, {dict(sorted(by_status.items()))}") -print(f"open findings by slice: {dict(sorted(by_slice.items()))}") -print(f"corpus digest: {digest.hexdigest()[:16]}") -print(f"protocol: {PINNED}") -if failures: - print(f"\nFAIL ({len(failures)}):") - for message in failures: - print(" - " + message) - sys.exit(1) -print("\nOK") + raw = open(path, encoding="utf-8").read() + front, body = raw.split("\n---\n", 1) + sid = re.search(r"^id: (\S+)", front, re.M).group(1) + spec_files[sid] = os.path.relpath(path, ROOT) + kind = re.search(r"^kind: (\w+)", front, re.M).group(1) + kinds[kind] += 1 + stated = re.search(r"^readiness: (\w+)", front, re.M).group(1) + readiness[stated] += 1 + if stated == "ready": + fail(f"{sid} states ready; only the owner states it, and STATE.md must record the ruling") + cited_extensions |= set(re.findall(r"\bE-(\d+)\b", raw)) + block = re.search(r"constrainedBy:(.*?)(?=\n [a-zA-Z]+:|\Z)", front, re.S) + block = block.group(1) if block else "" + linked_laws = {int(x) for x in re.findall(r"laws\.law(\d+)", block)} + linked_facts = {int(x) for x in re.findall(r"facts\.f(\d+)", block)} + section, cited_laws, cited_facts = None, set(), set() + for line in body.split("\n"): + heading = re.match(r"^#{2,3} (\S+)", line) + if heading: + section = heading.group(1) + continue + if section in TYPED_SECTIONS and line.startswith("- "): + cited_laws |= {int(x) for x in re.findall(r"\bLaw (\d+)", line)} + cited_facts |= {int(x) for x in re.findall(r"\bF(\d+)\b", line)} + family = sid.split(":")[1].split(".")[0] + exempt = kind in ("decision", "example") or family in ("laws", "facts") or sid == "spec:platform.vocabulary" + missing_laws, missing_facts = cited_laws - linked_laws, cited_facts - linked_facts + if not exempt and (missing_laws or missing_facts): + edge_gaps.append(f"{sid}: laws {sorted(missing_laws)} facts {sorted(missing_facts)}") + for gap in edge_gaps: + fail("cited without a constrainedBy edge (rubric 16): " + gap) + + # 2a. Open question sources resolve by address through the Protocol; closed ones keep their text. + register = json.load(open(os.path.join(DESIGN, "decisions/register.json"), encoding="utf-8")) + resolutions = question_resolutions(register["decisions"]) + for row in register["decisions"]: + for error in question_source_errors(row, questions, spec_files, resolutions): + fail(row["id"] + ": " + error) + + # 3. README.md against the Specs. + readme = open(os.path.join(DESIGN, "README.md"), encoding="utf-8").read() + expected_line = f"{n_specs} specs · {n_packs} packs · {n_anchors} anchors → {n_nodes} nodes · {n_edges} edges (0 errors, 0 warnings)" + if expected_line not in readme: + fail("README's expected validate output is stale; it should read: " + expected_line) + if f"validate: 0 errors · {n_warnings} warnings" not in readme: + fail(f"README's expected validate verdict is stale; it should read 0 errors and {n_warnings} warnings, each a listed prose mention") + registered = set(re.findall(r"^\| E-(\d+) \|", readme, re.M)) + if cited_extensions - registered: + fail(f"extensions cited in Specs but missing from README's register: E-{sorted(cited_extensions - registered, key=int)}") + if registered - cited_extensions: + fail(f"extensions registered in README but cited by no Spec: E-{sorted(registered - cited_extensions, key=int)}") + n_questions = sum(len(v) for v in questions.values()) + for link in re.findall(r"\]\(([^)#]+)\)", readme): + if not link.startswith("http") and not os.path.exists(os.path.join(DESIGN, link)): + fail(f"README link does not resolve: {link}") + + # 4. The review ledger. + ledger = json.load(open(os.path.join(DESIGN, "reviews", "consensus-ledger.json"), encoding="utf-8"))["ledger"] + by_status = collections.Counter() + for item in ledger: + if item["status"] not in LEDGER_STATUSES: + fail(f"ledger item {item['id']} has status {item['status']!r}; allowed: {sorted(LEDGER_STATUSES)}") + by_status[item["status"]] += 1 + + by_slice = collections.Counter(item.get("slice", "none") for item in ledger if item["status"] == "open") + + # 5. Each Pack page is what the tree generates. The page design/pages/.md is the page of pack:, + # and the generator's --check names the first line that differs. PACK_PAGES is the list of pages the corpus + # keeps, so a page deleted by mistake fails here instead of leaving nothing to check. + PACK_PAGES = ("history-view",) + for path in sorted(glob.glob(os.path.join(DESIGN, "pages", "*.md"))): + if os.path.splitext(os.path.basename(path))[0] not in PACK_PAGES: + fail(f"{os.path.relpath(path, ROOT)} is not in PACK_PAGES") + stale_pages = 0 + for name in PACK_PAGES: + path = os.path.join(DESIGN, "pages", name + ".md") + out = subprocess.run( + ["node", os.path.join(ROOT, "scripts", "pack-page.mjs"), "--pack", f"pack:{name}", "--out", os.path.relpath(path, ROOT), "--check"], + capture_output=True, text=True, cwd=ROOT, + ) + if out.returncode != 0: + stale_pages += 1 + message = (out.stderr + out.stdout).strip().splitlines() + fail(message[0] if message else f"scripts/pack-page.mjs exited {out.returncode} on {os.path.relpath(path, ROOT)}") + + digest = hashlib.sha256() + tracked = sorted(glob.glob(os.path.join(DESIGN, "specs", "**", "*.sdp.md"), recursive=True)) + tracked += [os.path.join(DESIGN, name) for name in ("README.md", "PLAN.md", "SESSIONS.md", "reviews/consensus-ledger.json", "tools/prose-mentions.json")] + for path in tracked: + digest.update(os.path.relpath(path, DESIGN).encode()) + digest.update(open(path, "rb").read()) + + print(expected_line) + print(f"validate: {n_errors} errors · {n_warnings} warnings, {len(listed)} prose mentions listed; readiness divergence: {diverging}") + print(f"entry addresses outside the Specs: {addresses}; dependsOn cycles: {cycles}") + print(f"architecture: {structure}; {isolation}") + print(f"open questions: {len(questions)} Specs, {n_questions} questions, {n_blocking} blocking; extensions registered: {len(registered)}") + print(f"stated readiness: {dict(sorted(readiness.items()))}") + print(f"ledger: {len(ledger)} findings, {dict(sorted(by_status.items()))}") + print(f"open findings by slice: {dict(sorted(by_slice.items()))}") + print(f"corpus digest: {digest.hexdigest()[:16]}") + print(f"Pack pages current: {len(PACK_PAGES) - stale_pages} of {len(PACK_PAGES)}") + print(f"protocol: {PINNED}") + if failures: + print(f"\nFAIL ({len(failures)}):") + for message in failures: + print(" - " + message) + sys.exit(1) + print("\nOK") + + +if __name__ == "__main__": + main() diff --git a/design/tools/decisions.py b/design/tools/decisions.py index b116292..50faf60 100644 --- a/design/tools/decisions.py +++ b/design/tools/decisions.py @@ -13,6 +13,7 @@ import os from pathlib import Path import re +import subprocess import sys import tempfile @@ -45,6 +46,78 @@ def date(value): raise argparse.ArgumentTypeError("expected a date in YYYY-MM-DD form") +def question_opening(text): + """Keep the authored text through its first sentence.""" + match = re.search(r"[.?!](?=\s|$)", text) + return text[:match.end()] if match else text + + +def normalize_question(text): + return " ".join(text.split()) + + +def graph_recipe(number, root=ROOT, addresses=None): + """Run a body from the pinned Protocol's shipped catalog.""" + catalog = (ROOT / "node_modules/@libar-dev/software-delivery-protocol/docs/agent-surface/recipes.md").read_text() + body = re.search(rf"## {number}\..*?```js\n(.*?)```", catalog, re.S).group(1) + if addresses is not None: + body = re.sub(r"const addresses = \[.*?\];", lambda _: "const addresses = " + json.dumps(addresses) + ";", body, count=1, flags=re.S) + result = subprocess.run(["npx", "sdp", "q", body, "--root", str(root), "--json"], + cwd=ROOT, capture_output=True, text=True) + if result.returncode: + raise ValueError(f"recipe {number} failed: {result.stderr or result.stdout}") + return json.loads(result.stdout) + + +def question_resolutions(rows, root=ROOT): + addresses = [source.get("address") for row in rows for source in row["sources"] + if source.get("kind") == "specQuestion" and source.get("open") is not False] + return {result["address"]: result for result in graph_recipe(25, root, addresses)["rows"] + if isinstance(result["address"], str)} + + +def question_source_errors(row, questions, files, resolutions=None): + """Resolve open sources through the Protocol; keep closed sources' text rule.""" + errors = [] + resolutions = resolutions or {} + for source in row["sources"]: + if source.get("kind") != "specQuestion": + continue + closed = source.get("open") is False + required = {"kind", "spec", "question", "file", "open"} if closed else {"kind", "address", "file"} + if set(source) != required: + errors.append("closed specQuestion requires kind, spec, question, file and open: false" if closed + else "open specQuestion requires kind, address and file") + if closed: + spec = source.get("spec") + question = source.get("question") + if not present(spec) or spec not in files: + errors.append("specQuestion names no Spec: " + repr(spec)) + continue + if source.get("file") != files[spec]: + errors.append("specQuestion file does not name its Spec: " + spec) + if not present(question): + errors.append("specQuestion requires non-empty question text") + continue + matches = sum(normalize_question(question_opening(text)) == normalize_question(question) + for text in questions.get(spec, [])) + if row["status"] not in ("decided", "folded"): + errors.append("open: false requires a decided or folded row") + if matches: + errors.append("open: false question resolves again: " + spec) + else: + address = source.get("address") + resolved = resolutions.get(address) if isinstance(address, str) else None + if not resolved or not resolved["resolves"] or resolved["section"] != "question": + reason = resolved["reason"] if resolved else "missing address" + errors.append(f"specQuestion address does not resolve to an open question: {address!r} ({reason})") + continue + spec = resolved["id"] + if source.get("file") != files.get(spec): + errors.append("specQuestion file does not name its Spec: " + spec) + return errors + + def row_errors(row, units, root, row_ids): """Validate the merged row and every patch refusal.""" errors = [] @@ -258,7 +331,8 @@ def matches(row): if args.cites and args.cites not in row["cites"]: return False if args.spec: - refs = [source["ref"] for source in row["sources"] if source["kind"] in ("specQuestion", "specLine")] + refs = [source.get("address", source.get("spec", "")) if source["kind"] == "specQuestion" else source["ref"] + for source in row["sources"] if source["kind"] in ("specQuestion", "specLine")] if not any(ref == args.spec or ref.split("#")[0] == args.spec for ref in refs): return False if (args.spec or args.family or args.cites) and row["status"] == "decided": diff --git a/design/tools/prose-mentions.json b/design/tools/prose-mentions.json new file mode 100644 index 0000000..477fc8f --- /dev/null +++ b/design/tools/prose-mentions.json @@ -0,0 +1,64 @@ +{ + "pairs": [ + { + "from": "spec:application.history-rebuild.write-pause-rebuild-abort", + "to": "spec:application.orders-inventory-example", + "reason": "An example keeps only its refines and verifies edges; the Verification bullet names the fixture view it runs on." + }, + { + "from": "spec:application.history-rebuild.write-pause-rebuild-interrupt", + "to": "spec:application.orders-inventory-example", + "reason": "An example keeps only its refines and verifies edges; the Verification bullet names the fixture view it runs on." + }, + { + "from": "spec:command.command-declaration.broken-argument-contract-fails-build", + "to": "spec:platform.acceptance-contract", + "reason": "An example keeps only its refines and verifies edges; the Verification bullet names where the compiled tier comes from." + }, + { + "from": "spec:command.command-declaration.renamed-handler-fails-build", + "to": "spec:platform.acceptance-contract", + "reason": "An example keeps only its refines and verifies edges; the Verification bullet names where the compiled tier comes from." + }, + { + "from": "spec:command.idempotency-and-receipts.ui-double-submit", + "to": "spec:application.parent-use-cases", + "reason": "An example keeps only its refines and verifies edges; the Verification bullet cites the call order the use case rules." + }, + { + "from": "spec:command.tenancy-and-authority.client-claims-agent-namespace", + "to": "spec:command.tenancy-and-authority.client-claims-system-namespace", + "reason": "The narrative names a sibling example as the first case; an example keeps only its refines and verifies edges." + }, + { + "from": "spec:command.tenancy-and-authority.client-claims-system-namespace", + "to": "spec:command.tenancy-and-authority.client-claims-agent-namespace", + "reason": "The narrative names a sibling example as the second case; an example keeps only its refines and verifies edges." + }, + { + "from": "spec:facts.probe-plan", + "to": "spec:facts.f04-nested-calls-cost-more-than-helpers.probe-3-hosted-function-calls", + "reason": "An open question points at the hosted example that runs Probe 3's quota half; the plan does not build on it." + }, + { + "from": "spec:kernel.decider-contract", + "to": "spec:command.actor-and-scope", + "reason": "The kernel's actor type is structural and imports nothing from the command family; the mention names what is passed." + }, + { + "from": "spec:operations.baseline-operations.broken-metrics-never-abort", + "to": "spec:operations.baseline-operations.broken-audit-aborts", + "reason": "A Verification bullet says two steps match a sibling example's; an example keeps only its refines and verifies edges." + }, + { + "from": "spec:platform.native-harness", + "to": "spec:command.tenancy-and-authority", + "reason": "An open question names the rule the fixture's stand-in entry bends; the harness does not build on it." + }, + { + "from": "spec:platform.vocabulary", + "to": "spec:command.actor-and-scope", + "reason": "A vocabulary term points at where the caller namespace union is pinned; the glossary depends on nothing here." + } + ] +} diff --git a/design/tools/test_decisions.py b/design/tools/test_decisions.py index 339fd75..f97003f 100644 --- a/design/tools/test_decisions.py +++ b/design/tools/test_decisions.py @@ -1,10 +1,12 @@ -"""Decision queries, patch refusals and the real register's graph coverage. +"""Decision queries, patch refusals, the real register's question sources and its rows found by Spec, +the two checks check.py runs through the Protocol's recipes, entry addresses outside the Specs and +dependsOn cycles, and check.py run whole over a sample tree, so that each of its checks reaches the +failures it prints and its exit status. Run: python3 -m unittest design/tools/test_decisions.py Fixtures and mutation copies live in operating-system temporary directories. """ -import collections import contextlib import copy import difflib @@ -15,6 +17,7 @@ from pathlib import Path import re import subprocess +import sys import tempfile import unittest @@ -24,6 +27,7 @@ SPEC = importlib.util.spec_from_file_location("decisions_under_test", TOOL_PATH) tool = importlib.util.module_from_spec(SPEC) SPEC.loader.exec_module(tool) +CHECK_PATH = Path(os.environ.get("CHECK_TOOL_UNDER_TEST", ROOT / "design/tools/check.py")) def lean(): @@ -40,7 +44,7 @@ def lean(): def row(identifier, **changes): result = { "id": identifier, "title": "Keep a bounded context call", - "sources": [{"kind": "specQuestion", "ref": "spec:context.tables#1", "file": "design/specs/context/tables.sdp.md", "line": 1}], + "sources": [{"kind": "specQuestion", "address": "spec:context.tables#question.boundedCall", "file": "design/specs/context/tables.sdp.md"}], "families": ["context"], "cites": ["E-32", "F13"], "provisionalReading": "One bounded call", "blocks": None, "advisor": "convex", "status": "unsorted", "foldedInto": [], "class": None, "reason": None, "lean": None, @@ -153,7 +157,7 @@ def test_spec_family_citation_commands_show_open_rows(self): for args in [("--spec", "spec:context.tables"), ("--family", "context"), ("--cites", "E-32")]: with self.subTest(args=args): self.assertEqual(self.ids(*args), ["OD-001", "OD-002", "OD-003", "OD-004", "OD-007"]) - self.assertEqual(self.ids("--spec", "spec:context.tables#1", "--decided"), ["OD-005", "OD-006"]) + self.assertEqual(self.ids("--spec", "spec:context.tables", "--decided"), ["OD-005", "OD-006"]) self.assertEqual(self.ids("--spec", "spec:context.table"), []) self.assertEqual(self.ids("--cites", "E-3"), []) @@ -340,49 +344,271 @@ def test_spec_line_sources_can_be_queried(self): self.assertEqual(self.ids("--spec", "spec:command.command-pipeline"), ["OD-007"]) +class QuestionSourceTests(unittest.TestCase): + def setUp(self): + self.row = row("OD-001") + self.spec = "spec:context.tables" + self.address = self.row["sources"][0]["address"] + self.files = {self.spec: "design/specs/context/tables.sdp.md"} + self.questions = {self.spec: ["Keep a bounded context call. Another sentence."]} + self.resolutions = {self.address: {"address": self.address, "resolves": True, + "id": self.spec, "section": "question", "key": "boundedCall", "reason": None}} + + def errors(self): + return tool.question_source_errors(self.row, self.questions, self.files, self.resolutions) + + def close_source(self): + self.row["sources"] = [{"kind": "specQuestion", "spec": self.spec, + "question": "Keep a bounded context call.", + "file": self.files[self.spec], "open": False}] + + def test_address_survives_question_text_changes(self): + self.questions[self.spec] = ["A different sentence."] + self.assertEqual(self.errors(), []) + + def test_missing_wrong_section_wrong_file_and_old_shape_are_refused(self): + for mutation in ("missing", "section", "file", "old", "line", "openTrue", "notString"): + with self.subTest(mutation=mutation): + saved_row = copy.deepcopy(self.row) + saved_resolutions = copy.deepcopy(self.resolutions) + source = self.row["sources"][0] + if mutation == "missing": self.resolutions[self.address].update(resolves=False, reason="entry") + elif mutation == "section": self.resolutions[self.address]["section"] = "design" + elif mutation == "file": source["file"] = "another.sdp.md" + elif mutation == "old": + source.pop("address") + source.update(spec=self.spec, question="Keep a bounded context call.") + elif mutation == "line": source["line"] = 1 + elif mutation == "openTrue": source["open"] = True + elif mutation == "notString": source["address"] = [] + self.assertTrue(self.errors()) + self.row, self.resolutions = saved_row, saved_resolutions + + def test_closed_opening_matching_normalizes_only_whitespace(self): + self.close_source() + self.row["status"] = "decided" + self.assertEqual(tool.question_opening("Keep `spec:context.tables` at 1.5 bytes. Why?"), + "Keep `spec:context.tables` at 1.5 bytes.") + self.row["sources"][0]["question"] = "Keep a bounded\n context call." + self.assertIn("open: false question resolves again", " ".join(self.errors())) + self.row["sources"][0]["question"] = "keep a bounded context call." + self.assertEqual(self.errors(), []) + + def test_removed_question_requires_closed_status_and_refuses_return(self): + self.close_source() + self.questions[self.spec] = [] + for status in tool.STATUSES: + self.row["status"] = status + with self.subTest(status=status): + self.assertEqual(bool(self.errors()), status not in ("decided", "folded")) + self.questions[self.spec] = ["Keep a bounded context call. Returned text."] + self.assertIn("open: false question resolves again", " ".join(self.errors())) + + def test_protocol_removed_key_fails_and_closed_source_keeps_passing(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + path = root / self.files[self.spec] + path.parent.mkdir(parents=True) + carrier = """--- +id: spec:context.tables +kind: rule +altitude: story +readiness: scoped +relations: {} +--- +# Bounded call + +## Intent +- outcome: Keep a bounded call. + +### Open questions +- [non-blocking #boundedCall] Keep a bounded context call. Another sentence. + +## Rule +- Keep the call bounded. + +## Verification — reviewed +- Review the bound. +""" + path.write_text(carrier) + self.resolutions = tool.question_resolutions([self.row], root) + self.assertEqual(self.errors(), []) + path.write_text(carrier.replace(" #boundedCall", "")) + self.resolutions = tool.question_resolutions([self.row], root) + self.assertIn("does not resolve", " ".join(self.errors())) + path.write_text(carrier.replace("- [non-blocking #boundedCall] Keep a bounded context call. Another sentence.\n", "")) + graph = tool.graph_recipe(20, root) + self.questions = {spec["id"]: [q["question"] for q in spec["questions"]] for spec in graph["specs"]} + self.close_source() + self.row["status"] = "decided" + before = copy.deepcopy(self.row["sources"]) + self.assertEqual(self.errors(), []) + self.assertEqual(self.row["sources"], before) + + class RealRegisterTests(unittest.TestCase): - def test_real_register_has_unique_rows_and_exact_recipe_20_coverage(self): + def test_real_register_has_unique_ids_and_resolving_question_sources(self): register = json.loads((ROOT / "design/decisions/register.json").read_text()) rows = register["decisions"] - self.assertGreaterEqual(len(rows), 147) ids = [r["id"] for r in rows] - self.assertEqual(ids, [f"OD-{i:03d}" for i in range(1, len(rows) + 1)]) - self.assertEqual(len(set(ids)), len(rows)) - for r in rows: - self.assertEqual(tool.row_errors(r, register["units"], ROOT, set(ids)), [], r["id"]) - self.assertTrue(all(target in ids for target in r["foldedInto"])) - if r["status"] in ("sorted", "waiting", "decided"): - self.assertIn(r["class"], tool.CLASSES) - self.assertTrue(tool.present(r["reason"])) - self.assertTrue(all(set(s) == {"kind", "ref", "file", "line"} for r in rows for s in r["sources"])) - products = [s["ref"] for r in rows for s in r["sources"] if s["kind"] == "productDecision"] - self.assertEqual(products, ["product/1", "product/2", "product/3"]) - self.assertEqual(len(register["possiblySame"]), 46) - self.assertEqual(register["schema"], 1) - self.assertEqual(register["units"], ["S3", "S4", "S5", "L3"]) - refs = collections.Counter(s["ref"] for r in rows for s in r["sources"] if s["kind"] == "specQuestion") + self.assertEqual(len(set(ids)), len(ids)) + graph = tool.graph_recipe(20) + resolutions = tool.question_resolutions(rows) + self.assertEqual(graph["malformed"], []) + questions = {spec["id"]: [q["question"] for q in spec["questions"]] for spec in graph["specs"]} + files = {} + for path in (ROOT / "design/specs").rglob("*.sdp.md"): + identifier = re.search(r"^id: (\S+)", path.read_text(), re.M).group(1) + files[identifier] = path.relative_to(ROOT).as_posix() for decision in rows: - for source in decision["sources"]: - if source["kind"] != "specQuestion": - continue - identifier, number = source["ref"].rsplit("#", 1) - text = (ROOT / source["file"]).read_text() - self.assertEqual(re.search(r"^id: (\S+)", text, re.M).group(1), identifier) - lines = text.splitlines() - start = lines.index("### Open questions") - end = next((i for i in range(start + 1, len(lines)) if lines[i].startswith("## ")), len(lines)) - locations = [i + 1 for i in range(start + 1, end) if re.match(r"^- \[(?:non-blocking|blocking)\] ", lines[i])] - self.assertEqual(source["line"], locations[int(number) - 1]) - recipes = (ROOT / "node_modules/@libar-dev/software-delivery-protocol/docs/agent-surface/recipes.md").read_text() - body = re.search(r"## 20\..*?```js\n(.*?)```", recipes, re.S).group(1) - result = subprocess.run(["npx", "sdp", "q", body, "--root", "design", "--json"], cwd=ROOT, capture_output=True, text=True) - self.assertEqual(result.returncode, 0, result.stderr) - graph = json.loads(result.stdout) - expected = collections.Counter(f"{spec['id']}#{i}" for spec in graph["specs"] for i, _ in enumerate(spec["questions"], 1)) - self.assertEqual(graph["totals"]["questions"], 144) - self.assertEqual(graph["totals"]["malformed"], 0) - self.assertEqual(refs, expected) - self.assertTrue(all(n == 1 for n in refs.values())) + with self.subTest(row=decision["id"]): + self.assertEqual(tool.row_errors(decision, register["units"], ROOT, set(ids)), []) + self.assertEqual(tool.question_source_errors(decision, questions, files, resolutions), []) + + def test_real_rows_are_found_by_the_spec_their_sources_name(self): + def ids(*args): + stdout = io.StringIO() + with contextlib.redirect_stdout(stdout): + code = tool.main([*args, "--json"], register_path=ROOT / "design/decisions/register.json", root=ROOT) + self.assertEqual(code, 0) + return {row["id"] for row in json.loads(stdout.getvalue())} + for spec, expected in [ + ("spec:application.projection-contract", {"OD-171"}), + ("spec:application.history-projection", {"OD-172", "OD-173", "OD-174"}), + ("spec:application.history-rebuild", {"OD-175"}), + ]: + with self.subTest(spec=spec): + self.assertLessEqual(expected, ids("--spec", spec) | ids("--spec", spec, "--decided")) + + +def sample_spec(name, relations="{}", key=" #keepIt"): + """A rule Spec of a sample corpus, with one Design entry and one open question, keyed unless key is empty.""" + return f"""--- +id: spec:sample.{name} +kind: rule +altitude: story +readiness: scoped +relations: {relations} +--- +# Sample {name} + +## Intent +- outcome: Keep {name}. + +### Open questions +- [non-blocking{key}] Keep {name} as it is. Another sentence. + +## Rule +- Keep {name}. + +## Design +- typeOne: `type One = string` + +## Verification — reviewed +- Review {name}. +""" + + +class CheckTests(unittest.TestCase): + """check.py's address and cycle checks over a sample corpus, through the shipped recipes 25 and 26.""" + + @classmethod + def setUpClass(cls): + sys.path.insert(0, str(ROOT / "design/tools")) + spec = importlib.util.spec_from_file_location("check_under_test", CHECK_PATH) + cls.check = importlib.util.module_from_spec(spec) + spec.loader.exec_module(cls.check) + + def setUp(self): + self.temporary = tempfile.TemporaryDirectory() + self.addCleanup(self.temporary.cleanup) + self.root = Path(self.temporary.name) + + def put(self, path, text): + (self.root / path).parent.mkdir(parents=True, exist_ok=True) + (self.root / path).write_text(text) + + def test_a_broken_address_fails_where_it_is_written(self): + self.put("design/specs/sample/one.sdp.md", sample_spec("one")) + self.put("src/cites.ts", "\n".join([ + "// spec:sample.one#design.typeOne resolves.", + "// spec:sample.one#question.keepIt resolves.", + "// spec:sample.one#design.typeTwo names no entry.", + "// spec:sample.none#design.typeOne names no Spec.", + "// spec:sample.one#design.Type-one is malformed.", + "// spec:sample.one#3 is a positional citation, kept as written.", + ]) + "\n") + found = self.check.address_tokens([self.root / "src/cites.ts"], self.root) + self.assertEqual([where for _, where in found], [f"src/cites.ts:{n}" for n in range(1, 6)]) + rows = tool.graph_recipe(25, self.root, [token for token, _ in found])["rows"] + self.assertEqual(self.check.address_failures(found, rows), [ + "entry address spec:sample.one#design.typeTwo at src/cites.ts:3 names no entry of its Spec", + "entry address spec:sample.none#design.typeOne at src/cites.ts:4 names no Spec", + "entry address spec:sample.one#design.Type-one at src/cites.ts:5 is outside the address grammar", + ]) + self.assertEqual(self.check.address_failures(found, rows[:-1]), + ["the address recipe answered 4 rows for 5 entry addresses"]) + + def test_a_planted_cycle_fails_and_a_chain_passes(self): + self.put("design/specs/sample/one.sdp.md", sample_spec("one", "\n dependsOn: spec:sample.two")) + self.put("design/specs/sample/two.sdp.md", sample_spec("two", "\n dependsOn: spec:sample.three")) + self.put("design/specs/sample/three.sdp.md", sample_spec("three")) + self.assertEqual(self.check.cycle_failures(tool.graph_recipe(26, self.root)), []) + self.put("design/specs/sample/three.sdp.md", sample_spec("three", "\n dependsOn: spec:sample.one")) + self.assertEqual(self.check.cycle_failures(tool.graph_recipe(26, self.root)), [ + "dependsOn cycle among spec:sample.one, spec:sample.three, spec:sample.two: " + "spec:sample.one -> spec:sample.two -> spec:sample.three -> spec:sample.one", + ]) + + + +class CheckCommandTests(unittest.TestCase): + """check.py run whole, as the command it is, over a sample tree in a temporary directory. + + The tree holds a copy of check.py and decisions.py under test, the pinned Protocol through a link to + this repository's node_modules, and a stand-in for the page generator. Four defects are planted at + once: an address that names no entry, a dependsOn cycle, an open question without a key, and a page + generator that exits 1. The run must print each one's failure, no other, and exit 1, so a check that + stops reaching its failures or the exit status fails here. + """ + + def put(self, path, text): + (self.root / path).parent.mkdir(parents=True, exist_ok=True) + (self.root / path).write_text(text) + + def test_each_check_reaches_its_failure_and_the_exit_status(self): + with tempfile.TemporaryDirectory() as temporary: + self.root = Path(temporary) + pin = json.loads((ROOT / "package.json").read_text())["devDependencies"]["@libar-dev/software-delivery-protocol"] + self.put("package.json", json.dumps({"devDependencies": {"@libar-dev/software-delivery-protocol": pin}})) + os.symlink(ROOT / "node_modules", self.root / "node_modules") + self.put("design/tools/check.py", CHECK_PATH.read_text()) + self.put("design/tools/decisions.py", TOOL_PATH.read_text()) + self.put("design/tools/prose-mentions.json", json.dumps({"pairs": []})) + self.put("design/decisions/register.json", json.dumps({"schema": 1, "units": [], "decisions": [], "possiblySame": []})) + self.put("design/reviews/consensus-ledger.json", json.dumps({"ledger": []})) + for name in ("STATE.md", "ROADMAP.md", "PLAN.md", "SESSIONS.md"): + self.put("design/" + name, "# Sample\n") + self.put("design/specs/sample/one.sdp.md", sample_spec("one", "\n dependsOn: spec:sample.two")) + self.put("design/specs/sample/two.sdp.md", sample_spec("two", "\n dependsOn: spec:sample.one", key="")) + self.put("src/cites.ts", "// spec:sample.one#design.typeOne resolves.\n// spec:sample.one#design.typeTwo names no entry.\n") + self.put("scripts/pack-page.mjs", 'console.error("pack-page: design/pages/history-view.md is behind the tree from line 1");\nprocess.exit(1);\n') + validate = subprocess.run(["node", str(ROOT / "node_modules/@libar-dev/software-delivery-protocol/dist/cli/sdp.js"), "validate", "."], + cwd=self.root, capture_output=True, text=True) + summary = re.search(r"\d+ specs · \d+ packs · \d+ anchors → \d+ nodes · \d+ edges", validate.stdout + validate.stderr) + self.assertIsNotNone(summary, validate.stdout + validate.stderr) + self.put("design/README.md", f"{summary.group(0)} (0 errors, 0 warnings)\n\nvalidate: 0 errors · 0 warnings\n") + run = subprocess.run([sys.executable, str(self.root / "design/tools/check.py")], cwd=self.root, capture_output=True, text=True, + env={**os.environ, "PYTHONDONTWRITEBYTECODE": "1"}) + printed = [line[3:] for line in run.stdout.splitlines() if line.startswith(" - ")] + self.assertEqual(printed, [ + "entry address spec:sample.one#design.typeTwo at src/cites.ts:2 names no entry of its Spec", + "dependsOn cycle among spec:sample.one, spec:sample.two: spec:sample.one -> spec:sample.two -> spec:sample.one", + "spec:sample.two carries an open question without a key", + "pack-page: design/pages/history-view.md is behind the tree from line 1", + ], run.stdout + run.stderr) + self.assertEqual(run.returncode, 1) + self.assertNotIn("\nOK", run.stdout) if __name__ == "__main__": diff --git a/docs/feedback/pr-12-annotations-section.md b/docs/feedback/pr-12-annotations-section.md new file mode 100644 index 0000000..f256d95 --- /dev/null +++ b/docs/feedback/pr-12-annotations-section.md @@ -0,0 +1,137 @@ +## Now: anchors in the code they bind, under the rulings of 2026-10-10 + +Until this arc, every implementation anchor of this corpus lived in a test. The platform's runtime code may not import the Protocol package, so `byFile` on a source file returned nothing, a hand-kept join (`design/tools/anchor-sites.json`) stood in for the binding, and code could name a design only by claiming to realize it. The Protocol's annotations arc, its plan 40 on the branch `feature/annotations-v0`, moves closer to the Protocol's original design: an anchor can be written as a doc comment, code can name a design it answers to without claiming to build it, and an anchor can say what architectural part its code plays. The owner ruled on the arc on 2026-10-10, and all three of its decisions now state `ready`. This branch pins the Protocol at `822aea4` (`b5fb659`), its `main` after the merge of #29, `feature/annotations-v0`, which carries the rulings, and applies them (`ecba393`); a second read by `gpt-6.1-sol` checked that pass against the code, and its fold-in follows `ed4f97f`. + +**1. Anchors in the code they bind.** A comment-form anchor is a top-level `/** … */` block of reserved `@sdp` tags. The source carries it with no Protocol import, so the runtime pays nothing. The 22 implementation anchors that lived in nine test files are 46 units in `src/`, `example/` and `fixture/`, one per realizing file or site, beside 12 component anchors; the harness keeps its 18 constant-form anchors. No test file keeps an implementation anchor. The file is the binding grain of both forms: a block's line locates the anchor and names no declaration. The online rebuild, at the top of [`src/read-model/rebuild.ts`](https://github.com/libar-dev/libar-platform/blob/unit/design-in-the-graph/src/read-model/rebuild.ts#L2): + +```ts +/** + * @sdpAnchor impl:application.rebuild + * @sdpLabel the online rebuild's operator entries, batch chains, tenant fill and batch bounds + * @sdpReferences spec:application.rebuild + * @sdpComponent component:platform.read-model + * @sdpRole service + */ +``` + +**2. A reference runs one way.** A unit satisfies a Spec when it and the other units that satisfy it realize the whole Spec. A unit references a Spec when it answers to that design without claiming to realize it: it follows the design or realizes part of it, so a change to the design asks the code to follow. A reference confers no delivery fact and moves no floor. A design that builds on existing code says so by its own `dependsOn` or `refines` to the Spec the code satisfies, never by a reference from the code. So the online rebuild no longer references `spec:application.history-rebuild`, a design that builds on it; the history rebuild already depends on `spec:application.rebuild`. A reference kept only to give `byFile` an answer is gone too. The graph holds 49 `references` edges from 41 units, to 22 Specs. + +**3. Whole Specs, honestly.** Anchor targets name whole Specs, so no unit satisfies a Spec while the Spec promises a part no unit builds, and a pinned declaration ahead of its code is such a part ([`design/PLAN.md`](https://github.com/libar-dev/libar-platform/blob/unit/design-in-the-graph/design/PLAN.md) 6.7 item 7). The rule, per Spec: list the entries no satisfying unit realizes; none, keep `satisfies`; a coherent part with its own trigger or step, split it into a child Spec and keep `satisfies` on the parent; otherwise write `references` until the code realizes what the Spec promises. The pass checked the twelve Specs the anchor check named against the code and found a thirteenth, `spec:command.receipt-table`; the second read found two more among the Specs the pass had left satisfied. Each holds a part no unit builds, running through entries the code realizes, so moving entries leaves none of them whole, and its units now reference it: + +| Spec | What no unit builds | +|---|---| +| `spec:context.journal` | the baseline migration, the stream rebuild, the derived mapping's parts | +| `spec:context.tables` | the derived mapping's `streamParts` table | +| `spec:context.persistence-adapter` | migrate on load (step 3a), the derived state | +| `spec:context.queries` | the `history`, `rebuild` and `byOperation` queries | +| `spec:context.context-component` | those queries and the baseline writer in its surface | +| `spec:command.command-declaration` | the history projection binding, the tombstone retention | +| `spec:command.idempotency-and-receipts` | tombstones | +| `spec:command.receipt-table` | the tombstone row and its horizon | +| `spec:command.command-pipeline` | the history projection's write at step 9 | +| `spec:application.read-models` | the history read model's write path | +| `spec:application.generation-registry` | `sourceIndex` in the batch cursor | +| `spec:application.rebuild` | history targets in the shared rebuild config | +| `spec:application.orders-inventory-example` | the order allocation history bound to its commands, journal inspection | +| `spec:application.parent-use-cases` | the business failure policy (`carryOrReject`), the import command | +| `spec:application.projection-contract` | the history writer's callers in its `callers` entry | + +`spec:kernel.outcome-model` is referenced for another reason. Its types in `src/kernel/outcome.ts` and its validators in `src/context/outcome.ts`, which gained an anchor, are built, but the model reserves three rejection codes for the adapter and gives every other code to a context, while the adapter also raises the boundary's `invalidInput`, as `spec:context.persistence-adapter` states. Two Specs disagree and the code follows one, so both units reference the model until the main thread rules which side changes. 9 Specs carry `implemented`, down from 24. [`docs/feedback/sdp-feedback-01.md`](https://github.com/libar-dev/libar-platform/blob/unit/design-in-the-graph/docs/feedback/sdp-feedback-01.md) holds the table with the entries, the code and the reason for each Spec. + +**4. Components with an architectural layer and a context.** Twelve `component:` anchors declare the architecture where it is realized: three in each of the `adapter`, `application`, `domain` and `infrastructure` layers, with the contexts `platform` (7), `orders` (2), `inventory` (2) and `orders-inventory` (1), and `uses` between them. [`design/PLAN.md`](https://github.com/libar-dev/libar-platform/blob/unit/design-in-the-graph/design/PLAN.md) section 2.1 lists them with the owner's two rules. The layer rule: `domain` is business meaning with no runtime import; `application` coordinates a use case; `adapter` realizes a kernel or command contract against Convex; `edge` is transport in; `infrastructure` is a shared mechanism that owns no use case. The context rule: the platform is the key bounded context, with its own language; orders-inventory is the supporting context and development tool, the composition of the two business contexts it models; nothing in the platform's core carries orders-inventory specifics. A component satisfies its seam's design Spec or binds nothing its members bind: `component:platform.gate` satisfies the write pause it and its members realize whole, and the components whose seam is only partly built bind nothing. + +**5. The isolation, read from the graph.** [`design/tools/check.py`](https://github.com/libar-dev/libar-platform/blob/unit/design-in-the-graph/design/tools/check.py) fails when a component in the context `platform` has a `uses` edge to a component in another context, each end resolved to its component directly or through `memberOf`. Today none of the 21 `uses` edges from the platform's seven components leaves it. + +**6. Roles, and the sidecar gone.** 37 units name the part they play, in a vocabulary this corpus owns; recipe 28 reads the taxonomy and `check.py` prints it and judges none of it. `design/tools/anchor-sites.json` and its test are deleted. [`tests/pure/byfile-coverage.test.ts`](https://github.com/libar-dev/libar-platform/blob/unit/design-in-the-graph/tests/pure/byfile-coverage.test.ts) reads the anchors in the tree, checks they are the code units the graph holds, and asserts that `byFile` names every anchored file's units and their Specs. [`scripts/pack-page.mjs`](https://github.com/libar-dev/libar-platform/blob/unit/design-in-the-graph/scripts/pack-page.mjs) reads code from the graph alone, and the [Pack page](https://github.com/libar-dev/libar-platform/blob/unit/design-in-the-graph/design/pages/history-view.md#1-history-projection) lists each unit at its file and line with its role, its component's architectural layer and context, and a "References" list per member. + +### The rulings + +The Protocol's branch rules the arc through three decision Specs, and the owner states `ready` on all three. The paths are in the Protocol's repository, on `main` since the merge of #29 (`822aea4`); the pinned package ships the same files under `node_modules/@libar-dev/software-delivery-protocol/`. + +| Ratified name | Ruling | Spec | +|---|---|---| +| the comment form (MD-36) | A top-level `/** … */` block of reserved `@sdp*` tags is a second form of an anchor, read into the same closed envelope as the constant form. No import is needed, comment prose authors nothing, and the file is the binding grain of both forms. Supersedes MD-35. | `specs/decisions/anchor-comment-form.sdp.md` | +| bindings are optional, plural, and may reference a design (MD-37) | `satisfies` is optional and plural and targets name whole Specs. The `references` edge says the code answers to a design without claiming to realize it, runs from code to design only, and confers no delivery fact. | `specs/decisions/anchor-binding-grain.sdp.md` | +| architectural significance is annotated where it is realized (MD-38) | A code anchor may carry a `role`, and a `component:` anchor a `layer` from a closed set of five and a `context`; the component anchor is the declared component. Roles stay a free vocabulary each corpus owns. Supersedes MD-30 and MD-34. | `specs/decisions/architectural-annotation.sdp.md` | + +### Try it + +From the repository root, after `npm ci`. Measured on 2026-10-10 at `b5fb659`, the pin at `822aea4`, after the fold of the sol check of the review repairs (`e2c77e8`) and the merge of #11 into `main` (`a758ba4`). The re-pin moved no number below. Re-run the commands rather than inherit them. + +```sh +npx sdp q 'return g.byFile("src/read-model/rebuild.ts")' --json +``` + +```json +{ + "path": "src/read-model/rebuild.ts", + "nodes": [{ "id": "impl:application.rebuild", "nodeType": "CodeNode", "line": 2 }], + "specs": ["spec:application.rebuild"] +} +``` + +Recipe 27 over `pack:history-view`, taken from the pinned package's catalog with only the Pack swapped in. Beside the three facts it now derives what each member builds on: the implemented Specs it depends on or refines, with their units. + +````sh +R=node_modules/@libar-dev/software-delivery-protocol/docs/agent-surface/recipes.md +npx sdp q "$(awk '/^## 27\./{f=1} /^## 28\./{f=0} f' "$R" | awk '/^```js/{c=1; next} /^```/{if (c) exit} c' | sed 's/pack:spec-studio-v1/pack:history-view/')" --json +```` + +```json +{ + "found": true, + "id": "pack:history-view", + "totals": { "members": 19, "withReferences": 10, "withImplementations": 2, "withVerifier": 7, "withBuildsOn": 11, "unbound": 7 }, + "rows": [ + { + "id": "spec:application.history-projection", + "resolved": true, + "referencedBy": [{ "id": "impl:application.order-allocation-history", "file": "example/domain/orderAllocationHistory.ts" }], + "implementedBy": [], + "hasVerifier": false, + "buildsOn": [ + { "id": "spec:context.event-envelope", "via": "dependsOn", "implementedBy": [{ "id": "impl:context.event-envelope", "file": "src/context/envelope.ts" }] } + ] + }, + … + ] +} +``` + +```sh +npx sdp validate . +``` + +``` +243 specs · 6 packs · 180 anchors → 429 nodes · 1499 edges (0 errors, 0 warnings) +… +validate: 0 errors · 12 warnings (conformance + honesty over the one graph) +``` + +The 12 warnings are the prose mentions listed in `design/tools/prose-mentions.json`. + +```sh +npm run check +``` + +``` +243 specs · 6 packs · 180 anchors → 429 nodes · 1499 edges (0 errors, 0 warnings) +validate: 0 errors · 12 warnings, 12 prose mentions listed; readiness divergence: [] +entry addresses outside the Specs: 497; dependsOn cycles: 0 +architecture: 76 code units; roles: codec 1, contract 6, decider 2, event-store 1, operator-entry 1, projection 2, query 5, read-model 1, schema 6, service 10, use-case 2; layers: adapter 3, application 3, domain 3, infrastructure 3; contexts: inventory 2, orders 2, orders-inventory 1, platform 7; 49 references edges from 41 units; platform isolation: 7 platform components, 21 uses edges from them, 0 into another context +… +Pack pages current: 1 of 1 +protocol: file:vendor/libar-dev-software-delivery-protocol-0.0.0-822aea4.tgz + +OK +``` + +`npm test` passed 1,151 tests in 75 files, with 10 skipped, the not-compared pairs. + +### For the owner + +The rulings of 2026-10-10 closed the layers, the contexts, `example/` as architecture, the reference direction and the whole-Spec satisfies. One item remains, from `docs/feedback/sdp-feedback-01.md`, section "2026-10-10, the rulings at `f23e358`": + +- **The capability steps' name.** "Layer" names one of the seven steps of capability in this corpus and, as the Protocol's `layer`, a component's architectural layer; `CONTEXT.md` now writes the second in full. Renaming the steps, with "stage" as the candidate, is the one-word-one-meaning answer. It touches a Spec identity (`spec:platform.layers-and-profiles`), 143 numbered uses across 65 carriers and the decisions document this corpus follows, so the owner makes it deliberately. + +Three leads go to a later pass (`design/STATE.md`, lead 47): seven of the sixteen Specs the pass and the second read moved hold a part with its own trigger or step that could split once its pinned declarations are narrowed to the code; two Spec corrections wait for the main thread, the adapter's `invalidInput` against the outcome model's reserved codes and the history writer's callers in the projection contract; and the platform's own Specs still carry orders-inventory worked examples in their Design entries, though its code uses nothing of the example. diff --git a/docs/feedback/sdp-feedback-01.md b/docs/feedback/sdp-feedback-01.md new file mode 100644 index 0000000..efc0832 --- /dev/null +++ b/docs/feedback/sdp-feedback-01.md @@ -0,0 +1,240 @@ +# Feedback to the Software Delivery Protocol from this corpus + +A running file, addressed to the Protocol's maintainers. Each `adopt` unit and each slice that learns something about the Protocol adds a dated section at the top. The first report, with its eight proposals, is [`sdp-development-from-application-platform.md`](../sdp-development-from-application-platform.md). + +Every number here was measured on this corpus on the date of its section. Re-run before relying on one. + +## 2026-10-10, the rulings at `f23e358` + +The owner ruled on the annotations arc on 2026-10-10, ten rulings from three advisors' answers to ten questions, with this guidance: prefer clean solutions; the amount of work never decides; the platform is the key context, orders-inventory is a supporting context and a development tool, and nothing in the platform's core carries orders-inventory specifics. The Protocol's branch `feature/annotations-v0` carries them at `f23e358`. The pin moved from `da56a34` to `f23e358` (`04a6aff`), packed from a clean clone as before, and on the pin alone nothing this corpus reads moved: 243 Specs, 428 nodes, 1,501 edges, 12 listed warnings. Then the corpus applied the rulings (`ecba393`). A second read by `gpt-6.1-sol` checked the pass against the code, and its fold-in moved three more Specs to references, named below. Every number below was measured after that fold-in. + +- **A reference runs one way.** A `references` edge from code says the code answers to a design: the unit follows it or realizes part of it, so a change to the design asks the code to follow. A design that builds on existing code says so by its own `dependsOn` or `refines`, never by a reference from the code. The online rebuild's reference to `spec:application.history-rebuild` came out; the history rebuild already depends on `spec:application.rebuild`. The reference from `src/kernel/outcome.ts` to `spec:kernel.decider-contract` came out too: its only reason was a `byFile` pair, and an edge kept for a test, a count or a `byFile` answer goes. The read-model tables keep their reference to `spec:application.aggregate-projection`, because `src/read-model/tables.ts` holds the aggregate form's `projectionMarkers` table that the Spec pins: the code realizes part of that design, the meaning the ruling keeps. The fixture's six depot units, the gate's operator entries, the allocation history's fold and the kernel, audit and operations components keep theirs for the same reason. Recipe 27 now adds a derived "builds on" column: for each member, the implemented Specs it depends on or refines, with their units. +- **The outcome model is referenced.** `src/kernel/outcome.ts` holds the outcome model's types, and `src/context/outcome.ts` its four validators, which the kernel may not hold. The first is now `impl:kernel.outcome-model`, renamed from `impl:kernel.decider-contract.outcome`, and the second gains the anchor `impl:kernel.outcome-model.validators`. The pass had both satisfy `spec:kernel.outcome-model`. The second read found that the model reserves three rejection codes for the adapter and gives every other code to a context, while the adapter's step 1 also raises `invalidInput`, a platform code of the outcome boundary, as `spec:context.persistence-adapter` states. The code follows the adapter's Spec, so the two Specs disagree, and both units reference the model until the main thread rules which side changes. +- **Anchor targets name whole Specs.** A unit satisfies a Spec only when the units that satisfy it together realize every entry it promises. `design/PLAN.md` section 6.7 item 7 no longer exempts a pinned declaration that is ahead of its code: a widened pin is a promise the code has not built. The pass below applied the ruling's rule to each Spec: list the Design entries no satisfying unit realizes; none, keep `satisfies`; a coherent part with its own trigger or step, split it into a child Spec that refines the parent, moving the entries verbatim, and keep `satisfies` on the parent; otherwise write `references` until the code realizes what the Spec promises. Astra's gap list was the input, and each gap was checked against the code. The second read found two more Specs the pass had left satisfied with a part no unit builds, the parent use cases and the projection contract. +- **Components.** A component satisfies its seam's design Spec or binds nothing its members bind. `component:platform.context`, `component:platform.command`, `component:platform.read-model` and `component:example.parent` referenced the Spec their members satisfied; their members now reference it and the component binds nothing. `component:example.orders-context` and `component:example.inventory-context` satisfied the example's Spec and bind nothing now. `component:platform.gate` still satisfies `spec:application.write-pause`, which it and its three members realize whole. The kernel, audit and operations components keep their references, to Specs no member binds. +- **The counts.** The graph derives 180 anchors into 429 nodes and 1,497 edges; validate reports 0 errors and the same 12 listed warnings. 9 Specs carry `implemented`, down from 24: the 12 of the pass, `spec:command.receipt-table`, `spec:application.parent-use-cases` and `spec:application.projection-contract` left it. There are 49 `references` edges from 41 units, to 22 Specs. Recipe 27 over `pack:history-view` gives 19 members: 10 referenced, 2 implemented, 7 with a verifier, 11 that build on implemented code, 7 with none of the first three. +- **Layers, contexts and `example/`.** The mapping stands as it was. `design/PLAN.md` section 2.1 lists the twelve components with the layer rule and the context rule in the rulings' words. `check.py` now fails when a component in the context `platform` has a `uses` edge to a component in another context, each end resolved to its component directly or through `memberOf`; its architecture line reads "platform isolation: 7 platform components, 21 uses edges from them, 0 into another context". `example/` keeps its five components and `fixture/` has none. +- **The word layer.** `CONTEXT.md` adds "architectural layer", the Protocol's `layer` on a component anchor, always written in full, and the Layer row avoids it. Renaming the capability steps is the owner's (below). +- **The coverage test.** `tests/pure/byfile-coverage.test.ts` no longer keeps the deleted sidecar's list. It reads the anchors in `src/`, `example/`, `fixture/` and `harness/`, comment form and constant form, checks that they are the code units the graph holds there, and asserts that `byFile` names, for every anchored file, each unit and each Spec it satisfies or references. +- **Tests.** `npm test` passed 1,141 tests in 74 files, with 10 skipped. + +### The per-Spec pass + +| Spec | What no unit builds, checked in the code | Outcome | Why | +|---|---|---|---| +| `spec:context.journal` | The baseline migration (`typeBaselineMigration`, `fnMigrateToSchemaVersion`, `fnDefineMaintenance`, `fnWriteBaseline` and its sequence, `baselineDriver`, `migrationWindow`), the stream rebuild (`fnRebuild`, `rebuildReads`, `limitFoldEvents`) and the derived mapping's parts in `fnLoad`. `src/context/journal.ts` has `createJournal`, `load` and `append`. | references | The migration has its own trigger, a deploy that raises a stream type's `stateSchemaVersion`, and the rebuild its own caller, but both run through `typeStreamRegistration`, `typeLoadedStream` and `fnLoad`, which the code realizes without them and the pinned module lists as ahead. No verbatim move leaves the journal whole. | +| `spec:context.tables` | `tableStreamParts`, its index use and `limitPartDocument`, the derived mapping's table. | references | The derived mapping is a part of its own, taken by a stream type that registers it, but `stateField`, `limitStreamDocument`, `limitIndexes` and `schemaFile` (three tables) state it inside entries the code realizes for the single mapping. | +| `spec:context.persistence-adapter` | `step3a`, migrate on load, and `migrationOnLoadCost`; the derived state in `step2` and `step9`. `src/context/adapter.ts` throws on a row saved under an older meaning. | references | Step 3a is a step of its own, but `step9`, `step10`, `typeStreamResult` and `limitDocumentsWrittenPerCall` count its baselines. | +| `spec:context.queries` | `fnDefineOthers`, `fnHistory`, `fnRebuild`, `typeRebuildResult`, `validatorRebuildResult`, `fnByOperation` and the history and rebuild limits. `src/context/queries.ts` has `defineGet`, `defineList` and `boundedPage`. | references | Journal inspection has its own callers, but `registeredFunctions` names its queries beside `get` and `list`, `fnGet` carries the derived mapping and the migration's transient refusal, and `fnDefineGet` and `fnDefineList` are ahead through `typeStreamRegistration`. | +| `spec:context.context-component` | The `history`, `rebuild` and `byOperation` queries and the `maintenance.writeBaseline` mutation of `convexSurface` and `registeredFunctions`. | references | The surface lists them in the same entries as the built operations and queries. | +| `spec:command.command-declaration` | The history projection in `typeReadModelBinding` and `"tombstone"` in `typeRetention`. | references | Both are pinned declarations ahead of the code. The history binding has no trigger or step of its own. | +| `spec:command.idempotency-and-receipts` | The tombstone class, its response and its retention (`typeReceiptClass`, `tombstoneResponse`, `retentionDefault`). `src/command/receipts.ts` deletes an expired receipt and never writes a tombstone. | references | Tombstones run through `typeReceiptClass` and the classification and insert entries the code realizes. | +| `spec:command.receipt-table`, found by this pass | `tombstoneShape` and the `tombstoneHorizon` of `expiryDefaults`. The table's `tombstone` field is always false. | references | The same tombstone capability, in entries the table otherwise realizes. | +| `spec:command.command-pipeline` | The history projection's write at step 9 (`readModelWrites`, `fnWriteReadModels`). | references | It runs inside step 9, with no trigger or step of its own, and `fnWriteReadModels` is ahead. | +| `spec:application.read-models` | The history read model's write path and its count in `limitReadModelWritesPerCommand`. `src/read-model/write.ts` calls only the per-entity projection. | references | Stated inside the per-entity entries, with no step of its own. | +| `spec:application.generation-registry` | `sourceIndex` in `validatorBatchCursor`, and `tableGenerationProgress` through it. | references | Both pins are ahead of the code. | +| `spec:application.rebuild` | History targets in `typeRebuildConfig`, `fnBatchSizeFor` and `fnResumeChain`, and the history compatibility check its shared entries name. | references | Ahead pairs in shared entries. The paused path itself already has its own Spec, `spec:application.history-rebuild`. | +| `spec:application.orders-inventory-example` | The order allocation history bound to `PlaceOrder` and `CancelOrder`, and journal inspection. The fold in `example/domain/orderAllocationHistory.ts` is bound to no command. | references | The Spec has no Design section; its promises are model terms, and `PlaceOrder`, the `CancelOrder` declaration and the order allocation history carry the binding in terms the code realizes otherwise. The fold now references this Spec too, for the part it realizes. | +| `spec:application.parent-use-cases`, found by the second read | The business failure policy (`typeBusinessFailurePolicy`, `fnCarryOrReject` and the flow steps that pass each context call through it) and the import command (`tableImports`, its indexes and `limitImportBatch`). `example/convex/ordering.ts` declares no business failure and throws a plain error on one, and no code writes an import row. | references | The import command has its own trigger and its entries could move verbatim, but the policy runs through the flows the code realizes, so a split alone leaves the parent partial. | +| `spec:application.projection-contract`, found by the second read | The history writer's callers in `callers`: step 9 and the history batches calling `applyHistoryProjection`. `src/read-model/write.ts` and the rebuild call only `applyProjection`. | references | The same missing path that moved the pipeline and the read models, stated in the shared `callers` entry with no step of its own. | +| `spec:kernel.outcome-model`, found by the second read | No entry is unbuilt, but the code contradicts `rejectionCodeContextOwned`: the adapter's step 1 raises `invalidInput`, which is neither one of the model's three reserved codes nor a context's own. | references, by two units | The pass had the two units satisfy it. The adapter follows `spec:context.persistence-adapter`, so the two Specs disagree, and the main thread rules which side changes. | + +No Spec of the pass split. Seven have a part with its own trigger or step: the journal (baseline migration, stream rebuild), the tables, the adapter and the context component (the derived mapping, migrate on load, the maintenance writer), the queries (journal inspection), and the receipts and the receipt table (tombstones). In each the part runs through entries the code realizes, mostly pinned declarations, so a split needs those entries narrowed to the code and the widening moved into the child, the pattern `spec:application.aggregate-projection` uses. That is a change to realized Spec text, beyond moving entries verbatim, so it waits for a pass the main thread rules. The parent use cases' import command could move verbatim, but the parent would keep the unbuilt policy. + +### What the owner ruled, and what remains + +Closed by the rulings: the layers (R5), the contexts (R6), whether `example/` is architecture (R7), the reference direction (R1), the whole-Spec satisfies (R2, the pass above), and the word layer on this corpus's side (R8). The Protocol closed `#referenceDirection` and `#entryAddressTargets` on `spec:decisions.anchor-binding-grain`, which states `ready` (R9), `#roleVocabulary` on `spec:decisions.architectural-annotation` (R3), and `#attachmentIdentity` on `spec:decisions.anchor-comment-form` (R4): the file is the binding grain of both forms. + +One owner item remains: renaming the capability steps to "stage". "Layer" names one of the seven steps of capability here and the architectural layer of a component in the Protocol. The rename is the one-word-one-meaning answer; it touches a Spec identity (`spec:platform.layers-and-profiles`), 143 numbered uses across 65 carriers and the decisions document the rubric binds this corpus to follow, so it is an adopter-language decision the owner makes deliberately. + +### What the pass leaves + +- **Splits by narrowing.** The seven Specs above, if the main thread rules a pass for them. +- **Two Spec corrections.** The outcome model and the persistence adapter disagree on the adapter's rejection codes: either `invalidInput` joins the codes the model lets the adapter raise, or the adapter stops raising it. The projection contract's `callers` entry names the callers of the history writer, which `spec:application.history-projection` holds; moving that clause there is a correction to realized Spec text. The main thread rules each; until then the code references the Spec. +- **Worked examples in the platform's Specs.** The isolation check reads code. The platform's own Specs still carry orders-inventory worked examples in their Design entries, such as `ordersExample`, `parentPassThrough` and `parentGet` in `spec:context.queries`, `operationExample` and `parentCall` in `spec:context.context-component`, and `contextCall` in `spec:command.command-pipeline`. Moving them to `spec:application.orders-inventory-example` would make the platform's design read without the example, as its code now does. + +## 2026-10-09, anchors in the source at `da56a34` + +The pin moved from `dfa1bd9` to `da56a34`, a build of the Protocol's branch `feature/annotations-v0` (its plan 40), packed from a clean clone as before. The graph schema is `0.8.0`. On the new pin alone nothing this corpus reads moved: 243 Specs, 393 nodes, 1,382 edges, 12 listed warnings. Then the corpus adopted the branch's comment form, and the gap the two sections below name as "an anchor that lives in a test" is closed. An independent read (`gpt-6.1-sol`) of the tree before the adoption found none of it there. A second read by the same model checked every comment-form anchor the adoption placed, and its fold-in moved two bindings, named below. Every number below was measured after that fold-in. + +- **Anchors at their sites.** A comment-form anchor is a `/** … */` block of reserved `@sdp` tags, so the source carries it with no Protocol import and its runtime pays nothing. The 22 implementation anchors that lived in nine test files are now 45 units in `src/`, `example/` and `fixture/`, one per realizing file or site, plus 12 component anchors. No test file keeps an implementation anchor; the 18 constant-form anchors under `harness/` stay. The graph derives 179 anchors into 428 nodes and 1,501 edges, and validate still reports 0 errors and the same 12 warnings. No delivery fact of the 243 Specs changed. +- **The join is gone.** `design/tools/anchor-sites.json`, its pure test and the page's reading of it are deleted. `tests/pure/byfile-coverage.test.ts` asserts that `byFile` names each of the 50 file and Spec pairs the join listed. `byFile("src/read-model/rebuild.ts")` names `spec:application.rebuild` and `spec:application.history-rebuild`. +- **Satisfies or references.** A unit satisfies a Spec its code realizes and references a design it names without realizing it; a reference confers nothing, and it may run either way, code written against a design or code a design rests on. There are 20 `references` edges from 18 units. The six depot units under `fixture/` only reference, because those compositions consume their targets or realize only part of them. A component that exports the implementation its member units satisfy realizes that design too, so `component:platform.gate` satisfies `spec:application.write-pause` beside its three members, and the Spec's delivery facts are unchanged. The rule is realization scope, not the owner-confirmation question both kernel Specs carry (`#deciderEventContract`, `#outcomeRejectionContract`). `src/kernel/decider.ts` holds the decider, its decision context and its fold helpers, so it satisfies `spec:kernel.decider-contract`. `src/kernel/outcome.ts` holds the outcome types the decider reuses: no part of the decider contract, and one part of the outcome model, whose validators sit in `src/context/outcome.ts` and whose reserved rejection codes sit in the adapter. It satisfies neither and references both. +- **The history view.** Recipe 27 over `pack:history-view` gives 19 members: 6 referenced, 11 implemented, 7 with a verifier, 6 with none of the three. `spec:application.history-projection` and `spec:application.history-rebuild` each show one referencing unit and no `implemented`. The page lists each unit at its own file and line, with its role and its component's layer and context, and a "References" list per member. +- **The architecture.** Twelve components, three in each of the `adapter`, `application`, `domain` and `infrastructure` layers; contexts `platform` 7, `orders` 2, `inventory` 2, `orders-inventory` 1. Roles, from recipe 28: service 10, schema 6, contract 5, query 5, decider 2, projection 2, use-case 2, codec 1, event-store 1, operator-entry 1, read-model 1. Component `uses` match the relative imports between the components, cycles included; the two context components reach their domain through `example/domain/index.ts`. `check.py` prints this taxonomy and judges none of it. +- **Tests.** `npm test` passed 1,140 tests in 74 files, with 10 skipped. + +### What the owner rules + +- The layers: the context library and the two context components are `adapter`, with `infrastructure` the alternative; the gate is `infrastructure`, with `application` the alternative. +- The contexts: every `src/` package is `platform`, and the parent composition is `orders-inventory` because it spans both bounded contexts. +- Whether `example/` is architecture. It is the production composition, so it has five components; as a sample it would keep its units and lose them. +- `impl:application.rebuild` references `spec:application.history-rebuild`, a design that rests on the online rebuild rather than one the rebuild was written against. That is the Protocol's open question `#referenceDirection` on `spec:decisions.anchor-binding-grain`, here in this corpus. +- **Whole-Spec satisfies on Specs with designed-but-unbuilt parts.** The second read named twelve Specs as satisfied by code that realizes only part of their Design: `spec:context.journal`, `spec:context.tables`, `spec:context.persistence-adapter`, `spec:context.queries`, `spec:context.context-component`, `spec:command.command-declaration`, `spec:command.idempotency-and-receipts`, `spec:command.command-pipeline`, `spec:application.read-models`, `spec:application.generation-registry`, `spec:application.rebuild` and `spec:application.orders-inventory-example`. The set of satisfied Specs is the one the test-resident anchors had, so this is the corpus's standing posture under `design/PLAN.md` section 6.7 item 7, not a change of the adoption. The owner decides per Spec whether to split it again, move its binding to references, or accept the whole-Spec claim. The Protocol's open question `#entryAddressTargets` on `spec:decisions.anchor-binding-grain` is the engine-side answer being held. + +### What still needs the Protocol + +- **A block binds by file and line.** Move the declaration under a block, or delete it, and the block still binds. The open question `#attachmentIdentity` on `spec:decisions.anchor-comment-form` asks whether a CodeNode records the symbol it binds. + +## 2026-10-04, adopting round 2 at `27b896a` + +The pin moved from `5e29c03` to `27b896a`, a build of the Protocol's branch `feature/adopter-round-2`, packed from a clean clone as before. The graph schema is `0.7.0`. The corpus derives 240 Specs, 6 Packs and 144 anchors into 390 nodes and 1,365 edges, validate reports 0 errors and 12 warnings, each a listed prose mention, and `npm test` passed 1,137 tests in 74 files, with 10 skipped. Of the five workarounds the first Pack page listed in the section below, four are gone. What the round changed here: + +- **Pack order.** `packContext` gives the members in the order the Pack file writes them. The page reads its order there and no longer parses the Pack file's front matter. +- **Question keys.** All 166 open questions in 72 Specs carry a key, and `check.py` fails on one without. The register cites 152 open questions by address. Seven sources keep their Spec and opening text, each on a row whose question was removed. The page joins a register row to a question by address: 47 of the 53 questions on the members of `pack:history-view` join a row, the same 47 the text join found. +- **The next rung.** `SpecContext.nextRungFailures` and recipe 9 name the unmet clauses of the rung above the floor, and a typed-dependency failure lists its targets. Here 22 of 240 Specs cannot reach the rung above the one they state: 10 hold a blocking question, and 12 are examples that state `defined` under a parent that states `scoped`. The other 218 have a floor above their stated rung. The page means the rung above the stated one, so when the floor sits higher it says nothing blocks that rung, and otherwise it prints the Protocol's first unmet clause with the targets and their rungs. Every member of the history view states `defined` with its floor at `ready`, so nothing on its page is held. +- **Recipes 25 and 26.** `check.py` sends the 489 entry addresses written outside the Specs to recipe 25 in one call, 337 Design addresses and 152 question addresses, and all of them resolve. Recipe 26 finds no `dependsOn` cycle, and `check.py` fails on any set it returns. That closes the gap of 2026-10-03, when a script outside the gates found nine Specs in loops. + +### The passes as readers of the page + +The domain pass (Opus 5.5) and the operator pass (`gpt-6-astra`) read `design/pages/history-view.md` first, with five tasks each: find the history contract and where each part is pinned; locate OD-157 and its question; say which members depended on the aggregate form and what still does after OD-162; find one declaration's type comparison; say whether a paused-rebuild example has native-backend evidence. + +The domain pass did the second, fourth and fifth in full and the other two in part. The page gave the history contract's declarations with their lines, and the Contract bullets on order independence and the version rule were only in the linked Spec. OD-162 and its ruling were not on the page. It also asked which command declarations bind the history view. + +The operator pass did all five in full, the third through the linked register, because the page gave no OD-162. It also needed the pinned module the type test imports, `generated/pinned/history-view.ts`, which the page did not link, and the restore's comparison contract behind a link outside the Pack. + +The page now links the pinned module, shows each open question's address, and has a section of the 22 register rows whose sources lie in the members, each with its status: 20 decided, each with the first sentence of its ruling, and 2 not decided, OD-147 and OD-171, whose sources cite no question. OD-162 is among the decided rows: its only source was an owner-queue item in `design/STATE.md`, and the register now also cites the three member lines its ruling changed. Member Contract text stays in the Spec files the page links. What the passes asked for and the page still lacks: + +- OD-163, whose one source is an owner-queue item and no Spec line. +- OD-166, whose question is on `spec:application.restore`, a Spec outside the Pack. +- The command declarations that bind a history view. `spec:application.orders-inventory-example` states them in the prose of three model terms, `PlaceOrder`, `ReceiveStock` and the `CancelOrder` declaration, and no declaration in the code binds the view yet. The page reads no prose, so it shows nothing for them. The gap is this corpus's: a keyed Design entry pinning each binding would reach the page through recipe 24. + +### What the adoption deleted + +From `scripts/pack-page.mjs`: `authoredMembers` (15 lines, the Pack file's front matter read by hand), `floorTable` (11 lines, the walk of the exported `readinessFloors` over `buildGraphIndex`), the `nextRung` that walked that table (10 lines), the `belowDefined` recomputation of the typed-dependency targets (5 lines), the 11 lines of `readGraph` that compared the Pack file with the graph, and the join of a question to a register row by its folded opening text. The imports of `buildGraphIndex` and `readinessFloors` went with them. In their place are `packMembers` (9 lines, which refuses a repeated or unresolved member) and a `nextRung` of 15 lines that reads `nextRungFailures` and compares the floor with the stated rung. The script still grew, from 1,114 to 1,185 lines, with the register section and the checks of the register's new source shapes. + +From `design/tools/check.py`: the address grammar, the two lines of its graph query that listed every Spec's Design and UI keys, and the 9-line lookup against them. `address_failures` reads recipe 25's rows in their place, and `cycle_failures` reads recipe 26's sets. Finding the addresses in files and naming each one's file and line stay this corpus's. Two tests in `design/tools/test_decisions.py` derive a sample corpus and show a broken address and a planted cycle failing. + +### What still needs the Protocol + +- **An entry's line.** The page still reads each member's carrier to find the line of every Design entry and open question, because the graph records the line of a binding and not of an entry. Any page that links an address to its line needs it. +- **A recipe's parameter as data.** Recipe 25 takes its addresses on its opening `const` line. `design/tools/decisions.py` rewrites that line with a regular expression and passes 489 addresses, 28,788 characters of JSON, on the command line. Recipe 4 takes its files as JSON in `SDP_CHANGED_FILES_JSON`. The same for recipe 25 would let an adopter run the body as shipped. +- **An anchor that lives in a test.** Unchanged since the first page: `design/tools/anchor-sites.json` is still the adopter's join, and `byFile` on `src/read-model/rebuild.ts` still returns no Spec. + +### Annotating every design pattern in code, an open question + +The owner on the first generation of this tooling: "I loved how annotations for architecturally significant patterns and relationships worked in gen 1." And: "AI agents were constantly against my approach. But that is the strongest value. It was misunderstood that the issue was typescript + gherking -> dual extraction. That was not the issue." And on the claim map of the Studio slice: "You would not be looking for shapes in the codebase that resemble the original desing. All of those from the docs would be annottated, all are significant." + +Two of the Protocol's decision records rule on this. `spec:decisions.architectural-significance-rides-primitives` rules that architectural significance is authored on the existing Spec kinds, relations and anchors, with no pattern kind or vocabulary, so an anchor satisfies only a realized Spec. `spec:decisions.jsdoc-graph-extraction-refused` rules that source commentary never authors graph content and that the anchor constant is the only path from code into the graph. The owner contests both records' account of gen 1. The rulings stand for this round. The Studio slice's claim map, 81 rows that an Opus agent found by reading the code for shapes that resemble each claim of the original design, is the manual work that annotating every design pattern in the code would turn into a projection, and it is the evidence for the later decision. This corpus shows the same question at a smaller scale: the declarations that bind a history view are a relationship the design states in prose and the code will realize, and a page can show a relationship only when the graph holds it. + +## 2026-10-04, a page per capability Pack, built by the adopter + +The unit `pack-page` built round 1 of `sdp-feedback-03-selection.md` beside this file. `scripts/pack-page.mjs` reads the graph in memory, through the package's exported `extract` and `createReader`. It writes `design/pages/history-view.md`, one committed page for `pack:history-view`: the members in authored order, each with its rung, floor, next rung and the clause that blocks it, its anchors and verifiers, its pinned pairs, the dependencies outside the Pack, and the open questions joined to the register. `check.py` fails when the page is behind the tree. Five things the page needed are not in the Protocol, and the page works around each: + +- **A floor at a rung above the stated one.** `evaluateReadinessFloor(node, index)` takes no target rung, and `floorFailures` covers only the stated rung. So recipe 9 answers `firstUnmetClause: null` for a Spec that stated `defined` and could not state `ready`. The page walks the exported `readinessFloors` with `buildGraphIndex` one rung up, using the Protocol's own predicates. A target rung on the evaluator would retire that walk, and recipe 9 could report the next rung's blocker. +- **Which targets fail the typed-dependency clause.** Its failure names the clause, not the dependencies that break it, so the page recomputes "a target stating below `defined`". +- **Pack order.** The reader's `packContext` and `graph.json` still sort the members, so the page reads the list in the `.pack.sdp.md` file. +- **An anchor that lives in a test.** The read-model library and the gate cannot import the Protocol, so their anchors sit in tests. `design/tools/anchor-sites.json` names the source files each such anchor stands for, and a pure test keeps it true. The graph does not hold that link: `byFile("src/read-model/rebuild.ts")` still returns no Spec, and recipe 4 does not see it. A companion binding on the anchor would let the graph see it. +- **A question's identity.** The register now cites an open question by its Spec and the opening of its text, verified by `check.py`, in place of a position that collided with the entry-address grammar and had drifted in 47 of 151 sources. An optional question key in the Protocol would make that an address. + +The page was tested against the reader this file's section of the same day describes. A `gpt-6-astra` job read only the page and the files it links to, and was given five tasks: find the history contract, locate OD-157, explain the aggregate dependency, find one declaration's type comparison, and say whether a paused-rebuild example has native-backend evidence. It did four in full. It found OD-157 but could not tell what "blocks S5" meant, and the page now says. That reader is the one plan 35 asks to be named before Spec Studio resumes. + +## 2026-10-04, the design as a person working in the repository reads it + +The unit `design-in-the-graph` gave this corpus the content a design ahead of its code needs. It has one maturity per Spec, `pack:history-view` in reading order, pinned declarations compiled and compared with the code, entry addresses in code, tests and the register, and two passes. Read through `npx sdp view` at `d39c89e` on the pin `5e29c03`, little of it reaches a person who navigates the repository and works on it. This section describes that reader and what the reader meets. It asks for no particular feature. + +- **A design reads as escaped JSON.** A Spec page prints `## Design` as one fenced JSON object. The history projection's ten entries, `HistoryProjection` and `applyHistoryProjection` among them, are escaped strings, each on one line. The corpus has 977 Design entries in 59 Specs, and its reviewers read the `.sdp.md` files instead. `spec:decisions.authored-entry-order` drafts a list rendering, and its blocking question waits for the owner to read ordered output. This corpus is that output. +- **A reader has no way in.** The index is one table with a row for each of the 239 Specs. Where to start comes from `design/STATE.md`, `design/ROADMAP.md` and memory, and no page answers it. +- **A Pack page shows no shape.** `pack:history-view` is authored in reading order with the history projection first, and its page lists the 15 members by id, every row alike. A reader cannot tell which members the design adds, which it rests on, or which built members its pins will change. The page also shows `spec:application.rebuild` with no implementation binding although `src/read-model/rebuild.ts` realizes its online path. That one is this corpus's missing anchor, lead 44 of `design/STATE.md`, and it misleads the reader all the same. +- **What the corpus knows about its own design is on no page.** In `tests/types/pinned.test-d.ts`, 53 pins equal their code, 14 lead it and 9 are not compared. `check.py` resolves 84 entry addresses outside the Specs, `design/decisions/register.json` joins open questions to decisions, and git holds the passes. A person learns any of it only by reading those files. The pinned module is the adopter's recipe under the Protocol's ruling, so presenting it is this corpus's work, but no shipped view has a place for the status of one entry. +- **Spec, code and test connect one way, and only for a whole Spec.** A Spec page lists each implementation anchor's file and line. The read-model library cannot import the Protocol, so its anchors sit in `tests/pure/read-model.test.ts`, and the link leads to a test and not to `src/read-model/`. From a file, the reader's `byFile` and recipe 4 name the Specs it touches, and no page does. Below the Spec, an entry address is a citation with no edge, as pull request 27 ruled. + +The data each of these needs is already in the graph or beside it: the authored member order, the delivery facts, the open questions, the examples with their tier in the opening line, the pinned module's three lists and the entry addresses. The Protocol's plan 35 defers Spec Studio until a recorded ruling names "the reader Markdown Design Review does not serve". The reader here is that one: a person inside the repository, working on it. + +## 2026-10-03, adopting the merge of pull request 27 at `5e29c03` + +The pin moved from `5993da7` to the merge commit of pull request 27, packed from a clean clone as before. The graph kept its shape (237 Specs, 384 nodes, 1,251 edges) and graph schema `0.6.0` changed nothing this corpus reads: `npm test` passed 1,050 and the typecheck was clean on the new pin with no other change. What the version gave, and what it asked of this corpus: + +- **The mention warnings were mostly missing edges.** Validate reported 83 `conformance/prose-mentions` pairs. Read one by one, 70 were dependencies the frontmatter never declared, now 56 relations (52 `dependsOn`, 3 `constrainedBy`, 1 `decidedBy`, one relation backing both directions where two Specs name each other); 12 are prose that no typed relation fits, eight of them in examples, which keep only `refines` and `verifies`; one moved with the text that carried it. The impact recipes read more of the corpus now. On the pull request's question whether "let the warning stand" teaches readers to skip warnings: this corpus answers it in its own gate. `check.py` accepts a pair warning only when the pair is listed with a one-line reason in `design/tools/prose-mentions.json`, and refuses a listed pair that no longer warns. No Protocol change is asked for. +- **Declaring the missing edges closed `dependsOn` cycles.** Two older edges ran backwards (the event envelope depended on the pipeline and the adapter, which check its bounds), and the new edges closed nine Specs into loops through them. Nothing in validate or recipe 21 reports a `dependsOn` cycle; a script found it. A cycle may be lawful data, but a recipe that lists the strongly connected sets of typed dependencies would let an adopter see one when it forms. +- **Entry addresses outside the Specs.** Validate checks an address in a Spec's prose, and it refused a broken one here as the Spec says. This corpus also cites Spec content from code comments, tests, the decision register, the ledger and the advisors' memos, and those cited lines: one test comment named `step5:125` of a Spec whose line 125 had held `step1` since an earlier edit. `check.py` now resolves every `spec:#design.` in those files through one `sdp q` body over `g.specContext(id).sections`. A shipped recipe that resolves a list of addresses would save every adopter writing that body. The register's own references to open questions, `spec:#`, use the `#` the Protocol now reserves; they live in JSON and nothing refuses them, but an open question has no address, and a register that cites one by index moves when a question above it moves. +- **Recipe 24 and the declarations module.** Recipe 24 listed 400 declarations in 37 Specs here at adoption, 401 in 39 after the split below. The module is the adopter's recipe, as the ruling says, and this corpus built it: `scripts/pinned.mjs` reads the recipe's body from the shipped catalog, takes one Pack's declarations and the closure of the names they use that other Specs pin, emits by this corpus's key prefixes (`type`, `fn`, `validator`, `table`), and `tests/types/pinned.test-d.ts` asserts the code's type equals the pin for each pair the code realizes. A changed pin fails `npm run test:types` at the pair's entry address. Three things the ruling's text did not foresee. First, a pin can lead its code on purpose: a design for coming work pins `ReadModelBinding` over three forms while the code admits one, so the test keeps such pairs in a list that asserts the two still differ, which fails the day the code catches up. Second, the first compile found a name pinned twice and sixteen names the design uses and no Spec pins, which is the module's whole value before code exists; two of them were the history design's own and are pinned now. Third, declarations that do not emit are common: `index*` fragments, `limit*` formulas, and the 51 `fn*` entries written as Convex registrations, `export const name = mutation({ ... })`, which no `declare` form takes. +- **Maturity is per Spec, so a Spec of mixed maturity had to split.** The projection contract held a per-entity form with code, a history form designed and not built, and an aggregate form deferred, under one readiness and one implementation binding. It is three Specs now, at `defined` with an anchor, `defined` without one, and `scoped` with its trigger, and the Pack of the history view shows the difference in its member table. The split also surfaced, through the `ready` floor's typed-dependency clause, that three implemented Specs promise the deferred form in their signatures. That is the graph working, and it is a design question for this corpus, not the Protocol. +- **The Design Review sorts a Pack's members by id.** The authoring skill says to preserve authored membership order, and `pack:history-view` is written in reading order, but its page lists members alphabetically. For a Pack that presents one capability's design, the authored order is the reading order. + +## 2026-10-03, round 2 of the roadmap + +- A model Spec now owns an example space: `orders-inventory-example.sdp.md` holds the `gwt-vocabulary` of its six examples, and the pinned Protocol validates it with 0 errors and 0 warnings, while the model template of this corpus's `design/PLAN.md` 6.1 lists only Intent and Model, so nothing told the author it was allowed. + +## 2026-10-01, slice S0, the first build on the Protocol at `5993da7` + +### A Spec with designed stubs is a maturity the Protocol cannot state + +The owner's direction during this slice: the Protocol should support Specs at any level of maturity, and that includes a Spec whose key abstractions are designed as stubs. The earlier generation of this tooling, architect, did that, and the Protocol is its successor. + +What happened here. Before a build job, a design agent decides the key abstractions of a Spec: the exported interface of a module, the shape of a record, the names. The slice's first design brief was this corpus's own mistake: it asked for a design that left the implementer nothing to decide, with no bound, and the agent wrote toward a long design file outside the repository and then prototyped the code, for about an hour. What the mistake showed is that the corpus had no home for the output of a design pass. A Spec that can hold the stubs is what bounds a design pass: the design is done when the Spec's key abstractions are stated as stubs, and the implementer writes the rest. + +What the Protocol offers today, and where it stops: + +1. **Readiness has no rung for it.** The rungs are `idea`, `scoped`, `defined` and `ready`, and they describe the intent. A Spec at `defined` with no design and a Spec at `defined` whose interfaces are settled look the same in the graph, so the build backlog cannot tell which Specs an implementer can start on. +2. **A stub in code reads as `implemented`.** The only binding from code to a Spec is a `codeAnchor` with `satisfies`, and the Protocol's `CONTEXT.md` defines `implemented` as one or more `satisfies` edges that resolve to the Spec, and the drift alarm as `implemented` without `ready`. A file of signatures with no bodies would claim the Spec is realized, and on a Spec below `ready` it would raise the drift alarm. There is no anchor, or flag on an anchor, that says "this is the designed surface, not the implementation". +3. **A stub in the Spec is one line of prose.** A `## Design` entry is `key: value` on one physical line, and fenced code is refused there. This corpus has 792 Design entries, counted as the bullets under every `## Design` heading on 2026-10-01, and 653 of them hold inline code; a multi-member interface on one line is hard to read and to review. Nothing checks a Design entry's signature against the exported one, so the two drift apart without a finding. Authored order for Design entries, already asked for below, matters more once the entries are stubs a reader walks in order. + +The owner's proposal for item 3: the Markdown carrier would accept a `ts` fenced block in a Spec, holding the design of its key contracts: the interfaces, types and function signatures a reader would otherwise find squeezed into one-line Design entries. Today the carrier accepts two fences, `gwt-vocabulary` under `## Example space` and `gwt` on an example, and refuses every other. + +What that leaves for the maintainers to decide: + +- Where the block is lawful: under `## Design`, under `## Contract`, or both, and whether one block per Spec or one per named contract. +- Whether the graph reads it. Read as data, each declared name could be an entry that entry search (recipe 23) finds, and a later check could compare a stub with the exported signature of the code anchored to the Spec. Read as content only, it is still what a reviewer and an implementer need. +- How a Spec with such a block shows in the graph, which is items 1 and 2: a `designed` fact or rung that the build backlog can filter on, and a code binding for a stub file that does not derive `implemented`. + +What this corpus does until then: the design agent proposes `## Design` entries and rules for the Specs a slice builds, signatures only, one line each, and the main thread writes them into the Spec. Stub files in code carry no anchor until they have bodies. If the carrier comes to accept a `ts` block, the Design entries that are declarations move into blocks, and the design pass writes its stubs there. + +### What the build learned about binding an example to a test + +Slice S0 bound eighteen examples to native tests with `bindExample` from the Vitest adapter. Seven things came up. The first two were read in `dist/adapters/vitest.js` of the pinned package, the others met during the build on 2026-10-01. + +1. **A step is bound by its exact text.** `planExample` looks a step up as `bindings[step.text]`. When a review rewords a Then, the test stops compiling, because the contract's step type changed, and a run that skips the compiler fails with `bindings[step.text] is not a function`. That is the right alarm when the meaning changed. It is noise when the wording was polished and the meaning stayed. This corpus now changes an example's steps and its test in the same commit. A stable key per step, beside its text, would let prose be reworded without touching the test, and the adapter could then say "no binding for step " in place of the type error. +2. **`bindExample`'s `after` runs only when every step passed.** The adapter awaits the plan and then `after`, with no `finally`. A test that owns a process cannot clean up there: a failed step would leave the process running. The harness registers its cleanup with Vitest's `onTestFinished` before the first await, and `after` is unused. A `finally` around the plan, or a sentence in the adapter's documentation, would make this plain. +3. **A test binds only to a Spec under the same root.** The corpus root was `design/`. The tests are outside it, so the root moved to the repository root, and `check.py` and every `sdp` call moved with it. +4. **A parent with no example space gets no registrar.** The harness Spec and the fact Specs have examples and no `## Example space`. Their tests import the example's step contract from `generated/contracts/` and call `bindExample`. That works, and it means `sdp build` has to run before the compiler: `npm run typecheck` and every test command here start with it. +5. **`specTest`'s `verifies` takes one `ref`.** A test that covers two examples needs two anchors. That fits one test per example, which is how every native test here is written, and is worth a line in the authoring skill. + +6. **The slice built code for a day before any of it carried a `codeAnchor`.** The tests were anchored from the first build, because `bindExample` needs the contract. Nothing asked for the other half, so the graph showed the harness Spec with verifiers and no implementation, the drift alarm (recipe 2) returned nothing, and recipe 4 answered `coverageUnknown` for every harness file. A side read of the corpus found it at the close. One anchor on the harness kept validate at 0 warnings and made recipe 2 list the Spec as built and not `ready`, which is this project's queue of Specs waiting for the owner. A warning for "a Spec whose examples are bound and that no code satisfies" would have asked at the right moment. +7. **The build backlog reads stated `ready`.** Here the owner states `ready` after a slice passes, so recipe 1 stays empty for the whole build. The same question at `defined` with a clear floor is a short local body and gave 80 Specs. A parameter for the rung would make recipe 1 serve both ways of working. + +One more, about evidence. Twice in this slice a reviewer's mutation showed a bound test that passed while proving less than its example said: of twenty mutations run on a native backend, nine left their test green, three on the first build and six on the second. The binding says which test claims an example. It cannot say whether the test would fail if the example were false. A corpus that leans on `verified` needs mutation runs beside it, and the Protocol's documentation could say so where it defines the term. + +## 2026-10-01, adopting pull request 26 at `5993da7` + +The corpus now depends on the Protocol at commit `5993da7b8f61ce00853b790cd9ffb428842b66d0`, the head of `feature/adopter-driven-hardening`. + +### What blocks an adopter + +1. **A dependency on the commit hash installs no CLI.** The pull request says the adopter can depend on the branch by commit hash. `npm install "github:libar-dev/software-delivery-protocol#5993da7…"` succeeds and installs `CONTEXT.md`, `docs/`, `specs/`, `.agents/` and `package.json`, with no `dist/` and no `node_modules/.bin/sdp`. `dist/` is ignored by git and the package has no `prepare` script, so nothing builds it on install. This corpus works around it: a clean clone at the commit, `npm ci`, `npm run build`, `npm pack`, and the tarball committed under `vendor/` as a `file:` dependency. A `prepare` script that runs the build would make the hash usable. So would a published prerelease. + +2. **The CLI cannot say which engine it is.** `sdp --version` prints the usage text. The package version is `0.0.0` at every commit, and `graph.json` carries `schemaVersion` and nothing about the engine. This corpus was pinned to a sibling checkout's `dist/` until today. When the unbound-example change landed there, the corpus's own check failed with not one Spec changed: 54 expected warnings became 0. An adopter could not tell the tool moving from the corpus moving. The check now prints the dependency string from `package.json`. A `--version` that prints the package version and the commit it was built from would let any adopter record what produced a result. + +### What worked, against the acceptance tests in plan 39 + +| Proposal | Test | Result | +|---|---|---| +| P2 | Unicode angle brackets become real ones and validate stays clean | 496 characters on 114 lines of 29 Specs replaced, every one inside an inline code span. `sdp validate design`: 0 errors, 0 warnings | +| P4 | Validate prints no warning on a corpus with no test anchors | 0 warnings, down from 54 | +| P3 | The assumed facts state `scoped`, and readiness divergence names a dependent that states `ready` | F14, F15 and F17 state `scoped` behind a blocking question that names the probe. They have 11 distinct `constrainedBy` dependents. On a scratch copy, stating `ready` on one of them makes recipe 7 return that Spec | +| P5 | The census sentence and the open-question table come from commands | Both hand-kept registers are deleted from the README. `sdp census` gives the counts. Recipe 20 gives 129 questions on 67 Specs, 12 of them blocking | + +The authoring skill's section on what is not settled was applied as written, in two places. Nine deferred Specs each carry a blocking question that names the trigger. Three assumed facts each carry a blocking question that names the probe. Both were one-word edits to questions the corpus already had, and the corpus's own rule that a blocking question forces `scoped` agreed with the floor without a change. + +Five checks left this corpus's check script because the Protocol now covers them: one warning per example, the README's warning count, the census sentence, the open-question tally and the open-question table. + +### Smaller things + +- **Recipe bodies are only in a Markdown file.** A script that wants recipe 20 has to cut the fenced block out of `recipes.md`. One file per body beside the catalog would let an adopter run `sdp q "$(cat …/20-open-question-register.js)"`. That adds no query verb, so it stays inside the agent-front-door ruling. +- **Recipe 22 on this corpus:** 153 mentions, 112 pairs, 34 backed, 78 unbacked, none unresolved. Of the 78, 28 are declared from the target's side and 50 from neither. The `declaredByTarget` column is what makes the list usable: it separates a parent naming its child from a relation nobody declared. Whether the 50 are missing relations or plain pointers is not known yet. The slices will say, one family at a time, and the answer bears on what the checked-mentions decision should warn about. +- **The derived step contracts are a head start.** `validate` writes 71 contract modules under `generated/contracts/`, one per example and per example space. The first tests in this repository will bind to them. + +### Still wanted from the open questions on the pull request + +- Authored order for `design` entries (`spec:extraction.open-section-order`). The Design Review still shows this corpus's pinned declarations alphabetized inside a JSON block, which is how a reviewer reads its 762 keyed Design entries today. +- Checked mentions (`spec:decisions.checked-mentions`). One concept stated in two Specs that disagree is still this corpus's largest class of defect, and the validator cannot see it. + +## 2026-10-02, from slice S1 + +Slice S1 built Layers 0 and 1 from the Specs' `## Design` entries with seven agents in an hour. What the Protocol did and did not carry: + +- **Pinned signatures were buildable.** Every kernel declaration compiled as written, and the builders reported each context and command declaration that did not, with file and line. A one-line Design entry was enough for an agent to build from. +- **A code anchor cannot live in code that may not import the Protocol.** The kernel imports nothing but a type, and the anchor builder comes from the Protocol's package, whose entry imports `fs`, `path` and `ts-morph`. The kernel's anchor sits in a test file, so the graph cannot say what a change under `src/kernel` reaches. An anchor that is a comment, or an import with no runtime, would fix it. +- **`sdp build` writes outside `generated/`.** For a suite that binds an example whose parent has an example space it writes a `*.test.generated.ts` file beside the suite. The repository ignores those files in git, Prettier and ESLint. +- **Build status has no home in the graph.** The thread recorded what the slice built as Design entries in the Specs, and the owner ruled that temporal information is not part of a Spec. What is built is derivable from anchors only for whole Specs; "this Spec is half built, these entries exist" has no carrier, so it lives in the project's state file. +- **An example written before its composition existed had to be rebound.** Eighteen examples bound values of an application that is not built yet (`PlaceOrder`) and were rebound to the fixture's commands. Nothing in the graph says which composition an example runs on; the corpus says it in prose. diff --git a/docs/feedback/sdp-feedback-02.md b/docs/feedback/sdp-feedback-02.md new file mode 100644 index 0000000..f27c6cc --- /dev/null +++ b/docs/feedback/sdp-feedback-02.md @@ -0,0 +1,183 @@ +**I would make the human reading experience the next SDP milestone.** The platform now has enough design content to support useful review, but the reader still has to reconstruct the design from several files and tools. The next work should let a person open one capability and understand its design, unresolved choices, relationship to code, and available proof. + +I read the updated [feedback](sdp-feedback-01.md), [platform PR #12](https://github.com/libar-dev/libar-platform/pull/12), and [SDP PR #27](https://github.com/libar-dev/software-delivery-protocol/pull/27). I also queried the platform at `d39c89e` using its installed SDP pin, `5e29c03`, and rendered the existing pages in memory. + +The current graph confirms 239 Specs, six Packs, 977 Design entries across 59 Specs, and 401 declaration entries across 39 Specs. All 15 members of `pack:history-view` state `defined`. That last fact explains why a maturity table alone cannot present this work well. + +Three implementation details affect the recommendations: + +- Pack membership retains authored order in the freshly derived edges, but `packContext` sorts members by ID, and graph serialization sorts the edges. Preserving reading order needs to survive the whole path. +- `byFile("src/read-model/rebuild.ts")` returns no Specs. A file browser would expose the existing binding gap unless that is addressed. +- `sdp validate` calls the build routine, which removes generated projection directories. A viewer needs a refresh path that does not remove the pages being read. These behaviors are visible in the installed engine (`node_modules/@libar-dev/software-delivery-protocol/dist/cli/sdp.js`, line 7756). + +Here are the changes I would consider, starting with the most immediately useful. + +The first group makes individual Specs readable. + +1. **Finish the proposed rendering of Design entries.** Render each keyed entry as a readable item in authored order. Show the key, its value, and a link to the source. Keep JSON only for structured values that need it. This has a carrying decision already, authored-entry-order (`node_modules/@libar-dev/software-delivery-protocol/specs/decisions/authored-entry-order.sdp.md`). It is the smallest change that improves all 977 entries at once. + +2. **Format declarations for reading in the generated view.** The source can retain the one-line declaration shape while the page displays a formatted code block, followed by the explanatory prose. For example, the existing declaration could appear as: + + ```ts + interface HistoryProjection { + name: string; + version: number; + table: string; + rowBudgetBytes?: number; + + keyOf( + tenantId: string, + event: EventEnvelope + ): string | null; + + project( + tenantId: string, + prior: Row | null, + events: EventEnvelope[] + ): Row; + } + ``` + + This would be a proposed presentation change. It would not require changing the authored carrier to accept multiline declarations. The platform can supply the TypeScript formatter because SDP deliberately treats declaration contents as language-independent text. Preserve the original text for copying, and fall back to it when formatting fails. + +3. **Give every addressable entry a working page link.** A citation such as `spec:application.history-projection#design.typeHistoryProjection` should open the corresponding entry, with a “Copy address” action. Generate the link from the existing address grammar. Entries that the grammar cannot address should remain visibly unaddressable rather than receiving an invented canonical identity. + +4. **Give a Spec a short table of contents.** A person should be able to jump to intent, questions, contract, declarations, examples, and bindings. A long Spec such as rebuild especially needs this. Entry keys can form a second level under Design, preserving authored order. + +5. **Separate declaration text from its behavioral explanation.** Many Design entries combine a signature with a long description of ordering, errors, bounds, and special cases. The page can present the opening code span as code and the remainder as prose without changing ownership or meaning. This would make `fnApplyHistoryProjection` substantially easier to review. + +6. **Make the page lead with what the reader needs to decide.** My suggested anatomy is: title and intent, stated readiness and structural floor, open questions, contract and design, examples and evidence, then the full relation and binding inventory. Keep all the content, but put exhaustive metadata after the material a reviewer needs first. + +7. **Show reference previews.** Hovering or focusing a Spec reference should show its title, intent, readiness, and relationship to the current Spec. An entry reference should preview that entry. This would reduce the constant navigation between the history projection, envelope, row conventions, and rebuild without duplicating their text in authored Specs. + +SDP #27 explicitly leaves readable Design rendering as a follow-up requiring a narrow reopening of the projection freeze. It also retains the one-line declaration ruling. Those are useful boundaries for the first implementation. [SDP PR #27](https://github.com/libar-dev/software-delivery-protocol/pull/27) + +The next group gives a reader a way into the corpus. + +8. **Make Packs the main entrance to active design work.** The home page should start with capability Packs and a brief description of what each lets the reader understand. The complete Spec table remains available as an inventory. The existing package Packs can continue serving broad orientation; smaller capability Packs serve decisions and implementation preparation. + +9. **Preserve Pack order through extraction, serialization, the reader, and rendering.** Add an explicit representation of authored order, either on the Pack or its membership data, and validate that it agrees with membership. The acceptance case should serialize and reload the graph before rendering. A fix that only reads the in-memory edge order would leave consumers of `graph.json` with different behavior. + +10. **Give a Pack an authored reading guide.** The [history-view Pack](../../design/specs/history-view.pack.sdp.md) already explains why each group belongs. Present that guide as short reading stages beside the member list. A future Pack extension could support named groups containing references, but those groups should organize reading only. They should not restate contracts or store implementation status. + +11. **Distinguish the design being considered from its supporting context.** A capability page should identify the focal Spec, shared contracts, examples, supporting facts, and shaping decisions. Start with presentation configuration or explicit reading groups. Derive “changed in this PR” from the selected Git comparison. Avoid permanently recording “new” or “already built” in a Pack, since those descriptions change. + +12. **Show dependencies outside the Pack.** `aggregate-projection` is an excellent example. It is outside `pack:history-view`, yet its `scoped` readiness affects three members’ ability to state `ready`. The page should show that dependency at the boundary, explain why it matters, and offer to expand it. Automatically adding every transitive dependency to Pack membership would make the Pack unwieldy. + +13. **Offer a refinement outline as well as Pack navigation.** Packs answer “what should I read together?” The `refines` graph answers “where does this promise belong?” Both are useful. Display the authored refinement hierarchy, with cross-links for other relations. Do not turn folder names into an additional authoritative hierarchy. + +14. **Give humans the same search entrances agents have.** One search box should accept a concept, title, Spec ID, entry key, declaration name, or file path. Results should say why they matched: title, Design entry, declaration, or source binding. Existing `findByConcept`, `byFile`, and recipes 23–24 provide most of the underlying data. Search for `HistoryProjection` should land on the declaration itself. + +15. **Split Specs when they need independent decisions, then reunite them in a Pack.** Continue the projection-contract split where it helps, but avoid splitting every type into its own Spec. A useful boundary is independent readiness, unresolved questions, behavioral ownership, or review scope. Shared content keeps one home. The Pack restores the readable capability narrative across those boundaries. + +A proposed history-view page could present its reading guide like this. These are suggested presentation groups, not new SDP states: + +| Reading purpose | Material | What the page should explain | +|---|---|---| +| Understand the proposed form | History projection | Shape, fold rules, bounds, and remaining questions | +| Understand shared conventions | Projection contract and read models | What the history form reuses | +| Understand operation | Rebuild, write pause, generation registry | How interruption, resume, abort, and switching work | +| Understand integration | Command declaration and pipeline | Which existing promises extend to the history form | +| Inspect proof obligations | Paused-rebuild examples | Required tier and whether a verifier is bound | +| Inspect assumptions | Event envelope, journal, queries, D9 | The contracts and decision this design rests on | +| Inspect an external dependency | Aggregate projection | Why it affects the readiness floor of three members | + +This is the shape missing from the current alphabetic table. + +The next group makes status and evidence useful without overstating them. + +16. **Keep readiness, design detail, bindings, and test results separate.** A page should be able to say: + + - Stated readiness: `defined`. + - Structural floor reached: `ready`. + - Declaration present: yes. + - Implementation binding: none recorded. + - Verifier binding: none recorded. + - Selected test result: unavailable. + + These facts answer different questions. A single “designed,” “implemented,” or percentage-complete badge would hide distinctions the platform needs. + +17. **Explain what prevents the next readiness rung.** Current `floorFailures` describes whether the stated rung is honest. It can be empty while a higher rung remains unavailable. I checked the three aggregate-dependent Specs: evaluating their prospective `ready` statement produces the typed-dependency failure. A human view should explain the target and clause, for example: “Cannot state ready because aggregate projection states scoped.” This can begin as a recipe using the existing floor evaluator, without editing the Spec. + +18. **Put the pinned type-test inventory beside the declaration.** The current [type-test file](../../tests/types/pinned.test-d.ts#L42) declares 53 matching pairs, 14 expected differences, and nine pairs it does not compare. A platform adapter can expose each pair’s address, comparison mode, code symbol, and test location. A declaration page can then show “expected to match” or “expected to differ,” with the actual execution result loaded separately. + +19. **Expose what a type comparison leaves out.** “Compared after removing the context parameter” matters. So do placeholder declarations, imported fallback types, and declarations the emitter skips. Display these qualifications beside the comparison. Also distinguish an entry with no comparison from an entry that the emitter cannot emit. The [emitter](../../scripts/pinned.mjs) already contains much of this information. + +20. **Strengthen selected “ahead” checks around the intended difference.** An inequality assertion shows that the types differ. It does not establish that they differ for the expected reason. For consequential pairs, add compiled-tier assertions for the supported common shape and the specific proposed extension. For example, check the intended projection forms explicitly. This is worth doing selectively where unrelated drift could otherwise remain green. + +21. **Show behavioral proof beside type compatibility.** A declaration matching its implementation does not prove event interleaving, cancellation behavior, or interruption recovery. The page should place relevant examples and their required tiers beside the signature. This creates a useful distinction between “the shapes agree” and “the promised behavior has evidence.” + +22. **Repair missing bindings before using their absence to guide work.** Rebuild and write pause are already recorded as binding gaps. Address those, while checking whether each carrying Spec’s scope is narrow enough for an honest binding. A broad workflow with only one realized path may need a better boundary. Until then, render “no implementation binding recorded,” which is what the graph actually knows. + +23. **Represent the binding location and implementation location separately.** The read-model anchors live in [a test file](../../tests/pure/read-model.test.ts#L17) because deployed code cannot import SDP. A useful page needs to show both where that assertion is made and which implementation it points to. Start with an explicit platform mapping or an extractor-owned companion-binding design. If SDP generalizes this, preserve the authored binding and validate its target. Do not silently treat a test import as proof that the imported module satisfies the Spec. + +24. **Add a reverse file view.** Opening a source file should show the Specs bound to it, relevant declarations, examples, and affected Packs. Use `byFile` for recorded coverage, and display “coverage unknown” when it has none. This is especially useful during implementation and review, where the reader often arrives holding a filename rather than a Spec ID. + +25. **Attach test results as a separate, clearly identified input.** The platform already has run records. Join them by verifier identity and show commit, tier, composition, target, and result. Keep compiled, pure test, `convex-test`, and native backend distinct. If the selected run predates the source being read, say so. This should consume existing test output rather than introduce another manually maintained proof log. + +The next group supports repeated design passes. + +26. **Present unresolved choices as an actionable decision list.** Join recipe 20 with [the decision register](../../design/decisions/register.json#L6403). Show the question, provisional reading, class, advisor, affected Specs, and downstream consequences. Preserve the difference between a question that blocks an SDP readiness floor and a register decision that blocks a platform unit. OD-157 is a concrete case: its Spec question is non-blocking for readiness, while the register assigns an owner decision affecting S5. + +27. **Give open questions stable identities.** Positional references such as `spec:application.rebuild#7` are fragile and are not current SDP entry addresses. A small SDP extension could add optional question keys and a ruled address form. Migration should validate uniqueness and detect missing references. Until then, the existing `OD-*` identity can be the stable presentation link, with its Spec association checked by platform tooling. Any proposed question-address syntax should remain explicitly a proposal until ruled. + +28. **Make a review lens a filter over the same capability.** Domain, Convex, operator, and implementation views can reorder or emphasize relevant questions, entries, examples, and constraints. They should open the same carrying Specs. The existing [pass task](../../design/advisors/task-pass.md) already supplies the working pattern. A lens does not need a new Spec kind or a compulsory stage sequence. + +29. **Generate the next pass’s reading material.** Given a Pack and lens, produce a disposable view containing the reading order, open questions, dependency concerns, relevant declarations, and examples. This could be copied into an advisor brief or read directly by a person. Start with a platform recipe. Generalize it only after repeated use shows which joins are stable. + +30. **Show changes at Spec-entry granularity.** Compare two graph snapshots by Spec ID, section, and entry key. Distinguish changed declaration text, changed explanatory prose, moved entries, readiness edits, relation edits, and new or removed examples. A Git diff remains available underneath. For the stacked platform PR, compare against its PR base so the reader sees the design adoption separately from the much larger underlying branch. + +31. **Show what changed since a design pass.** If a pass read a known commit, compare that version with the current one. Display “this declaration changed since the operator pass” or “a dependency gained a blocking question.” This is review context derived from Git, not a new approval fact in the graph. Begin at Spec granularity; entry-level precision can follow. + +32. **Keep decision rationale close to the affected design.** A decision panel should show the chosen or provisional option, the do-nothing option, alternatives, consequences, and the Specs it shapes. This is particularly useful for the paused verify pass: the reader needs the tradeoff and affected scenarios, not just an `OD-157` link. Draw from the existing register and decision Specs rather than maintaining another summary by hand. + +Several supporting changes would make these views dependable. + +33. **Add a dependency-cycle recipe.** Report strongly connected sets for selected relation types, including self-loops, and show an actual path through each set. Begin with `dependsOn`; offer the broader typed-dependency view separately. A cycle should be visible without automatically becoming an error, since its acceptability depends on what the relations mean. + +34. **Provide batch address resolution through the existing query mechanism.** Accept a list of addresses and return the resolved Spec, section, key, value, and a precise failure reason. This would serve the platform’s external citations, a viewer’s links, and decision-register checks. It can be a shipped recipe first. Preserve the exact mention/address grammar instead of teaching every adopter a slightly different regular expression. + +35. **Show accepted prose mentions with their reasons.** The platform’s exception policy is already useful. Present the exception reason in the relationship view, along with new unclassified pairs and stale exceptions. This helps readers distinguish “there is intentionally no typed relation” from “someone forgot the dependency.” + +36. **Separate inspection from artifact publication.** A local viewer should use in-memory extraction, validation, and rendering. It should not call the build path on every refresh. A future CLI cleanup could provide the same separation for checks, but the first viewer can use the exported APIs. When extraction fails, display the diagnostics and clearly identify any retained page as the last successful snapshot. + +37. **Keep changing context outside the pure Design Review.** The current Design Review is specified as a function of the graph alone, without timestamps or commit identities. A repository browser can surround that content with the selected commit, engine pin, Git comparison, and test run. That separation lets the pure pages stay deterministic while the browser answers practical questions about freshness. The current boundary is explicit in the Design Review Spec (`node_modules/@libar-dev/software-delivery-protocol/specs/consumers/design-review.sdp.md`, line 19). + +38. **Build a small read-only browser with static output first.** It needs a navigation column, a readable content column, search, filters, entry links, and an optional context panel. Static HTML with client-side search is sufficient for an initial export; local refresh can be added around the same renderer. Keyboard navigation, readable line lengths, text labels beside colors, and print output should be part of that first cut. + +39. **Make PR review open on the rendered capability.** CI can generate the changed Packs and Specs as a downloadable review artifact, with a short job summary linking to it. Include the selected comparison base and generation provenance in the artifact wrapper. Keep generated status out of authored Specs. This would make #12’s design work inspectable without requiring reviewers to install and navigate the repository. + +40. **Test the human tasks that motivated the work.** Use the history view as the acceptance corpus. Ask a reader to find the history contract, locate OD-157, explain the aggregate dependency, find the type comparison for one declaration, and determine whether a paused-rebuild example has native-backend evidence. Record where they get stuck. Renderer snapshots are useful, but they cannot establish that someone can find those answers. + +There is a natural division of responsibility. + +| Responsibility | Best home | +|---|---| +| Readable Design entries, stable entry links, preserved Pack order | SDP | +| Generic Spec, Pack, relation, and file navigation | SDP reader and human view | +| Question identity and reusable address resolution | SDP, with explicit model decisions | +| TypeScript declaration formatting and comparison details | Platform adapter initially | +| Decision-register classes, owner queue, and roadmap associations | Platform | +| Tier, composition, target, and test-run presentation | Platform adapter | +| Git comparison and review freshness | Repository browser context | +| Authored intent | Existing Specs | +| Work history | Git and existing work records | + +That division avoids making SDP understand Convex-specific declarations, platform decision classes, or the platform’s test harness. + +I would package the work into a short sequence of reviewable PRs: + +| Order | Change | Concrete completion test | +|---|---|---| +| 1 | **Readable Design Review.** Narrowly reopen the rendering rule; render entries and address links. | The history projection’s ten entries can be read without opening JSON or the carrier file. | +| 2 | **Ordered capability navigation.** Preserve Pack order end to end; improve the Pack page and entrance page. | History projection remains first after graph serialization and reload; external dependencies are visible. | +| 3 | **Platform design context.** Expose declaration comparisons and decision-register associations; resolve the known binding gaps. | From a declaration, a reader can reach its comparison and code; from OD-157, the affected design and examples. | +| 4 | **Small repository browser.** Add search, file navigation, lens filters, and a read-only refresh path. | The acceptance tasks above can be completed without querying the CLI manually. | +| 5 | **Iteration review.** Add entry-level comparisons, changes since a pass, and CI review artifacts. | A reviewer can identify what changed in the design since the previous pass and inspect the source diff. | +| 6 | **Reusable integrity improvements.** Add question identities, cycle reporting, and batch address resolution. | Reordering questions preserves references; dependency loops and broken external citations are explained directly. | + +The current projection freeze and Studio deferral should be addressed explicitly in those proposals. The freeze requires a superseding decision for changes to the shipped views. Plan 35 (`plans/35-agent-surface-adoption-and-self-binding.md` in the Protocol's repository, line 80) also asks for a named underserved reader and a package home before Studio resumes. Your updated feedback supplies a concrete reader: a person working inside this repository who needs to inspect and iterate on a capability. I would recommend `@libar-dev/` as the proposed home for that developer tool and keep the first scope to reading and review. + +**My first choice would be PRs 1–3, followed promptly by the small browser.** Formatting the pages will provide immediate value; navigation and the platform adapters will make them useful for actual design decisions. I would postpone new maturity rungs and changes to the declaration carrier until that readable view has been used for the remaining history-view passes. + +This session remained read-only. I used graph queries, source inspection, and in-memory rendering; I did not rerun the platform’s proof tiers. The existing edit to the feedback file is unchanged. \ No newline at end of file diff --git a/docs/feedback/sdp-feedback-03-selection.md b/docs/feedback/sdp-feedback-03-selection.md new file mode 100644 index 0000000..5a81c71 --- /dev/null +++ b/docs/feedback/sdp-feedback-03-selection.md @@ -0,0 +1,67 @@ +I'm done verifying. Writing the refined selection now, with what each check found. + +**Two facts about the checkouts first.** The local SDP checkout at `~/dev-libar/software-delivery-protocol` is on `main` at the merge of pull request 26 and has not fetched the merge of 27, so I verified every SDP claim against the platform's pinned tarball under `node_modules` instead. In the platform, `docs/sdp-feedback.md` is deleted in the working tree and its content now sits in `docs/feedback/` as three files, the third being my earlier answer. Nothing below depends on that move. + +**What I had wrong or unclear in the first answer** + +- I wrote that the declaration ruling "leaves" formatting to the platform. The ruling says only that the Protocol parses no language inside the span. It is silent on rendering. Formatting is platform work because SDP is language-independent, not because a ruling assigns it. +- I presented the platform's own reading of Pack order and the SDP fix as if they were alternatives. They are one sequence: the platform reads the `.pack.sdp.md` list today, and the SDP fix retires that workaround. +- Step 1 looked like a small Spec Studio while I said Studio should wait. The difference is that step 1 emits static Markdown from existing data and keeps the Design Review untouched. It is the probe that gives plan 35 the named reader it asks for before Studio resumes. + +## Round 1, platform, in order + +1. **One generated page per capability Pack.** A script reads the member list from the Pack file in authored order, because both the reader and the serialized graph sort it: `packContext` sorts members by code units, and `generated/graph.json` already lists the history view's `belongsTo` edges alphabetically with the generation registry first. Per member it prints the stated rung, the floor, the clause blocking the next rung, the implementation and verifier bindings, and the pair's state from the three describe blocks of the pinned type test. Below the members: dependencies outside the Pack, and open questions joined to register rows. The page may carry the commit and the engine pin, which the Design Review may not, since its Spec rules that the page set carries no timestamp, commit or run identity. +2. **Two owner rulings the page depends on.** Both already sit in the STATE.md owner queue. First, what `ready` states. I recommend the Protocol's reading, approved to build, stated on the history projection after the two remaining passes. It makes recipe 1 answer the backlog instead of the local `defined` query in lead 10, and it changes what roadmap unit 4e, `ready` by family, means. Second, the aggregate form's reach into three built Specs. Narrow those pins to the forms the code admits, or the page will show three built members whose floor waits on a deferred Spec for the whole history view's life. +3. **Close the two binding gaps.** STATE.md records that rebuild and write pause show no implementation binding although code realizes both. Anchor them, and add a small mapping from each anchor that lives in a test to the implementation file it stands for, since the read-model anchors sit in a pure test because `src/` cannot import the Protocol. +4. **Stop citing questions by position.** The register holds 137 references of the form `spec:#`. That form now collides with the entry-address grammar, and one is already broken: OD-006 cites a question that no longer exists, which is why `test_decisions.py` fails. Make the OD number the link, store the question's opening text beside it, and let `check.py` verify text rather than index. +5. **Run the domain and operator passes against the page.** The roadmap already orders them before 5d. Add one line to the task: read the page before the recipes, and report what it could not answer. That report is the acceptance test for step 1 and the evidence for round 2. + +## Round 2, SDP, as four pull requests + +| Order | Change | What I verified | +| ----- | --------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | Ratify `spec:decisions.authored-entry-order` | Its blocking question asks whether to reopen the freeze now or after `open-section-order` has landed and the ordered JSON has been read. Both conditions are met: order landed at schema 0.6.0, and the 2026-10-04 feedback section is the read. The record needs `supersedes` on the freeze and the three-part ADR test, as its own text says. | +| 2 | Pack authored order through the graph | The loss is in serialization as well as the reader, so a reader-only fix would leave `graph.json` consumers different. The authoring skill already tells authors to keep membership order. | +| 3 | Two shipped recipes: batch address resolution, and `dependsOn` strongly connected sets with one path each | Neither exists. Recipe 21 has no cycle text, and the catalog ends at 24. Recipes add no query verb, so plan 35's exclusions do not apply. | +| 4 | Optional question keys with a ruled address form | The platform's 137 positional references are the use case. The grammar question, what `#` means after a Spec id, needs a model ruling. | + +**Not now.** A `designed` rung, the `ts` fenced block, declaration formatting inside SDP, and Spec Studio. The first two change the model before step 5 shows what a reader needs. Studio waits on the ruling plan 35 names, and step 1 is what produces that evidence. + +**Order.** Step 1 and SDP PR 1 start together. Steps 2 to 4 are each a day and gate step 5. SDP PRs 2 to 4 follow step 5's report. + + +--- + + +Go. The updated selection is the right approach. It's narrow. It uses work you already have to do next, the domain and operator passes before 5d, as the acceptance test. And it starts the one SDP change that helps every page, readable Design entries, with a single ruling. Treat the long Astra-style list (02) as the backlog behind it, not as the plan. + +## Why this one over 02 + +02 is good analysis, and its 40 items are mostly correct. Its sequence, though, builds a browser in PR 4 and diff-based review in PR 5 before anyone has learned what a reader needs. 03 learns that first from the two passes, then lets SDP PRs 2 to 4 follow the report. That's cheaper, and it gives plan 35 the evidence it asks for before Spec Studio is reconsidered. The revision also fixed most of what I raised. The order holds now, `ready` and the aggregate's reach are rulings rather than steps, and Pack order is clearly a workaround that SDP PR 2 retires. + +## Conditions for the go + +1. **Split before anchoring `rebuild`.** This is still my one real disagreement. Step 3 anchors it as it stands, which tells the page and the drift alarm that the history path is built. That's the mixed-maturity problem this PR just fixed for the projection contract. Either move the history path out first, or have the page say which path the anchor covers. 02 kept this caveat (item 22), and 03 lost it. `write-pause` can be anchored now. +2. **Two data sources in step 1 don't exist yet.** + - **Pinned state.** Reading it from the describe blocks of `pinned.test-d.ts` means parsing test names. Move the three lists into a small data module that both the test and the page import. + - **The next-rung clause.** Recipe 9 doesn't give "the clause blocking the next rung". For `command-declaration` it returns floor `defined`, next rung `ready` and `firstUnmetClause: null`, because it only reports failures of the stated rung. The page needs its own floor evaluation for the next rung, and recipe 9's behavior is worth a line in round 2. +3. **Say where the page lives.** `sdp validate` wipes `generated/`, and git ignores it. Put the page somewhere else, or regenerate it after validate. Also decide between committing it with a staleness check and publishing it as a CI artifact only. +4. **Make step 5's acceptance concrete.** Use 02's task list (item 40). A reader should be able to: + - find the history contract; + - locate OD-157; + - explain the aggregate dependency; + - find one declaration's type comparison; + - say whether a paused-rebuild example has native-backend evidence. + + "Report what it could not answer" is too open to compare across the two passes. + +## One cheap addition + +`generated/pinned/history-view.ts` already holds every pinned declaration of the Pack, each one under its entry address. They're one line each. If `pinned.mjs` ran Prettier on the file as it writes it, the Pack would have a readable contract listing on day one. The page could link to it by address, and nothing would wait on SDP PR 1. Formatting doesn't change any type the test compares. + +## Two facts to correct in 03 + +- **The register's references.** I count 151 references of the form `spec:#`, not 137. Six already point past the last question of their Spec, not only OD-006: OD-006, OD-007, OD-064 twice, OD-074 and OD-123. +- **SDP PR 1.** "Ratify" is your statement of `ready` on the decision record, with `supersedes` on the freeze. The rendering change follows it, so it's two acts, the ruling and then the build. + +Its "not now" list matches mine: no `designed` rung, no `ts` fence, no declaration formatting inside SDP, no Studio yet. \ No newline at end of file diff --git a/docs/sdp-feedback.md b/docs/sdp-feedback.md deleted file mode 100644 index e9ba105..0000000 --- a/docs/sdp-feedback.md +++ /dev/null @@ -1,92 +0,0 @@ -# Feedback to the Software Delivery Protocol from this corpus - -A running file, addressed to the Protocol's maintainers. Each `adopt` unit and each slice that learns something about the Protocol adds a dated section at the top. The first report, with its eight proposals, is [`sdp-development-from-application-platform.md`](sdp-development-from-application-platform.md). - -Every number here was measured on this corpus on the date of its section. Re-run before relying on one. - -## 2026-10-03, round 2 of the roadmap - -- A model Spec now owns an example space: `orders-inventory-example.sdp.md` holds the `gwt-vocabulary` of its six examples, and the pinned Protocol validates it with 0 errors and 0 warnings, while the model template of this corpus's `design/PLAN.md` 6.1 lists only Intent and Model, so nothing told the author it was allowed. - -## 2026-10-01, slice S0, the first build on the Protocol at `5993da7` - -### A Spec with designed stubs is a maturity the Protocol cannot state - -The owner's direction during this slice: the Protocol should support Specs at any level of maturity, and that includes a Spec whose key abstractions are designed as stubs. The earlier generation of this tooling, architect, did that, and the Protocol is its successor. - -What happened here. Before a build job, a design agent decides the key abstractions of a Spec: the exported interface of a module, the shape of a record, the names. The slice's first design brief was this corpus's own mistake: it asked for a design that left the implementer nothing to decide, with no bound, and the agent wrote toward a long design file outside the repository and then prototyped the code, for about an hour. What the mistake showed is that the corpus had no home for the output of a design pass. A Spec that can hold the stubs is what bounds a design pass: the design is done when the Spec's key abstractions are stated as stubs, and the implementer writes the rest. - -What the Protocol offers today, and where it stops: - -1. **Readiness has no rung for it.** The rungs are `idea`, `scoped`, `defined` and `ready`, and they describe the intent. A Spec at `defined` with no design and a Spec at `defined` whose interfaces are settled look the same in the graph, so the build backlog cannot tell which Specs an implementer can start on. -2. **A stub in code reads as `implemented`.** The only binding from code to a Spec is a `codeAnchor` with `satisfies`, and the Protocol's `CONTEXT.md` defines `implemented` as one or more `satisfies` edges that resolve to the Spec, and the drift alarm as `implemented` without `ready`. A file of signatures with no bodies would claim the Spec is realized, and on a Spec below `ready` it would raise the drift alarm. There is no anchor, or flag on an anchor, that says "this is the designed surface, not the implementation". -3. **A stub in the Spec is one line of prose.** A `## Design` entry is `key: value` on one physical line, and fenced code is refused there. This corpus has 792 Design entries, counted as the bullets under every `## Design` heading on 2026-10-01, and 653 of them hold inline code; a multi-member interface on one line is hard to read and to review. Nothing checks a Design entry's signature against the exported one, so the two drift apart without a finding. Authored order for Design entries, already asked for below, matters more once the entries are stubs a reader walks in order. - -The owner's proposal for item 3: the Markdown carrier would accept a `ts` fenced block in a Spec, holding the design of its key contracts: the interfaces, types and function signatures a reader would otherwise find squeezed into one-line Design entries. Today the carrier accepts two fences, `gwt-vocabulary` under `## Example space` and `gwt` on an example, and refuses every other. - -What that leaves for the maintainers to decide: - -- Where the block is lawful: under `## Design`, under `## Contract`, or both, and whether one block per Spec or one per named contract. -- Whether the graph reads it. Read as data, each declared name could be an entry that entry search (recipe 23) finds, and a later check could compare a stub with the exported signature of the code anchored to the Spec. Read as content only, it is still what a reviewer and an implementer need. -- How a Spec with such a block shows in the graph, which is items 1 and 2: a `designed` fact or rung that the build backlog can filter on, and a code binding for a stub file that does not derive `implemented`. - -What this corpus does until then: the design agent proposes `## Design` entries and rules for the Specs a slice builds, signatures only, one line each, and the main thread writes them into the Spec. Stub files in code carry no anchor until they have bodies. If the carrier comes to accept a `ts` block, the Design entries that are declarations move into blocks, and the design pass writes its stubs there. - -### What the build learned about binding an example to a test - -Slice S0 bound eighteen examples to native tests with `bindExample` from the Vitest adapter. Seven things came up. The first two were read in `dist/adapters/vitest.js` of the pinned package, the others met during the build on 2026-10-01. - -1. **A step is bound by its exact text.** `planExample` looks a step up as `bindings[step.text]`. When a review rewords a Then, the test stops compiling, because the contract's step type changed, and a run that skips the compiler fails with `bindings[step.text] is not a function`. That is the right alarm when the meaning changed. It is noise when the wording was polished and the meaning stayed. This corpus now changes an example's steps and its test in the same commit. A stable key per step, beside its text, would let prose be reworded without touching the test, and the adapter could then say "no binding for step " in place of the type error. -2. **`bindExample`'s `after` runs only when every step passed.** The adapter awaits the plan and then `after`, with no `finally`. A test that owns a process cannot clean up there: a failed step would leave the process running. The harness registers its cleanup with Vitest's `onTestFinished` before the first await, and `after` is unused. A `finally` around the plan, or a sentence in the adapter's documentation, would make this plain. -3. **A test binds only to a Spec under the same root.** The corpus root was `design/`. The tests are outside it, so the root moved to the repository root, and `check.py` and every `sdp` call moved with it. -4. **A parent with no example space gets no registrar.** The harness Spec and the fact Specs have examples and no `## Example space`. Their tests import the example's step contract from `generated/contracts/` and call `bindExample`. That works, and it means `sdp build` has to run before the compiler: `npm run typecheck` and every test command here start with it. -5. **`specTest`'s `verifies` takes one `ref`.** A test that covers two examples needs two anchors. That fits one test per example, which is how every native test here is written, and is worth a line in the authoring skill. - -6. **The slice built code for a day before any of it carried a `codeAnchor`.** The tests were anchored from the first build, because `bindExample` needs the contract. Nothing asked for the other half, so the graph showed the harness Spec with verifiers and no implementation, the drift alarm (recipe 2) returned nothing, and recipe 4 answered `coverageUnknown` for every harness file. A side read of the corpus found it at the close. One anchor on the harness kept validate at 0 warnings and made recipe 2 list the Spec as built and not `ready`, which is this project's queue of Specs waiting for the owner. A warning for "a Spec whose examples are bound and that no code satisfies" would have asked at the right moment. -7. **The build backlog reads stated `ready`.** Here the owner states `ready` after a slice passes, so recipe 1 stays empty for the whole build. The same question at `defined` with a clear floor is a short local body and gave 80 Specs. A parameter for the rung would make recipe 1 serve both ways of working. - -One more, about evidence. Twice in this slice a reviewer's mutation showed a bound test that passed while proving less than its example said: of twenty mutations run on a native backend, nine left their test green, three on the first build and six on the second. The binding says which test claims an example. It cannot say whether the test would fail if the example were false. A corpus that leans on `verified` needs mutation runs beside it, and the Protocol's documentation could say so where it defines the term. - -## 2026-10-01, adopting pull request 26 at `5993da7` - -The corpus now depends on the Protocol at commit `5993da7b8f61ce00853b790cd9ffb428842b66d0`, the head of `feature/adopter-driven-hardening`. - -### What blocks an adopter - -1. **A dependency on the commit hash installs no CLI.** The pull request says the adopter can depend on the branch by commit hash. `npm install "github:libar-dev/software-delivery-protocol#5993da7…"` succeeds and installs `CONTEXT.md`, `docs/`, `specs/`, `.agents/` and `package.json`, with no `dist/` and no `node_modules/.bin/sdp`. `dist/` is ignored by git and the package has no `prepare` script, so nothing builds it on install. This corpus works around it: a clean clone at the commit, `npm ci`, `npm run build`, `npm pack`, and the tarball committed under `vendor/` as a `file:` dependency. A `prepare` script that runs the build would make the hash usable. So would a published prerelease. - -2. **The CLI cannot say which engine it is.** `sdp --version` prints the usage text. The package version is `0.0.0` at every commit, and `graph.json` carries `schemaVersion` and nothing about the engine. This corpus was pinned to a sibling checkout's `dist/` until today. When the unbound-example change landed there, the corpus's own check failed with not one Spec changed: 54 expected warnings became 0. An adopter could not tell the tool moving from the corpus moving. The check now prints the dependency string from `package.json`. A `--version` that prints the package version and the commit it was built from would let any adopter record what produced a result. - -### What worked, against the acceptance tests in plan 39 - -| Proposal | Test | Result | -|---|---|---| -| P2 | Unicode angle brackets become real ones and validate stays clean | 496 characters on 114 lines of 29 Specs replaced, every one inside an inline code span. `sdp validate design`: 0 errors, 0 warnings | -| P4 | Validate prints no warning on a corpus with no test anchors | 0 warnings, down from 54 | -| P3 | The assumed facts state `scoped`, and readiness divergence names a dependent that states `ready` | F14, F15 and F17 state `scoped` behind a blocking question that names the probe. They have 11 distinct `constrainedBy` dependents. On a scratch copy, stating `ready` on one of them makes recipe 7 return that Spec | -| P5 | The census sentence and the open-question table come from commands | Both hand-kept registers are deleted from the README. `sdp census` gives the counts. Recipe 20 gives 129 questions on 67 Specs, 12 of them blocking | - -The authoring skill's section on what is not settled was applied as written, in two places. Nine deferred Specs each carry a blocking question that names the trigger. Three assumed facts each carry a blocking question that names the probe. Both were one-word edits to questions the corpus already had, and the corpus's own rule that a blocking question forces `scoped` agreed with the floor without a change. - -Five checks left this corpus's check script because the Protocol now covers them: one warning per example, the README's warning count, the census sentence, the open-question tally and the open-question table. - -### Smaller things - -- **Recipe bodies are only in a Markdown file.** A script that wants recipe 20 has to cut the fenced block out of `recipes.md`. One file per body beside the catalog would let an adopter run `sdp q "$(cat …/20-open-question-register.js)"`. That adds no query verb, so it stays inside the agent-front-door ruling. -- **Recipe 22 on this corpus:** 153 mentions, 112 pairs, 34 backed, 78 unbacked, none unresolved. Of the 78, 28 are declared from the target's side and 50 from neither. The `declaredByTarget` column is what makes the list usable: it separates a parent naming its child from a relation nobody declared. Whether the 50 are missing relations or plain pointers is not known yet. The slices will say, one family at a time, and the answer bears on what the checked-mentions decision should warn about. -- **The derived step contracts are a head start.** `validate` writes 71 contract modules under `generated/contracts/`, one per example and per example space. The first tests in this repository will bind to them. - -### Still wanted from the open questions on the pull request - -- Authored order for `design` entries (`spec:extraction.open-section-order`). The Design Review still shows this corpus's pinned declarations alphabetized inside a JSON block, which is how a reviewer reads its 762 keyed Design entries today. -- Checked mentions (`spec:decisions.checked-mentions`). One concept stated in two Specs that disagree is still this corpus's largest class of defect, and the validator cannot see it. - -## 2026-10-02, from slice S1 - -Slice S1 built Layers 0 and 1 from the Specs' `## Design` entries with seven agents in an hour. What the Protocol did and did not carry: - -- **Pinned signatures were buildable.** Every kernel declaration compiled as written, and the builders reported each context and command declaration that did not, with file and line. A one-line Design entry was enough for an agent to build from. -- **A code anchor cannot live in code that may not import the Protocol.** The kernel imports nothing but a type, and the anchor builder comes from the Protocol's package, whose entry imports `fs`, `path` and `ts-morph`. The kernel's anchor sits in a test file, so the graph cannot say what a change under `src/kernel` reaches. An anchor that is a comment, or an import with no runtime, would fix it. -- **`sdp build` writes outside `generated/`.** For a suite that binds an example whose parent has an example space it writes a `*.test.generated.ts` file beside the suite. The repository ignores those files in git, Prettier and ESLint. -- **Build status has no home in the graph.** The thread recorded what the slice built as Design entries in the Specs, and the owner ruled that temporal information is not part of a Spec. What is built is derivable from anchors only for whole Specs; "this Spec is half built, these entries exist" has no carrier, so it lives in the project's state file. -- **An example written before its composition existed had to be rebound.** Eighteen examples bound values of an application that is not built yet (`PlaceOrder`) and were rebound to the fixture's commands. Nothing in the graph says which composition an example runs on; the corpus says it in prose. diff --git a/example/convex/convex.config.ts b/example/convex/convex.config.ts index 7892837..bacaba3 100644 --- a/example/convex/convex.config.ts +++ b/example/convex/convex.config.ts @@ -1,3 +1,10 @@ +/** + * @sdpAnchor component:example.parent + * @sdpLabel the production composition's parent deployment over Orders and Inventory + * @sdpUses component:platform.command, component:platform.context, component:platform.read-model, component:platform.gate, component:platform.audit, component:example.orders-context, component:example.inventory-context, component:example.orders-domain, component:example.inventory-domain + * @sdpLayer application + * @sdpContext orders-inventory + */ import { defineApp } from "convex/server"; import inventory from "./inventory/convex.config.js"; import orders from "./orders/convex.config.js"; diff --git a/example/convex/inventory/convex.config.ts b/example/convex/inventory/convex.config.ts index dc99bba..304aa63 100644 --- a/example/convex/inventory/convex.config.ts +++ b/example/convex/inventory/convex.config.ts @@ -1,3 +1,10 @@ +/** + * @sdpAnchor component:example.inventory-context + * @sdpLabel the Inventory context component + * @sdpUses component:platform.context, component:example.inventory-domain + * @sdpLayer adapter + * @sdpContext inventory + */ import { defineComponent } from "convex/server"; // The Inventory context. Its mount name is its contextId. export default defineComponent("inventory"); diff --git a/example/convex/inventory/operations.ts b/example/convex/inventory/operations.ts index 4ba25a8..c4bc57b 100644 --- a/example/convex/inventory/operations.ts +++ b/example/convex/inventory/operations.ts @@ -1,4 +1,11 @@ // The Inventory context's sanctioned operations. Each takes a list, so a parent use case makes one call. +/** + * @sdpAnchor impl:application.orders-inventory-example.inventory-operations + * @sdpLabel the Inventory sanctioned operations + * @sdpReferences spec:application.orders-inventory-example + * @sdpComponent component:example.inventory-context + * @sdpRole service + */ import { v, type Infer, type ObjectType } from "convex/values"; import { defineOperation, diff --git a/example/convex/inventory/queries/stockItem.ts b/example/convex/inventory/queries/stockItem.ts index 8d7de7d..76a18fb 100644 --- a/example/convex/inventory/queries/stockItem.ts +++ b/example/convex/inventory/queries/stockItem.ts @@ -1,4 +1,11 @@ // The Inventory context's queries over its stock item stream: one definer each. +/** + * @sdpAnchor impl:application.orders-inventory-example.inventory-stock-item-query + * @sdpLabel the stock item query + * @sdpReferences spec:application.orders-inventory-example + * @sdpComponent component:example.inventory-context + * @sdpRole query + */ import { defineGet, defineList } from "../../../../src/context/index.js"; import { journal, stockItemStream } from "../streams.js"; export const get = defineGet(journal, stockItemStream); diff --git a/example/convex/inventory/schema.ts b/example/convex/inventory/schema.ts index e88f586..d254805 100644 --- a/example/convex/inventory/schema.ts +++ b/example/convex/inventory/schema.ts @@ -1,3 +1,10 @@ +/** + * @sdpAnchor impl:application.orders-inventory-example.inventory-schema + * @sdpLabel the Inventory context schema + * @sdpReferences spec:application.orders-inventory-example + * @sdpComponent component:example.inventory-context + * @sdpRole schema + */ import { defineSchema } from "convex/server"; import { contextTables } from "../../../src/context/index.js"; // A context component's schema holds the context library's tables and nothing else. diff --git a/example/convex/inventory/streams.ts b/example/convex/inventory/streams.ts index 5d80170..61df4a0 100644 --- a/example/convex/inventory/streams.ts +++ b/example/convex/inventory/streams.ts @@ -1,4 +1,10 @@ // The Inventory context's journal and the registration of its one stream type, the stock item. +/** + * @sdpAnchor impl:application.orders-inventory-example.inventory-streams + * @sdpLabel the stock item stream registration + * @sdpReferences spec:application.orders-inventory-example + * @sdpComponent component:example.inventory-context + */ import { v, type Infer } from "convex/values"; import { createJournal, diff --git a/example/convex/orderSummary.ts b/example/convex/orderSummary.ts index dae6019..17687a0 100644 --- a/example/convex/orderSummary.ts +++ b/example/convex/orderSummary.ts @@ -1,6 +1,13 @@ // The order summary, the essential summary of the Orders and Inventory application: a per-entity read // model with the order's status, line count, total and the time it was placed, written at the // pipeline's step 9 from the order's DTO by every command that writes the order. +/** + * @sdpAnchor impl:application.orders-inventory-example.order-summary + * @sdpLabel the order summary read model + * @sdpReferences spec:application.orders-inventory-example + * @sdpComponent component:example.parent + * @sdpRole read-model + */ import { v, type Infer } from "convex/values"; import { defaultRowBudgetBytes, diff --git a/example/convex/ordering.ts b/example/convex/ordering.ts index f7c419d..ce7ef8d 100644 --- a/example/convex/ordering.ts +++ b/example/convex/ordering.ts @@ -5,6 +5,13 @@ // order is answered entityExists before Inventory decides, and CancelOrder because the order's own // state refuses a cancel before Inventory decides. A rejection or a throw in Inventory rolls back the // Orders call with the whole mutation, so neither executor catches anything. +/** + * @sdpAnchor impl:application.parent-use-cases + * @sdpLabel PlaceOrder and CancelOrder over Orders and then Inventory + * @sdpReferences spec:application.parent-use-cases, spec:application.orders-inventory-example + * @sdpComponent component:example.parent + * @sdpRole use-case + */ import { v, type Infer } from "convex/values"; import { internalCommand, diff --git a/example/convex/orders/convex.config.ts b/example/convex/orders/convex.config.ts index 3fcf167..9eb9ee3 100644 --- a/example/convex/orders/convex.config.ts +++ b/example/convex/orders/convex.config.ts @@ -1,3 +1,10 @@ +/** + * @sdpAnchor component:example.orders-context + * @sdpLabel the Orders context component + * @sdpUses component:platform.context, component:example.orders-domain + * @sdpLayer adapter + * @sdpContext orders + */ import { defineComponent } from "convex/server"; // The Orders context. Its mount name is its contextId. export default defineComponent("orders"); diff --git a/example/convex/orders/operations.ts b/example/convex/orders/operations.ts index 68d9014..d74eba9 100644 --- a/example/convex/orders/operations.ts +++ b/example/convex/orders/operations.ts @@ -1,4 +1,11 @@ // The Orders context's sanctioned operations. Each takes a list, so a parent use case makes one call. +/** + * @sdpAnchor impl:application.orders-inventory-example.orders-operations + * @sdpLabel the Orders sanctioned operations + * @sdpReferences spec:application.orders-inventory-example + * @sdpComponent component:example.orders-context + * @sdpRole service + */ import { v, type Infer, type ObjectType } from "convex/values"; import { defineOperation, diff --git a/example/convex/orders/queries/order.ts b/example/convex/orders/queries/order.ts index 88fa0bf..972b1be 100644 --- a/example/convex/orders/queries/order.ts +++ b/example/convex/orders/queries/order.ts @@ -1,4 +1,11 @@ // The Orders context's queries over its order stream: one definer each. +/** + * @sdpAnchor impl:application.orders-inventory-example.orders-order-query + * @sdpLabel the order query + * @sdpReferences spec:application.orders-inventory-example + * @sdpComponent component:example.orders-context + * @sdpRole query + */ import { defineGet, defineList } from "../../../../src/context/index.js"; import { journal, orderStream } from "../streams.js"; export const get = defineGet(journal, orderStream); diff --git a/example/convex/orders/schema.ts b/example/convex/orders/schema.ts index e88f586..cc7725d 100644 --- a/example/convex/orders/schema.ts +++ b/example/convex/orders/schema.ts @@ -1,3 +1,10 @@ +/** + * @sdpAnchor impl:application.orders-inventory-example.orders-schema + * @sdpLabel the Orders context schema + * @sdpReferences spec:application.orders-inventory-example + * @sdpComponent component:example.orders-context + * @sdpRole schema + */ import { defineSchema } from "convex/server"; import { contextTables } from "../../../src/context/index.js"; // A context component's schema holds the context library's tables and nothing else. diff --git a/example/convex/orders/streams.ts b/example/convex/orders/streams.ts index 9bcb476..324a644 100644 --- a/example/convex/orders/streams.ts +++ b/example/convex/orders/streams.ts @@ -1,5 +1,11 @@ // The Orders context's journal and the registration of its one stream type, the order. The parent // reads the order's DTO validator from here: a DTO's shape belongs to the context that returns it. +/** + * @sdpAnchor impl:application.orders-inventory-example.orders-streams + * @sdpLabel the order stream registration + * @sdpReferences spec:application.orders-inventory-example + * @sdpComponent component:example.orders-context + */ import { v, type Infer } from "convex/values"; import { createJournal, diff --git a/example/convex/receiving.ts b/example/convex/receiving.ts index f6734c6..ffff9d7 100644 --- a/example/convex/receiving.ts +++ b/example/convex/receiving.ts @@ -1,6 +1,13 @@ // ReceiveStock, the command that receives stock: one call to the Inventory context's receive operation, // which creates a stock item that has no stream and adds to the quantity on hand. Every stock item // comes to exist through it. +/** + * @sdpAnchor impl:application.parent-use-cases.receiving + * @sdpLabel ReceiveStock + * @sdpReferences spec:application.parent-use-cases, spec:application.orders-inventory-example + * @sdpComponent component:example.parent + * @sdpRole use-case + */ import { v, type Infer } from "convex/values"; import { internalCommand, diff --git a/example/domain/order.ts b/example/domain/order.ts index d1763e5..dcb2933 100644 --- a/example/domain/order.ts +++ b/example/domain/order.ts @@ -1,5 +1,19 @@ // The Orders context's order stream: an order is placed once with its lines, its total is the sum of // quantity times unit price over its lines, and a placed order is cancelled once. +/** + * @sdpAnchor component:example.orders-domain + * @sdpLabel the Orders context's pure domain code + * @sdpUses component:platform.kernel + * @sdpLayer domain + * @sdpContext orders + */ +/** + * @sdpAnchor impl:application.orders-inventory-example + * @sdpLabel the order decider + * @sdpReferences spec:application.orders-inventory-example + * @sdpComponent component:example.orders-domain + * @sdpRole decider + */ import type { DecideResult, Decider, diff --git a/example/domain/orderAllocationHistory.ts b/example/domain/orderAllocationHistory.ts index 49dc70e..bfd7b7f 100644 --- a/example/domain/orderAllocationHistory.ts +++ b/example/domain/orderAllocationHistory.ts @@ -1,10 +1,17 @@ // The order allocation history's rules: keyOf maps an event to the order it belongs to, and project -// folds one order's events into its row (spec:application.projection-contract, -// orderAllocationHistoryFold). Each field of the row is written by the events of one stream, -// placedAt and cancelledAt by the order's and each allocation by its stock item's, and the -// allocations are kept in stock item ID order, so the row is one value whatever the order in which -// the events of different streams arrive. The parent's HistoryProjection holds these rules; no -// context mounts them, so index.ts does not export them. +// folds one order's events into its row +// (spec:application.history-projection#design.orderAllocationHistoryFold). Each field of the row +// is written by the events of one stream, placedAt and cancelledAt by the order's and each +// allocation by its stock item's, and the allocations are kept in stock item ID order, so the row +// is one value whatever the order in which the events of different streams arrive. The parent's +// HistoryProjection holds these rules; no context mounts them, so index.ts does not export them. +/** + * @sdpAnchor impl:application.order-allocation-history + * @sdpLabel the order allocation history's keyOf and project rules + * @sdpReferences spec:application.history-projection, spec:application.orders-inventory-example + * @sdpComponent component:example.parent + * @sdpRole projection + */ import type { OrderEvent } from "./order.js"; import type { StockItemEvent } from "./stockItem.js"; // The part of an event envelope the rules read; the context library's EventEnvelope has each field. diff --git a/example/domain/stockItem.ts b/example/domain/stockItem.ts index 699387d..b5c7796 100644 --- a/example/domain/stockItem.ts +++ b/example/domain/stockItem.ts @@ -1,6 +1,20 @@ // The Inventory context's stock item stream: the quantity on hand and the quantity allocated to // orders, two totals and nothing per order. The quantity available is the first less the second, an // allocation never passes it, and a release never passes the quantity allocated. +/** + * @sdpAnchor component:example.inventory-domain + * @sdpLabel the Inventory context's pure domain code + * @sdpUses component:platform.kernel + * @sdpLayer domain + * @sdpContext inventory + */ +/** + * @sdpAnchor impl:application.orders-inventory-example.stock-item + * @sdpLabel the stock item decider + * @sdpReferences spec:application.orders-inventory-example + * @sdpComponent component:example.inventory-domain + * @sdpRole decider + */ import type { DecideResult, Decider, diff --git a/fixture/convex/depot/convex.config.ts b/fixture/convex/depot/convex.config.ts index 33249ce..66bc236 100644 --- a/fixture/convex/depot/convex.config.ts +++ b/fixture/convex/depot/convex.config.ts @@ -1,3 +1,8 @@ +/** + * @sdpAnchor impl:context.context-component.depot-config + * @sdpLabel the depot fixture component's definition + * @sdpReferences spec:context.context-component + */ import { defineComponent } from "convex/server"; import migrations from "@convex-dev/migrations/convex.config.js"; // The mount name is the context's contextId, the constant in streams.ts. The migrations component is diff --git a/fixture/convex/depot/operations.ts b/fixture/convex/depot/operations.ts index 66eb70d..1af018b 100644 --- a/fixture/convex/depot/operations.ts +++ b/fixture/convex/depot/operations.ts @@ -1,4 +1,9 @@ // The depot's sanctioned operations. Each takes a list, so a parent use case makes one call. +/** + * @sdpAnchor impl:context.context-component.depot-operations + * @sdpLabel the depot fixture's list-shaped operations + * @sdpReferences spec:context.context-component, spec:context.batch-shaped-api + */ import { ConvexError, v, type Infer, type ObjectType } from "convex/values"; import { defineOperation, diff --git a/fixture/convex/depot/queries/document.ts b/fixture/convex/depot/queries/document.ts index d7d2089..ffc11f4 100644 --- a/fixture/convex/depot/queries/document.ts +++ b/fixture/convex/depot/queries/document.ts @@ -1,4 +1,9 @@ // The depot's queries over its document stream, as a context registers them: one definer each. +/** + * @sdpAnchor impl:context.context-component.depot-document-query + * @sdpLabel the depot fixture's document query + * @sdpReferences spec:context.context-component + */ import { defineGet, defineList } from "../../../../src/context/index.js"; import { documentStream, journal } from "../streams.js"; export const get = defineGet(journal, documentStream); diff --git a/fixture/convex/depot/schema.ts b/fixture/convex/depot/schema.ts index 710b69e..0654ee2 100644 --- a/fixture/convex/depot/schema.ts +++ b/fixture/convex/depot/schema.ts @@ -1,3 +1,8 @@ +/** + * @sdpAnchor impl:context.context-component.depot-schema + * @sdpLabel the depot fixture's schema + * @sdpReferences spec:context.context-component + */ import { defineSchema, defineTable } from "convex/server"; import { contextTables } from "../../../src/context/index.js"; import { v } from "convex/values"; diff --git a/fixture/convex/depot/streams.ts b/fixture/convex/depot/streams.ts index 201ade7..0a07d3b 100644 --- a/fixture/convex/depot/streams.ts +++ b/fixture/convex/depot/streams.ts @@ -1,5 +1,10 @@ // The depot's journal and the registrations of its three stream types. The fixture's faults live // here, in the parts the composition supplies to the library: a mapping, a toDto and a decider. +/** + * @sdpAnchor impl:context.context-component.depot-streams + * @sdpLabel the depot fixture's stream registrations + * @sdpReferences spec:context.context-component + */ import { v, type Value } from "convex/values"; import { createJournal, diff --git a/fixture/convex/depotCommands.ts b/fixture/convex/depotCommands.ts index 0921670..15e7229 100644 --- a/fixture/convex/depotCommands.ts +++ b/fixture/convex/depotCommands.ts @@ -1,5 +1,10 @@ // The fixture composition's commands over the depot context. Each is one declaration and the two static // exports the command library builds from it; each makes one call to a depot operation. +/** + * @sdpAnchor impl:command.command-declaration.depot-commands + * @sdpLabel the depot fixture's command declarations + * @sdpReferences spec:command.command-declaration + */ import { v, type Infer } from "convex/values"; import { internalCommand, diff --git a/package-lock.json b/package-lock.json index 5ff4802..3699c3b 100644 --- a/package-lock.json +++ b/package-lock.json @@ -7,12 +7,13 @@ "": { "name": "libar-platform", "version": "0.0.0", + "license": "Apache-2.0", "devDependencies": { "@convex-dev/eslint-plugin": "5.0.0", "@convex-dev/migrations": "0.3.6", "@edge-runtime/vm": "5.0.0", "@eslint/js": "10.0.1", - "@libar-dev/software-delivery-protocol": "file:vendor/libar-dev-software-delivery-protocol-0.0.0-5993da7.tgz", + "@libar-dev/software-delivery-protocol": "file:vendor/libar-dev-software-delivery-protocol-0.0.0-822aea4.tgz", "@types/node": "24.19.0", "@types/react": "19.3.0", "@types/react-dom": "19.3.0", @@ -800,8 +801,8 @@ }, "node_modules/@libar-dev/software-delivery-protocol": { "version": "0.0.0", - "resolved": "file:vendor/libar-dev-software-delivery-protocol-0.0.0-5993da7.tgz", - "integrity": "sha512-NR0Cyq7AIepjToh1uR/2D4BwmCGDDWeIHXDh2yb60+sBvdRpdYZTrOmBs/p0S2HPxnpZzSLxtP/fkfoLoW48ng==", + "resolved": "file:vendor/libar-dev-software-delivery-protocol-0.0.0-822aea4.tgz", + "integrity": "sha512-brKD2HWM4+I+qc/nL9iSDudnlCurvU4dQIRHfK32xkym/yRtv/4R6eUGGwRwkQqXdiIvqrmKlXyKJFX7dIYKmg==", "dev": true, "license": "Apache-2.0", "dependencies": { diff --git a/package.json b/package.json index 8b33f28..ba76227 100644 --- a/package.json +++ b/package.json @@ -14,7 +14,8 @@ "validate": "sdp validate", "view": "sdp view", "census": "sdp census", - "sdp:build": "npx sdp build", + "sdp:build": "npx sdp build && node scripts/pinned.mjs --pack pack:history-view --name history-view", + "pages": "node scripts/pack-page.mjs --pack pack:history-view --out design/pages/history-view.md", "typecheck": "npm run sdp:build && tsc", "lint": "eslint src fixture example harness tests scripts vitest.config.ts vitest.hosted.config.ts eslint.config.js", "format:check": "prettier --check src fixture example harness tests scripts evidence/README.md README.md package.json tsconfig.json convex.json vitest.config.ts vitest.hosted.config.ts eslint.config.js .github/workflows/ci.yml .github/workflows/hosted.yml .prettierrc.json", @@ -35,7 +36,7 @@ "@convex-dev/migrations": "0.3.6", "@edge-runtime/vm": "5.0.0", "@eslint/js": "10.0.1", - "@libar-dev/software-delivery-protocol": "file:vendor/libar-dev-software-delivery-protocol-0.0.0-5993da7.tgz", + "@libar-dev/software-delivery-protocol": "file:vendor/libar-dev-software-delivery-protocol-0.0.0-822aea4.tgz", "@types/node": "24.19.0", "@types/react": "19.3.0", "@types/react-dom": "19.3.0", diff --git a/scripts/pack-page.mjs b/scripts/pack-page.mjs new file mode 100644 index 0000000..ce9be66 --- /dev/null +++ b/scripts/pack-page.mjs @@ -0,0 +1,1201 @@ +// Writes the page of one capability Pack, which a person opens to understand its design: +// `node scripts/pack-page.mjs --pack --out ` writes the page, and with `--check` it +// writes nothing and compares the page at --out with what the tree generates, naming the first line +// that differs. It reads the graph in memory through the Protocol package's exported extractor and +// reader, never through validate, view, census or build, and runs recipes 20 and 24 as the pinned +// Protocol ships them in recipes.md. Two inputs join it: the pinned pairs +// (tests/types/pinned-pairs.json) and the rows of the decision register +// (design/decisions/register.json); a missing input is an error that names its path. The code a +// member binds comes from the graph alone: each anchor at its own site, with its role and its +// component. The members come in the order the reader's Pack context gives them, which is the order +// the Pack file writes them. The next rung is the rung above the stated one: the reader's +// `nextRungFailures` are the unmet clauses of the rung above the floor, so when the floor sits above +// the stated rung nothing blocks the next rung, and otherwise its first unmet clause is the first of +// them, with the targets that break a typed-dependency clause. +// Exit 0: written, or the page is current. 1: --check found the page behind the tree. 2: it cannot +// generate. +import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs"; +import { dirname, join, relative, resolve, sep } from "node:path"; +import { fileURLToPath } from "node:url"; +import { parseArgs } from "node:util"; +import { + SPEC_KINDS, + SPEC_READINESS, + createReader, + extract, +} from "@libar-dev/software-delivery-protocol"; +import { format, resolveConfig } from "prettier"; + +const root = join(import.meta.dirname, ".."); +const engine = "@libar-dev/software-delivery-protocol"; +const inputs = { + pairs: "tests/types/pinned-pairs.json", + register: "design/decisions/register.json", +}; +// The relations a Spec rests on, as recipe 21 and the floor's typed-dependency clause read them. +const typedRelations = ["refines", "dependsOn", "constrainedBy", "decidedBy"]; +const typedClause = "typed-dependency-targets-are-defined"; +const blockingClause = "no-blocking-open-questions"; +// The module scripts/pinned.mjs emits for a Pack, by default named after the Pack's id. +const pinnedModule = (pack) => + `generated/pinned/${pack.replace(/^pack:/, "")}.ts`; + +export class PageError extends Error {} +function cannot(reason) { + throw new PageError(reason); +} + +// --------------------------------------------------------------------------------------------- +// Inputs. One reader per input, each to the shape the rulings fix; a file that is missing or off +// that shape is an error that names the path. + +function readJson(base, path) { + const file = join(base, path); + if (!existsSync(file)) cannot(`${path} is missing`); + try { + return JSON.parse(readFileSync(file, "utf8")); + } catch (error) { + return cannot(`${path} is not JSON: ${error.message}`); + } +} +const isText = (value) => typeof value === "string" && value.length > 0; + +// The pinned pairs: what the type test compares for each pinned declaration, by entry address. +export function readPinnedPairs(base, pack) { + const path = inputs.pairs; + const data = readJson(base, path); + if (data?.pack !== pack) + cannot(`${path} names ${JSON.stringify(data?.pack)}, not ${pack}`); + if (!isText(data.test) || !Array.isArray(data.pairs)) + cannot(`${path} needs a test path and a list of pairs`); + const pairs = new Map(); + data.pairs.forEach((pair, at) => { + const where = `${path} pairs[${at}]`; + if (!isText(pair?.address)) cannot(`${where} has no address`); + if (!["bound", "ahead", "notCompared"].includes(pair.state)) + cannot(`${where} has the state ${JSON.stringify(pair.state)}`); + if (!["type", "typeWithoutCtx", "table", "none"].includes(pair.comparison)) + cannot(`${where} has the comparison ${JSON.stringify(pair.comparison)}`); + if (pair.code !== null && !/^[^#\s]+#[\w$]+$/.test(pair.code ?? "")) + cannot(`${where} has the code ${JSON.stringify(pair.code)}`); + if (pair.state === "bound" ? pair.reason !== null : !isText(pair.reason)) + cannot(`${where} has the reason ${JSON.stringify(pair.reason)}`); + if (pairs.has(pair.address)) cannot(`${where} repeats ${pair.address}`); + pairs.set(pair.address, pair); + }); + return { test: data.test, pairs }; +} + +// The register: every row with the fields the page shows and its sources as the register writes +// them. An open question source cites its question by address; a closed one keeps its Spec and the +// question's opening text, as design/advisors/register.md rules. +export function readRegister(base) { + const path = inputs.register; + const data = readJson(base, path); + if (!Array.isArray(data?.decisions)) + cannot(`${path} has no list of decisions`); + return data.decisions.map((decision, at) => { + if (!isText(decision?.id)) cannot(`${path} decisions[${at}] has no id`); + const where = `${path} ${decision.id}`; + if (!Array.isArray(decision.sources)) + cannot(`${where} has no list of sources`); + for (const source of decision.sources) { + if (!isText(source?.kind) || !isText(source.file)) + cannot(`${where}: a source needs a kind and a file`); + if (source.kind !== "specQuestion") continue; + if (source.open === false) { + if (![source.spec, source.question].every(isText)) + cannot( + `${where}: a closed specQuestion source needs spec, question and file`, + ); + } else if (!isText(source.address)) + cannot(`${where}: an open specQuestion source needs address and file`); + } + return { + id: decision.id, + title: decision.title ?? null, + class: decision.class ?? null, + status: decision.status ?? null, + advisor: decision.advisor ?? null, + blocks: decision.blocks ?? null, + decidedBy: decision.decidedBy ?? null, + decidedOn: decision.decidedOn ?? null, + ruling: decision.ruling ?? null, + sources: decision.sources, + }; + }); +} + +// One entry for each open question source: the row it belongs to and the address it cites. +export function questionSources(rows) { + return rows.flatMap((row) => + row.sources + .filter( + (source) => source.kind === "specQuestion" && source.open !== false, + ) + .map((source) => ({ ...row, address: source.address })), + ); +} + +// --------------------------------------------------------------------------------------------- +// The pure parts. + +// The members of a Pack in the order the reader's Pack context gives them, each once. +export function packMembers(context) { + const members = context.members.map((member) => member.id); + if (members.length === 0) cannot(`${context.id} lists no specs`); + const repeated = members.find((id, at) => members.indexOf(id) !== at); + if (repeated !== undefined) cannot(`${context.id} lists ${repeated} twice`); + const unresolved = context.members.find((member) => !member.resolved); + if (unresolved !== undefined) cannot(`${unresolved.id} is not in the graph`); + return members; +} + +// The rung above the stated one and the clauses that fail at it, from a Spec context of the reader. +// `nextRungFailures` are the unmet clauses of the rung above the floor, cumulative in table order. +// When the floor sits above the stated rung, the next rung holds; otherwise the rung above the floor +// is at or below the next rung, so its first unmet clause is the first that fails at the next rung. +// A kind the Protocol does not ratify has no floor, and null says the next rung is not evaluated. +export function nextRung(context) { + if (!SPEC_KINDS.includes(context.specKind)) return null; + const stated = SPEC_READINESS.indexOf(context.statedReadiness); + const rung = SPEC_READINESS[stated + 1] ?? null; + if (rung === null) return { rung, below: false, failing: [] }; + const floor = + context.derivedReadiness === undefined + ? -1 + : SPEC_READINESS.indexOf(context.derivedReadiness); + return { + rung, + below: floor < stated, + failing: floor > stated ? [] : context.nextRungFailures, + }; +} + +// The address of an open question, or null when its marker carries no key. +export const questionAddress = (spec, question) => + question.key === null || question.key === undefined + ? null + : `${spec}#question.${question.key}`; + +// The register rows of one Spec joined to its open questions by address: a row joins the question +// its source cites. `left` holds the rows whose address names the Spec and none of its questions. +export function joinQuestions(spec, questions, rows) { + const own = rows.filter((row) => row.address.startsWith(`${spec}#`)); + const joined = questions.map((question) => { + const address = questionAddress(spec, question); + return { + ...question, + address, + rows: + address === null ? [] : own.filter((row) => row.address === address), + }; + }); + const left = own.filter( + (row) => !joined.some((question) => question.rows.includes(row)), + ); + return { questions: joined, left }; +} + +// The register rows whose sources lie in a member's file, beside what the open questions show: every +// decided row, and every other row none of whose sources is an open question. `files` is keyed by +// the members' files; each row keeps only its sources in a member, in the register's order. +export function memberRows(rows, files) { + return rows.flatMap((row) => { + const inside = row.sources.filter((source) => files.has(source.file)); + if (inside.length === 0) return []; + const asks = row.sources.some( + (source) => source.kind === "specQuestion" && source.open !== false, + ); + if (row.status !== "decided" && asks) return []; + return [{ ...row, sources: inside }]; + }); +} + +// Prettier formats a declaration where it parses as TypeScript, as written or as the form its key +// implies: a call signature as a function, an index as a call chained on a name, a property as an +// interface member. Otherwise the declaration stays as the Spec writes it. +const wrappers = [ + { applies: () => true, wrap: (text) => text, unwrap: (text) => text }, + { + applies: (text) => /^[\w$]+\s*[<(]/.test(text), + wrap: (text) => `function ${text} {}`, + unwrap: (text) => + text.startsWith("function ") && text.endsWith(" {}") + ? text.slice("function ".length, -" {}".length) + : null, + }, + { + applies: (text) => text.startsWith("."), + wrap: (text) => `x${text}`, + unwrap: (text) => (text.startsWith("x.") ? text.slice(1) : null), + }, + { + applies: (text) => /^[\w$]+\??:/.test(text), + wrap: (text) => `interface Pinned {\n${text}\n}`, + unwrap: (text) => { + const lines = text.split("\n"); + const inner = lines.slice(1, -1); + if ( + lines[0] !== "interface Pinned {" || + lines.at(-1) !== "}" || + !inner.every((line) => line === "" || line.startsWith(" ")) + ) + return null; + return inner.map((line) => line.slice(2)).join("\n"); + }, + }, +]; +export async function formatDeclaration(text, options = {}) { + for (const { applies, wrap, unwrap } of wrappers) { + if (!applies(text)) continue; + let formatted; + try { + formatted = await format(wrap(text), { + ...options, + parser: "typescript", + }); + } catch { + continue; + } + const body = unwrap(formatted.trimEnd()); + if (body !== null) return { text: body.replace(/;$/, ""), formatted: true }; + } + return { text, formatted: false }; +} + +// The table entry each index entry is chained on, by entry address, as the pinned module chains it: +// an `index*` entry of the form `.index(...)` belongs to the `table*` entry of the form +// `name: defineTable(...)` that its Spec wrote last before it. `rows` are recipe 24's rows in order. +export function indexTables(rows) { + const tables = new Map(); + let table = null; + for (const row of rows) { + const prefix = /^[a-z]+/.exec(row.key)?.[0]; + const declaration = row.declaration.trim(); + if (prefix === "table" && /^[\w$]+:\s*defineTable\(/.test(declaration)) + table = row; + else if ( + prefix === "index" && + declaration.startsWith(".index(") && + table?.spec === row.spec + ) + tables.set( + `${row.spec}#design.${row.key}`, + `${table.spec}#design.${table.key}`, + ); + } + return tables; +} + +// The line of each Design entry and of each open question in a Spec's Markdown carrier. +export function specLines(text) { + const design = new Map(); + const questions = []; + let section = null; + text.split("\n").forEach((line, at) => { + const heading = /^#{2,3} (.+)$/.exec(line); + if (heading !== null) section = heading[1]; + else if (section === "Design") { + const key = /^- ([a-z][A-Za-z0-9]*): /.exec(line); + if (key !== null) design.set(key[1], at + 1); + } else if ( + section === "Open questions" && + /^- \[(?:non-)?blocking(?: #[a-z][A-Za-z0-9]*)?\] /.test(line) + ) + questions.push(at + 1); + }); + return { design, questions }; +} + +// A question's opening, to the end of its first sentence and at most `limit` characters, cut at a +// space outside any code span. +export function opening(text, limit = 220) { + let code = 0; + let sentence = text.length; + let cut = -1; + for (let at = 0; at < text.length; at++) { + if (text[at] === "`") { + let run = 1; + while (text[at + run] === "`") run++; + code = code === 0 ? run : code === run ? 0 : code; + at += run - 1; + } else if ( + code === 0 && + text[at] === "." && + /\s/.test(text[at + 1] ?? " ") + ) { + sentence = at + 1; + break; + } else if (code === 0 && text[at] === " " && at <= limit) cut = at; + } + if (sentence <= limit || cut === -1) return text.slice(0, sentence); + return `${text.slice(0, cut)} …`; +} + +// The first line at which two texts differ, or null when they are equal. +export function firstDifference(committed, generated) { + if (committed === generated) return null; + const left = committed.split("\n"); + const right = generated.split("\n"); + let at = 0; + while (at < left.length && at < right.length && left[at] === right[at]) at++; + return { + line: at + 1, + committed: left[at] ?? null, + generated: right[at] ?? null, + }; +} + +// The tier an example's opening line names, as in "Sc L2-6 · native backend tier · the resume case.". +export function tierOf(openingLine) { + for (const part of openingLine.split("·")) { + const tier = /^\s*(.+?) tier\.?\s*$/.exec(part); + if (tier !== null) return tier[1]; + } + return null; +} + +// --------------------------------------------------------------------------------------------- +// Markdown. + +function span(text) { + const runs = text.match(/`+/g) ?? []; + const fence = "`".repeat(Math.max(0, ...runs.map((run) => run.length)) + 1); + const pad = text.startsWith("`") || text.endsWith("`") ? " " : ""; + return `${fence}${pad}${text}${pad}${fence}`; +} +const cell = (text) => text.replaceAll("|", "\\|").replaceAll("\n", " "); +const table = (head, rows) => [ + `| ${head.join(" | ")} |`, + `|${head.map(() => "---").join("|")}|`, + ...rows.map((row) => `| ${row.map(cell).join(" | ")} |`), +]; +function slug(heading) { + return heading + .toLowerCase() + .replace(/[^\p{L}\p{M}\p{N}\p{Pc} -]/gu, "") + .replaceAll(" ", "-"); +} +const plural = (count, one, many = `${one}s`) => + `${count} ${count === 1 ? one : many}`; +const list = (items) => + items.length <= 1 + ? (items[0] ?? "") + : `${items.slice(0, -1).join(", ")} and ${items.at(-1)}`; +const compared = { + type: "the type", + typeWithoutCtx: "the type, its ctx parameter erased", + table: "the table's document and indexes", + none: "nothing", +}; +const states = { bound: "bound", ahead: "ahead", notCompared: "not compared" }; + +// --------------------------------------------------------------------------------------------- +// The page. + +const AsyncFunction = (async () => {}).constructor; +function recipe(number) { + const recipes = readFileSync( + join(root, "node_modules", engine, "docs/agent-surface/recipes.md"), + "utf8", + ); + const section = recipes + .split(/^## /m) + .find((part) => part.startsWith(`${number}. `)); + const fence = /^```js\n([\s\S]*?)^```$/m.exec(section ?? ""); + if (fence === null) + cannot(`recipes.md has no recipe ${number} with a js fence`); + return fence[1]; +} + +async function readGraph(pack, base) { + const extracted = extract({ root: base }); + const error = extracted.report.findings.find( + (finding) => finding.severity === "error", + ); + if (error !== undefined) + cannot(`the graph does not derive: ${error.message.split("\n")[0]}`); + const { graph } = extracted; + const reader = createReader(graph); + const summary = + reader.packs().find((candidate) => candidate.id === pack) ?? + cannot(`the graph has no ${pack}`); + return { + graph, + reader, + summary, + members: packMembers(reader.packContext(pack)), + questions: await new AsyncFunction("g", "graph", "report", recipe(20))( + reader, + graph, + undefined, + ), + declarations: await new AsyncFunction("g", "graph", "report", recipe(24))( + reader, + graph, + undefined, + ), + }; +} + +// `base` is the repository the page describes: its graph, its inputs and its links. The recipes +// come from the Protocol this script's repository pins. +export async function generate(pack, out, base = root) { + const pin = JSON.parse(readFileSync(join(base, "package.json"), "utf8")); + const enginePin = + pin.devDependencies?.[engine] ?? + pin.dependencies?.[engine] ?? + cannot(`package.json does not depend on ${engine}`); + const { graph, reader, summary, members, questions, declarations } = + await readGraph(pack, base); + const chained = indexTables(declarations.rows); + const pairs = readPinnedPairs(base, pack); + const register = readRegister(base); + const sources = questionSources(register); + const prettierOptions = + (await resolveConfig(join(base, "package.json"))) ?? {}; + + const pageDirectory = dirname(out); + const link = (path, line) => { + const target = relative(pageDirectory, join(base, path)) + .split(sep) + .join("/"); + if (line === undefined || line === null) return target; + return `${target}${path.endsWith(".md") ? "?plain=1" : ""}#L${line}`; + }; + const specs = new Map(reader.specs().map((spec) => [spec.id, spec])); + const rank = (rung) => SPEC_READINESS.indexOf(rung); + const memberSet = new Set(members); + const openingLine = (id) => (specs.get(id)?.narrative ?? "").split("\n")[0]; + const specLink = (id) => { + const spec = specs.get(id); + return spec === undefined ? span(id) : `[${span(id)}](${link(spec.file)})`; + }; + + // Everything the page says about one member. + const model = await Promise.all( + members.map(async (id, at) => { + const context = reader.specContext(id); + const next = nextRung(context); + const lines = specLines(readFileSync(join(base, context.file), "utf8")); + const typed = context.relationsOut.filter((relation) => + typedRelations.includes(relation.type), + ); + const own = ( + questions.specs.find((row) => row.id === id)?.questions ?? [] + ).map((question, position) => ({ + ...question, + position: position + 1, + line: lines.questions[position] ?? null, + })); + const joined = joinQuestions(id, own, sources); + const entries = await Promise.all( + declarations.rows + .filter((row) => row.spec === id) + .map(async (row) => { + const address = `${id}#design.${row.key}`; + const own = pairs.pairs.get(address) ?? null; + // An index has no pair of its own: its table's pair compares the table's indexes. + const table = own === null ? (chained.get(address) ?? null) : null; + return { + key: row.key, + address, + line: lines.design.get(row.key) ?? null, + pair: own ?? pairs.pairs.get(table) ?? null, + table: + table !== null && pairs.pairs.has(table) + ? table.slice(table.indexOf("#design.") + "#design.".length) + : null, + declaration: await formatDeclaration( + row.declaration, + prettierOptions, + ), + }; + }), + ); + const declared = new Set(entries.map((entry) => entry.key)); + return { + id, + number: at + 1, + context, + heading: `${at + 1}. ${context.title ?? id}`, + next, + typed, + blockers: + next?.failing.find((clause) => clause.clauseId === typedClause) + ?.targets ?? [], + entries, + otherEntries: [...lines.design] + .filter(([key]) => !declared.has(key)) + .map(([key, line]) => ({ key, line })), + questions: joined.questions, + leftRows: joined.left, + }; + }), + ); + const byId = new Map(model.map((member) => [member.id, member])); + const memberLink = (id) => { + const member = byId.get(id); + return `[${member.context.title ?? id}](#${slug(member.heading)})`; + }; + const nextText = (member) => { + if (member.next === null) return "not evaluated: the kind is not ratified"; + if (member.next.rung === null) return "none above ready"; + if (member.next.failing.length === 0) + return `${member.next.rung}, which its floor holds`; + return `${member.next.rung}, held by ${span(member.next.failing[0].clauseId)}`; + }; + const pairCounts = (member) => { + const count = (state) => + member.entries.filter((entry) => entry.pair?.state === state).length; + const parts = [ + [count("bound"), "bound"], + [count("ahead"), "ahead"], + [count("notCompared"), "not compared"], + [ + member.entries.filter((entry) => entry.pair === null).length, + "without a pair", + ], + ].filter(([n]) => n > 0); + return parts.length === 0 + ? "no pinned declaration" + : parts.map(([n, word]) => `${n} ${word}`).join(", "); + }; + const examplesOf = (member) => + member.context.verifiers.filter((verifier) => verifier.via === "example"); + const testsOf = (member) => + member.context.verifiers.filter( + (verifier) => verifier.via === "test-anchor", + ); + const evidenceText = (member) => { + if (member.context.specKind === "example") { + const tier = tierOf(openingLine(member.id)); + const bound = testsOf(member).length > 0; + return `${tier === null ? "no tier named" : `${tier} tier`}, ${bound ? "a verifier is bound" : "no verifier bound"}`; + } + const examples = examplesOf(member); + const parts = []; + if (examples.length > 0) + parts.push( + `${examples.filter((example) => example.enabled).length} of ${plural(examples.length, "example")} with a verifier`, + ); + if (testsOf(member).length > 0) + parts.push(plural(testsOf(member).length, "test")); + return parts.length === 0 ? "none" : parts.join(", "); + }; + + const page = []; + const write = (...lines) => page.push(...lines); + const blockingCount = (questions) => { + const count = questions.filter((question) => question.blocking).length; + return count === 0 ? "none blocking" : `${count} blocking`; + }; + const site = (file, line) => + file === undefined + ? "no site" + : `[${span(file)}${line === undefined ? "" : ` line ${line}`}](${link(file, line)})`; + // The code a member binds, from the graph alone: the units that satisfy it and the units that + // reference it, each at its own site. + const codeText = (context) => { + const parts = []; + if (context.implementations.length > 0) + parts.push(plural(context.implementations.length, "anchor")); + if (context.references.length > 0) + parts.push(`referenced by ${plural(context.references.length, "unit")}`); + return parts.length === 0 ? "none" : parts.join(", "); + }; + const codeNodes = new Map( + graph.nodes + .filter((node) => node.nodeType === "CodeNode") + .map((node) => [node.id, node]), + ); + const componentOf = new Map( + graph.edges + .filter((edge) => edge.type === "memberOf") + .map((edge) => [edge.from, edge.to]), + ); + // One code unit: its id, its site and its label, then what the anchors declare of its structure: + // its role, and the layer and bounded context of the component it is or belongs to. + const placement = (node) => + list( + [ + node?.layer === undefined ? null : `the ${node.layer} layer`, + node?.context === undefined ? null : `the ${node.context} context`, + ].filter((part) => part !== null), + ); + const unitLine = (binding) => { + const facts = []; + if (binding.role !== undefined) facts.push(`Its role is ${binding.role}.`); + const own = placement(binding); + if (own !== "") facts.push(`It is a component in ${own}.`); + const component = componentOf.get(binding.codeId); + if (component !== undefined) { + const where = placement(codeNodes.get(component)); + facts.push( + `It belongs to ${span(component)}${where === "" ? "" : `, in ${where}`}.`, + ); + } + return `- ${span(binding.codeId)} at ${site(binding.file, binding.line)}${binding.label === undefined ? "" : `: ${binding.label}`}.${facts.length === 0 ? "" : ` ${facts.join(" ")}`}`; + }; + + // Head. + write( + `# ${summary.title ?? pack}`, + "", + "", + "", + ); + if (summary.framing !== undefined) write(summary.framing, "", ""); + const pinned = pinnedModule(pack); + write( + `This page is generated from the Specs of ${span(pack)} by ${span("scripts/pack-page.mjs")}, with two inputs: the pinned pairs in [${span(inputs.pairs)}](${link(inputs.pairs)}) and the register rows in [${span(inputs.register)}](${link(inputs.register)}). Do not edit it by hand. Change a Spec or an input, then run ${span("npm run pages")}; ${span("npm run pages -- --check")} fails while the page is behind the tree. The Protocol is pinned at ${span(`${engine}@${enginePin}`)}.${summary.modelRefs.length > 0 ? ` The members speak the vocabulary of ${list(summary.modelRefs.map(specLink))}.` : ""}`, + "", + "## How to read it", + "", + "Start with the members and with what holds a member below its next rung: together they say where the design stands and what keeps it there. Then come the Specs outside the Pack that the members rest on, the open questions with their register rows, the register rows whose sources lie in the members with their status and rulings, and each member in full with its code, its examples and verifiers, and its pinned declarations. Long lists are folded; open them where you need them.", + "", + "- The stated rung is the readiness a Spec's author states: idea, scoped, defined or ready.", + "- A register row blocks the first unit of work that needs its decision, a build slice such as S5 named in `design/STATE.md`. That is apart from a question marked blocking, which holds its Spec at scoped, and apart from the floor.", + "- The floor is the highest rung whose clauses all hold in the graph. It can sit above the stated rung.", + "- The next rung is the one above the stated rung, with the first floor clause that fails at it. A rung whose clauses all hold still waits for its author to state it.", + "- An open question is cited by its address, the Spec's id, `#question.` and the key its marker carries.", + "- An implementation anchor binds code to a Spec at its site, the file and line it is written at. Code that satisfies a Spec realizes it, and the graph derives implemented from it. Code that references a Spec answers to its design without claiming to realize it: the unit follows the design or realizes part of it, so a change to the design asks the code to follow. A design that builds on existing code says so by its own relation to the Spec that code satisfies. A reference confers nothing. Each unit gives its role and its component, with the component's layer and bounded context, where the anchor declares them.", + `- A pair is a pinned declaration and the code a type test compares it with: bound when the two are equal, ahead when the design leads the code, not compared when the test compares nothing. The test imports every pinned declaration of the Pack from [${span(pinned)}](${link(pinned)}), which ${span("npm run sdp:build")} emits and git does not keep.`, + "", + ); + + // Members. + write( + "## Members", + "", + `${plural(members.length, "Spec")}, in the order the Pack reads them.`, + "", + ...table( + [ + "#", + "Spec", + "Kind", + "Stated", + "Floor", + "Next rung", + "Code", + "Examples and verifiers", + "Pairs", + "Open questions", + ], + model.map((member) => [ + `[${member.number}](#${slug(member.heading)})`, + specLink(member.id), + member.context.specKind, + member.context.statedReadiness, + member.context.derivedReadiness ?? "none", + nextText(member), + codeText(member.context), + evidenceText(member), + pairCounts(member), + member.questions.length === 0 + ? "none" + : `${member.questions.length}, ${blockingCount(member.questions)}`, + ]), + ), + "", + ); + + // What holds a member below its next rung. + write("## What holds a member below its next rung", ""); + const held = model.filter((member) => member.next?.failing.length > 0); + const clear = model.filter( + (member) => member.next?.rung !== null && member.next?.failing.length === 0, + ); + if (held.length === 0) write("No member is held below its next rung.", ""); + for (const member of held) { + const [first, ...rest] = member.next.failing; + const floor = member.context.derivedReadiness; + write( + `- ${memberLink(member.id)} states ${member.context.statedReadiness} and cannot reach ${member.next.rung}.${member.next.below ? ` Its floor, ${floor ?? "none"}, is below its stated rung.` : ""} The first clause that fails is ${span(first.clauseId)}, which the Protocol states as "${first.description}"${rest.length > 0 ? ` After it, ${list(rest.map((clause) => span(clause.clauseId)))} ${rest.length === 1 ? "fails" : "fail"} too.` : ""}`, + ); + if (member.blockers.length > 0) + write( + ` It rests on ${list(member.blockers.map((target) => `${specLink(target.id)} by ${target.type}, which ${target.statedReadiness === undefined ? "is not a Spec" : `states ${target.statedReadiness}`}`))}.`, + ); + const blocking = member.questions.filter((question) => question.blocking); + if ( + member.next.failing.some( + (clause) => clause.clauseId === blockingClause, + ) && + blocking.length > 0 + ) + write( + ` Its blocking ${blocking.length === 1 ? "question is" : "questions are"} ${list(blocking.map((question) => (question.address === null ? `question ${question.position}` : span(question.address))))}, under its open questions below.`, + ); + } + if (held.length > 0) write(""); + if (clear.length > 0) + write( + `The floor holds at the next rung for ${list(clear.map((member) => memberLink(member.id)))}. Each waits for its author to state that rung.`, + "", + ); + const top = model.filter((member) => member.next?.rung === null); + if (top.length > 0) + write( + `${list(top.map((member) => memberLink(member.id)))} state ready, the top rung.`, + "", + ); + + // Outside the Pack. + const outside = new Map(); + for (const member of model) + for (const relation of member.typed) { + if (memberSet.has(relation.otherId)) continue; + const target = outside.get(relation.otherId) ?? { + id: relation.otherId, + from: [], + blocks: [], + }; + target.from.push(`${span(member.id)} ${relation.type}`); + if ( + member.blockers.some((blocker) => blocker.id === relation.otherId) && + !target.blocks.includes(member) + ) + target.blocks.push(member); + outside.set(relation.otherId, target); + } + const targets = [...outside.values()].sort((left, right) => + left.id.localeCompare(right.id), + ); + const rungOf = (id) => specs.get(id)?.statedReadiness ?? "not in the graph"; + const floorOf = (id) => + specs.has(id) + ? (specs.get(id).derivedReadiness ?? "none") + : "not in the graph"; + const blocking = targets.filter((target) => target.blocks.length > 0); + write( + "## Outside the Pack", + "", + "### What the members rest on", + "", + `The members rest on ${plural(targets.length, "Spec")} outside the Pack through refines, dependsOn, constrainedBy or decidedBy. A Spec states ready only when every such target states at least defined, so a target below defined holds each member that rests on it below ready.`, + "", + ); + if (blocking.length === 0) + write("No target outside the Pack holds a member below its next rung.", ""); + else + write( + ...table( + ["Spec", "Stated", "Floor", "Holds below ready", "Rested on by"], + blocking.map((target) => [ + specLink(target.id), + rungOf(target.id), + floorOf(target.id), + list(target.blocks.map((member) => memberLink(member.id))), + target.from.join(", "), + ]), + ), + "", + ); + write( + `
All ${plural(targets.length, "Spec")} outside the Pack that the members rest on`, + "", + ...table( + ["Spec", "Stated", "Floor", "Rested on by"], + targets.map((target) => [ + specLink(target.id), + rungOf(target.id), + floorOf(target.id), + target.from.join(", "), + ]), + ), + "", + "
", + "", + ); + const restingOn = new Map(); + for (const member of model) + for (const relation of member.context.relationsIn) { + if ( + memberSet.has(relation.otherId) || + !typedRelations.includes(relation.type) + ) + continue; + if ( + member.context.verifiers.some( + (verifier) => verifier.verifierId === relation.otherId, + ) + ) + continue; + const source = restingOn.get(relation.otherId) ?? { + id: relation.otherId, + to: [], + members: [], + }; + source.to.push(`${relation.type} ${span(member.id)}`); + source.members.push(member.id); + restingOn.set(relation.otherId, source); + } + const dependents = [...restingOn.values()].sort((left, right) => + left.id.localeCompare(right.id), + ); + write( + "### What rests on the members", + "", + `${plural(dependents.length, "Spec")} outside the Pack ${dependents.length === 1 ? "rests" : "rest"} on a member. An example that verifies a member is listed with that member instead.`, + "", + ); + const designedOnly = dependents.filter( + (source) => rank(specs.get(source.id)?.statedReadiness) < rank("defined"), + ); + if (designedOnly.length > 0) + write( + `${designedOnly.length === 1 ? "One of them states" : `${designedOnly.length} of them state`} less than defined:`, + "", + ...designedOnly.map( + (source) => + `- ${specLink(source.id)} states ${rungOf(source.id)} and rests on ${list( + [...new Set(source.members)].map((id) => memberLink(id)), + )}.`, + ), + "", + ); + if (dependents.length > 0) + write( + `
The ${plural(dependents.length, "Spec")} that rest on a member`, + "", + ...table( + ["Spec", "Stated", "Floor", "Rests on"], + dependents.map((source) => [ + specLink(source.id), + rungOf(source.id), + floorOf(source.id), + source.to.join(", "), + ]), + ), + "", + "
", + "", + ); + + // Open questions. + const allQuestions = model.flatMap((member) => member.questions); + const unjoined = allQuestions.filter( + (question) => question.rows.length === 0, + ); + write( + "## Open questions", + "", + `${plural(allQuestions.length, "open question")} on ${plural(model.filter((member) => member.questions.length > 0).length, "member")}, ${blockingCount(allQuestions)}. ${allQuestions.length - unjoined.length} join a register row by their address; ${unjoined.length} join none. A row gives its OD id, its class, its status, the advisor whose lens it falls under and what it blocks. Each question shows its address, which ends in its key, and its opening, at most its first sentence, and links to its line in the Spec for the rest.`, + "", + ); + const rowText = (row) => + `**${row.id}** class ${row.class ?? "none"}, status ${row.status ?? "none"}, advisor ${row.advisor ?? "none"}, ${row.blocks ? `blocks the unit ${row.blocks}` : "blocks no unit"}`; + for (const member of model) { + if (member.questions.length === 0 && member.leftRows.length === 0) continue; + write(`**${memberLink(member.id)}**, ${span(member.id)}`, ""); + for (const question of member.questions) { + const name = + question.address === null + ? `question ${question.position}, which carries no key` + : span(question.address); + const where = + question.line === null + ? name + : `[${name}](${link(member.context.file, question.line)})`; + const rows = + question.rows.length === 0 + ? "No register row" + : question.rows.map(rowText).join("; "); + write( + `- ${rows}. ${where}, ${question.blocking ? "[blocking]" : "[non-blocking]"}: ${opening(question.question)}`, + ); + } + for (const row of member.leftRows) + write( + `- ${rowText(row)}. Its source cites ${span(row.address)}, which names none of this Spec's open questions.`, + ); + write(""); + } + + // The register rows whose sources lie in the members, beside those the open questions show. + const memberFiles = new Map( + model.map((member) => [member.context.file, member]), + ); + const onMembers = memberRows(register, memberFiles); + const decided = onMembers.filter((row) => row.status === "decided"); + const questionLines = new Map( + model.flatMap((member) => + member.questions + .filter((question) => question.address !== null) + .map((question) => [ + question.address, + { file: member.context.file, line: question.line }, + ]), + ), + ); + const sourceText = (source) => { + const member = memberFiles.get(source.file); + const on = `on ${memberLink(member.id)}`; + if (source.kind === "specQuestion" && source.open === false) + return `a question ${on} that the Spec no longer holds`; + if (source.kind === "specQuestion") { + const at = questionLines.get(source.address); + return `${at === undefined || at.line === null ? span(source.address) : `[${span(source.address)}](${link(at.file, at.line)})`} ${on}`; + } + const name = span(source.ref ?? source.kind); + return `${source.line === undefined ? `[${name}](${link(source.file)})` : `[${name}](${link(source.file, source.line)})`} ${on}`; + }; + write( + "## Register rows whose sources lie in the members", + "", + `${plural(onMembers.length, "register row")} ${onMembers.length === 1 ? "has" : "have"} a source in a member's file: every decided row, and every row not decided whose sources cite no open question; a row not decided that cites one is under the open questions above. ${decided.length} ${decided.length === 1 ? "is" : "are"} decided and ${onMembers.length - decided.length} ${onMembers.length - decided.length === 1 ? "is" : "are"} not. Each row gives its status and its sources in the members; a decided row gives the first sentence of its ruling, and a row not decided gives its title. A source other than a question links to the line where it stood when the row was loaded. ${span("python3 design/tools/decisions.py --id ")} prints a row in full.`, + "", + ); + for (const row of onMembers) + write( + `- ${rowText(row)}, from ${list(row.sources.map(sourceText))}. ${ + row.status === "decided" + ? `Decided by ${row.decidedBy ?? "no one named"} on ${row.decidedOn ?? "no date"}. Ruling: ${opening(row.ruling ?? "none recorded.")}` + : `No ruling. Its title: ${opening(row.title ?? "none.")}` + }`, + ); + if (onMembers.length > 0) write(""); + + // Each member. + write("## Each member", ""); + for (const member of model) { + const { context } = member; + write( + `### ${member.heading}`, + "", + `${specLink(member.id)} · ${context.specKind} · stated ${context.statedReadiness} · floor ${context.derivedReadiness ?? "none"} · next rung ${nextText(member)}.`, + "", + ); + const inside = member.typed + .filter((relation) => memberSet.has(relation.otherId)) + .sort( + (left, right) => + typedRelations.indexOf(left.type) - + typedRelations.indexOf(right.type) || + members.indexOf(left.otherId) - members.indexOf(right.otherId), + ); + if (inside.length > 0) + write( + `Inside the Pack it rests on ${list(inside.map((relation) => `${memberLink(relation.otherId)} by ${relation.type}`))}.`, + "", + ); + + const isExample = context.specKind === "example"; + const tests = testsOf(member); + if (isExample) { + const line = openingLine(member.id); + const tier = tierOf(line); + write( + `**Tier and verifier.** Its opening line is ${span(line)}, which ${tier === null ? "names no tier" : `names the ${tier} tier`}. ${tests.length === 0 ? "No verifier is bound to it." : `${plural(tests.length, "verifier is", "verifiers are")} bound to it.`}`, + "", + ); + } + + if (context.implementations.length > 0) { + write("**Code.**", ""); + for (const binding of context.implementations) write(unitLine(binding)); + write(""); + } else if (!isExample) + write("**Code.** No implementation anchor binds it.", ""); + if (context.references.length > 0) { + write( + `**References.** ${plural(context.references.length, "unit references", "units reference")} this design. A reference says the code answers to the design without claiming to realize it, and it confers no delivery fact and moves no floor.`, + "", + ); + for (const binding of context.references) write(unitLine(binding)); + write(""); + } + + const examples = examplesOf(member); + if (examples.length > 0) + write( + "**Examples.**", + "", + ...table( + ["Example", "Tier", "Verifier", "Title"], + examples.map((example) => [ + specLink(example.verifierId), + tierOf(openingLine(example.verifierId)) ?? "not named", + example.enabled ? "bound" : "none bound", + example.label ?? "", + ]), + ), + "", + ); + if (tests.length > 0) { + write("**Verifiers.**", ""); + for (const test of tests) + write( + `- ${span(test.verifierId)} at ${site(test.file, test.line)}${test.label === undefined ? "" : `: ${test.label}`}.`, + ); + write(""); + } + if (!isExample && examples.length === 0 && tests.length === 0) + write("**Examples and verifiers.** None is bound to it.", ""); + + const entryLink = (entry) => + entry.line === null + ? span(entry.address) + : `[${span(entry.address)}](${link(context.file, entry.line)})`; + const tableLink = (key) => { + const line = member.entries.find((entry) => entry.key === key)?.line; + return line === null || line === undefined + ? span(key) + : `[${span(key)}](${link(context.file, line)})`; + }; + if (member.entries.length > 0) { + write( + `**Pinned declarations.** ${plural(member.entries.length, "Design entry", "Design entries")} open with a code span. Each pair comes from [${span(inputs.pairs)}](${link(inputs.pairs)}) and is the test named by its address in [${span(pairs.test)}](${link(pairs.test)}).`, + "", + ...table( + ["Design entry", "Pair", "Compared", "Code", "Why"], + member.entries.map((entry) => { + if (entry.pair === null) + return [ + entryLink(entry), + "no pair", + "", + "", + `${span(inputs.pairs)} lists no pair for it`, + ]; + const [file, name] = entry.pair.code?.split("#") ?? []; + const comparedText = + entry.table === null + ? compared[entry.pair.comparison] + : entry.pair.comparison === "table" + ? `the indexes, through its table ${tableLink(entry.table)}` + : `${compared[entry.pair.comparison]}, through its table ${tableLink(entry.table)}`; + return [ + entryLink(entry), + states[entry.pair.state], + comparedText, + file === undefined + ? "none" + : `${span(name)} in [${span(file)}](${link(file)})`, + entry.pair.reason ?? "", + ]; + }), + ), + "", + `
The ${plural(member.entries.length, "declaration")} as the Spec pins them`, + "", + ); + for (const entry of member.entries) + write( + `${entryLink(entry)}${entry.declaration.formatted ? "" : ", as the Spec writes it, which Prettier does not parse"}`, + "", + "```ts", + entry.declaration.text, + "```", + "", + ); + write("
", ""); + } else if (!isExample) write("**Pinned declarations.** It pins none.", ""); + if (member.otherEntries.length > 0) + write( + `**Other Design entries.** ${plural(member.otherEntries.length, "entry pins", "entries pin")} no declaration: ${member.otherEntries + .map(({ key, line }) => `[${span(key)}](${link(context.file, line)})`) + .join(", ")}.`, + "", + ); + } + + // Pairs that name no pinned declaration of a member. + const shown = new Set( + model.flatMap((member) => member.entries.map((entry) => entry.address)), + ); + const strays = [...pairs.pairs.values()].filter( + (pair) => !shown.has(pair.address), + ); + const addressLink = (address) => { + const [id, key] = address.split("#design."); + const spec = specs.get(id); + if (spec === undefined) return span(address); + const line = specLines( + readFileSync(join(base, spec.file), "utf8"), + ).design.get(key); + return line === undefined + ? `[${span(address)}](${link(spec.file)})` + : `[${span(address)}](${link(spec.file, line)})`; + }; + if (strays.length > 0) + write( + "## Pairs without a member's declaration", + "", + `${plural(strays.length, "pair")} in ${span(inputs.pairs)} ${strays.length === 1 ? "names" : "name"} no pinned declaration of a member.`, + "", + ...table( + ["Address", "Pair", "Compared", "Code", "Why"], + strays.map((pair) => [ + addressLink(pair.address), + states[pair.state], + compared[pair.comparison], + pair.code === null ? "none" : span(pair.code), + pair.reason ?? "", + ]), + ), + "", + ); + + return `${page + .join("\n") + .replace(/\n{3,}/g, "\n\n") + .trimEnd()}\n`; +} + +// --------------------------------------------------------------------------------------------- +// The command. It takes the arguments after the script's path and the repository it reads, so a +// test runs it over a sample tree; it answers the exit status. + +export async function main(args = process.argv.slice(2), base = root) { + const { values } = parseArgs({ + args, + options: { + pack: { type: "string" }, + out: { type: "string" }, + check: { type: "boolean", default: false }, + }, + }); + if (values.pack === undefined || values.out === undefined) + cannot("name the Pack and the page: --pack --out "); + const out = resolve(values.out); + const path = relative(base, out).split(sep).join("/"); + if (path.startsWith("..") || !path.endsWith(".md")) + cannot( + `the page must be a .md file inside the repository, not ${values.out}`, + ); + const generated = await generate(values.pack, out, base); + if (!values.check) { + mkdirSync(dirname(out), { recursive: true }); + writeFileSync(out, generated); + console.error(`pack-page: wrote ${path}`); + return 0; + } + if (!existsSync(out)) { + console.error(`pack-page: ${path} is missing; run npm run pages`); + return 1; + } + const difference = firstDifference(readFileSync(out, "utf8"), generated); + if (difference === null) return 0; + console.error( + `pack-page: ${path} is behind the tree from line ${difference.line}: the page has ${JSON.stringify(difference.committed)}, the tree generates ${JSON.stringify(difference.generated)}; run npm run pages`, + ); + return 1; +} + +if ( + process.argv[1] !== undefined && + resolve(process.argv[1]) === fileURLToPath(import.meta.url) +) + main().then( + (code) => process.exit(code), + (error) => { + console.error( + `pack-page: cannot generate · ${error instanceof PageError ? error.message : (error?.stack ?? error)}`, + ); + process.exit(2); + }, + ); diff --git a/scripts/pinned.mjs b/scripts/pinned.mjs new file mode 100644 index 0000000..c0a728c --- /dev/null +++ b/scripts/pinned.mjs @@ -0,0 +1,408 @@ +// Emits the pinned module of one Pack, which the type tests import: +// `node scripts/pinned.mjs --pack [--name ]` writes generated/pinned/.ts, by +// default the Pack's id without `pack:`. It runs the body of recipe 24, "Pinned declarations", read +// from the pinned Protocol's recipes.md, with the pinned CLI. spec:extraction.contract-declarations +// rules the module: a projection, regenerated and never edited, which earns its place when a +// changed pin fails the compiled tier's typecheck against the code. +// +// This project's policy, by the prefix of the entry's key: `type*` opening with `type ` or +// `interface ` and `validator*` opening with `const ` are exported as written; `fn*` that is a call +// signature `name<...>(...): R` becomes `export declare function`, where a default value makes its +// parameter optional, or, when a required parameter follows, keeps it required with a type that +// admits `undefined`; `table*` of the form `name: defineTable(...)` is a member of +// `export const tables`, and each `index*` of the form `.index(...)` is chained onto the table +// entry its Spec wrote last before it. Every other entry is skipped and counted by its prefix on +// stderr. Each pin of another Spec that a declaration uses is pulled in, marked. A name pinned in two +// entries, in any form, is refused, as is a preamble name a Spec pins. +// Exit 0: written. 2: it cannot emit. The pure parts are exported for tests/pure/pinned.test.ts. +import { execFileSync } from "node:child_process"; +import { mkdirSync, readFileSync, writeFileSync } from "node:fs"; +import { join, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { parseArgs } from "node:util"; +import * as prettier from "prettier"; +import ts from "typescript"; + +const root = join(import.meta.dirname, ".."); +const protocol = join( + root, + "node_modules/@libar-dev/software-delivery-protocol", +); +function cannot(reason) { + console.error(`pinned: cannot emit · ${reason}`); + process.exit(2); +} + +const preamble = `import { + defineTable, + type FunctionReference, + type GenericDataModel, + type GenericDatabaseReader, + type GenericDatabaseWriter, + type GenericMutationCtx, + type GenericQueryCtx, + type PaginationOptions, + type PaginationResult, + type RegisteredMutation, + type RegisteredQuery, +} from "convex/server"; +import { v, type GenericId, type Validator, type Value } from "convex/values"; + +// Provided by Convex or by a composition's generated code. +type MutationCtx = GenericMutationCtx; // _generated/server.js, over the composition's data model. +type QueryCtx = GenericQueryCtx; // _generated/server.js, over the composition's data model. +type Id = GenericId; // _generated/dataModel.js.`; + +// Names the design uses that no Spec pins as a declaration: the code's module that declares each, +// or the type that stands in when the code declares none. +const gaps = { + AppendResult: "src/context", + BatchRef: "src/read-model", + DiagnosticSink: "src/operations", + FailedCall: "src/command", + FillBatchRef: "src/read-model", + GateDataModel: "src/gate", + ListArgs: "src/context", + ReadModelDataModel: "src/read-model", + RowDataModel: "src/read-model", + SubjectRef: "src/command", + WriteBaselineArgs: "Record", + WriteBaselineResult: "unknown", + outcomeKindValidator: "src/command", +}; +function gapDeclaration(name) { + const source = gaps[name]; + const code = `import("../../${source}/index.js").${name}`; + if (!source.startsWith("src/")) + return `type ${name} = ${source}; // nothing in the code stands in.`; + return /^[a-z]/.test(name) + ? `declare const ${name}: typeof ${code};` + : `type ${name} = ${code};`; +} + +function query(body) { + const sdp = join(protocol, "dist/cli/sdp.js"); + try { + const out = execFileSync( + process.execPath, + [sdp, "q", body, "--root", root, "--json"], + { cwd: root, encoding: "utf8", maxBuffer: 1 << 27 }, + ); + return JSON.parse(out); + } catch (error) { + cannot(`the query failed: ${error.message.split("\n")[0]}`); + } +} + +// The body of recipe 24 as the Protocol ships it: the first js fence under its heading. +function recipe24() { + const recipes = readFileSync( + join(protocol, "docs/agent-surface/recipes.md"), + "utf8", + ); + const section = recipes.split(/^## /m).find((part) => /^24\. /.test(part)); + const fence = /^```js\n([\s\S]*?)^```$/m.exec(section ?? ""); + if (fence === null) cannot("recipes.md has no recipe 24 with a js fence"); + return fence[1]; +} + +const parse = (text) => + ts.createSourceFile("pin.ts", text, ts.ScriptTarget.Latest, true); +const children = (node) => { + const list = []; + ts.forEachChild(node, (child) => { + list.push(child); + }); + return list; +}; + +// A type that `| undefined` would otherwise bind inside. +const grouped = (type) => + ts.isFunctionTypeNode(type) || + ts.isConstructorTypeNode(type) || + ts.isConditionalTypeNode(type); + +// A call signature parses alone as one bodiless function declaration with a return type. A default +// value becomes a comment on its parameter: a trailing default makes the parameter optional, and a +// default before a required parameter, which no optional parameter may precede, keeps it required +// with a type that admits `undefined`. +export function callSignature(span) { + if (!/^[\w$]+\s*[<(]/.test(span)) return null; + const file = parse(`export declare function ${span}`); + const [statement] = file.statements; + if ( + file.parseDiagnostics.length > 0 || + file.statements.length !== 1 || + !ts.isFunctionDeclaration(statement) || + statement.body !== undefined || + statement.type === undefined + ) + return null; + const required = (parameter) => + parameter.initializer === undefined && + parameter.questionToken === undefined && + parameter.dotDotDotToken === undefined; + const parameters = [...statement.parameters]; + let text = file.text; + for (const [at, { name, type, initializer }] of [ + ...parameters.entries(), + ].reverse()) { + if (initializer === undefined) continue; + const beforeRequired = parameters.slice(at + 1).some(required); + const annotation = + type === undefined + ? "" + : beforeRequired + ? `: ${grouped(type) ? `(${type.getText(file)})` : type.getText(file)} | undefined` + : text.slice(name.end, type.end); + text = `${text.slice(0, name.end)}${beforeRequired ? "" : "?"}${annotation} /* = ${initializer.getText(file)} */${text.slice(initializer.end)}`; + } + return { name: statement.name.text, text }; +} + +// What one entry declares under the policy, or kind null. +const exported = { + type: [/^(?:type|interface)\s+([\w$]+)/, "type"], + validator: [/^const\s+([\w$]+)/, "value"], +}; +export function declarationOf(row) { + const prefix = /^[a-z]+/.exec(row.key)[0]; + const span = row.declaration.trim(); + const base = { ...row, prefix, address: `${row.spec}#design.${row.key}` }; + const named = exported[prefix]?.[0].exec(span)?.[1]; + if (named !== undefined) + return { + ...base, + kind: exported[prefix][1], + name: named, + text: `export ${span}`, + }; + const signature = prefix === "fn" ? callSignature(span) : null; + if (signature !== null) return { ...base, kind: "type", ...signature }; + if (prefix === "table" && /^[\w$]+:\s*defineTable\(/.test(span)) + return { ...base, kind: "table", name: "", text: span, indexes: [] }; + if (prefix === "index" && /^\.index\(/.test(span)) + return { ...base, kind: "index", name: "", text: span }; + return { ...base, kind: null }; +} + +// The names a node declares for what is inside it, by space: its type parameters and a mapped +// type's key are types, and its parameters are values. The type parameters a conditional type +// infers in its extends clause are types inside its true branch and nowhere else, which is where +// TypeScript sees them; a nested conditional type's are its own. +const named = (declaration) => + ts.isIdentifier(declaration.name) ? [declaration.name.text] : []; +function declaredBy(node) { + return { + types: [ + ...(ts.isMappedTypeNode(node) ? named(node.typeParameter) : []), + ...(node.typeParameters ?? []).flatMap(named), + ], + values: (node.parameters ?? []).flatMap(named), + }; +} +const inferredBy = (type) => + ts.isConditionalTypeNode(type) + ? [] + : [ + ...(ts.isInferTypeNode(type) ? named(type.typeParameter) : []), + ...children(type).flatMap(inferredBy), + ]; + +// Whether an identifier names a type: the name of a type reference, of a heritage clause or of an +// import type, or the head of a qualified name under one. `typeof x` and every expression name a +// value. +function namesType(node) { + let at = node; + while (ts.isQualifiedName(at.parent)) at = at.parent; + return ( + ts.isTypeReferenceNode(at.parent) || + ts.isExpressionWithTypeArguments(at.parent) || + ts.isImportTypeNode(at.parent) + ); +} + +// The names a declaration uses: identifiers that are not a property, a member, what is declared, or +// a name in scope. A name is resolved in its own space, so a value parameter hides no type of its +// name and a type parameter hides no value; and a parameter or type parameter is in scope inside +// the node that declares it and nowhere else, so a nested `` hides no outer `Outcome`. +export function namesUsedBy(declaration) { + const used = new Set(); + const extended = (scope, names) => + names.length === 0 ? scope : new Set([...scope, ...names]); + const visit = (node, types, values) => { + const parent = node.parent; + if ( + ts.isIdentifier(node) && + !(namesType(node) ? types : values).has(node.text) && + (ts.isShorthandPropertyAssignment(parent) || + (ts.isPropertyAccessExpression(parent) + ? parent.expression === node + : ts.isQualifiedName(parent) + ? parent.left === node + : parent.name !== node)) + ) + used.add(node.text); + const declared = declaredBy(node); + const innerTypes = extended(types, declared.types); + const innerValues = extended(values, declared.values); + const inTrueBranch = ts.isConditionalTypeNode(node) + ? extended(innerTypes, inferredBy(node.extendsType)) + : innerTypes; + for (const child of children(node)) + visit( + child, + child === node.trueType ? inTrueBranch : innerTypes, + innerValues, + ); + }; + const own = new Set([declaration.name]); + visit( + parse( + declaration.kind === "table" + ? `const tables = { ${declaration.text} };` + : declaration.text, + ), + own, + own, + ); + return [...used]; +} + +// The scope's own entries, then the pin of each name they use that another Spec pins, marked with +// the first address that uses it; and each name used that no Spec pins, with the addresses that use +// it. +export function pulledIn(own, pinnedBy) { + const emitted = own.filter( + (row) => row.kind !== null && row.kind !== "index", + ); + const gapUses = new Map(); + for (let at = 0; at < emitted.length; at++) { + const { address } = emitted[at]; + for (const used of namesUsedBy(emitted[at])) { + if (used in gaps) + gapUses.set(used, [...(gapUses.get(used) ?? []), address]); + const [pin] = pinnedBy.get(used) ?? []; + if (pin !== undefined && !emitted.includes(pin)) + emitted.push(Object.assign(pin, { why: `${address} uses ${used}` })); + } + } + return { emitted, gapUses }; +} + +async function main(args = process.argv.slice(2)) { + const { values: options } = parseArgs({ + args, + options: { pack: { type: "string" }, name: { type: "string" } }, + }); + if (options.pack === undefined) cannot("name the Pack: --pack "); + const name = options.name ?? options.pack.replace(/^pack:/, ""); + const scope = query( + `return g.packContext(${JSON.stringify(options.pack)})?.members.map((member) => member.id) ?? null;`, + ); + if (scope === null) cannot(`the graph has no ${options.pack}`); + + const all = query(recipe24()).rows.map(declarationOf); + let table; + for (const row of all) + if (row.kind === "table") table = row; + else if (row.kind === "index" && table?.spec === row.spec) + table.indexes.push(row); + else if (row.kind === "index") row.kind = null; + const pinnedBy = Map.groupBy( + all.filter((row) => row.kind === "type" || row.kind === "value"), + (row) => row.name, + ); + const addresses = (pins) => pins.map((pin) => pin.address).join(" and "); + for (const [twice, pins] of pinnedBy) + if (pins.length > 1) + cannot(`${twice} is pinned in two entries, ${addresses(pins)}`); + for (const provided of [ + "MutationCtx", + "QueryCtx", + "Id", + ...Object.keys(gaps), + ]) + if (pinnedBy.has(provided)) + cannot( + `the preamble declares ${provided}, which ${addresses(pinnedBy.get(provided))} pins: take it out of the preamble`, + ); + + const own = all.filter((row) => scope.includes(row.spec)); + const { emitted, gapUses } = pulledIn(own, pinnedBy); + + // A value is declared after the values it uses. + const values = []; + const place = (declaration, path = new Set()) => { + if (values.includes(declaration) || path.has(declaration)) return; + path.add(declaration); + for (const used of namesUsedBy(declaration)) + for (const pin of pinnedBy.get(used) ?? []) + if (pin.kind === "value") place(pin, path); + values.push(declaration); + }; + emitted.filter((row) => row.kind === "value").forEach((row) => place(row)); + + const comment = (row) => + `// ${row.address}${row.why === undefined ? "" : ` (pulled in: ${row.why})`}`; + const block = (rows) => rows.flatMap((row) => [comment(row), row.text]); + const member = (table) => + [...block([table]), ...block(table.indexes).map((line) => ` ${line}`)].map( + (line, at, lines) => ` ${line}${at === lines.length - 1 ? "," : ""}`, + ); + const lines = [ + `// The pinned declarations of ${options.pack}, emitted by scripts/pinned.mjs from recipe 24 of the`, + "// pinned Protocol. Never edit it: change the Spec, then run `npm run sdp:build`. Each declaration", + "// follows the address of its Design entry.", + preamble, + "", + "// Used by the design and pinned by no Spec. Where the code declares one, its declaration stands in.", + ...[...gapUses.keys()].sort().flatMap((gap) => { + const skippedPin = all.find( + (row) => row.kind === null && row.declaration.startsWith(`${gap} =`), + ); + return [ + `// ${gap}: used by ${[...new Set(gapUses.get(gap))].join(", ")}.${skippedPin === undefined ? "" : ` ${skippedPin.address} opens with it, without const.`}`, + gapDeclaration(gap), + ]; + }), + "", + ...block(emitted.filter((row) => row.kind === "type")), + ...block(values), + "export const tables = {", + ...emitted.filter((row) => row.kind === "table").flatMap(member), + "};", + ]; + const outputPath = join(root, "generated/pinned", `${name}.ts`); + let formatted; + try { + const config = JSON.parse( + readFileSync(join(root, ".prettierrc.json"), "utf8"), + ); + formatted = await prettier.format(`${lines.join("\n")}\n`, { + ...config, + filepath: outputPath, + }); + } catch (error) { + cannot(`${outputPath}: Prettier failed: ${error.message}`); + } + mkdirSync(join(root, "generated/pinned"), { recursive: true }); + writeFileSync(outputPath, formatted); + + const counted = (rows) => + [...Map.groupBy(rows, (row) => row.prefix)] + .sort(([left], [right]) => left.localeCompare(right)) + .map(([prefix, group]) => `${prefix} ${group.length}`) + .join(", "); + const ownEmitted = own.filter((row) => row.kind !== null); + const skipped = own.filter((row) => row.kind === null); + console.error( + `pinned: generated/pinned/${name}.ts · ${scope.length} Specs · emitted ${ownEmitted.length} (${counted(ownEmitted)}) and ${emitted.length - emitted.filter((row) => row.why === undefined).length} pulled in · skipped ${skipped.length} (${counted(skipped)}) · ${gapUses.size} names pinned by no Spec`, + ); +} + +if ( + process.argv[1] !== undefined && + resolve(process.argv[1]) === fileURLToPath(import.meta.url) +) + await main(); diff --git a/src/audit/index.ts b/src/audit/index.ts index 1651224..612c77b 100644 --- a/src/audit/index.ts +++ b/src/audit/index.ts @@ -1,4 +1,12 @@ // The audit library: the audit record of a command and the operator audit record of a gate change. +/** + * @sdpAnchor component:platform.audit + * @sdpLabel the audit library: the audit record of a command and of a gate change + * @sdpReferences spec:operations.baseline-operations + * @sdpUses component:platform.command, component:platform.context + * @sdpLayer infrastructure + * @sdpContext platform + */ export { auditDecisionValidator, auditKindValidator, diff --git a/src/command/actor-and-scope.ts b/src/command/actor-and-scope.ts index 139981f..3f84ba0 100644 --- a/src/command/actor-and-scope.ts +++ b/src/command/actor-and-scope.ts @@ -3,6 +3,13 @@ // entry states. Validators, types and the operator's check only, so the context library can import the // actor's validator without carrying the parent's helpers; those, authorize, establishActor and the // grant helpers, are in authority.ts. +/** + * @sdpAnchor impl:command.actor-and-scope + * @sdpLabel the actor, scope, namespace, authority and operator shapes + * @sdpSatisfies spec:command.actor-and-scope + * @sdpComponent component:platform.command + * @sdpRole contract + */ import { v } from "convex/values"; import { limitActorIdLength, utf8Length } from "../context/text.js"; export type ActorKind = "human" | "service" | "agent" | "reviewer" | "operator"; diff --git a/src/command/authority.ts b/src/command/authority.ts index 6031f5c..205ec95 100644 --- a/src/command/authority.ts +++ b/src/command/authority.ts @@ -3,6 +3,13 @@ // authorizeQuery, which a parent query calls before it discloses anything, the plain helpers an // operator's function uses to write and delete a grant, and nextTenant, which reads the tenant list. // None is registered here. +/** + * @sdpAnchor impl:command.tenancy-and-authority + * @sdpLabel establishActor, authorize, authorizeQuery and the grant helpers + * @sdpSatisfies spec:command.tenancy-and-authority, spec:command.actor-and-scope + * @sdpComponent component:platform.command + * @sdpRole service + */ import type { GenericDatabaseReader, GenericQueryCtx } from "convex/server"; import { ConvexError } from "convex/values"; import { limitIdLength, utf8Length } from "../context/text.js"; diff --git a/src/command/declaration.ts b/src/command/declaration.ts index de6c423..1a69c4b 100644 --- a/src/command/declaration.ts +++ b/src/command/declaration.ts @@ -1,5 +1,12 @@ // The command declaration of spec:command.command-declaration and its two composition helpers, which // turn one declaration into the pipeline's public and internal entries. +/** + * @sdpAnchor impl:command.command-declaration + * @sdpLabel CommandDeclaration, publicCommand and internalCommand + * @sdpReferences spec:command.command-declaration + * @sdpComponent component:platform.command + * @sdpRole contract + */ import { internalMutationGeneric, mutationGeneric, diff --git a/src/command/index.ts b/src/command/index.ts index 635c6a8..d7ecaae 100644 --- a/src/command/index.ts +++ b/src/command/index.ts @@ -1,4 +1,11 @@ // Layer 1's command library: what runs in the parent deployment. +/** + * @sdpAnchor component:platform.command + * @sdpLabel the command library that runs in the parent deployment + * @sdpUses component:platform.kernel, component:platform.context, component:platform.gate, component:platform.read-model, component:platform.operations, component:platform.audit + * @sdpLayer application + * @sdpContext platform + */ export { actorKindValidator, actorValidator, diff --git a/src/command/outcome-boundary.ts b/src/command/outcome-boundary.ts index 5681ee3..fc1c084 100644 --- a/src/command/outcome-boundary.ts +++ b/src/command/outcome-boundary.ts @@ -1,5 +1,12 @@ // The outcome boundary of spec:command.outcome-boundary: the wire shape of a rejection and of a // transient refusal, the helpers that throw them, and the one rethrow at the pipeline's outermost level. +/** + * @sdpAnchor impl:command.outcome-boundary + * @sdpLabel reject, refuseTransient, normalizeThrown and classifyThrown + * @sdpSatisfies spec:command.outcome-boundary + * @sdpComponent component:platform.command + * @sdpRole codec + */ import { ConvexError, getConvexSize, v } from "convex/values"; import { validate } from "convex-helpers/validators"; import { rejectionValidator } from "../context/outcome.js"; diff --git a/src/command/pipeline.ts b/src/command/pipeline.ts index 428baa6..da55169 100644 --- a/src/command/pipeline.ts +++ b/src/command/pipeline.ts @@ -1,5 +1,12 @@ // The command pipeline of spec:command.command-pipeline: the steps between a caller's request and the // committed outcome, in one top-level mutation. Its only failure signal is a throw. +/** + * @sdpAnchor impl:command.command-pipeline + * @sdpLabel runPipeline and the command response + * @sdpReferences spec:command.command-pipeline + * @sdpComponent component:platform.command + * @sdpRole service + */ import type { GenericDatabaseReader, GenericDatabaseWriter, diff --git a/src/command/receipts.ts b/src/command/receipts.ts index 65997f9..8568c9f 100644 --- a/src/command/receipts.ts +++ b/src/command/receipts.ts @@ -1,6 +1,13 @@ // Receipts of spec:command.idempotency-and-receipts: the key, the fingerprint, the lookup, the // classification and the insert, helpers inside the pipeline's mutation; and the sweep and its loop // over tenants, handlers that each composition registers as internal mutations for operations. +/** + * @sdpAnchor impl:command.idempotency-and-receipts + * @sdpLabel the receipt key, fingerprint, lookup, classification, insert and sweep + * @sdpReferences spec:command.idempotency-and-receipts + * @sdpComponent component:platform.command + * @sdpRole service + */ import { paginator } from "convex-helpers/server/pagination"; import { defineSchema, type GenericDatabaseReader } from "convex/server"; import { convexToJson, v, type JSONValue, type Value } from "convex/values"; diff --git a/src/command/tables.ts b/src/command/tables.ts index 4944462..c378e10 100644 --- a/src/command/tables.ts +++ b/src/command/tables.ts @@ -1,6 +1,14 @@ // The parent's tables of spec:command.receipt-table and spec:command.actor-and-scope, receipts, grants // and the tenant list, as a fragment the app's schema.ts spreads into defineSchema. Every index leads // with tenantId. +/** + * @sdpAnchor impl:command.receipt-table + * @sdpLabel the receipts, grants and tenant tables fragment + * @sdpSatisfies spec:command.actor-and-scope + * @sdpReferences spec:command.receipt-table + * @sdpComponent component:platform.command + * @sdpRole schema + */ import { defineTable, type DataModelFromSchemaDefinition, diff --git a/src/context/adapter.ts b/src/context/adapter.ts index f506e4c..234299d 100644 --- a/src/context/adapter.ts +++ b/src/context/adapter.ts @@ -92,6 +92,12 @@ export type OperationArgs = { input: I; facts?: Readonly>; }; +/** + * @sdpAnchor impl:context.batch-shaped-api + * @sdpLabel the per-call operation bounds and runOperation over a plan + * @sdpSatisfies spec:context.batch-shaped-api + * @sdpComponent component:platform.context + */ const mebibyte = 1024 * 1024; export const limitStreamsPerCall = 256; export const limitStreamBytesPerCall = 8 * mebibyte; @@ -102,6 +108,13 @@ export const limitReturnBytesPerCall = 8 * mebibyte; function reject(rejection: Rejection): never { throw new ConvexError(rejection); } +/** + * @sdpAnchor impl:context.persistence-adapter + * @sdpLabel execute: load, decide, fold, append and save one stream + * @sdpReferences spec:context.persistence-adapter + * @sdpComponent component:platform.context + * @sdpRole service + */ // Load, decide, fold, append and save one stream, in that order, in the caller's sub-transaction. export async function execute( ctx: MutationCtx, diff --git a/src/context/envelope.ts b/src/context/envelope.ts index d7989ee..d43a9dd 100644 --- a/src/context/envelope.ts +++ b/src/context/envelope.ts @@ -1,4 +1,11 @@ // The event envelope of spec:context.event-envelope: the fifteen fields every event carries. +/** + * @sdpAnchor impl:context.event-envelope + * @sdpLabel the envelope types and validators + * @sdpSatisfies spec:context.event-envelope + * @sdpComponent component:platform.context + * @sdpRole contract + */ import { v } from "convex/values"; import { actorValidator, type Actor } from "../command/actor-and-scope.js"; export type CausedBy = diff --git a/src/context/index.ts b/src/context/index.ts index 2b26c72..2249676 100644 --- a/src/context/index.ts +++ b/src/context/index.ts @@ -1,4 +1,11 @@ // Layer 1's context library: what runs inside a context component. +/** + * @sdpAnchor component:platform.context + * @sdpLabel the context library that runs inside every context component + * @sdpUses component:platform.kernel, component:platform.command + * @sdpLayer adapter + * @sdpContext platform + */ export { causedByValidator, eventEnvelopeValidator, diff --git a/src/context/journal.ts b/src/context/journal.ts index e7328dc..3627569 100644 --- a/src/context/journal.ts +++ b/src/context/journal.ts @@ -1,4 +1,11 @@ // The journal of spec:context.journal: a context's events, the stream registration, load and append. +/** + * @sdpAnchor impl:context.journal + * @sdpLabel createJournal, load and append + * @sdpReferences spec:context.journal + * @sdpComponent component:platform.context + * @sdpRole event-store + */ import type { DocumentByName as Doc } from "convex/server"; import { getConvexSize, diff --git a/src/context/operation.ts b/src/context/operation.ts index 3b31cee..84d1500 100644 --- a/src/context/operation.ts +++ b/src/context/operation.ts @@ -29,6 +29,13 @@ export const operationArgsValidators = ( // A component's generated mutation builder is mutationGeneric typed with the component's data model. // The library types it with the tables it owns, which every context component's schema holds. const mutation: MutationBuilder = mutationGeneric; +/** + * @sdpAnchor impl:context.context-component + * @sdpLabel defineOperation, one registered mutation per sanctioned operation + * @sdpReferences spec:context.context-component + * @sdpComponent component:platform.context + * @sdpRole service + */ export function defineOperation( journal: Journal, declaration: OperationDeclaration, diff --git a/src/context/outcome.ts b/src/context/outcome.ts index b8e1d87..7168149 100644 --- a/src/context/outcome.ts +++ b/src/context/outcome.ts @@ -1,6 +1,13 @@ // The validators of the kernel's outcome model (spec:kernel.outcome-model), which the kernel may not // hold because it imports nothing from Convex at run time, and the operation outcome's validator of // spec:context.persistence-adapter. +/** + * @sdpAnchor impl:kernel.outcome-model.validators + * @sdpLabel the outcome model's validators at the context boundary, and the operation outcome's + * @sdpReferences spec:kernel.outcome-model, spec:context.persistence-adapter + * @sdpComponent component:platform.context + * @sdpRole contract + */ import { v, type Validator } from "convex/values"; import { eventEnvelopeValidator } from "./envelope.js"; export const affectedRefValidator = v.object({ diff --git a/src/context/queries.ts b/src/context/queries.ts index b73cdd4..2c77fb4 100644 --- a/src/context/queries.ts +++ b/src/context/queries.ts @@ -1,5 +1,12 @@ // The context queries of spec:context.queries: one definer per query a context serves, each // registering a public query of the component that the parent calls through the component API. +/** + * @sdpAnchor impl:context.queries + * @sdpLabel defineGet, defineList and boundedPage + * @sdpReferences spec:context.queries + * @sdpComponent component:platform.context + * @sdpRole query + */ import { paginator } from "convex-helpers/server/pagination"; import { defineSchema, diff --git a/src/context/tables.ts b/src/context/tables.ts index caceb0e..5492043 100644 --- a/src/context/tables.ts +++ b/src/context/tables.ts @@ -1,5 +1,12 @@ // The tables of spec:context.tables, as a fragment a context component's schema.ts spreads into // defineSchema. Single mapping only: no streamParts table. +/** + * @sdpAnchor impl:context.tables + * @sdpLabel the context tables fragment + * @sdpReferences spec:context.tables + * @sdpComponent component:platform.context + * @sdpRole schema + */ import { defineTable, type DataModelFromSchemaDefinition, diff --git a/src/gate/entries.ts b/src/gate/entries.ts index 1af0d53..9ed6eb4 100644 --- a/src/gate/entries.ts +++ b/src/gate/entries.ts @@ -4,6 +4,14 @@ // reaches it, that reads no grant and no ctx.auth; the two that change the gate check the stated // operator first and record the trimmed text it returns. An operator entry closes and resumes only "all" or a tenant scope; a // source scope is closed and reopened by the rebuild alone. Every refusal is a plain Error. +/** + * @sdpAnchor impl:application.write-pause.entries + * @sdpLabel the four gate operator entries + * @sdpSatisfies spec:application.write-pause + * @sdpReferences spec:command.actor-and-scope + * @sdpComponent component:platform.gate + * @sdpRole operator-entry + */ import { internalMutationGeneric, internalQueryGeneric, diff --git a/src/gate/gate.ts b/src/gate/gate.ts index c841da6..be90864 100644 --- a/src/gate/gate.ts +++ b/src/gate/gate.ts @@ -5,6 +5,13 @@ // nothing else. closeScope and resumeScope are the only writers of the gate document, and each writes // its operator audit record in the caller's mutation with no catch around it, so a failed insert // changes nothing. +/** + * @sdpAnchor impl:application.write-pause + * @sdpLabel gateAllows, assertWritable and the two scope writers + * @sdpSatisfies spec:application.write-pause + * @sdpComponent component:platform.gate + * @sdpRole service + */ import type { GenericDatabaseReader, GenericDatabaseWriter, diff --git a/src/gate/index.ts b/src/gate/index.ts index 896bddf..6a73267 100644 --- a/src/gate/index.ts +++ b/src/gate/index.ts @@ -1,5 +1,13 @@ // The maintenance gate: its table, the writer check every command and background batch makes, the two // gate writers and the four operator entries a composition re-exports from its gate module. +/** + * @sdpAnchor component:platform.gate + * @sdpLabel the maintenance gate: its table, the writer check, the two writers and the operator entries + * @sdpSatisfies spec:application.write-pause + * @sdpUses component:platform.command, component:platform.context, component:platform.audit, component:platform.read-model + * @sdpLayer infrastructure + * @sdpContext platform + */ export { closeGate, getGate, diff --git a/src/gate/tables.ts b/src/gate/tables.ts index ad0651e..c52f890 100644 --- a/src/gate/tables.ts +++ b/src/gate/tables.ts @@ -1,6 +1,13 @@ // The maintenance gate's table of spec:application.write-pause, as a fragment every composition that // registers a command spreads into defineSchema, because the pipeline's step 7 reads it. It holds at // most one document, whose closed entries are in the order they were closed. +/** + * @sdpAnchor impl:application.write-pause.tables + * @sdpLabel the gate document's table fragment + * @sdpSatisfies spec:application.write-pause + * @sdpComponent component:platform.gate + * @sdpRole schema + */ import { defineTable, type DataModelFromSchemaDefinition, diff --git a/src/kernel/decider.ts b/src/kernel/decider.ts index dd764be..d23fdb8 100644 --- a/src/kernel/decider.ts +++ b/src/kernel/decider.ts @@ -1,4 +1,11 @@ // The decider contract of spec:kernel.decider-contract: the shapes a context's pure domain code fits, and its helpers. +/** + * @sdpAnchor impl:kernel.decider-contract + * @sdpLabel the decider contract's shapes and its fold, rebuild, invariant and transition helpers + * @sdpSatisfies spec:kernel.decider-contract + * @sdpComponent component:platform.kernel + * @sdpRole contract + */ import type { Rejection } from "./outcome.js"; // The adapter adds the envelope; a decider returns only these fields. export type DomainEvent = { diff --git a/src/kernel/index.ts b/src/kernel/index.ts index 6f2810c..6c692ea 100644 --- a/src/kernel/index.ts +++ b/src/kernel/index.ts @@ -1,4 +1,11 @@ // Layer 0, the pure domain kernel. Its one Convex import is the type Value, which the compiler erases. +/** + * @sdpAnchor component:platform.kernel + * @sdpLabel the pure domain kernel: the decider contract and the outcome types + * @sdpReferences spec:kernel.domain-kernel + * @sdpLayer domain + * @sdpContext platform + */ export type { AffectedRef, CommittedOutcome, diff --git a/src/kernel/outcome.ts b/src/kernel/outcome.ts index d9ebba4..dbc983d 100644 --- a/src/kernel/outcome.ts +++ b/src/kernel/outcome.ts @@ -1,4 +1,11 @@ // The outcome model of spec:kernel.outcome-model. Types only; the validators belong to the context boundary. +/** + * @sdpAnchor impl:kernel.outcome-model + * @sdpLabel the outcome model's types + * @sdpReferences spec:kernel.outcome-model + * @sdpComponent component:platform.kernel + * @sdpRole contract + */ import type { Value } from "convex/values"; export type StreamVersion = { tenantId: string; diff --git a/src/operations/index.ts b/src/operations/index.ts index 9444750..cd73eb9 100644 --- a/src/operations/index.ts +++ b/src/operations/index.ts @@ -1,4 +1,12 @@ // The operations library: the diagnostic record a command writes beside its outcome. +/** + * @sdpAnchor component:platform.operations + * @sdpLabel the operations library: the diagnostic record a command writes + * @sdpReferences spec:operations.baseline-operations + * @sdpUses component:platform.kernel, component:platform.context, component:platform.command + * @sdpLayer infrastructure + * @sdpContext platform + */ export { consoleSink, diagnosticGapLine, diff --git a/src/read-model/generations.ts b/src/read-model/generations.ts index bcf19bf..058ca2c 100644 --- a/src/read-model/generations.ts +++ b/src/read-model/generations.ts @@ -1,4 +1,11 @@ // The registry reads a command and a query make. None is registered here. +/** + * @sdpAnchor impl:application.generation-registry + * @sdpLabel generationsToWrite and activeGeneration + * @sdpReferences spec:application.generation-registry + * @sdpComponent component:platform.read-model + * @sdpRole query + */ import type { WritableGeneration } from "./projection.js"; import { registryReader, diff --git a/src/read-model/index.ts b/src/read-model/index.ts index 2f008d8..d6afbe9 100644 --- a/src/read-model/index.ts +++ b/src/read-model/index.ts @@ -1,4 +1,11 @@ // The read-model library: what the parent uses to declare, rebuild, write and read a read model. +/** + * @sdpAnchor component:platform.read-model + * @sdpLabel the read-model library: declare, rebuild, write and read a read model + * @sdpUses component:platform.kernel, component:platform.context, component:platform.command, component:platform.gate + * @sdpLayer application + * @sdpContext platform + */ export { activeGeneration, generationsToWrite } from "./generations.js"; export { applyProjection, diff --git a/src/read-model/projection.ts b/src/read-model/projection.ts index f57cee1..caf8c04 100644 --- a/src/read-model/projection.ts +++ b/src/read-model/projection.ts @@ -1,5 +1,12 @@ // The projection contract of spec:application.projection-contract for a per-entity read model: its // declaration, its projection, and applyProjection, the one writer of its rows. +/** + * @sdpAnchor impl:application.projection-contract + * @sdpLabel ReadModel, Projection and applyProjection + * @sdpReferences spec:application.projection-contract + * @sdpComponent component:platform.read-model + * @sdpRole projection + */ import { getConvexSize, type Value } from "convex/values"; import type { GenericDataModel } from "convex/server"; import type { StreamVersion } from "../kernel/index.js"; diff --git a/src/read-model/queries.ts b/src/read-model/queries.ts index d2db04f..51cb0a6 100644 --- a/src/read-model/queries.ts +++ b/src/read-model/queries.ts @@ -1,5 +1,12 @@ // What a parent query over a read model returns, and the bound on a list page of one // (spec:application.read-models). +/** + * @sdpAnchor impl:application.read-models.queries + * @sdpLabel readModelView, pageInGeneration and limitReadModelList + * @sdpReferences spec:application.read-models + * @sdpComponent component:platform.read-model + * @sdpRole query + */ import { limitListBytes, limitListPage, diff --git a/src/read-model/rebuild.ts b/src/read-model/rebuild.ts index 01bc07c..2b8624f 100644 --- a/src/read-model/rebuild.ts +++ b/src/read-model/rebuild.ts @@ -1,4 +1,11 @@ // The operator entries and bounded batch chains of the online read-model rebuild. +/** + * @sdpAnchor impl:application.rebuild + * @sdpLabel the online rebuild's operator entries, batch chains, tenant fill and batch bounds + * @sdpReferences spec:application.rebuild + * @sdpComponent component:platform.read-model + * @sdpRole service + */ import type { FunctionReference, GenericMutationCtx, @@ -642,7 +649,7 @@ export async function getGenerations( config: RebuildConfig, args: { readModel: string }, ) { - // rebuild.sdp.md:137: a read model the composition does not declare has no generations to list. + // spec:application.rebuild#design.fnGetGenerations: a read model the composition does not declare has no generations to list. if ( !config.targets.some(({ readModel }) => readModel.name === args.readModel) ) diff --git a/src/read-model/tables.ts b/src/read-model/tables.ts index 497a618..62a8444 100644 --- a/src/read-model/tables.ts +++ b/src/read-model/tables.ts @@ -2,6 +2,13 @@ // aggregate markers and the tenant fill's row, and the conventions every // per-entity read-model row follows (spec:application.projection-contract), as fragments the parent's // schema.ts spreads into defineSchema. +/** + * @sdpAnchor impl:application.generation-registry.tables + * @sdpLabel the generations, progress, marker and tenant fill tables and rowConventions + * @sdpReferences spec:application.projection-contract, spec:application.generation-registry, spec:application.aggregate-projection + * @sdpComponent component:platform.read-model + * @sdpRole schema + */ import { defineTable, type DataModelFromSchemaDefinition, diff --git a/src/read-model/write.ts b/src/read-model/write.ts index b8107e4..c4a143c 100644 --- a/src/read-model/write.ts +++ b/src/read-model/write.ts @@ -1,5 +1,12 @@ // The read-model half of step 9 of spec:command.command-pipeline: the read models a command declares, // written from the streams entries its executor returned. +/** + * @sdpAnchor impl:application.read-models + * @sdpLabel writeReadModels, step 9's read-model half + * @sdpReferences spec:application.read-models, spec:command.command-pipeline + * @sdpComponent component:platform.read-model + * @sdpRole service + */ import type { StreamDto } from "../context/adapter.js"; import type { StreamVersion } from "../kernel/index.js"; import { generationsToWrite } from "./generations.js"; diff --git a/tests/native/generation-registry.first-rebuild-installs-read-model.test.ts b/tests/native/generation-registry.first-rebuild-installs-read-model.test.ts index 72dfca2..ab42d18 100644 --- a/tests/native/generation-registry.first-rebuild-installs-read-model.test.ts +++ b/tests/native/generation-registry.first-rebuild-installs-read-model.test.ts @@ -86,7 +86,8 @@ export async function plainNativeMessage(call: Promise) { } async function rebuild(world: World) { const backend = required(world.backend, "the backend"); - // generation-registry.first-rebuild-installs-read-model.sdp.md:30; rebuild.sdp.md:123: the closed tenant gate parks the first batch. + // generation-registry.first-rebuild-installs-read-model.sdp.md:30; + // spec:application.rebuild#design.step3: the closed tenant gate parks the first batch. await backend.admin.run(getFunctionName(internal.gate.closeGate), { scopeKey: `tenant:${tenantId}`, reason: "observe the first generation", @@ -121,7 +122,8 @@ async function rebuild(world: World) { operator, }), ); - // generation-registry.first-rebuild-installs-read-model.sdp.md:30; rebuild.sdp.md:115: exact in-flight refusal. + // generation-registry.first-rebuild-installs-read-model.sdp.md:30; + // spec:application.rebuild#design.fnStartGeneration: exact in-flight refusal. expect(world.second).toBe( "Read model documentSummary already has generation 1 in building", ); diff --git a/tests/native/read-models.test.ts b/tests/native/read-models.test.ts index 8a80bc4..7315d76 100644 --- a/tests/native/read-models.test.ts +++ b/tests/native/read-models.test.ts @@ -77,7 +77,8 @@ test("native: a command whose read model has no generation fails as a technical events: [], summaries: [], }); - // generation-registry.sdp.md:37; rebuild.sdp.md:115,133: the first rebuild installs generation 1. + // generation-registry.sdp.md:37; spec:application.rebuild#design.fnStartGeneration and + // spec:application.rebuild#design.fnSwitchGeneration: the first rebuild installs generation 1. await install(backend); const registry = await backend.admin.readTable("generations"); expect(registry).toMatchObject([ @@ -89,7 +90,8 @@ test("native: a command whose read model has no generation fails as a technical startedBy: operator, }, ]); - // rebuild.sdp.md:115: an active generation does not refuse the next rebuild. + // spec:application.rebuild#design.fnStartGeneration: an active generation does not refuse the + // next rebuild. const nextId = await backend.admin.run( getFunctionName(internal.rebuild.startGeneration), { @@ -100,7 +102,7 @@ test("native: a command whose read model has no generation fails as a technical const next = (await backend.admin.readTable("generations")).find( (row) => row._id === nextId, ); - // rebuild.sdp.md:115: start takes the next generation number. + // spec:application.rebuild#design.fnStartGeneration: start takes the next generation number. expect(next).toMatchObject({ generation: 2, projectionVersion: 1, diff --git a/tests/native/rebuild-install.ts b/tests/native/rebuild-install.ts index e64309b..55274c6 100644 --- a/tests/native/rebuild-install.ts +++ b/tests/native/rebuild-install.ts @@ -20,7 +20,8 @@ export type Progress = Record & { }; export type GenerationEntry = { generation: Generation; progress: Progress }; -// rebuild.sdp.md:137: the operator query returns each generation with its progress row. +// spec:application.rebuild#design.fnGetGenerations: the operator query returns each generation with +// its progress row. export async function generations(backend: Backend, readModel: string) { return (await backend.admin.run("rebuild:getGenerations", { readModel, @@ -65,7 +66,9 @@ export async function waitForGeneration( } } -// generation-registry.sdp.md:37; rebuild.sdp.md:115,127,133: install by the first rebuild. +// generation-registry.sdp.md:37; spec:application.rebuild#design.fnStartGeneration, +// spec:application.rebuild#design.verifySteps and +// spec:application.rebuild#design.fnSwitchGeneration: install by the first rebuild. async function install( backend: Backend, readModel: string, @@ -81,7 +84,8 @@ async function install( if (typeof id !== "string") throw new Error("startGeneration did not return an ID"); const verified = await waitForGeneration(backend, readModel, id); - // rebuild.sdp.md:115: the installed generation is at the version the setup stated. + // spec:application.rebuild#design.fnStartGeneration: the installed generation is at the version + // the setup stated. expect(verified.generation.projectionVersion).toBe(projectionVersion); await backend.admin.run(refs.switch, { generationId: id, operator }); return id; diff --git a/tests/native/rebuild.online-rebuild-interrupt-resume.test.ts b/tests/native/rebuild.online-rebuild-interrupt-resume.test.ts index a441948..5f057bb 100644 --- a/tests/native/rebuild.online-rebuild-interrupt-resume.test.ts +++ b/tests/native/rebuild.online-rebuild-interrupt-resume.test.ts @@ -657,8 +657,9 @@ bindExample( { generationId: id2(w), operator: "interrupt operator" }, ); await stopLive(w); - // rebuild.online-rebuild-interrupt-resume.sdp.md:33; rebuild.sdp.md step1:125: the batch the interrupt - // replaced records the interruption, with the operator from the generation row, and schedules nothing. + // rebuild.online-rebuild-interrupt-resume.sdp.md:33; spec:application.rebuild#design.step1: + // the batch the interrupt replaced records the interruption, with the operator from the + // generation row, and schedules nothing. w.interrupted = await waitForGeneration( backendOf(w), readModel, @@ -713,7 +714,8 @@ bindExample( (row) => row.tenantId === tenants[0] && row.key === w.existingOrder, )?.status, ).toBe("cancelled"); - // rebuild.sdp.md:69: an existing subject without a building row remains absent until a batch writes it. + // spec:application.rebuild, its rule that live commands also write the generation being + // built: an existing subject without a building row remains absent until a batch writes it. expect( (await rows(w, 2)).find( (row) => row.tenantId === tenants[1] && row.key === w.missingOrder, @@ -746,7 +748,8 @@ bindExample( "changedBy", ]), ); - // rebuild.sdp.md:134: the interrupt replaces the fence, marks it as its own and records the operator. + // spec:application.rebuild#design.fnInterruptGeneration: the interrupt replaces the fence, + // marks it as its own and records the operator. expect(w.interrupted.generation).toMatchObject({ fence: required(w.oldFence, "the original fence") + 1, interruptedFence: required(w.oldFence, "the original fence") + 1, @@ -767,7 +770,8 @@ bindExample( "changedBy", ]), ); - // rebuild.sdp.md:131: resume replaces the fence and keeps the batch size. + // spec:application.rebuild#design.fnResumeChain: resume replaces the fence and keeps the + // batch size. expect(w.resumed).toMatchObject({ generation: { fence: w.interrupted.generation.fence + 1, @@ -775,7 +779,7 @@ bindExample( }, progress: { batchSize: 1 }, }); - // rebuild.sdp.md:131: resume clears the interrupt error. + // spec:application.rebuild#design.fnResumeChain: resume clears the interrupt error. expect(w.resumed.progress.lastError).toBeUndefined(); // rebuild.online-rebuild-interrupt-resume.sdp.md:36: each operator change records a new time. expect(Number(w.interrupted.generation.changedAt)).toBeGreaterThan( @@ -858,13 +862,16 @@ bindExample( "the live writes before rollback", ); const retiredRows = await rows(w, 1); - // rebuild.online-rebuild-interrupt-resume.sdp.md:37; rebuild.sdp.md:74: the retired generation was not written by the new order. + // rebuild.online-rebuild-interrupt-resume.sdp.md:37; spec:application.rebuild, its rule that + // a retired generation is not written: the retired generation was not written by the new + // order. expect( retiredRows.find( (row) => row.tenantId === tenants[0] && row.key === "after-switch", ), ).toBeUndefined(); - // rebuild.online-rebuild-interrupt-resume.sdp.md:37; rebuild.sdp.md:74: the retired row still needs its cancellation repaired. + // rebuild.online-rebuild-interrupt-resume.sdp.md:37; spec:application.rebuild, its rule that + // a retired generation is not written: the retired row still needs its cancellation repaired. expect( retiredRows.find( (row) => row.tenantId === tenants[0] && row.key === cancelled, @@ -901,7 +908,8 @@ bindExample( (record) => record.identifier === "rebuild:backfillBatch", ), ).toEqual([]); - // rebuild.sdp.md:134: rollback retains generation 1's batch size and cumulative counts. + // spec:application.rebuild#design.fnRollbackGeneration: rollback retains generation 1's batch + // size and cumulative counts. expect(verified.progress.batchSize).toBe(before.progress.batchSize); // rebuild.online-rebuild-interrupt-resume.sdp.md:37: every checkpoint in rollback is one committed verify batch. expect(verified.progress.batchesDone - before.progress.batchesDone).toBe( @@ -912,7 +920,9 @@ bindExample( record.error === null, ).length, ); - // rebuild.online-rebuild-interrupt-resume.sdp.md:37; rebuild.sdp.md:127: exactly one verify pass, at generation 1's retained size. + // rebuild.online-rebuild-interrupt-resume.sdp.md:37; + // spec:application.rebuild#design.verifySteps: exactly one verify pass, at generation 1's + // retained size. expect(verified.progress.batchesDone - before.progress.batchesDone).toBe( expectedBatches, ); diff --git a/tests/pure/byfile-coverage.test.ts b/tests/pure/byfile-coverage.test.ts new file mode 100644 index 0000000..8f05c2c --- /dev/null +++ b/tests/pure/byfile-coverage.test.ts @@ -0,0 +1,127 @@ +import { + createReader, + type GraphSchema, +} from "@libar-dev/software-delivery-protocol"; +import { readFile, readdir } from "node:fs/promises"; +import { join, relative } from "node:path"; +import { expect, test } from "vitest"; + +// The anchors as the source states them, read from the tree rather than from a list kept by hand. +// The comment form is a top-level /** … */ block with @sdpAnchor, @sdpSatisfies and @sdpReferences; +// the constant form, used under harness/, is a codeAnchor({ … }) call whose satisfies and references +// hold ref("spec:…") targets. The graph is the one `npm run sdp:build` derives before the tests run. +const root = join(import.meta.dirname, "../.."); +const sourceRoots = ["src", "example", "fixture", "harness"]; + +type ScannedAnchor = { id: string; file: string; targets: string[] }; + +async function sourceFiles(directory: string): Promise { + const entries = await readdir(join(root, directory), { withFileTypes: true }); + const files = await Promise.all( + entries.map(async (entry) => { + const path = join(directory, entry.name); + if (entry.isDirectory()) + return entry.name === "_generated" ? [] : sourceFiles(path); + return /\.tsx?$/.test(entry.name) ? [path] : []; + }), + ); + return files.flat(); +} + +const list = (value: string | undefined): string[] => + value === undefined ? [] : value.split(",").map((item) => item.trim()); + +function commentAnchors(file: string, text: string): ScannedAnchor[] { + const anchors: ScannedAnchor[] = []; + for (const [block] of text.matchAll(/\/\*\*[\s\S]*?\*\//g)) { + const tag = (name: string) => + new RegExp(`^\\s*\\*\\s*@${name}\\s+(.+?)\\s*$`, "m").exec(block)?.[1]; + const id = tag("sdpAnchor"); + if (id === undefined) continue; + anchors.push({ + id, + file, + targets: [...list(tag("sdpSatisfies")), ...list(tag("sdpReferences"))], + }); + } + return anchors; +} + +function constantAnchors(file: string, text: string): ScannedAnchor[] { + const anchors: ScannedAnchor[] = []; + for (const [call] of text.matchAll(/codeAnchor\(\{[\s\S]*?\n\}\);/g)) { + const id = /codeAnchorId\("([^"]+)"\)/.exec(call)?.[1]; + if (id === undefined) continue; + const field = (name: string) => + new RegExp(`${name}:\\s*(\\[[^\\]]*\\]|ref\\([^)]*\\))`).exec(call)?.[1]; + const targets = [field("satisfies"), field("references")].flatMap( + (value) => + value === undefined + ? [] + : [...value.matchAll(/ref\("([^"]+)"\)/g)].flatMap((match) => + match[1] === undefined ? [] : [match[1]], + ), + ); + anchors.push({ id, file, targets }); + } + return anchors; +} + +async function scanAnchors(): Promise { + const files = (await Promise.all(sourceRoots.map(sourceFiles))).flat(); + const anchors = await Promise.all( + files.map(async (path) => { + const text = await readFile(join(root, path), "utf8"); + const file = relative(root, join(root, path)); + return [...commentAnchors(file, text), ...constantAnchors(file, text)]; + }), + ); + return anchors.flat(); +} + +async function loadGraph(): Promise { + return JSON.parse( + await readFile(join(root, "generated/graph.json"), "utf8"), + ) as GraphSchema; +} + +test("pure: the scan of the source finds the code units the graph holds there", async () => { + const graph = await loadGraph(); + const scanned = (await scanAnchors()).map(({ id, file }) => `${id} ${file}`); + const derived = graph.nodes + .filter( + (node) => + node.nodeType === "CodeNode" && + sourceRoots.some((top) => node.file.startsWith(`${top}/`)), + ) + .map((node) => `${node.id} ${node.file}`); + expect(scanned.length).toBeGreaterThan(0); + expect([...scanned].sort()).toEqual([...derived].sort()); +}); + +test("pure: byFile answers for every anchored source file with its units and their Specs", async () => { + const reader = createReader(await loadGraph()); + const missing = (await scanAnchors()).flatMap(({ id, file, targets }) => { + const answer = reader.byFile(file); + const ids = answer.nodes.map((node) => node.id); + return [ + ...(ids.includes(id) ? [] : [`${file}: ${id}`]), + ...targets + .filter((target) => !answer.specs.includes(target)) + .map((target) => `${file}: ${id} → ${target}`), + ]; + }); + expect(missing).toEqual([]); +}); + +test("pure: no implementation anchor sits in a test file", async () => { + const graph = await loadGraph(); + expect( + graph.nodes + .filter( + (node) => + node.nodeType === "CodeNode" && node.file.startsWith("tests/"), + ) + .map((node) => node.id), + ).toEqual([]); +}); diff --git a/tests/pure/command.test.ts b/tests/pure/command.test.ts index 17d7acb..8f9f2d6 100644 --- a/tests/pure/command.test.ts +++ b/tests/pure/command.test.ts @@ -1,8 +1,3 @@ -import { - codeAnchor, - codeAnchorId, - ref, -} from "@libar-dev/software-delivery-protocol"; import { validate } from "convex-helpers/validators"; import { ConvexError, getConvexSize, v, type Value } from "convex/values"; import { createHash } from "node:crypto"; @@ -20,53 +15,6 @@ import { reject, type Receipt, } from "../../src/command/index.js"; -// Binds the command library under src/command/ to its Specs. The library is bundled into the parent -// deployment, so it carries no Protocol import itself; these tests carry its anchors. -const anchorPipeline = codeAnchor({ - id: codeAnchorId("impl:command.command-pipeline"), - label: "runPipeline and the command response in src/command", - satisfies: ref("spec:command.command-pipeline"), -}); -const anchorBoundary = codeAnchor({ - id: codeAnchorId("impl:command.outcome-boundary"), - label: "reject, refuseTransient, normalizeThrown and classifyThrown", - satisfies: ref("spec:command.outcome-boundary"), -}); -const anchorReceipts = codeAnchor({ - id: codeAnchorId("impl:command.idempotency-and-receipts"), - label: "the receipt key, fingerprint, lookup, classification and insert", - satisfies: ref("spec:command.idempotency-and-receipts"), -}); -const anchorReceiptTable = codeAnchor({ - id: codeAnchorId("impl:command.receipt-table"), - label: "the receipts table in src/command", - satisfies: ref("spec:command.receipt-table"), -}); -const anchorAuthority = codeAnchor({ - id: codeAnchorId("impl:command.tenancy-and-authority"), - label: "establishActor, authorize and the grant helpers", - satisfies: ref("spec:command.tenancy-and-authority"), -}); -const anchorActorAndScope = codeAnchor({ - id: codeAnchorId("impl:command.actor-and-scope"), - label: "the actor, scope, namespace, authority and grants shapes", - satisfies: ref("spec:command.actor-and-scope"), -}); -const anchorDeclaration = codeAnchor({ - id: codeAnchorId("impl:command.command-declaration"), - label: - "CommandDeclaration, publicCommand, internalCommand and the depot commands", - satisfies: ref("spec:command.command-declaration"), -}); -void [ - anchorPipeline, - anchorBoundary, - anchorReceipts, - anchorReceiptTable, - anchorAuthority, - anchorActorAndScope, - anchorDeclaration, -]; test("pure: canonicalJson sorts object keys at every depth, keeps array order and omits undefined members", () => { expect(canonicalJson({ b: 1, a: { d: [3, 1], c: "x" }, e: undefined })).toBe( diff --git a/tests/pure/context-queries.test.ts b/tests/pure/context-queries.test.ts index 417e497..b0e7fce 100644 --- a/tests/pure/context-queries.test.ts +++ b/tests/pure/context-queries.test.ts @@ -1,8 +1,3 @@ -import { - codeAnchor, - codeAnchorId, - ref, -} from "@libar-dev/software-delivery-protocol"; import type { PaginationOptions } from "convex/server"; import type { Validator, Value } from "convex/values"; import { expect, test } from "vitest"; @@ -19,13 +14,6 @@ import { journal, stockStream, } from "../../fixture/convex/depot/streams.js"; -// Binds the context queries under src/context/ to their Spec; the library carries no Protocol import. -const anchor = codeAnchor({ - id: codeAnchorId("impl:context.queries"), - label: "defineGet, defineList and boundedPage in src/context", - satisfies: ref("spec:context.queries"), -}); -void anchor; // What Convex reads of a registered function and of a validator, which their types keep internal. type Exported = { exportArgs(): string; exportReturns(): string }; const exported = (fn: unknown) => fn as Exported; diff --git a/tests/pure/context.test.ts b/tests/pure/context.test.ts index 953a215..5969d50 100644 --- a/tests/pure/context.test.ts +++ b/tests/pure/context.test.ts @@ -1,8 +1,3 @@ -import { - codeAnchor, - codeAnchorId, - ref, -} from "@libar-dev/software-delivery-protocol"; import { v } from "convex/values"; import { expect, test } from "vitest"; import { @@ -20,47 +15,6 @@ import { type StockResult, type StockState, } from "../../fixture/domain/index.js"; -// Binds the context library under src/context/ to its Specs. The library is bundled into every -// context component, so it carries no Protocol import itself; these tests carry its anchors. -const anchorComponent = codeAnchor({ - id: codeAnchorId("impl:context.context-component"), - label: "defineOperation and the depot fixture context", - satisfies: ref("spec:context.context-component"), -}); -const anchorAdapter = codeAnchor({ - id: codeAnchorId("impl:context.persistence-adapter"), - label: "execute, planned and runOperation in src/context", - satisfies: ref("spec:context.persistence-adapter"), -}); -const anchorJournal = codeAnchor({ - id: codeAnchorId("impl:context.journal"), - label: "createJournal, load and append in src/context", - satisfies: ref("spec:context.journal"), -}); -const anchorTables = codeAnchor({ - id: codeAnchorId("impl:context.tables"), - label: "the context tables in src/context", - satisfies: ref("spec:context.tables"), -}); -const anchorEnvelope = codeAnchor({ - id: codeAnchorId("impl:context.event-envelope"), - label: "the envelope types and validators in src/context", - satisfies: ref("spec:context.event-envelope"), -}); -const anchorBatch = codeAnchor({ - id: codeAnchorId("impl:context.batch-shaped-api"), - label: - "the operation bounds in src/context and the depot's list-shaped operations", - satisfies: ref("spec:context.batch-shaped-api"), -}); -void [ - anchorComponent, - anchorAdapter, - anchorJournal, - anchorTables, - anchorEnvelope, - anchorBatch, -]; const stock: StreamRegistration< StockState, StockCommand, diff --git a/tests/pure/example-domain.test.ts b/tests/pure/example-domain.test.ts index c081bb0..e5e03a5 100644 --- a/tests/pure/example-domain.test.ts +++ b/tests/pure/example-domain.test.ts @@ -1,8 +1,3 @@ -import { - codeAnchor, - codeAnchorId, - ref, -} from "@libar-dev/software-delivery-protocol"; import { validate } from "convex-helpers/validators"; import { expect, test } from "vitest"; import { @@ -30,15 +25,6 @@ import { } from "../../example/convex/orders/streams.js"; import { orderSummary } from "../../example/convex/orderSummary.js"; import type { StreamMeta } from "../../src/context/index.js"; -// Binds the production composition under example/ to the Spec of its domain. The composition is -// bundled into a deployment, so it carries no Protocol import itself; this test carries its anchor. -const anchor = codeAnchor({ - id: codeAnchorId("impl:application.orders-inventory-example"), - label: - "the order and stock item deciders, the Orders and Inventory contexts, PlaceOrder, CancelOrder, ReceiveStock and the order summary", - satisfies: ref("spec:application.orders-inventory-example"), -}); -void anchor; const context: DecisionContext = { now: Date.UTC(2026, 9, 1, 12), actor: { kind: "human", id: "example-user" }, diff --git a/tests/pure/example-use-cases.test.ts b/tests/pure/example-use-cases.test.ts index ad88207..4735dbd 100644 --- a/tests/pure/example-use-cases.test.ts +++ b/tests/pure/example-use-cases.test.ts @@ -1,8 +1,3 @@ -import { - codeAnchor, - codeAnchorId, - ref, -} from "@libar-dev/software-delivery-protocol"; import { getFunctionAddress } from "convex/server"; import { expect, test } from "vitest"; import { @@ -13,14 +8,6 @@ import { receiveStockDeclaration } from "../../example/convex/receiving.js"; import type { MutationCtx } from "../../src/command/index.js"; import type { OperationRef, StreamDto } from "../../src/context/index.js"; import type { StreamVersion } from "../../src/kernel/index.js"; -// Binds PlaceOrder, CancelOrder and ReceiveStock, the production composition's use cases, to their Spec. -const anchor = codeAnchor({ - id: codeAnchorId("impl:application.parent-use-cases"), - label: - "PlaceOrder and CancelOrder over Orders and then Inventory, and ReceiveStock", - satisfies: ref("spec:application.parent-use-cases"), -}); -void anchor; type Call = { reference: string; args: Record }; const version = ( contextId: string, diff --git a/tests/pure/kernel.test.ts b/tests/pure/kernel.test.ts index 65b6740..2799d1e 100644 --- a/tests/pure/kernel.test.ts +++ b/tests/pure/kernel.test.ts @@ -1,8 +1,3 @@ -import { - codeAnchor, - codeAnchorId, - ref, -} from "@libar-dev/software-delivery-protocol"; import { expect, test, vi } from "vitest"; import { checkInvariants, @@ -13,14 +8,6 @@ import { type DomainEvent, type Transitions, } from "../../src/kernel/index.js"; -// Binds the kernel under src/kernel/ to its contract. The kernel imports no package at run time, -// so it cannot carry the anchor itself; its unit tests carry it for it. -const anchor = codeAnchor({ - id: codeAnchorId("impl:kernel.decider-contract"), - label: "the domain kernel in src/kernel", - satisfies: ref("spec:kernel.decider-contract"), -}); -void anchor; type Added = DomainEvent<"added", { amount: number }>; function counter(initial: number): Decider { return { diff --git a/tests/pure/order-allocation-history-row-size.test.ts b/tests/pure/order-allocation-history-row-size.test.ts new file mode 100644 index 0000000..3ad6fe3 --- /dev/null +++ b/tests/pure/order-allocation-history-row-size.test.ts @@ -0,0 +1,120 @@ +// The largest row of the order allocation history measures 55,803 bytes with getConvexSize, within +// the row budget the view declares, the ceiling of 65,536 +// (spec:application.history-projection#design.tableOrderAllocationHistory). The row is what +// applyProjection measures before a write, without the system fields _id and _creationTime, which +// the stored document adds: the row conventions every read-model row carries +// (spec:application.projection-contract#design.validatorRowConventions) beside the fields the fold +// writes, with the tenant and order IDs at their bound, the most stock items an order names, each +// ID at the example's bound, and every allocation released. +import { getConvexSize, type ObjectType } from "convex/values"; +import { expect, test } from "vitest"; +import { journal as inventoryJournal } from "../../example/convex/inventory/streams.js"; +import { maxOrderLines } from "../../example/convex/ordering.js"; +import { journal as ordersJournal } from "../../example/convex/orders/streams.js"; +import { maxStockItemIdBytes } from "../../example/convex/receiving.js"; +import { + orderDecider, + stockItemDecider, + type OrderEvent, + type StockItemEvent, +} from "../../example/domain/index.js"; +import { + projectOrderAllocationHistory, + type HistoryEvent, + type OrderAllocationHistoryFields, +} from "../../example/domain/orderAllocationHistory.js"; +import { limitIdLength, utf8Length } from "../../src/command/index.js"; +import type { StreamVersion } from "../../src/kernel/index.js"; +import { + limitRowBudgetBytes, + rowConventions, +} from "../../src/read-model/index.js"; +const tenantId = "t".padEnd(limitIdLength, "x"); +const orderId = "o".padEnd(limitIdLength, "x"); +// One distinct stock item per line of the largest order. +const stockItemIds = Array.from({ length: maxOrderLines }, (_, i) => + String(i).padStart(3, "0").padEnd(maxStockItemIdBytes, "x"), +); +const historyEvent = ( + streamId: string, + recordedAt: number, + { eventType, payload }: OrderEvent | StockItemEvent, +): HistoryEvent => ({ streamId, eventType, recordedAt, payload }); +// The order placed and cancelled, and every stock item allocated to it and released. +const fields: OrderAllocationHistoryFields = projectOrderAllocationHistory( + tenantId, + null, + [ + historyEvent(orderId, 1, { + eventType: "OrderPlaced", + eventSchemaVersion: 1, + payload: { + lines: stockItemIds.map((stockItemId) => ({ + stockItemId, + quantity: 1, + unitPrice: 0, + })), + total: 0, + }, + }), + ...stockItemIds.map((stockItemId) => + historyEvent(stockItemId, 2, { + eventType: "StockAllocated", + eventSchemaVersion: 1, + payload: { orderId, quantity: 1 }, + }), + ), + ...stockItemIds.map((stockItemId) => + historyEvent(stockItemId, 3, { + eventType: "AllocationReleased", + eventSchemaVersion: 1, + payload: { orderId, quantity: 1 }, + }), + ), + historyEvent(orderId, 4, { + eventType: "OrderCancelled", + eventSchemaVersion: 1, + payload: {}, + }), + ], +); +// The highest version folded of the order's stream and of each stock item's. +const sourceVersions: StreamVersion[] = [ + { + tenantId, + contextId: ordersJournal.contextId, + streamType: orderDecider.streamType, + streamId: orderId, + version: 2, + }, + ...stockItemIds.map((streamId) => ({ + tenantId, + contextId: inventoryJournal.contextId, + streamType: stockItemDecider.streamType, + streamId, + version: 3, + })), +]; +const row: ObjectType & OrderAllocationHistoryFields = { + ...fields, + tenantId, + generation: 1, + key: orderId, + projectionVersion: 1, + sourceVersions, +}; + +test("pure: the largest order allocation history row measures 55,803 bytes, within the view's row budget of 65,536", () => { + expect([tenantId, orderId].map(utf8Length)).toStrictEqual([256, 256]); + expect(new Set(stockItemIds.map(utf8Length))).toStrictEqual(new Set([64])); + expect(row.allocations).toHaveLength(100); + expect( + row.allocations.every(({ releasedAt }) => releasedAt !== undefined), + ).toBe(true); + expect([row.placedAt, row.cancelledAt]).toStrictEqual([1, 4]); + expect(row.sourceVersions).toHaveLength(101); + const size = getConvexSize(row); + expect(size).toBe(55803); + expect(limitRowBudgetBytes).toBe(65536); + expect(size).toBeLessThanOrEqual(limitRowBudgetBytes); +}); diff --git a/tests/pure/order-allocation-history.test.ts b/tests/pure/order-allocation-history.test.ts index 269db31..86405a6 100644 --- a/tests/pure/order-allocation-history.test.ts +++ b/tests/pure/order-allocation-history.test.ts @@ -1,6 +1,6 @@ // The order allocation history's fold gives one row whatever the order in which the events of // different streams arrive and however they are split between calls -// (spec:application.projection-contract, orderAllocationHistoryFold). +// (spec:application.history-projection#design.orderAllocationHistoryFold). import { expect, test } from "vitest"; import { orderAllocationHistoryKeyOf, diff --git a/tests/pure/pack-page.test.ts b/tests/pure/pack-page.test.ts new file mode 100644 index 0000000..95c4ee8 --- /dev/null +++ b/tests/pure/pack-page.test.ts @@ -0,0 +1,900 @@ +// scripts/pack-page.mjs, the generator of a Pack's page: its pure parts, and one page generated from +// a sample corpus in a temporary directory, whose graph the Protocol's extractor derives. The sample +// shows the members in the Pack's order, the clause that holds a member below its next rung with +// the targets that break it, each open question's address, and the register rows on the members. +import { spawnSync } from "node:child_process"; +import { + copyFile, + mkdir, + mkdtemp, + readFile, + rm, + symlink, + writeFile, +} from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { expect, test, vi } from "vitest"; + +type Failure = { + clauseId: string; + description: string; + targets?: { type: string; id: string; statedReadiness?: string }[]; +}; +type Context = { + specKind: string; + statedReadiness: string; + derivedReadiness?: string; + nextRungFailures: Failure[]; +}; +type Question = { blocking: boolean; question: string; key: string | null }; +type Source = Record & { kind: string; file: string }; +type Row = { + id: string; + title: string | null; + class: string | null; + status: string | null; + advisor: string | null; + blocks: string | null; + decidedBy: string | null; + decidedOn: string | null; + ruling: string | null; + sources: Source[]; +}; +type PackPage = { + packMembers(context: { + id: string; + members: { id: string; resolved: boolean }[]; + }): string[]; + nextRung( + context: Context, + ): { rung: string | null; below: boolean; failing: Failure[] } | null; + joinQuestions( + spec: string, + questions: Question[], + rows: (Row & { address: string })[], + ): { + questions: (Question & { address: string | null; rows: Row[] })[]; + left: Row[]; + }; + questionSources(rows: Row[]): (Row & { address: string })[]; + memberRows(rows: Row[], files: Map): Row[]; + formatDeclaration( + text: string, + ): Promise<{ text: string; formatted: boolean }>; + readPinnedPairs(base: string, pack: string): unknown; + readRegister(base: string): Row[]; + indexTables( + rows: { spec: string; key: string; declaration: string }[], + ): Map; + specLines(text: string): { design: Map; questions: number[] }; + opening(text: string, limit?: number): string; + tierOf(openingLine: string): string | null; + firstDifference( + committed: string, + generated: string, + ): { + line: number; + committed: string | null; + generated: string | null; + } | null; + generate(pack: string, out: string, base?: string): Promise; + main(args: string[], base?: string): Promise; +}; +// The generator is a script with no declaration file, so it is loaded by URL and typed here. +const page = (await import( + new URL("../../scripts/pack-page.mjs", import.meta.url).href +)) as PackPage; + +// Entry addresses are built, so that the check of entry addresses outside the Specs, which reads +// this file, finds no address of the sample corpus. +const at = (spec: string, section: string, key: string) => + `${spec}#${section}.${key}`; + +test("the members come in the order the reader's Pack context gives them, each once", () => { + const member = (id: string, resolved = true) => ({ id, resolved }); + const members = [ + member("spec:sample.second"), + member("spec:sample.first"), + member("spec:other.third"), + ]; + expect(page.packMembers({ id: "pack:sample", members })).toEqual([ + "spec:sample.second", + "spec:sample.first", + "spec:other.third", + ]); + expect(() => + page.packMembers({ + id: "pack:sample", + members: [...members, member("spec:sample.first")], + }), + ).toThrow("pack:sample lists spec:sample.first twice"); + expect(() => + page.packMembers({ + id: "pack:sample", + members: [member("spec:sample.absent", false)], + }), + ).toThrow("spec:sample.absent is not in the graph"); + expect(() => page.packMembers({ id: "pack:empty", members: [] })).toThrow( + "pack:empty lists no specs", + ); +}); + +const typed: Failure = { + clauseId: "typed-dependency-targets-are-defined", + description: "Every typed target states at least defined.", + targets: [ + { type: "dependsOn", id: "spec:sample.beta", statedReadiness: "scoped" }, + ], +}; +const anchors: Failure = { + clauseId: "anchors-resolve", + description: "Any anchors present resolve.", +}; +function context( + statedReadiness: string, + derivedReadiness: string | undefined, + nextRungFailures: Failure[] = [], +): Context { + return { + specKind: "contract", + statedReadiness, + ...(derivedReadiness === undefined ? {} : { derivedReadiness }), + nextRungFailures, + }; +} + +test("at the floor, the next rung is held by the Protocol's next-rung failures, the first one first", () => { + expect( + page.nextRung(context("defined", "defined", [typed, anchors])), + ).toEqual({ rung: "ready", below: false, failing: [typed, anchors] }); +}); + +test("a floor above the stated rung leaves nothing to hold the next rung", () => { + // The Protocol reports the rung above the floor, ready here; the page's next rung is defined. + expect(page.nextRung(context("scoped", "defined", [typed]))).toEqual({ + rung: "defined", + below: false, + failing: [], + }); +}); + +test("a floor below the stated rung is said, and its first unmet clause holds the next rung", () => { + const blocking: Failure = { + clauseId: "no-blocking-open-questions", + description: "Spec has no blocking open question in intent.openQuestions.", + }; + expect(page.nextRung(context("defined", "scoped", [blocking]))).toEqual({ + rung: "ready", + below: true, + failing: [blocking], + }); +}); + +test("ready has no next rung, and a kind the Protocol does not ratify is not evaluated", () => { + expect(page.nextRung(context("ready", "ready"))).toEqual({ + rung: null, + below: false, + failing: [], + }); + expect( + page.nextRung({ ...context("defined", undefined), specKind: "essay" }), + ).toBeNull(); +}); + +function row(id: string, changes: Partial = {}): Row { + return { + id, + title: `The title of ${id}.`, + class: "owner", + status: "sorted", + advisor: "domain", + blocks: null, + decidedBy: null, + decidedOn: null, + ruling: null, + sources: [], + ...changes, + }; +} +const question = (address: string, file = "design/specs/sample/one.sdp.md") => + ({ kind: "specQuestion", address, file }) as Source; + +test("a register row joins the open question its source cites by address", () => { + const spec = "spec:sample.rebuild"; + const questions: Question[] = [ + { blocking: false, question: "The fold.", key: "fold" }, + { blocking: true, question: "The verify pass.", key: "verifyPass" }, + { blocking: false, question: "A question with no key.", key: null }, + ]; + const sources = page.questionSources([ + row("OD-157", { sources: [question(at(spec, "question", "verifyPass"))] }), + row("OD-158", { sources: [question(at(spec, "question", "fold"))] }), + row("OD-159", { + sources: [question(at("spec:sample.rebuild.child", "question", "fold"))], + }), + row("OD-160", { sources: [question(at(spec, "question", "removed"))] }), + row("OD-161", { + sources: [ + { + kind: "specQuestion", + spec, + question: "A closed question.", + file: "x", + open: false, + }, + ], + }), + ]); + expect(sources.map((source) => source.id)).toEqual([ + "OD-157", + "OD-158", + "OD-159", + "OD-160", + ]); + const joined = page.joinQuestions(spec, questions, sources); + expect(joined.questions.map((entry) => entry.rows.map((r) => r.id))).toEqual([ + ["OD-158"], + ["OD-157"], + [], + ]); + expect(joined.questions.map((entry) => entry.address)).toEqual([ + at(spec, "question", "fold"), + at(spec, "question", "verifyPass"), + null, + ]); + expect(joined.left.map((r) => r.id)).toEqual(["OD-160"]); +}); + +test("the rows on the members are every decided row and every other row that cites no open question", () => { + const file = "design/specs/sample/one.sdp.md"; + const elsewhere = { kind: "ownerQueue", ref: "O1", file: "design/STATE.md" }; + const line = { kind: "specLine", ref: "sample.one#rule", file, line: 9 }; + const rows = [ + row("OD-001", { status: "decided", sources: [elsewhere, line] }), + row("OD-002", { status: "decided", sources: [question("q", file)] }), + row("OD-003", { status: "sorted", sources: [question("q", file)] }), + row("OD-004", { status: "sorted", sources: [line] }), + row("OD-005", { status: "decided", sources: [elsewhere] }), + ]; + const chosen = page.memberRows(rows, new Map([[file, {}]])); + expect(chosen.map((r) => r.id)).toEqual(["OD-001", "OD-002", "OD-004"]); + expect(chosen[0]?.sources).toEqual([line]); +}); + +test("a declaration Prettier parses is formatted, as written or in the form its key implies", async () => { + expect( + await page.formatDeclaration("type Pair = { left: string; right: number }"), + ).toEqual({ + text: "type Pair = { left: string; right: number }", + formatted: true, + }); + const signature = await page.formatDeclaration( + "applyHistoryProjection(ctx: MutationCtx, projection: HistoryProjection, input: { tenantId: string; events: EventEnvelope[] }): Promise", + ); + expect(signature.formatted).toBe(true); + expect(signature.text.split("\n")).toEqual([ + "applyHistoryProjection(", + " ctx: MutationCtx,", + " projection: HistoryProjection,", + " input: { tenantId: string; events: EventEnvelope[] },", + "): Promise", + ]); + expect( + await page.formatDeclaration('.index("by_key", ["tenantId", "key"])'), + ).toEqual({ text: '.index("by_key", ["tenantId", "key"])', formatted: true }); + expect( + await page.formatDeclaration('audit?: { kind: "security" | "business" }'), + ).toEqual({ + text: 'audit?: { kind: "security" | "business" }', + formatted: true, + }); +}); + +test("a declaration Prettier does not parse stays as the Spec writes it", async () => { + for (const text of [ + "limitListPage(budgetBytes) = max(1, min(100, floor(4 MiB / budgetBytes)))", + "@convex-dev/migrations", + "[extension] type VersionedResult = { data: T }", + ]) + expect(await page.formatDeclaration(text)).toEqual({ + text, + formatted: false, + }); +}); + +test("a missing input is an error that names its path", async () => { + const base = await mkdtemp(join(tmpdir(), "pack-page-")); + try { + expect(() => page.readPinnedPairs(base, "pack:history-view")).toThrow( + "tests/types/pinned-pairs.json is missing", + ); + expect(() => page.readRegister(base)).toThrow( + "design/decisions/register.json is missing", + ); + } finally { + await rm(base, { recursive: true, force: true }); + } +}); + +test("an input off its ruled shape is refused", async () => { + const base = await mkdtemp(join(tmpdir(), "pack-page-")); + try { + await mkdir(join(base, "tests/types"), { recursive: true }); + await mkdir(join(base, "design/decisions"), { recursive: true }); + const pair = { + address: at( + "spec:application.generation-registry", + "design", + "typeRegistryReader", + ), + state: "bound", + comparison: "type", + code: "src/sample/index.ts#One", + reason: null, + }; + const pairs = (pairs: object[]) => + writeFile( + join(base, "tests/types/pinned-pairs.json"), + JSON.stringify({ + pack: "pack:sample", + test: "tests/types/x.test-d.ts", + pairs, + }), + ); + await pairs([pair]); + expect(() => page.readPinnedPairs(base, "pack:other")).toThrow( + 'names "pack:sample", not pack:other', + ); + await pairs([{ ...pair, reason: "a bound pair has no reason" }]); + expect(() => page.readPinnedPairs(base, "pack:sample")).toThrow( + "pairs[0] has the reason", + ); + await pairs([pair, pair]); + expect(() => page.readPinnedPairs(base, "pack:sample")).toThrow( + `pairs[1] repeats ${pair.address}`, + ); + const register = (source: object) => + writeFile( + join(base, "design/decisions/register.json"), + JSON.stringify({ decisions: [{ id: "OD-001", sources: [source] }] }), + ); + // An open question source that cites its question by opening text, the form the rulings retire. + await register({ + kind: "specQuestion", + spec: "spec:sample.one", + question: "The opening.", + file: "design/specs/sample/one.sdp.md", + }); + expect(() => page.readRegister(base)).toThrow( + "OD-001: an open specQuestion source needs address and file", + ); + await register({ + kind: "specQuestion", + spec: "spec:sample.one", + file: "design/specs/sample/one.sdp.md", + open: false, + }); + expect(() => page.readRegister(base)).toThrow( + "OD-001: a closed specQuestion source needs spec, question and file", + ); + await register({ kind: "specLine", ref: "sample.one#rule" }); + expect(() => page.readRegister(base)).toThrow( + "OD-001: a source needs a kind and a file", + ); + } finally { + await rm(base, { recursive: true, force: true }); + } +}); + +test("the line of each Design entry and open question is read from the carrier", () => { + const spec = [ + "---", + "id: spec:sample.one", + "---", + "# One", + "", + "## Intent", + "", + "- outcome: Pin one shape (E-1)", + "", + "### Open questions", + "", + "- [non-blocking #first] Extension E-1: the first question (E-1)", + "- [blocking] The second question (E-1)", + "", + "## Design", + "", + "A paragraph that names typeOne: in prose.", + "", + "- typeOne: `type One = { a: string }` (E-1)", + "- limitOne: 4 items (E-1)", + ].join("\n"); + const lines = page.specLines(spec); + expect([...lines.design]).toEqual([ + ["typeOne", 19], + ["limitOne", 20], + ]); + expect(lines.questions).toEqual([12, 13]); +}); + +test("a question opens to the end of its first sentence, cut outside code spans", () => { + expect(page.opening("The first sentence. The second.")).toBe( + "The first sentence.", + ); + expect(page.opening("It reads `a. b` and 0.1.124 here. Then more.")).toBe( + "It reads `a. b` and 0.1.124 here.", + ); + expect(page.opening("One two `three four five` six seven.", 15)).toBe( + "One two …", + ); +}); + +test("an example's tier is read from its opening line", () => { + expect(page.tierOf("Sc L2-6 · native backend tier · the resume case.")).toBe( + "native backend", + ); + expect(page.tierOf("Sc L1-4 · pure test tier.")).toBe("pure test"); + expect(page.tierOf("Story · Detail: full · Traces: D9.")).toBeNull(); +}); + +test("the check names the first line where the committed page differs", () => { + expect(page.firstDifference("a\nb\n", "a\nb\n")).toBeNull(); + expect(page.firstDifference("a\nb\nc\n", "a\nB\nc\n")).toEqual({ + line: 2, + committed: "b", + generated: "B", + }); + expect(page.firstDifference("a\n", "a\nb\n")).toEqual({ + line: 2, + committed: "", + generated: "b", + }); +}); + +test("an index belongs to the table its Spec wrote last before it, as the pinned module chains it", () => { + const rows = [ + { + spec: "spec:sample.one", + key: "tableItems", + declaration: "items: defineTable({ name: v.string() })", + }, + { + spec: "spec:sample.one", + key: "validatorItem", + declaration: "const itemValidator = v.object({})", + }, + { + spec: "spec:sample.one", + key: "indexItemsByName", + declaration: '.index("by_name", ["name"])', + }, + { + spec: "spec:sample.one", + key: "indexItemsByNameUse", + declaration: "by_name serves the list", + }, + { + spec: "spec:sample.one", + key: "tableMarks", + declaration: "marks: defineTable({ at: v.number() })", + }, + { + spec: "spec:sample.one", + key: "indexMarksByAt", + declaration: '.index("by_at", ["at"])', + }, + { + spec: "spec:sample.two", + key: "indexOrphan", + declaration: '.index("by_key", ["key"])', + }, + ]; + expect([...page.indexTables(rows)]).toEqual([ + [ + at("spec:sample.one", "design", "indexItemsByName"), + at("spec:sample.one", "design", "tableItems"), + ], + [ + at("spec:sample.one", "design", "indexMarksByAt"), + at("spec:sample.one", "design", "tableMarks"), + ], + ]); +}); + +// A decision Spec of the sample corpus: `relations` are its front-matter lines, `questions` the +// lines of its open questions. +function decision( + name: string, + readiness: string, + relations: string[], + questions: string[] = [], +) { + return [ + "---", + `id: spec:sample.${name}`, + "kind: decision", + "altitude: feature", + `readiness: ${readiness}`, + "relations:", + ...relations.map((relation) => ` ${relation}`), + "---", + `# The ${name} decision`, + "", + "## Intent", + "", + `- outcome: Settle ${name}`, + ...(questions.length === 0 + ? [] + : ["", "### Open questions", "", ...questions]), + "", + "## Decision", + "", + `- context: A sample needs ${name}`, + `- decision: Take ${name}`, + "", + ].join("\n"); +} + +test("the page of a sample Pack reads its order, its blockers and its register rows from the graph", async () => { + const base = await mkdtemp(join(tmpdir(), "pack-page-")); + const put = async (path: string, text: string) => { + await mkdir(dirname(join(base, path)), { recursive: true }); + await writeFile(join(base, path), text); + }; + const file = (name: string) => `design/specs/sample/${name}.sdp.md`; + try { + await put( + "package.json", + JSON.stringify({ + devDependencies: { + "@libar-dev/software-delivery-protocol": "file:sample.tgz", + }, + }), + ); + await put( + "tests/types/pinned-pairs.json", + JSON.stringify({ + pack: "pack:sample", + test: "tests/types/sample.test-d.ts", + pairs: [], + }), + ); + // The Pack lists its members out of id order. + await put( + "design/specs/sample.pack.sdp.md", + [ + "---", + "id: pack:sample", + "specs:", + " - spec:sample.gamma", + " - spec:sample.alpha", + " - spec:sample.beta", + "---", + "# Sample", + "", + "The Specs of a sample, in reading order.", + "", + ].join("\n"), + ); + // Alpha states defined and rests on two Specs that state scoped, one of them outside the Pack. + await put( + file("alpha"), + decision("alpha", "defined", [ + "dependsOn:", + " - spec:sample.delta", + " - spec:sample.beta", + ]), + ); + await put( + file("beta"), + decision( + "beta", + "scoped", + ["refines: spec:sample.gamma"], + ["- [blocking #betaChoice] Which option does beta take. More text."], + ), + ); + await put( + file("gamma"), + decision("gamma", "scoped", ["refines: spec:sample.alpha"]), + ); + await put( + file("delta"), + decision("delta", "scoped", ["refines: spec:sample.gamma"]), + ); + const betaChoice = at("spec:sample.beta", "question", "betaChoice"); + await put( + "design/decisions/register.json", + JSON.stringify({ + decisions: [ + row("OD-001", { + sources: [question(betaChoice, file("beta"))], + blocks: "S5", + }), + row("OD-002", { + status: "decided", + decidedBy: "owner", + decidedOn: "2026-10-04", + ruling: "Take the first option. Then the reasons.", + sources: [ + { + kind: "specLine", + ref: "sample.alpha#take", + file: file("alpha"), + line: 19, + }, + ], + }), + row("OD-003", { + status: "decided", + decidedBy: "owner", + decidedOn: "2026-10-04", + ruling: "A ruling outside the members.", + sources: [ + { + kind: "ownerQueue", + ref: "O1", + file: "design/STATE.md", + line: 1, + }, + ], + }), + row("OD-004", { + sources: [ + { + kind: "specLine", + ref: "sample.gamma#term", + file: file("gamma"), + line: 13, + }, + ], + }), + ], + }), + ); + const text = await page.generate( + "pack:sample", + join(base, "design/pages/sample.md"), + base, + ); + const lines = text.split("\n"); + + const memberIds = lines + .filter((line) => /^\| \[\d+\]\(#/.test(line)) + .map((line) => /`(spec:[^`]+)`/.exec(line)?.[1]); + expect(memberIds).toEqual([ + "spec:sample.gamma", + "spec:sample.alpha", + "spec:sample.beta", + ]); + + const alpha = lines.findIndex((line) => + line.startsWith("- [The alpha decision]"), + ); + expect(lines[alpha]).toContain( + "states defined and cannot reach ready. The first clause that fails is `typed-dependency-targets-are-defined`", + ); + expect(lines[alpha + 1]).toBe( + " It rests on [`spec:sample.beta`](../specs/sample/beta.sdp.md) by dependsOn, which states scoped and [`spec:sample.delta`](../specs/sample/delta.sdp.md) by dependsOn, which states scoped.", + ); + const beta = lines.findIndex((line) => + line.startsWith("- [The beta decision]"), + ); + expect(lines[beta]).toContain( + "The first clause that fails is `no-blocking-open-questions`", + ); + expect(lines[beta + 1]).toBe( + ` Its blocking question is \`${betaChoice}\`, under its open questions below.`, + ); + expect(text).toContain( + "The floor holds at the next rung for [The gamma decision](#1-the-gamma-decision).", + ); + expect(text).toContain( + "| [`spec:sample.delta`](../specs/sample/delta.sdp.md) | scoped | defined | [The alpha decision](#2-the-alpha-decision) | `spec:sample.alpha` dependsOn |", + ); + + expect(text).toContain( + `- **OD-001** class owner, status sorted, advisor domain, blocks the unit S5. [\`${betaChoice}\`](../specs/sample/beta.sdp.md?plain=1#L17), [blocking]: Which option does beta take.`, + ); + expect(text).toContain( + "- **OD-002** class owner, status decided, advisor domain, blocks no unit, from [`sample.alpha#take`](../specs/sample/alpha.sdp.md?plain=1#L19) on [The alpha decision](#2-the-alpha-decision). Decided by owner on 2026-10-04. Ruling: Take the first option.", + ); + expect(text).not.toContain("Then the reasons."); + expect(text).not.toContain("OD-003"); + expect(text).toContain("## Register rows whose sources lie in the members"); + expect(text).toContain( + "- **OD-004** class owner, status sorted, advisor domain, blocks no unit, from [`sample.gamma#term`](../specs/sample/gamma.sdp.md?plain=1#L13) on [The gamma decision](#1-the-gamma-decision). No ruling. Its title: The title of OD-004.", + ); + expect(text).toContain( + "[`generated/pinned/sample.ts`](../../generated/pinned/sample.ts)", + ); + } finally { + await rm(base, { recursive: true, force: true }); + } +}); + +// A sample tree of two Specs in one Pack, with every input the generator reads, in base. +async function smallSample(base: string) { + const put = async (path: string, text: string) => { + await mkdir(dirname(join(base, path)), { recursive: true }); + await writeFile(join(base, path), text); + }; + await put( + "package.json", + JSON.stringify({ + devDependencies: { + "@libar-dev/software-delivery-protocol": "file:sample.tgz", + }, + }), + ); + await put( + "tests/types/pinned-pairs.json", + JSON.stringify({ + pack: "pack:sample", + test: "tests/types/sample.test-d.ts", + pairs: [], + }), + ); + await put( + "design/decisions/register.json", + JSON.stringify({ decisions: [] }), + ); + await put( + "design/specs/sample.pack.sdp.md", + [ + "---", + "id: pack:sample", + "specs:", + " - spec:sample.alpha", + " - spec:sample.beta", + "---", + "# Sample", + "", + ].join("\n"), + ); + await put( + "design/specs/sample/alpha.sdp.md", + decision("alpha", "scoped", ["refines: spec:sample.beta"]), + ); + await put( + "design/specs/sample/beta.sdp.md", + decision("beta", "scoped", []).replace("relations:\n", "relations: {}\n"), + ); +} + +// The code of a member comes from the graph alone: a comment-form anchor in a source file binds at its +// own site, and the page names its role and its component's layer and context. +test("the page names each code unit at its own site, with its role and component, and lists the units that reference a member", async () => { + const base = await mkdtemp(join(tmpdir(), "pack-page-")); + const put = async (path: string, text: string) => { + await mkdir(dirname(join(base, path)), { recursive: true }); + await writeFile(join(base, path), text); + }; + try { + await smallSample(base); + await put( + "src/sample/index.ts", + [ + "/**", + " * @sdpAnchor component:sample.core", + " * @sdpLabel the sample's core", + " * @sdpLayer domain", + " * @sdpContext sample", + " */", + "export {};", + "", + ].join("\n"), + ); + await put( + "src/sample/rules.ts", + [ + "// The sample's rules.", + "/**", + " * @sdpAnchor impl:sample.rules", + " * @sdpLabel the sample's rules", + " * @sdpReferences spec:sample.alpha", + " * @sdpComponent component:sample.core", + " * @sdpRole decider", + " */", + "export const rules = 1;", + "", + ].join("\n"), + ); + const text = await page.generate( + "pack:sample", + join(base, "design/pages/sample.md"), + base, + ); + expect(text).not.toContain("anchor-sites"); + expect(text).toMatch( + /^\| \[1\]\(#1-the-alpha-decision\) .*\| referenced by 1 unit \|/m, + ); + expect(text).toContain( + "**References.** 1 unit references this design. A reference says the code answers to the design without claiming to realize it, and it confers no delivery fact and moves no floor.", + ); + expect(text).toContain( + "- `impl:sample.rules` at [`src/sample/rules.ts` line 2](../../src/sample/rules.ts#L2): the sample's rules. Its role is decider. It belongs to `component:sample.core`, in the domain layer and the sample context.", + ); + } finally { + await rm(base, { recursive: true, force: true }); + } +}); + +test("the command's --check exits 1 for a page that is missing or behind the tree, and 0 for a current one", async () => { + const base = await mkdtemp(join(tmpdir(), "pack-page-")); + const said: string[] = []; + const error = vi + .spyOn(console, "error") + .mockImplementation((message: string) => void said.push(message)); + try { + await smallSample(base); + const out = join(base, "design/pages/sample.md"); + const command = ["--pack", "pack:sample", "--out", out]; + const check = () => page.main([...command, "--check"], base); + + expect(await check()).toBe(1); + expect(said.at(-1)).toBe( + "pack-page: design/pages/sample.md is missing; run npm run pages", + ); + expect(await page.main(command, base)).toBe(0); + expect(await check()).toBe(0); + const written = await readFile(out, "utf8"); + await writeFile(out, `${written}A line the tree does not generate.\n`); + expect(await check()).toBe(1); + expect(said.at(-1)).toContain( + "pack-page: design/pages/sample.md is behind the tree from line", + ); + await writeFile(out, written); + expect(await check()).toBe(0); + } finally { + error.mockRestore(); + await rm(base, { recursive: true, force: true }); + } +}); + +// The script run as a process, from a copy in a sample tree that links this repository's +// node_modules, so the exit status a caller such as check.py reads is the one asserted. +test("the generator run as a process exits non-zero with --check on a stale page", async () => { + const base = await mkdtemp(join(tmpdir(), "pack-page-")); + try { + await smallSample(base); + await mkdir(join(base, "scripts"), { recursive: true }); + await copyFile( + fileURLToPath(new URL("../../scripts/pack-page.mjs", import.meta.url)), + join(base, "scripts/pack-page.mjs"), + ); + await symlink( + fileURLToPath(new URL("../../node_modules", import.meta.url)), + join(base, "node_modules"), + ); + const run = (...flags: string[]) => + spawnSync( + process.execPath, + [ + "scripts/pack-page.mjs", + "--pack", + "pack:sample", + "--out", + "design/pages/sample.md", + ...flags, + ], + { cwd: base, encoding: "utf8" }, + ); + expect(run().status).toBe(0); + expect(run("--check").status).toBe(0); + const out = join(base, "design/pages/sample.md"); + await writeFile( + out, + `${await readFile(out, "utf8")}A line the tree does not generate.\n`, + ); + const stale = run("--check"); + expect(stale.status).toBe(1); + expect(stale.stderr).toContain( + "pack-page: design/pages/sample.md is behind the tree from line", + ); + } finally { + await rm(base, { recursive: true, force: true }); + } +}); diff --git a/tests/pure/pinned-pairs.test.ts b/tests/pure/pinned-pairs.test.ts new file mode 100644 index 0000000..d157b33 --- /dev/null +++ b/tests/pure/pinned-pairs.test.ts @@ -0,0 +1,177 @@ +import { readFileSync } from "node:fs"; +import { dirname, resolve } from "node:path"; +import ts from "typescript"; +import { expect, test } from "vitest"; + +const root = resolve(import.meta.dirname, "../.."); +const testFile = "tests/types/pinned.test-d.ts"; +const source = ts.createSourceFile( + testFile, + readFileSync(process.env.PINNED_TEST_FILE ?? resolve(root, testFile), "utf8"), + ts.ScriptTarget.Latest, + true, +); +const titles = new Map([ + ["bound:", "bound"], + ["ahead:", "ahead"], + ["not compared:", "notCompared"], +]); + +type Pair = { + address: string; + state: string; + comparison: string; + code: string | null; + reason: string | null; +}; + +const imports = new Map(); +for (const statement of source.statements) { + if ( + !ts.isImportDeclaration(statement) || + !ts.isStringLiteral(statement.moduleSpecifier) + ) + continue; + const clause = statement.importClause; + const bindings = clause?.namedBindings; + const alias = + clause?.name?.text ?? + (bindings && ts.isNamespaceImport(bindings) + ? bindings.name.text + : undefined); + if (alias && alias !== "pinned") + imports.set( + alias, + resolve(root, dirname(testFile), statement.moduleSpecifier.text) + .slice(root.length + 1) + .replace(/\.js$/, ".ts"), + ); +} + +function readTests(): Pair[] { + const pairs: Pair[] = []; + const found = new Set(); + for (const statement of source.statements) { + if ( + !ts.isExpressionStatement(statement) || + !ts.isCallExpression(statement.expression) || + statement.expression.expression.getText(source) !== "describe" + ) + continue; + const [title, callback] = statement.expression.arguments; + if (!title || !ts.isStringLiteral(title)) continue; + const match = [...titles].find(([prefix]) => title.text.startsWith(prefix)); + if (!match) throw new Error(`Unknown pinned pair block: ${title.text}`); + const [prefix, state] = match; + if (found.has(prefix)) + throw new Error(`Duplicate pinned pair block: ${prefix}`); + found.add(prefix); + if ( + !callback || + !ts.isArrowFunction(callback) || + !ts.isBlock(callback.body) + ) + throw new Error(`No test block: ${title.text}`); + for (const entry of callback.body.statements) { + if ( + !ts.isExpressionStatement(entry) || + !ts.isCallExpression(entry.expression) + ) + continue; + const call = entry.expression; + if (!["test", "test.skip"].includes(call.expression.getText(source))) + continue; + const [name, body] = call.arguments; + if (!name || !ts.isStringLiteral(name)) + throw new Error(`No test name in ${title.text}`); + const address = name.text.split(": ")[0]!; + const reason = name.text.split(": ").slice(1).join(": "); + const text = body?.getText(source) ?? ""; + let code: string | null = null; + for (const [alias, file] of imports) { + const member = text.match( + new RegExp(`\\b${alias}\\.([A-Za-z_$][\\w$]*)`), + ); + if (member) { + code = `${file}#${member[1]}`; + break; + } + if (new RegExp(`\\b${alias}\\b`).test(text)) { + code = `${file}#default`; + break; + } + } + pairs.push({ + address, + state, + comparison: + state === "notCompared" + ? "none" + : text.includes("WithoutCtx<") + ? "typeWithoutCtx" + : text.includes("TableShape<") + ? "table" + : "type", + code, + reason: + state === "bound" + ? null + : reason.includes("outside the Pack") + ? `The pin is outside pack:history-view and is not emitted; ${reason}` + : reason, + }); + } + } + for (const prefix of titles.keys()) + if (!found.has(prefix)) + throw new Error(`Missing pinned pair block: ${prefix}`); + return pairs; +} + +test("pure: pinned pairs match the compiled tier's test names, states and order", () => { + // A scratch JSON path lets the pure test tier check drift without editing the repository. + const pairsFile = + process.env.PINNED_PAIRS_FILE ?? + resolve(root, "tests/types/pinned-pairs.json"); + const listing = JSON.parse(readFileSync(pairsFile, "utf8")) as { + pack: string; + test: string; + pairs: Pair[]; + }; + expect(listing.pack).toBe("pack:history-view"); + expect(listing.test).toBe(testFile); + const tests = readTests(); + const addresses = new Set(); + for (const pair of listing.pairs) { + expect(addresses.has(pair.address), `${pair.address}: duplicate pair`).toBe( + false, + ); + addresses.add(pair.address); + expect( + tests.some((entry) => entry.address === pair.address), + `${pair.address}: pair absent from test`, + ).toBe(true); + expect( + pair.reason === null || + (pair.reason.length > 0 && !/[\r\n]/.test(pair.reason)), + `${pair.address}: reason must be one line or null`, + ).toBe(true); + expect( + pair.state === "bound" ? pair.reason === null : pair.reason !== null, + `${pair.address}: reason follows state`, + ).toBe(true); + } + for (const [index, entry] of tests.entries()) { + const pair = listing.pairs.find((pair) => pair.address === entry.address); + expect(pair, `${entry.address}: test absent from JSON`).toBeDefined(); + expect(pair?.state, `${entry.address}: wrong state`).toBe(entry.state); + expect(listing.pairs[index]?.address, `${entry.address}: wrong order`).toBe( + entry.address, + ); + expect(pair?.comparison, `${entry.address}: wrong comparison`).toBe( + entry.comparison, + ); + expect(pair?.code, `${entry.address}: wrong code export`).toBe(entry.code); + expect(pair?.reason, `${entry.address}: wrong reason`).toBe(entry.reason); + } +}); diff --git a/tests/pure/pinned.test.ts b/tests/pure/pinned.test.ts new file mode 100644 index 0000000..3d7ce03 --- /dev/null +++ b/tests/pure/pinned.test.ts @@ -0,0 +1,255 @@ +// scripts/pinned.mjs, the emitter of a Pack's pinned module: its pure parts. A call signature with +// a default before a required parameter emits a required parameter whose type admits undefined, +// since TypeScript refuses a required parameter after an optional one (TS1016). A parameter or +// type parameter is in scope inside the node that declares it and nowhere else, a name resolves +// in its own space, so a value parameter hides no type, and a name a conditional type infers is +// in scope in its true branch only; so a name a declaration uses outside that scope is used, and +// the pin of another Spec is pulled in. Each emitted text is compiled with the TypeScript API. +import ts from "typescript"; +import { expect, test } from "vitest"; + +type Row = { spec: string; key: string; declaration: string }; +type Declaration = Row & { + prefix: string; + address: string; + kind: "type" | "value" | "table" | "index" | null; + name: string; + text: string; + why?: string; +}; +type Pinned = { + callSignature(span: string): { name: string; text: string } | null; + declarationOf(row: Row): Declaration; + namesUsedBy(declaration: Declaration): string[]; + pulledIn( + own: Declaration[], + pinnedBy: Map, + ): { emitted: Declaration[]; gapUses: Map }; +}; +// The emitter is a script with no declaration file, so it is loaded by URL and typed here. +const pinned = (await import( + new URL("../../scripts/pinned.mjs", import.meta.url).href +)) as Pinned; + +// The diagnostics of one module compiled alone under this repository's strictness, each as +// `TS: `. The library files are read once. +const options: ts.CompilerOptions = { + strict: true, + exactOptionalPropertyTypes: true, + noUncheckedIndexedAccess: true, + target: ts.ScriptTarget.ES2022, + module: ts.ModuleKind.ESNext, + moduleResolution: ts.ModuleResolutionKind.Bundler, + types: [], + skipLibCheck: true, + noEmit: true, +}; +const libraries = new Map(); +function diagnostics(text: string): string[] { + const fileName = "/pinned.ts"; + const host = ts.createCompilerHost(options, true); + const { fileExists, readFile, getSourceFile } = host; + host.fileExists = (path) => path === fileName || fileExists.call(host, path); + host.readFile = (path) => + path === fileName ? text : readFile.call(host, path); + host.getSourceFile = (path, language, onError) => { + if (path === fileName) + return ts.createSourceFile(path, text, language, true); + if (!libraries.has(path)) + libraries.set(path, getSourceFile.call(host, path, language, onError)); + return libraries.get(path); + }; + const program = ts.createProgram([fileName], options, host); + return ts + .getPreEmitDiagnostics(program) + .map( + (diagnostic) => + `TS${diagnostic.code}: ${ts.flattenDiagnosticMessageText(diagnostic.messageText, "\n")}`, + ); +} + +test("a default before a required parameter stays required and admits undefined; a trailing default is optional", () => { + const read = pinned.callSignature( + "read(limit: number = 100, tenantId: string, after: string | null = null): number", + ); + expect(read).toEqual({ + name: "read", + text: "export declare function read(limit: number | undefined /* = 100 */, tenantId: string, after?: string | null /* = null */): number", + }); + expect(diagnostics(read!.text)).toEqual([]); + // The emission before the fix: TypeScript refuses it. + expect( + diagnostics( + "export declare function read(limit?: number /* = 100 */, tenantId: string): number", + ), + ).toEqual([ + "TS1016: A required parameter cannot follow an optional parameter.", + ]); + // A function type is grouped before `| undefined`, which would otherwise bind inside it. + const run = pinned.callSignature( + "run(onDone: () => void = noop, name: string): void", + ); + expect(run?.text).toBe( + "export declare function run(onDone: (() => void) | undefined /* = noop */, name: string): void", + ); + expect(diagnostics(run!.text)).toEqual([]); +}); + +const row = ( + key: string, + declaration: string, + spec = "spec:sample.handler", +): Row => ({ spec, key, declaration }); +// Entry addresses are built, so that the check of entry addresses outside the Specs, which reads +// this file, finds no address of the sample corpus. +const at = (spec: string, key: string) => `${spec}#design.${key}`; + +test("a parameter or type parameter is in scope inside the node that declares it and nowhere else", () => { + const usedBy = (key: string, declaration: string) => + pinned.namesUsedBy(pinned.declarationOf(row(key, declaration))); + // The method's own hides nothing outside the method. + expect( + usedBy( + "typeHandler", + "type Handler = { run(outcome: Outcome): Outcome; fallback: Outcome }", + ), + ).toEqual(["Outcome"]); + // Every use is inside the node that declares the name. + expect( + usedBy( + "typeLocal", + "type Local = { run(outcome: Outcome, next: (outcome: Outcome) => Outcome): Outcome }", + ), + ).toEqual([]); + expect( + usedBy("fnHandle", "handle(outcome: Outcome): Promise"), + ).toEqual(["Promise"]); + expect(usedBy("typeKeys", "type Keys = { [K in keyof T]: K }")).toEqual( + [], + ); + expect( + usedBy( + "typeUnwrap", + "type Unwrap = T extends Promise ? U : Outcome", + ), + ).toEqual(["Promise", "Outcome"]); + // A name a conditional type infers is in scope in its true branch and nowhere else. + expect( + usedBy( + "typeUnwrapOutcome", + "type Unwrap = T extends Promise ? Outcome : Outcome", + ), + ).toEqual(["Promise", "Outcome"]); + // A value parameter hides no type of its name, and a type parameter hides no value. + expect(usedBy("fnRun", "run(Outcome: Outcome): Outcome")).toEqual([ + "Outcome", + ]); + expect( + usedBy( + "typeChecked", + "type Checked = { check(outcomeValidator: unknown): typeof outcomeValidator }", + ), + ).toEqual([]); + expect( + usedBy( + "typeOf", + "type Of = outcomeValidator | typeof outcomeValidator", + ), + ).toEqual(["outcomeValidator"]); + expect( + usedBy( + "validatorOutcome", + "const outcomeValidator = v.union(outcomeKindValidator, v.null())", + ), + ).toEqual(["v", "outcomeKindValidator"]); +}); + +test("an outer dependency pinned by another Spec is pulled in, and the module compiles", () => { + const handler = pinned.declarationOf( + row( + "typeHandler", + "type Handler = { run(outcome: Outcome): Outcome; fallback: Outcome }", + ), + ); + const local = pinned.declarationOf( + row( + "typeLocal", + "type Local = { run(outcome: Outcome): Outcome }", + ), + ); + const outcome = pinned.declarationOf( + row( + "typeOutcome", + 'type Outcome = { kind: "committed" | "rejected" }', + "spec:sample.outcome", + ), + ); + const pinnedBy = new Map( + [handler, local, outcome].map((pin) => [pin.name, [pin]]), + ); + const { emitted, gapUses } = pinned.pulledIn([handler], pinnedBy); + const handlerAt = at("spec:sample.handler", "typeHandler"); + expect(emitted.map(({ address, why }) => ({ address, why }))).toEqual([ + { address: handlerAt, why: undefined }, + { + address: at("spec:sample.outcome", "typeOutcome"), + why: `${handlerAt} uses Outcome`, + }, + ]); + expect(gapUses.size).toBe(0); + expect(diagnostics(emitted.map((pin) => pin.text).join("\n"))).toEqual([]); + // Without the pull-in the module names a type it never declares. + expect(diagnostics(handler.text)).toEqual([ + "TS2304: Cannot find name 'Outcome'.", + ]); + // A nested type parameter that covers every use pulls nothing in. + expect(pinned.pulledIn([local], pinnedBy).emitted).toEqual([local]); + expect(diagnostics(local.text)).toEqual([]); +}); + +test("an inferred name and a value parameter hide no outer type, and the module compiles", () => { + const outcome = () => + pinned.declarationOf( + row( + "typeOutcome", + 'type Outcome = { kind: "committed" | "rejected" }', + "spec:sample.outcome", + ), + ); + const valueAsType = + "TS2749: 'Outcome' refers to a value, but is being used as a type here. Did you mean 'typeof Outcome'?"; + const cases: [Row, string[]][] = [ + [ + row( + "typeUnwrap", + "type Unwrap = T extends Promise ? Outcome : Outcome", + ), + ["TS2304: Cannot find name 'Outcome'."], + ], + [ + row("fnRun", "run(Outcome: Outcome): Outcome"), + [valueAsType, valueAsType], + ], + ]; + for (const [sample, alone] of cases) { + const own = pinned.declarationOf(sample); + const pin = outcome(); + const { emitted } = pinned.pulledIn( + [own], + new Map([ + [own.name, [own]], + [pin.name, [pin]], + ]), + ); + expect(emitted.map(({ address, why }) => ({ address, why }))).toEqual([ + { address: own.address, why: undefined }, + { + address: at("spec:sample.outcome", "typeOutcome"), + why: `${own.address} uses Outcome`, + }, + ]); + expect(diagnostics(emitted.map((pin) => pin.text).join("\n"))).toEqual([]); + // Alone, the module names a type it never declares, or uses a value where a type is needed. + expect(diagnostics(own.text)).toEqual(alone); + } +}); diff --git a/tests/pure/read-model.test.ts b/tests/pure/read-model.test.ts index 0feb980..d065543 100644 --- a/tests/pure/read-model.test.ts +++ b/tests/pure/read-model.test.ts @@ -1,8 +1,3 @@ -import { - codeAnchor, - codeAnchorId, - ref, -} from "@libar-dev/software-delivery-protocol"; import { expect, test } from "vitest"; import { documentSummary } from "../../fixture/convex/summaries.js"; import { limitListBytes } from "../../src/context/index.js"; @@ -14,24 +9,6 @@ import { limitRowBudgetBytes, readModelView, } from "../../src/read-model/index.js"; -// Binds the read-model library under src/read-model/ to its Specs. The library is bundled into the -// parent deployment, so it carries no Protocol import itself; these tests carry its anchors. -const anchorProjection = codeAnchor({ - id: codeAnchorId("impl:application.projection-contract"), - label: "ReadModel, Projection, rowConventions and applyProjection", - satisfies: ref("spec:application.projection-contract"), -}); -const anchorRegistry = codeAnchor({ - id: codeAnchorId("impl:application.generation-registry"), - label: "the generations table, generationsToWrite and activeGeneration", - satisfies: ref("spec:application.generation-registry"), -}); -const anchorReadModels = codeAnchor({ - id: codeAnchorId("impl:application.read-models"), - label: "writeReadModels, readModelView and limitReadModelList", - satisfies: ref("spec:application.read-models"), -}); -void [anchorProjection, anchorRegistry, anchorReadModels]; const version = { tenantId: "t-1", contextId: "depot", diff --git a/tests/pure/rebuild.test.ts b/tests/pure/rebuild.test.ts index b062e76..a3a493e 100644 --- a/tests/pure/rebuild.test.ts +++ b/tests/pure/rebuild.test.ts @@ -49,7 +49,8 @@ test("pure: the fixture's documentTitle versions differ only on the deleted docu ).toThrow(); }); -// rebuild.sdp.md fnBatchSizeFor:116, limitBatchSize:146, limitPurgeBatch:147. +// spec:application.rebuild#design.fnBatchSizeFor, spec:application.rebuild#design.limitBatchSize and +// spec:application.rebuild#design.limitPurgeBatch. test.each([ [16384, 100, 256], [65536, 32, 64], @@ -105,10 +106,10 @@ test.each(projectionCases)( } }, ); -// rebuild.sdp.md step6:126 and verifySteps:127 expose no pure cursor function. +// spec:application.rebuild#design.step6 and spec:application.rebuild#design.verifySteps expose no pure cursor function. // The direct batch tests in tests/simulator/rebuild.test.ts prove those cursor steps. -// rebuild.sdp.md limitTenantFillBatch:142, limitGenerationsListed:151. +// spec:application.rebuild#design.limitTenantFillBatch and spec:application.rebuild#design.limitGenerationsListed. test("pure: tenant fill and generation listing have their pinned bounds", async () => { const { limitTenantFillBatch, limitGenerationsListed } = await import("../../src/read-model/rebuild.js"); diff --git a/tests/simulator/example.test.ts b/tests/simulator/example.test.ts index 4e73857..82fb44a 100644 --- a/tests/simulator/example.test.ts +++ b/tests/simulator/example.test.ts @@ -283,7 +283,9 @@ test( entry: "PlaceOrder", message: "An order needs at least one line", }); - // rebuild.sdp.md:115,133: installing returns a generation ID; its number is on the row. + // spec:application.rebuild#design.fnStartGeneration and + // spec:application.rebuild#design.fnSwitchGeneration: installing returns a generation ID; its + // number is on the row. await install(t); expect( await t.run((ctx) => ctx.db.query("generations").collect()), @@ -374,7 +376,8 @@ test( message: "A quantity must be a positive whole number, not 0", details: { quantity: 0 }, }); - // rebuild.sdp.md:115: an unknown read model is refused with the exact message. + // spec:application.rebuild#design.fnStartGeneration: an unknown read model is refused with the + // exact message. expect( String( await failure( diff --git a/tests/simulator/gate-support.ts b/tests/simulator/gate-support.ts index 8b3d2e7..586d85d 100644 --- a/tests/simulator/gate-support.ts +++ b/tests/simulator/gate-support.ts @@ -117,7 +117,9 @@ export const inspection = ( export const depotRows = (t: GateApp) => t.run((ctx) => ctx.runQuery(inspection, {})); -// generation-registry.sdp.md:37; rebuild.sdp.md:115,127,133: scheduled batches finish before switch. +// generation-registry.sdp.md:37; spec:application.rebuild#design.fnStartGeneration, +// spec:application.rebuild#design.verifySteps and +// spec:application.rebuild#design.fnSwitchGeneration: scheduled batches finish before switch. export async function installFixtureReadModel( t: GateApp, readModel = "documentSummary", @@ -138,7 +140,8 @@ export async function installFixtureReadModel( generationId, operator, }); - // rebuild.sdp.md:115: the active generation is at the version the setup stated. + // spec:application.rebuild#design.fnStartGeneration: the active generation is at the version + // the setup stated. expect(await t.run((ctx) => ctx.db.get(generationId))).toMatchObject({ state: "active", projectionVersion, @@ -166,7 +169,8 @@ export async function installOrderSummary( generationId, operator, }); - // rebuild.sdp.md:115: the active generation is at the version the setup stated. + // spec:application.rebuild#design.fnStartGeneration: the active generation is at the version + // the setup stated. expect(await t.run((ctx) => ctx.db.get(generationId))).toMatchObject({ state: "active", projectionVersion, diff --git a/tests/simulator/read-model.test.ts b/tests/simulator/read-model.test.ts index 6eaa18f..07f3b64 100644 --- a/tests/simulator/read-model.test.ts +++ b/tests/simulator/read-model.test.ts @@ -214,7 +214,8 @@ describe("the first rebuild", () => { async () => { const t = app(); const generationId = await install(t); - // generation-registry.sdp.md:37; rebuild.sdp.md:115,133: installation is the first rebuild. + // generation-registry.sdp.md:37; spec:application.rebuild#design.fnStartGeneration and + // spec:application.rebuild#design.fnSwitchGeneration: installation is the first rebuild. expect(await generations(t)).toMatchObject([ { _id: generationId, @@ -230,7 +231,8 @@ describe("the first rebuild", () => { readModel: "documentSummary", operator, }); - // rebuild.sdp.md:115: active does not refuse a start; building does, by this exact message. + // spec:application.rebuild#design.fnStartGeneration: active does not refuse a start; + // building does, by this exact message. await expect( t.mutation(internal.rebuild.startGeneration, { readModel: "documentSummary", @@ -239,7 +241,8 @@ describe("the first rebuild", () => { ).rejects.toThrow( /^Read model documentSummary already has generation 2 in building$/, ); - // rebuild.sdp.md:115: the refused start inserted no third generation. + // spec:application.rebuild#design.fnStartGeneration: the refused start inserted no third + // generation. expect(await generations(t)).toHaveLength(2); await t.finishAllScheduledFunctions(() => vi.runOnlyPendingTimers()); await t.mutation(internal.rebuild.switchGeneration, { @@ -259,7 +262,8 @@ describe("the first rebuild", () => { const t = app(); await registry(t, [[3, "aborted"]]); const generationId = await install(t); - // rebuild.sdp.md:115: the next generation is one above the last, including an aborted one. + // spec:application.rebuild#design.fnStartGeneration: the next generation is one above the + // last, including an aborted one. expect(await generations(t)).toContainEqual( expect.objectContaining({ _id: generationId, @@ -267,7 +271,8 @@ describe("the first rebuild", () => { state: "active", }), ); - // rebuild.sdp.md:115: an unknown read model is refused before registration. + // spec:application.rebuild#design.fnStartGeneration: an unknown read model is refused before + // registration. await expect( t.mutation(internal.rebuild.startGeneration, { readModel: "nope", diff --git a/tests/simulator/rebuild.test.ts b/tests/simulator/rebuild.test.ts index 1e28998..b16ec5a 100644 --- a/tests/simulator/rebuild.test.ts +++ b/tests/simulator/rebuild.test.ts @@ -86,7 +86,8 @@ async function lastSchedule(t: App, name: string, args: object) { }); } -// rebuild.sdp.md fnStartGeneration:115; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnStartGeneration; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test.each([false, true])( "convex-test: install on an empty deployment, tenant present=%s", async (present) => { @@ -175,7 +176,7 @@ test.each([false, true])( }, ); -// rebuild.sdp.md fnStartGeneration:115; projection-contract.sdp.md typeReadModel:67. +// spec:application.rebuild#design.fnStartGeneration; projection-contract.sdp.md typeReadModel:67. test("convex-test: the default projection is the first and generation numbers increase past aborted rows", async () => { const t = rebuildApp(); const first = await t.mutation(internal.rebuild.startGeneration, { @@ -192,7 +193,9 @@ test("convex-test: the default projection is the first and generation numbers in expect((await read(t, second)).generation.generation).toBe(2); }); -// rebuild.sdp.md fnStartGeneration:115, fnBatchSizeFor:116, errorUnknownGeneration:143. +// spec:application.rebuild#design.fnStartGeneration, +// spec:application.rebuild#design.fnBatchSizeFor, +// spec:application.rebuild#design.errorUnknownGeneration. test("convex-test: start refusals follow target, projection, batch size, in-flight, empty list and fill order", async () => { const t = rebuildApp(); await grantsWithoutTenants(t, 1); @@ -243,7 +246,8 @@ test("convex-test: start refusals follow target, projection, batch size, in-flig } }); -// rebuild.sdp.md fnStartGeneration:115, fnBatchSizeFor:116. +// spec:application.rebuild#design.fnStartGeneration, +// spec:application.rebuild#design.fnBatchSizeFor. test.each([0, -1, 1.5, NaN, Infinity])( "convex-test: start refuses batch size %s with its full message", async (batchSize) => { @@ -256,7 +260,8 @@ test.each([0, -1, 1.5, NaN, Infinity])( }, ); -// rebuild.sdp.md fnStartGeneration:115, limitBatchSize:146. +// spec:application.rebuild#design.fnStartGeneration, +// spec:application.rebuild#design.limitBatchSize. test.each([100, 101])( "convex-test: start stores the capped batch size for requested %i", async (batchSize) => { @@ -266,8 +271,13 @@ test.each([100, 101])( }, ); -// rebuild.sdp.md errorUnknownGeneration:143, fnInterruptGeneration:130, fnResumeGeneration:132, -// fnSwitchGeneration:133, fnRollbackGeneration:134, fnAbortGeneration:135, fnPurgeGeneration:136. +// spec:application.rebuild#design.errorUnknownGeneration, +// spec:application.rebuild#design.fnInterruptGeneration, +// spec:application.rebuild#design.fnResumeGeneration, +// spec:application.rebuild#design.fnSwitchGeneration, +// spec:application.rebuild#design.fnRollbackGeneration, +// spec:application.rebuild#design.fnAbortGeneration, +// spec:application.rebuild#design.fnPurgeGeneration. test("convex-test: every generation entry refuses an unknown ID and every batch leaves it alone", async () => { const t = rebuildApp(); const id = await start(t); @@ -309,7 +319,8 @@ test("convex-test: every generation entry refuses an unknown ID and every batch expect(await snapshot(t)).toStrictEqual(before); }); -// rebuild.sdp.md Design:107, the generation with no progress row; fnInterruptGeneration:134. +// spec:application.rebuild, its Design narrative on a generation with no progress row; +// spec:application.rebuild#design.fnInterruptGeneration. test("convex-test: every entry that reads the progress row and every batch over a generation with none throws, and the interrupt reads none", async () => { const t = rebuildApp(); const id = await start(t); @@ -361,7 +372,8 @@ test("convex-test: every entry that reads the progress row and every batch over }); }); -// actor-and-scope.sdp.md fnAssertOperator:91; rebuild.sdp.md fnStartGeneration:115 through fnFillTenants:139. +// actor-and-scope.sdp.md fnAssertOperator:91; spec:application.rebuild#design.fnStartGeneration +// through spec:application.rebuild#design.fnFillTenants. test.each(["", " \n\t ", " ".repeat(513), "x".repeat(513), "é".repeat(257)])( "convex-test: every operator entry refuses invalid operator %j before a lookup", async (stated) => { @@ -405,7 +417,7 @@ test.each(["", " \n\t ", " ".repeat(513), "x".repeat(513), "é".repeat(257)])( }, ); -// actor-and-scope.sdp.md fnAssertOperator:91; rebuild.sdp.md convexSurface:112. +// actor-and-scope.sdp.md fnAssertOperator:91; spec:application.rebuild#design.convexSurface. test("convex-test: every plain entry checks the operator before touching ctx", async () => { const t = rebuildApp(); const id = await start(t); @@ -455,7 +467,8 @@ test("convex-test: every plain entry checks the operator before touching ctx", a }); }); -// rebuild.sdp.md step6:126, verifySteps:127, fnEnumerateCall:120; actor-and-scope.sdp.md fnNextTenant:89. +// spec:application.rebuild#design.step6, spec:application.rebuild#design.verifySteps, +// spec:application.rebuild#design.fnEnumerateCall; actor-and-scope.sdp.md fnNextTenant:89. test("convex-test: page cursors continue, tenants cross in tenant-ID order and verify restarts at the first tenant", async () => { const t = rebuildApp(); await create(t, "z", "z", "tenant-z"); @@ -520,7 +533,8 @@ test("convex-test: page cursors continue, tenants cross in tenant-ID order and v await lastSchedule(t, "verifyBatch", { generationId: id, fence: 1 }); }); -// rebuild.sdp.md step2:122, step6:126. The empty tenant is an independent tenant-list obligation. +// spec:application.rebuild#design.step2, spec:application.rebuild#design.step6. The empty tenant is +// an independent tenant-list obligation. test("convex-test: an empty tenant consumes a batch and advances to the next tenant", async () => { const t = rebuildApp(); await tenant(t, "tenant-a"); @@ -545,7 +559,10 @@ test("convex-test: an empty tenant consumes a batch and advances to the next ten ]); }); -// rebuild.sdp.md rule:81, step6:126, verifySteps:127. The tenant behind the cursor is an independent obligation. +// spec:application.rebuild, its rule that a tenant entering the tenant list behind a backfill's +// cursor is not visited by that pass; spec:application.rebuild#design.step6, +// spec:application.rebuild#design.verifySteps. The tenant behind the cursor is an independent +// obligation. test("convex-test: a tenant inserted between crossed tenants is skipped by backfill and visited by verify", async () => { const t = rebuildApp(); await install(t); @@ -580,7 +597,9 @@ test("convex-test: a tenant inserted between crossed tenants is skipped by backf ); }); -// rebuild.sdp.md step1:125, verifySteps:131, purgeSteps:132, limitOutstandingBatches:152. +// spec:application.rebuild#design.step1, spec:application.rebuild#design.verifySteps, +// spec:application.rebuild#design.purgeSteps, +// spec:application.rebuild#design.limitOutstandingBatches. test.each(["backfill", "verify", "purge"] as const)( "convex-test: in %s the batch the interrupt replaced records it once, and after a resume writes nothing", async (pass) => { @@ -639,7 +658,10 @@ test.each(["backfill", "verify", "purge"] as const)( }, ); -// rebuild.sdp.md fnInterruptGeneration:134, fnResumeChain:135, fnResumeGeneration:136; rebuild.online-rebuild-interrupt-resume.sdp.md:38. +// spec:application.rebuild#design.fnInterruptGeneration, +// spec:application.rebuild#design.fnResumeChain, +// spec:application.rebuild#design.fnResumeGeneration; +// rebuild.online-rebuild-interrupt-resume.sdp.md:38. test("convex-test: interrupt after exactly three batches and resume at the stored size with a new fence", async () => { const t = rebuildApp(); for (let i = 0; i < 6; i++) await create(t, `d${i}`); @@ -698,7 +720,8 @@ test("convex-test: interrupt after exactly three batches and resume at the store expect(next.progress.cursor).not.toEqual(resumed.progress.cursor); }); -// rebuild.sdp.md fnInterruptGeneration:134, step1:125; rebuild.online-rebuild-interrupt-resume.sdp.md:38. +// spec:application.rebuild#design.fnInterruptGeneration, spec:application.rebuild#design.step1; +// rebuild.online-rebuild-interrupt-resume.sdp.md:38. test("convex-test: an interrupt while a batch is scheduled leaves the progress row untouched, and that batch records the interruption and schedules nothing", async () => { const t = rebuildApp(); for (let i = 0; i < 3; i++) await create(t, `d${i}`); @@ -740,7 +763,8 @@ test("convex-test: an interrupt while a batch is scheduled leaves the progress r expect(await scheduled(t)).toStrictEqual(queued); }); -// rebuild.sdp.md step1:125, fnInterruptGeneration:134, fnResumeChain:135; generation-registry.sdp.md tableGenerations. +// spec:application.rebuild#design.step1, spec:application.rebuild#design.fnInterruptGeneration, +// spec:application.rebuild#design.fnResumeChain; generation-registry.sdp.md tableGenerations. test("convex-test: a batch pending across an interrupt and a resume, or across a resume alone, writes nothing", async () => { const t = rebuildApp(); for (let i = 0; i < 3; i++) await create(t, `d${i}`); @@ -763,7 +787,8 @@ test("convex-test: a batch pending across an interrupt and a resume, or across a } }); -// rebuild.sdp.md step1:125, fnInterruptGeneration:134; generation-registry.sdp.md tableGenerations. +// spec:application.rebuild#design.step1, spec:application.rebuild#design.fnInterruptGeneration; +// generation-registry.sdp.md tableGenerations. test("convex-test: a batch pending across two interrupts writes the note of the second once", async () => { const t = rebuildApp(); for (let i = 0; i < 3; i++) await create(t, `d${i}`); @@ -796,7 +821,8 @@ test("convex-test: a batch pending across two interrupts writes the note of the expect(await snapshot(t)).toStrictEqual(recorded); }); -// rebuild.sdp.md fnInterruptGeneration:134, fnResumeChain:135. +// spec:application.rebuild#design.fnInterruptGeneration, +// spec:application.rebuild#design.fnResumeChain. test.each(states)( "convex-test: an idle pass in %s cannot be resumed, and the interrupt decides from the state alone", async (state) => { @@ -840,7 +866,8 @@ test.each(states)( }, ); -// rebuild.sdp.md step3:123, fnResumeChain:131; write-pause.sdp.md fnGateAllows, restoreDoor. +// spec:application.rebuild#design.step3, spec:application.rebuild#design.fnResumeChain; +// write-pause.sdp.md fnGateAllows, restoreDoor. test.each(["tenant", "restore"])( "convex-test: a parked batch under %s records its error without checkpointing and resumes after an interrupt", async (gate) => { @@ -928,8 +955,8 @@ test.each(projectionCases)( }, ); -// projection-contract.sdp.md step3:92; generation-registry.sdp.md typeWritableGeneration; -// rebuild.sdp.md rule:69, rule:70, step5:125. +// spec:application.projection-contract#design.step3, spec:application.generation-registry#design.writableGenerationRole, +// spec:application.rebuild#design.step5 and its rule that live commands also write the generation being built. test.each(["building", "verifying", "verified"] as const)( "convex-test: live commands create and update rows in %s with that state's role", async (state) => { @@ -1040,7 +1067,7 @@ test("convex-test: active version one and building version two project live writ ).toEqual(["deleted-before", "deleted-live", "live"]); }); -// rebuild.sdp.md verifySteps:127; projection-contract.sdp.md step1:90, step3:92. +// spec:application.rebuild#design.verifySteps; projection-contract.sdp.md step1:90, step3:92. test("convex-test: verify inserts a missing row, updates an older row and deletes a row projected to null", async () => { const t = rebuildApp(); for (const key of ["missing", "older", "deleted"]) @@ -1139,7 +1166,7 @@ test.each(["active", "building", "verifying"] as const)( }, ); -// rebuild.sdp.md fnSwitchGeneration:133; generation-registry.sdp.md transitionCost. +// spec:application.rebuild#design.fnSwitchGeneration; generation-registry.sdp.md transitionCost. test.each([undefined, 0, 2592000000])( "convex-test: switch patches both generations atomically with rollback period %s", async (rollbackPeriodMs) => { @@ -1179,7 +1206,7 @@ test.each([undefined, 0, 2592000000])( }, ); -// rebuild.sdp.md fnSwitchGeneration:133. +// spec:application.rebuild#design.fnSwitchGeneration. test.each(states.filter((s) => s !== "verified"))( "convex-test: switch refuses %s before an invalid rollback period", async (state) => { @@ -1198,7 +1225,7 @@ test.each(states.filter((s) => s !== "verified"))( }, ); -// rebuild.sdp.md fnSwitchGeneration:133. +// spec:application.rebuild#design.fnSwitchGeneration. test.each([-1, 0.5, 2592000001, NaN, Infinity])( "convex-test: switch refuses rollback period %s without changing either row", async (rollbackPeriodMs) => { @@ -1219,7 +1246,8 @@ test.each([-1, 0.5, 2592000001, NaN, Infinity])( }, ); -// rebuild.sdp.md fnSwitchGeneration:133, fnAbortGeneration:135. +// spec:application.rebuild#design.fnSwitchGeneration, +// spec:application.rebuild#design.fnAbortGeneration. test("convex-test: abort cannot remove the active generation while another awaits switch", async () => { const t = rebuildApp(); const active = await install(t); @@ -1247,7 +1275,8 @@ test("convex-test: abort cannot remove the active generation while another await expect((await read(t, id)).generation.state).toBe("active"); }); -// rebuild.sdp.md fnRollbackGeneration:134, verifySteps:127; rebuild.online-rebuild-interrupt-resume.sdp.md:38. +// spec:application.rebuild#design.fnRollbackGeneration, +// spec:application.rebuild#design.verifySteps; rebuild.online-rebuild-interrupt-resume.sdp.md:38. test("convex-test: rollback at version two repairs intervening creates, updates and a deletion before switch", async () => { const t = rebuildApp(); await install(t, "documentSummary"); @@ -1362,7 +1391,8 @@ test("convex-test: rollback at version two repairs intervening creates, updates expect((await read(t, id)).generation.state).toBe("retired"); }); -// rebuild.sdp.md fnRollbackGeneration:134, limitRollbackPeriod:150. +// spec:application.rebuild#design.fnRollbackGeneration, +// spec:application.rebuild#design.limitRollbackPeriod. test.each(states.filter((s) => s !== "retired"))( "convex-test: rollback refuses %s", async (state) => { @@ -1380,7 +1410,8 @@ test.each(states.filter((s) => s !== "retired"))( }, ); -// rebuild.sdp.md fnRollbackGeneration:134. The remaining five refusals are checked in their pinned order. +// spec:application.rebuild#design.fnRollbackGeneration. The remaining five refusals are checked in +// their pinned order. test("convex-test: rollback checks period, purge, pause, in-flight generation and projection in order", async () => { const t = rebuildApp(); const old = await install(t); @@ -1418,8 +1449,10 @@ test("convex-test: rollback checks period, purge, pause, in-flight generation an ); await patchGeneration(t, old, {}, { pass: "idle" }); const other = await start(t); - // rebuild.sdp.md:84, :134: a fill and a reopened generation exclude each other; the pause refusal - // precedes the fill refusal, which precedes the in-flight refusal. + // spec:application.rebuild, its rule that a tenant fill and a generation in flight exclude each + // other, and spec:application.rebuild#design.fnRollbackGeneration: a fill and a reopened + // generation exclude each other; the pause refusal precedes the fill refusal, which precedes the + // in-flight refusal. const fillRow = { fence: 1, cursor: null, @@ -1485,7 +1518,7 @@ test("convex-test: rollback checks period, purge, pause, in-flight generation an ); }); -// rebuild.sdp.md fnAbortGeneration:135. +// spec:application.rebuild#design.fnAbortGeneration. test.each(["building", "verifying", "verified"] as const)( "convex-test: abort from %s keeps its cursor and counts and ends the pass", async (state) => { @@ -1527,7 +1560,7 @@ test.each(["building", "verifying", "verified"] as const)( }, ); -// rebuild.sdp.md fnAbortGeneration:135. +// spec:application.rebuild#design.fnAbortGeneration. test.each(["", "x".repeat(257), "é".repeat(129)])( "convex-test: abort refuses reason %j before checking state", async (reason) => { @@ -1546,7 +1579,7 @@ test.each(["", "x".repeat(257), "é".repeat(129)])( }, ); -// rebuild.sdp.md fnAbortGeneration:135. +// spec:application.rebuild#design.fnAbortGeneration. test.each(["active", "retired", "aborted", "purged"] as const)( "convex-test: abort refuses %s", async (state) => { @@ -1569,7 +1602,8 @@ test.each(["active", "retired", "aborted", "purged"] as const)( }, ); -// rebuild.sdp.md fnRollbackGeneration:134, fnAbortGeneration:135. +// spec:application.rebuild#design.fnRollbackGeneration, +// spec:application.rebuild#design.fnAbortGeneration. test("convex-test: aborting a rollback leaves the serving generation active", async () => { const t = rebuildApp(); const old = await install(t); @@ -1603,7 +1637,7 @@ test("convex-test: aborting a rollback leaves the serving generation active", as }); }); -// rebuild.sdp.md fnPurgeGeneration:136. +// spec:application.rebuild#design.fnPurgeGeneration. test.each(["building", "verifying", "verified", "active", "purged"] as const)( "convex-test: purge refuses %s", async (state) => { @@ -1621,7 +1655,9 @@ test.each(["building", "verifying", "verified", "active", "purged"] as const)( }, ); -// rebuild.sdp.md fnPurgeGeneration:136, fnRollbackGeneration:134, limitRollbackPeriod:150. +// spec:application.rebuild#design.fnPurgeGeneration, +// spec:application.rebuild#design.fnRollbackGeneration, +// spec:application.rebuild#design.limitRollbackPeriod. test("convex-test: purge is refused before retireAfter and allowed at it while rollback is refused at it", async () => { const t = rebuildApp(); const old = await install(t); @@ -1699,7 +1735,8 @@ async function purgeRows( }); } -// rebuild.sdp.md purgeSteps:128, fnPurgeGeneration:136; generation-registry.sdp.md tableGenerationProgress. +// spec:application.rebuild#design.purgeSteps, spec:application.rebuild#design.fnPurgeGeneration; +// generation-registry.sdp.md tableGenerationProgress. test("convex-test: purge crosses tenants and checkpoints only progress until the pass ends", async () => { const t = rebuildApp(); const id = await start(t, { batchSize: 2 }); @@ -1768,7 +1805,8 @@ test("convex-test: purge crosses tenants and checkpoints only progress until the expect(await scheduled(t)).toStrictEqual(queue); }); -// rebuild.sdp.md purgeSteps:128, limitPurgeBatch:147. The purge near its bound is an independent obligation. +// spec:application.rebuild#design.purgeSteps, spec:application.rebuild#design.limitPurgeBatch. The +// purge near its bound is an independent obligation. test("convex-test: one more row than the purge cap takes exactly two batches on the tenant", async () => { const t = rebuildApp(); const id = await start(t, { batchSize: 2 }); @@ -1809,8 +1847,9 @@ test("convex-test: one more row than the purge cap takes exactly two batches on }); }); -// rebuild.sdp.md purgeSteps:128, limitPurgeBatch:147. A valid start caps batchSize below this limit; -// these stored progress rows exercise the defensive cap with a size above it. +// spec:application.rebuild#design.purgeSteps, spec:application.rebuild#design.limitPurgeBatch. A +// valid start caps batchSize below this limit; these stored progress rows exercise the defensive cap +// with a size above it. test.each([256, 257])( "convex-test: the read-model purge limit caps an oversized stored batch at 256 for %i rows", async (count) => { @@ -1857,7 +1896,9 @@ test.each([256, 257])( }, ); -// rebuild.sdp.md fnPurgeGeneration:136, fnInterruptGeneration:130, fnResumeGeneration:132, purgeSteps:128. +// spec:application.rebuild#design.fnPurgeGeneration, +// spec:application.rebuild#design.fnInterruptGeneration, +// spec:application.rebuild#design.fnResumeGeneration, spec:application.rebuild#design.purgeSteps. test.each(["retired", "aborted"] as const)( "convex-test: an interrupted purge of %s resumes at its stored batch size", async (state) => { @@ -1955,7 +1996,7 @@ test.each(["retired", "aborted"] as const)( }, ); -// rebuild.sdp.md fnStartGeneration:115, fnFillTenants:139. +// spec:application.rebuild#design.fnStartGeneration, spec:application.rebuild#design.fnFillTenants. test("convex-test: a grant beside an empty tenant list refuses start with the fillTenants message", async () => { const t = rebuildApp(); await grantsWithoutTenants(t, 1); @@ -1966,7 +2007,8 @@ test("convex-test: a grant beside an empty tenant list refuses start with the fi ); }); -// rebuild.sdp.md tableTenantFill:138, fnFillTenants:139, fillSteps:141, limitTenantFillBatch:142. +// spec:application.rebuild#design.tableTenantFill, spec:application.rebuild#design.fnFillTenants, +// spec:application.rebuild#design.fillSteps, spec:application.rebuild#design.limitTenantFillBatch. test.each([8, 9])( "convex-test: fill walks %i distinct tenants with a bound of eight and a second fill inserts none", async (count) => { @@ -2066,7 +2108,7 @@ test.each([8, 9])( }, ); -// rebuild.sdp.md fnFillTenants:139, fnStartGeneration:115. +// spec:application.rebuild#design.fnFillTenants, spec:application.rebuild#design.fnStartGeneration. test.each(["building", "verifying", "verified"] as const)( "convex-test: fillTenants refuses a generation in %s", async (state) => { @@ -2081,7 +2123,8 @@ test.each(["building", "verifying", "verified"] as const)( }, ); -// rebuild.sdp.md fnStartGeneration:115, fnFillTenants:139, fillSteps:141. +// spec:application.rebuild#design.fnStartGeneration, spec:application.rebuild#design.fnFillTenants, +// spec:application.rebuild#design.fillSteps. test("convex-test: start refuses an unfinished fill even after the tenant list has rows", async () => { const t = rebuildApp(); await grantsWithoutTenants(t, 9); @@ -2097,7 +2140,8 @@ test("convex-test: start refuses an unfinished fill even after the tenant list h expect((await read(t, id)).generation.state).toBe("building"); }); -// rebuild.sdp.md fillSteps:141, fnFillTenants:139; write-pause.sdp.md fnGateAllows. +// spec:application.rebuild#design.fillSteps, spec:application.rebuild#design.fnFillTenants; +// write-pause.sdp.md fnGateAllows. test("convex-test: fill ignores a tenant gate, parks under the deployment gate and keeps its cursor on resumption", async () => { const t = rebuildApp(); await grantsWithoutTenants(t, 9); @@ -2155,7 +2199,7 @@ test("convex-test: fill ignores a tenant gate, parks under the deployment gate a }); }); -// rebuild.sdp.md fnFillTenants:139, fillSteps:141. +// spec:application.rebuild#design.fnFillTenants, spec:application.rebuild#design.fillSteps. test("convex-test: fill refuses exactly the restore door and its batch parks there", async () => { const t = rebuildApp(); await grantsWithoutTenants(t, 1); @@ -2181,7 +2225,7 @@ test("convex-test: fill refuses exactly the restore door and its batch parks the expect(await scheduled(t)).toStrictEqual(queue); }); -// rebuild.sdp.md fillSteps:141. +// spec:application.rebuild#design.fillSteps. test("convex-test: fill batches with no row, a stale fence or an idle pass write nothing", async () => { const t = rebuildApp(); const empty = await snapshot(t); @@ -2197,7 +2241,8 @@ test("convex-test: fill batches with no row, a stale fence or an idle pass write expect(await snapshot(t)).toStrictEqual(idle); }); -// rebuild.sdp.md fnGetGenerations:137, limitGenerationsListed:151. +// spec:application.rebuild#design.fnGetGenerations, +// spec:application.rebuild#design.limitGenerationsListed. test("convex-test: getGenerations returns the newest twenty complete row pairs from twenty-one generations", async () => { const t = rebuildApp(); const ids: GenerationId[] = []; @@ -2229,7 +2274,8 @@ test("convex-test: getGenerations returns the newest twenty complete row pairs f expect(await snapshot(t)).toStrictEqual(before); }); -// actor-and-scope.sdp.md fnAssertOperator:91; rebuild.sdp.md fnStartGeneration:115 through fnFillTenants:139. +// actor-and-scope.sdp.md fnAssertOperator:91; spec:application.rebuild#design.fnStartGeneration +// through spec:application.rebuild#design.fnFillTenants. test("convex-test: every operator entry accepts and records the trimmed 512-byte bound", async () => { const t = rebuildApp(); const atBound = "é".repeat(256); @@ -2279,7 +2325,9 @@ test("convex-test: every operator entry accepts and records the trimmed 512-byte expect((await fill(t))?.changedBy).toBe(atBound); }); -// rebuild.sdp.md convexSurface:112, fnStartGeneration:115, fnBackfillBatch:117, fnVerifyBatch:118, fnSwitchGeneration:133. +// spec:application.rebuild#design.convexSurface, spec:application.rebuild#design.fnStartGeneration, +// spec:application.rebuild#design.fnBackfillBatch, spec:application.rebuild#design.fnVerifyBatch, +// spec:application.rebuild#design.fnSwitchGeneration. test("convex-test: the example composition installs orderSummary through its registered rebuild entries", async () => { const t = productionTest(); const id = await t.mutation(exampleInternal.rebuild.startGeneration, { @@ -2330,7 +2378,8 @@ test("convex-test: the example composition installs orderSummary through its reg }); }); -// rebuild.sdp.md step1:121, verifySteps:127, purgeSteps:128. +// spec:application.rebuild#design.step1, spec:application.rebuild#design.verifySteps, +// spec:application.rebuild#design.purgeSteps. test.each([ ["backfillBatch", "active", "backfill"], ["backfillBatch", "building", "verify"], @@ -2351,7 +2400,8 @@ test.each([ }, ); -// rebuild.sdp.md fnResumeChain:131, fnResumeGeneration:132, verifySteps:127. +// spec:application.rebuild#design.fnResumeChain, +// spec:application.rebuild#design.fnResumeGeneration, spec:application.rebuild#design.verifySteps. test("convex-test: resuming verify preserves its counts and cursor and schedules verify with the new fence", async () => { const t = rebuildApp(); for (let i = 0; i < 3; i++) await create(t, `document-${i}`); @@ -2413,7 +2463,8 @@ test("convex-test: resuming verify preserves its counts and cursor and schedules }); }); -// rebuild.sdp.md step3:123, verifySteps:127, purgeSteps:128; write-pause.sdp.md fnGateAllows:66. +// spec:application.rebuild#design.step3, spec:application.rebuild#design.verifySteps, +// spec:application.rebuild#design.purgeSteps; write-pause.sdp.md fnGateAllows:66. test.each(["verify", "purge"] as const)( "convex-test: %s also parks under a closed tenant without touching rows or checkpoint counts", async (pass) => { @@ -2455,7 +2506,8 @@ test.each(["verify", "purge"] as const)( }, ); -// rebuild.sdp.md fnStartGeneration:115, fnSwitchGeneration:133; generation-registry.sdp.md transitionCost. +// spec:application.rebuild#design.fnStartGeneration, +// spec:application.rebuild#design.fnSwitchGeneration; generation-registry.sdp.md transitionCost. test("convex-test: a failed activation commits neither half of the switch", async () => { const t = rebuildApp(true); const old = await install(t); @@ -2473,7 +2525,8 @@ test("convex-test: a failed activation commits neither half of the switch", asyn expect((await read(t, id)).generation.state).toBe("verified"); }); -// rebuild.sdp.md fnStartGeneration:115, Design:107; projection-contract.sdp.md readModelRowShape. +// spec:application.rebuild#design.fnStartGeneration, its Design narrative; +// projection-contract.sdp.md readModelRowShape. test("convex-test: startGeneration schedules exactly one batch and an install leaves one row per subject and no marker", async () => { const t = rebuildApp(); for (const documentId of ["a1", "a2", "a3"]) await create(t, documentId); @@ -2503,7 +2556,7 @@ test("convex-test: startGeneration schedules exactly one batch and an install le ).toStrictEqual([]); }); -// rebuild.sdp.md fnGetGenerations:137: only a declared read model's rows are listed. +// spec:application.rebuild#design.fnGetGenerations: only a declared read model's rows are listed. test("convex-test: getGenerations answers [] for an undeclared read model even when rows name it", async () => { const t = rebuildApp(); const id = await start(t); diff --git a/tests/types/pinned-pairs.json b/tests/types/pinned-pairs.json new file mode 100644 index 0000000..c22c28f --- /dev/null +++ b/tests/types/pinned-pairs.json @@ -0,0 +1,538 @@ +{ + "pack": "pack:history-view", + "test": "tests/types/pinned.test-d.ts", + "pairs": [ + { + "address": "spec:application.generation-registry#design.typeRegistryReader", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#RegistryReader", + "reason": null + }, + { + "address": "spec:application.generation-registry#design.fnGenerationsToWrite", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#generationsToWrite", + "reason": null + }, + { + "address": "spec:application.generation-registry#design.fnActiveGeneration", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#activeGeneration", + "reason": null + }, + { + "address": "spec:application.generation-registry#design.validatorGenerationState", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#generationStateValidator", + "reason": null + }, + { + "address": "spec:application.generation-registry#design.validatorProgressPass", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#progressPassValidator", + "reason": null + }, + { + "address": "spec:application.generation-registry#design.tableGenerations", + "state": "bound", + "comparison": "table", + "code": "src/read-model/index.ts#readModelTables", + "reason": null + }, + { + "address": "spec:application.history-projection#design.typeOrderAllocation", + "state": "bound", + "comparison": "type", + "code": "example/domain/orderAllocationHistory.ts#OrderAllocation", + "reason": null + }, + { + "address": "spec:application.history-projection#design.typeOrderAllocationHistoryFields", + "state": "bound", + "comparison": "type", + "code": "example/domain/orderAllocationHistory.ts#OrderAllocationHistoryFields", + "reason": null + }, + { + "address": "spec:application.projection-contract#design.typeProjection", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#Projection", + "reason": null + }, + { + "address": "spec:application.projection-contract#design.typeReadModel", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#ReadModel", + "reason": null + }, + { + "address": "spec:application.projection-contract#design.fnProjectionOf", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#projectionOf", + "reason": null + }, + { + "address": "spec:application.projection-contract#design.typeWritableGeneration", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#WritableGeneration", + "reason": null + }, + { + "address": "spec:application.projection-contract#design.typeRowWriter", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#RowWriter", + "reason": null + }, + { + "address": "spec:application.projection-contract#design.typeApplyMode", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#ApplyMode", + "reason": null + }, + { + "address": "spec:application.projection-contract#design.typeApplyResult", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#ApplyResult", + "reason": null + }, + { + "address": "spec:application.projection-contract#design.fnApplyProjection", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#applyProjection", + "reason": null + }, + { + "address": "spec:application.projection-contract#design.validatorRowConventions", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#rowConventions", + "reason": null + }, + { + "address": "spec:application.projection-contract#design.validatorOrderSummaryFields", + "state": "bound", + "comparison": "type", + "code": "example/convex/orderSummary.ts#orderSummaryFields", + "reason": null + }, + { + "address": "spec:application.projection-contract#design.tableOrderSummaries", + "state": "bound", + "comparison": "table", + "code": "example/convex/schema.ts#default", + "reason": null + }, + { + "address": "spec:application.read-models#design.fnPageInGeneration", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#pageInGeneration", + "reason": null + }, + { + "address": "spec:application.read-models#design.fnReadModelView", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#readModelView", + "reason": null + }, + { + "address": "spec:application.rebuild#design.typeRebuildTarget", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#RebuildTarget", + "reason": null + }, + { + "address": "spec:application.rebuild#design.tableTenantFill", + "state": "bound", + "comparison": "table", + "code": "src/read-model/index.ts#readModelTables", + "reason": null + }, + { + "address": "spec:application.write-pause#design.typeScopeKey", + "state": "bound", + "comparison": "type", + "code": "src/gate/index.ts#ScopeKey", + "reason": null + }, + { + "address": "spec:application.write-pause#design.fnGateAllows", + "state": "bound", + "comparison": "type", + "code": "src/gate/index.ts#gateAllows", + "reason": null + }, + { + "address": "spec:application.write-pause#design.fnAssertWritable", + "state": "bound", + "comparison": "type", + "code": "src/gate/index.ts#assertWritable", + "reason": null + }, + { + "address": "spec:application.write-pause#design.fnScopesOfUseCase", + "state": "bound", + "comparison": "type", + "code": "src/gate/index.ts#scopesOfUseCase", + "reason": null + }, + { + "address": "spec:application.write-pause#design.validatorClosedEntry", + "state": "bound", + "comparison": "type", + "code": "src/gate/index.ts#closedEntryValidator", + "reason": null + }, + { + "address": "spec:application.write-pause#design.tableMaintenanceGates", + "state": "bound", + "comparison": "table", + "code": "src/gate/index.ts#gateTables", + "reason": null + }, + { + "address": "spec:command.command-declaration#design.typeSourceRef", + "state": "bound", + "comparison": "type", + "code": "src/read-model/index.ts#SourceRef", + "reason": null + }, + { + "address": "spec:command.command-declaration#design.typePermissionPolicy", + "state": "bound", + "comparison": "type", + "code": "src/command/index.ts#PermissionPolicy", + "reason": null + }, + { + "address": "spec:command.command-declaration#design.typeExecutorResult", + "state": "bound", + "comparison": "type", + "code": "src/command/index.ts#ExecutorResult", + "reason": null + }, + { + "address": "spec:command.command-declaration#design.typeBounds", + "state": "bound", + "comparison": "type", + "code": "src/command/index.ts#Bounds", + "reason": null + }, + { + "address": "spec:command.command-pipeline#design.typePipelineCall", + "state": "bound", + "comparison": "type", + "code": "src/command/index.ts#PipelineCall", + "reason": null + }, + { + "address": "spec:command.command-pipeline#design.typeCommandResponse", + "state": "bound", + "comparison": "type", + "code": "src/command/index.ts#CommandResponse", + "reason": null + }, + { + "address": "spec:command.command-pipeline#design.validatorCommandResponse", + "state": "bound", + "comparison": "type", + "code": "src/command/index.ts#commandResponseValidator", + "reason": null + }, + { + "address": "spec:context.event-envelope#design.typeCausedBy", + "state": "bound", + "comparison": "type", + "code": "src/context/index.ts#CausedBy", + "reason": null + }, + { + "address": "spec:context.event-envelope#design.validatorCausedBy", + "state": "bound", + "comparison": "type", + "code": "src/context/index.ts#causedByValidator", + "reason": null + }, + { + "address": "spec:context.event-envelope#design.typeOperationRef", + "state": "bound", + "comparison": "type", + "code": "src/context/index.ts#OperationRef", + "reason": null + }, + { + "address": "spec:context.event-envelope#design.validatorOperationRef", + "state": "bound", + "comparison": "type", + "code": "src/context/index.ts#operationRefValidator", + "reason": null + }, + { + "address": "spec:context.event-envelope#design.typeEventEnvelope", + "state": "bound", + "comparison": "type", + "code": "src/context/index.ts#EventEnvelope", + "reason": null + }, + { + "address": "spec:context.event-envelope#design.validatorEventEnvelope", + "state": "bound", + "comparison": "type", + "code": "src/context/index.ts#eventEnvelopeValidator", + "reason": null + }, + { + "address": "spec:context.event-envelope#design.typeEnvelopeInput", + "state": "bound", + "comparison": "type", + "code": "src/context/index.ts#EnvelopeInput", + "reason": null + }, + { + "address": "spec:context.journal#design.fnCreateJournal", + "state": "bound", + "comparison": "type", + "code": "src/context/index.ts#createJournal", + "reason": null + }, + { + "address": "spec:context.journal#design.typeJournal", + "state": "bound", + "comparison": "type", + "code": "src/context/index.ts#Journal", + "reason": null + }, + { + "address": "spec:context.journal#design.typeStreamMeta", + "state": "bound", + "comparison": "type", + "code": "src/context/index.ts#StreamMeta", + "reason": null + }, + { + "address": "spec:context.queries#design.typeQueryArgs", + "state": "bound", + "comparison": "type", + "code": "src/context/index.ts#GetArgs", + "reason": null + }, + { + "address": "spec:context.queries#design.typePageLimit", + "state": "bound", + "comparison": "type", + "code": "src/context/index.ts#PageLimit", + "reason": null + }, + { + "address": "spec:context.queries#design.fnBoundedPage", + "state": "bound", + "comparison": "type", + "code": "src/context/index.ts#boundedPage", + "reason": null + }, + { + "address": "spec:application.generation-registry#design.validatorBatchCursor", + "state": "ahead", + "comparison": "type", + "code": "src/read-model/index.ts#batchCursorValidator", + "reason": "the history view's build adds sourceIndex" + }, + { + "address": "spec:application.generation-registry#design.tableGenerationProgress", + "state": "ahead", + "comparison": "table", + "code": "src/read-model/index.ts#readModelTables", + "reason": "its cursor, through validatorBatchCursor" + }, + { + "address": "spec:application.history-projection#design.typeHistoryEvent", + "state": "ahead", + "comparison": "type", + "code": "example/domain/orderAllocationHistory.ts#HistoryEvent", + "reason": "the design adds `eventSchemaVersion`; the code carries it in 5d" + }, + { + "address": "spec:application.history-projection#design.fnOrderAllocationHistoryKeyOf", + "state": "ahead", + "comparison": "type", + "code": "example/domain/orderAllocationHistory.ts#orderAllocationHistoryKeyOf", + "reason": "the design adds `eventSchemaVersion`; the code carries it in 5d" + }, + { + "address": "spec:application.history-projection#design.fnProjectOrderAllocationHistory", + "state": "ahead", + "comparison": "type", + "code": "example/domain/orderAllocationHistory.ts#projectOrderAllocationHistory", + "reason": "the design adds `eventSchemaVersion`; the code carries it in 5d" + }, + { + "address": "spec:application.rebuild#design.typeRebuildConfig", + "state": "ahead", + "comparison": "type", + "code": "src/read-model/index.ts#RebuildConfig", + "reason": "the history view's build adds history targets" + }, + { + "address": "spec:application.rebuild#design.fnBatchSizeFor", + "state": "ahead", + "comparison": "type", + "code": "src/read-model/index.ts#batchSizeFor", + "reason": "the history view's build takes a history projection" + }, + { + "address": "spec:application.rebuild#design.fnResumeChain", + "state": "ahead", + "comparison": "typeWithoutCtx", + "code": "src/read-model/index.ts#resumeChain", + "reason": "through typeRebuildConfig, its ctx erased" + }, + { + "address": "spec:command.command-declaration#design.typeReadModelBinding", + "state": "ahead", + "comparison": "type", + "code": "src/read-model/index.ts#ReadModelBinding", + "reason": "the history view's build binds history projections" + }, + { + "address": "spec:command.command-pipeline#design.fnWriteReadModels", + "state": "ahead", + "comparison": "type", + "code": "src/read-model/index.ts#writeReadModels", + "reason": "through typeReadModelBinding" + }, + { + "address": "spec:command.command-declaration#design.typeRetention", + "state": "ahead", + "comparison": "type", + "code": "src/command/index.ts#Retention", + "reason": "a receipt's tombstone expiry" + }, + { + "address": "spec:context.journal#design.typeStreamRegistration", + "state": "ahead", + "comparison": "type", + "code": "src/context/index.ts#StreamRegistration", + "reason": "baselines and migration, and the deferred streamParts for the derived mapping" + }, + { + "address": "spec:context.journal#design.typeLoadedStream", + "state": "ahead", + "comparison": "type", + "code": "src/context/index.ts#LoadedStream", + "reason": "the deferred streamParts" + }, + { + "address": "spec:context.journal#design.fnLoad", + "state": "ahead", + "comparison": "typeWithoutCtx", + "code": "src/context/index.ts#load", + "reason": "through typeStreamRegistration and typeLoadedStream, its ctx erased" + }, + { + "address": "spec:context.journal#design.fnAppend", + "state": "ahead", + "comparison": "typeWithoutCtx", + "code": "src/context/index.ts#append", + "reason": "through typeLoadedStream, its ctx erased" + }, + { + "address": "spec:context.queries#design.fnDefineGet", + "state": "ahead", + "comparison": "type", + "code": "src/context/index.ts#defineGet", + "reason": "through typeStreamRegistration" + }, + { + "address": "spec:context.queries#design.fnDefineList", + "state": "ahead", + "comparison": "type", + "code": "src/context/index.ts#defineList", + "reason": "through typeStreamRegistration" + }, + { + "address": "spec:command.command-declaration#design.typeExecutor", + "state": "notCompared", + "comparison": "none", + "code": null, + "reason": "differs only in ctx" + }, + { + "address": "spec:command.command-declaration#design.typeAdmissionPolicy", + "state": "notCompared", + "comparison": "none", + "code": null, + "reason": "differs only in ctx" + }, + { + "address": "spec:application.write-pause#design.fnCloseScope", + "state": "notCompared", + "comparison": "none", + "code": null, + "reason": "differs only in ctx" + }, + { + "address": "spec:application.write-pause#design.fnResumeScope", + "state": "notCompared", + "comparison": "none", + "code": null, + "reason": "differs only in ctx" + }, + { + "address": "spec:command.command-declaration#design.typeCommandDeclaration", + "state": "notCompared", + "comparison": "none", + "code": null, + "reason": "leads the code and differs in ctx, in its executor and admission" + }, + { + "address": "spec:command.command-declaration#design.fnPublicCommand", + "state": "notCompared", + "comparison": "none", + "code": null, + "reason": "leads the code and differs in ctx, through typeCommandDeclaration" + }, + { + "address": "spec:command.command-declaration#design.fnInternalCommand", + "state": "notCompared", + "comparison": "none", + "code": null, + "reason": "leads the code and differs in ctx, through typeCommandDeclaration" + }, + { + "address": "spec:command.command-pipeline#design.fnRunPipeline", + "state": "notCompared", + "comparison": "none", + "code": null, + "reason": "leads the code and differs in ctx, through typeCommandDeclaration" + }, + { + "address": "spec:command.command-pipeline#design.fnRelayFailure", + "state": "notCompared", + "comparison": "none", + "code": null, + "reason": "leads the code and differs in ctx, through typeCommandDeclaration" + }, + { + "address": "spec:application.aggregate-projection#design.tableProjectionMarkers", + "state": "notCompared", + "comparison": "none", + "code": null, + "reason": "The pin is outside pack:history-view and is not emitted; the aggregate form's table, outside the Pack, while readModelTables still holds it" + } + ] +} diff --git a/tests/types/pinned.test-d.ts b/tests/types/pinned.test-d.ts new file mode 100644 index 0000000..8b43000 --- /dev/null +++ b/tests/types/pinned.test-d.ts @@ -0,0 +1,383 @@ +// pack:history-view's pinned declarations against the code, through the module scripts/pinned.mjs +// emits from recipe 24. Each test is named by the Design entry it binds. Bound: the code equals the +// pin, so a changed pin fails the typecheck; a table's pair holds its indexes too. Ahead: the design +// leads the code, so the day the code catches up the assertion fails and the pair moves to bound; a +// function's ctx is erased first where the pin's and the code's differ. Not compared: the pair +// differs only in the ctx, or carries a ctx inside a type no erasing reaches, or its pin belongs to a +// Spec outside the Pack, so the module does not emit it; nothing is asserted. +import type { TableDefinition } from "convex/server"; +import { describe, expectTypeOf, test } from "vitest"; +import type * as pinned from "../../generated/pinned/history-view.js"; +import type * as orderSummary from "../../example/convex/orderSummary.js"; +import type schema from "../../example/convex/schema.js"; +import type * as allocationHistory from "../../example/domain/orderAllocationHistory.js"; +import type * as command from "../../src/command/index.js"; +import type * as context from "../../src/context/index.js"; +import type * as gate from "../../src/gate/index.js"; +import type * as readModel from "../../src/read-model/index.js"; + +// Representative arguments for the generic declarations. +type Dto = { orderId: string; lineCount: number }; +type Row = { orderId: string; total: number }; +type Input = { orderId: string }; +type Result = { allocated: boolean }; +type State = { status: "open" | "closed" }; +type Command = { kind: "close" }; +type Event = { eventType: "closed"; eventSchemaVersion: 1; payload: null }; +type Document = { _id: string; _creationTime: number; generation: number }; +// A table's document and its indexes, names and fields, which comparing the table type does not reach. +type TableShape
= + Table extends TableDefinition< + infer Document, + infer Indexes, + infer Search, + infer Vector + > + ? { document: Document; indexes: Indexes; search: Search; vector: Vector } + : never; +// A function without its first parameter, the ctx, for a pair whose ctx notation differs. +type WithoutCtx = F extends (ctx: never, ...rest: infer Rest) => infer Return + ? (...rest: Rest) => Return + : never; + +describe("bound: the code equals the pin", () => { + test("spec:application.generation-registry#design.typeRegistryReader", () => { + expectTypeOf().toEqualTypeOf(); + expectTypeOf< + readModel.RegistryReader + >().toEqualTypeOf>(); + }); + test("spec:application.generation-registry#design.fnGenerationsToWrite", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.generationsToWrite + >(); + }); + test("spec:application.generation-registry#design.fnActiveGeneration", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.activeGeneration + >(); + }); + test("spec:application.generation-registry#design.validatorGenerationState", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.generationStateValidator + >(); + }); + test("spec:application.generation-registry#design.validatorProgressPass", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.progressPassValidator + >(); + }); + test("spec:application.generation-registry#design.tableGenerations", () => { + expectTypeOf< + TableShape<(typeof readModel.readModelTables)["generations"]> + >().toEqualTypeOf>(); + }); + test("spec:application.history-projection#design.typeOrderAllocation", () => { + expectTypeOf().toEqualTypeOf(); + }); + test("spec:application.history-projection#design.typeOrderAllocationHistoryFields", () => { + expectTypeOf().toEqualTypeOf(); + }); + test("spec:application.projection-contract#design.typeProjection", () => { + expectTypeOf>().toEqualTypeOf< + pinned.Projection + >(); + }); + test("spec:application.projection-contract#design.typeReadModel", () => { + expectTypeOf>().toEqualTypeOf< + pinned.ReadModel + >(); + }); + test("spec:application.projection-contract#design.fnProjectionOf", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.projectionOf + >(); + }); + test("spec:application.projection-contract#design.typeWritableGeneration", () => { + expectTypeOf().toEqualTypeOf(); + }); + test("spec:application.projection-contract#design.typeRowWriter", () => { + expectTypeOf().toEqualTypeOf(); + expectTypeOf< + readModel.RowWriter + >().toEqualTypeOf>(); + }); + test("spec:application.projection-contract#design.typeApplyMode", () => { + expectTypeOf().toEqualTypeOf(); + }); + test("spec:application.projection-contract#design.typeApplyResult", () => { + expectTypeOf().toEqualTypeOf(); + }); + test("spec:application.projection-contract#design.fnApplyProjection", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.applyProjection + >(); + }); + test("spec:application.projection-contract#design.validatorRowConventions", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.rowConventions + >(); + }); + test("spec:application.projection-contract#design.validatorOrderSummaryFields", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.orderSummaryFields + >(); + }); + test("spec:application.projection-contract#design.tableOrderSummaries", () => { + expectTypeOf< + TableShape<(typeof schema)["tables"]["orderSummaries"]> + >().toEqualTypeOf>(); + }); + test("spec:application.read-models#design.fnPageInGeneration", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.pageInGeneration + >(); + }); + test("spec:application.read-models#design.fnReadModelView", () => { + expectTypeOf>().toEqualTypeOf< + typeof pinned.readModelView + >(); + }); + test("spec:application.rebuild#design.typeRebuildTarget", () => { + expectTypeOf>().toEqualTypeOf< + pinned.RebuildTarget + >(); + }); + test("spec:application.rebuild#design.tableTenantFill", () => { + expectTypeOf< + TableShape<(typeof readModel.readModelTables)["tenantFill"]> + >().toEqualTypeOf>(); + }); + test("spec:application.write-pause#design.typeScopeKey", () => { + expectTypeOf().toEqualTypeOf(); + }); + test("spec:application.write-pause#design.fnGateAllows", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.gateAllows + >(); + }); + test("spec:application.write-pause#design.fnAssertWritable", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.assertWritable + >(); + }); + test("spec:application.write-pause#design.fnScopesOfUseCase", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.scopesOfUseCase + >(); + }); + test("spec:application.write-pause#design.validatorClosedEntry", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.closedEntryValidator + >(); + }); + test("spec:application.write-pause#design.tableMaintenanceGates", () => { + expectTypeOf< + TableShape<(typeof gate.gateTables)["maintenanceGates"]> + >().toEqualTypeOf>(); + }); + test("spec:command.command-declaration#design.typeSourceRef", () => { + expectTypeOf().toEqualTypeOf(); + }); + test("spec:command.command-declaration#design.typePermissionPolicy", () => { + expectTypeOf>().toEqualTypeOf< + pinned.PermissionPolicy + >(); + }); + test("spec:command.command-declaration#design.typeExecutorResult", () => { + expectTypeOf>().toEqualTypeOf< + pinned.ExecutorResult + >(); + }); + test("spec:command.command-declaration#design.typeBounds", () => { + expectTypeOf().toEqualTypeOf(); + }); + test("spec:command.command-pipeline#design.typePipelineCall", () => { + expectTypeOf>().toEqualTypeOf< + pinned.PipelineCall + >(); + }); + test("spec:command.command-pipeline#design.typeCommandResponse", () => { + expectTypeOf>().toEqualTypeOf< + pinned.CommandResponse + >(); + }); + test("spec:command.command-pipeline#design.validatorCommandResponse", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.commandResponseValidator + >(); + }); + test("spec:context.event-envelope#design.typeCausedBy", () => { + expectTypeOf().toEqualTypeOf(); + }); + test("spec:context.event-envelope#design.validatorCausedBy", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.causedByValidator + >(); + }); + test("spec:context.event-envelope#design.typeOperationRef", () => { + expectTypeOf().toEqualTypeOf(); + }); + test("spec:context.event-envelope#design.validatorOperationRef", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.operationRefValidator + >(); + }); + test("spec:context.event-envelope#design.typeEventEnvelope", () => { + expectTypeOf().toEqualTypeOf(); + expectTypeOf>().toEqualTypeOf< + pinned.EventEnvelope + >(); + }); + test("spec:context.event-envelope#design.validatorEventEnvelope", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.eventEnvelopeValidator + >(); + }); + test("spec:context.event-envelope#design.typeEnvelopeInput", () => { + expectTypeOf().toEqualTypeOf(); + }); + test("spec:context.journal#design.fnCreateJournal", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.createJournal + >(); + }); + test("spec:context.journal#design.typeJournal", () => { + expectTypeOf().toEqualTypeOf(); + }); + test("spec:context.journal#design.typeStreamMeta", () => { + expectTypeOf().toEqualTypeOf(); + }); + test("spec:context.queries#design.typeQueryArgs", () => { + expectTypeOf().toEqualTypeOf(); + }); + test("spec:context.queries#design.typePageLimit", () => { + expectTypeOf().toEqualTypeOf(); + }); + test("spec:context.queries#design.fnBoundedPage", () => { + expectTypeOf().toEqualTypeOf< + typeof pinned.boundedPage + >(); + }); +}); + +describe("ahead: the design leads the code", () => { + test("spec:application.generation-registry#design.validatorBatchCursor: the history view's build adds sourceIndex", () => { + expectTypeOf().not.toEqualTypeOf< + typeof pinned.batchCursorValidator + >(); + }); + test("spec:application.generation-registry#design.tableGenerationProgress: its cursor, through validatorBatchCursor", () => { + expectTypeOf< + TableShape<(typeof readModel.readModelTables)["generationProgress"]> + >().not.toEqualTypeOf< + TableShape<(typeof pinned.tables)["generationProgress"]> + >(); + }); + test("spec:application.history-projection#design.typeHistoryEvent: the design adds `eventSchemaVersion`; the code carries it in 5d", () => { + expectTypeOf().not.toEqualTypeOf(); + // The pin says every envelope is a HistoryEvent, so the code's envelope carries each field the pin declares. + expectTypeOf().toExtend(); + }); + test("spec:application.history-projection#design.fnOrderAllocationHistoryKeyOf: the design adds `eventSchemaVersion`; the code carries it in 5d", () => { + expectTypeOf< + typeof allocationHistory.orderAllocationHistoryKeyOf + >().not.toEqualTypeOf(); + }); + test("spec:application.history-projection#design.fnProjectOrderAllocationHistory: the design adds `eventSchemaVersion`; the code carries it in 5d", () => { + expectTypeOf< + typeof allocationHistory.projectOrderAllocationHistory + >().not.toEqualTypeOf(); + }); + test("spec:application.rebuild#design.typeRebuildConfig: the history view's build adds history targets", () => { + expectTypeOf().not.toEqualTypeOf(); + }); + test("spec:application.rebuild#design.fnBatchSizeFor: the history view's build takes a history projection", () => { + expectTypeOf().not.toEqualTypeOf< + typeof pinned.batchSizeFor + >(); + }); + test("spec:application.rebuild#design.fnResumeChain: through typeRebuildConfig, its ctx erased", () => { + expectTypeOf>().not.toEqualTypeOf< + WithoutCtx + >(); + }); + test("spec:command.command-declaration#design.typeReadModelBinding: the history view's build binds history projections", () => { + expectTypeOf().not.toEqualTypeOf(); + }); + test("spec:command.command-pipeline#design.fnWriteReadModels: through typeReadModelBinding", () => { + expectTypeOf().not.toEqualTypeOf< + typeof pinned.writeReadModels + >(); + }); + test("spec:command.command-declaration#design.typeRetention: a receipt's tombstone expiry", () => { + expectTypeOf().not.toEqualTypeOf(); + }); + test("spec:context.journal#design.typeStreamRegistration: baselines and migration, and the deferred streamParts for the derived mapping", () => { + expectTypeOf< + context.StreamRegistration + >().not.toEqualTypeOf< + pinned.StreamRegistration + >(); + }); + test("spec:context.journal#design.typeLoadedStream: the deferred streamParts", () => { + expectTypeOf>().not.toEqualTypeOf< + pinned.LoadedStream + >(); + }); + test("spec:context.journal#design.fnLoad: through typeStreamRegistration and typeLoadedStream, its ctx erased", () => { + expectTypeOf< + WithoutCtx> + >().not.toEqualTypeOf< + WithoutCtx> + >(); + }); + test("spec:context.journal#design.fnAppend: through typeLoadedStream, its ctx erased", () => { + expectTypeOf< + WithoutCtx> + >().not.toEqualTypeOf>>(); + }); + test("spec:context.queries#design.fnDefineGet: through typeStreamRegistration", () => { + expectTypeOf().not.toEqualTypeOf< + typeof pinned.defineGet + >(); + }); + test("spec:context.queries#design.fnDefineList: through typeStreamRegistration", () => { + expectTypeOf().not.toEqualTypeOf< + typeof pinned.defineList + >(); + }); +}); + +describe("not compared: the pin takes Convex's generic MutationCtx and the code a composition's own ctx, or the pin is outside the Pack", () => { + test.skip( + "spec:command.command-declaration#design.typeExecutor: differs only in ctx", + ); + test.skip( + "spec:command.command-declaration#design.typeAdmissionPolicy: differs only in ctx", + ); + test.skip( + "spec:application.write-pause#design.fnCloseScope: differs only in ctx", + ); + test.skip( + "spec:application.write-pause#design.fnResumeScope: differs only in ctx", + ); + test.skip( + "spec:command.command-declaration#design.typeCommandDeclaration: leads the code and differs in ctx, in its executor and admission", + ); + test.skip( + "spec:command.command-declaration#design.fnPublicCommand: leads the code and differs in ctx, through typeCommandDeclaration", + ); + test.skip( + "spec:command.command-declaration#design.fnInternalCommand: leads the code and differs in ctx, through typeCommandDeclaration", + ); + test.skip( + "spec:command.command-pipeline#design.fnRunPipeline: leads the code and differs in ctx, through typeCommandDeclaration", + ); + test.skip( + "spec:command.command-pipeline#design.fnRelayFailure: leads the code and differs in ctx, through typeCommandDeclaration", + ); + test.skip( + "spec:application.aggregate-projection#design.tableProjectionMarkers: the aggregate form's table, outside the Pack, while readModelTables still holds it", + ); +}); diff --git a/tests/types/rebuild.test-d.ts b/tests/types/rebuild.test-d.ts index eea8313..89928c2 100644 --- a/tests/types/rebuild.test-d.ts +++ b/tests/types/rebuild.test-d.ts @@ -68,7 +68,8 @@ type Progress = { updatedAt: number; }; -// rebuild.sdp.md fnStartGeneration:115; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnStartGeneration; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: example startGeneration arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -114,7 +115,8 @@ test("compiled: example startGeneration arguments, visibility and return", () => void [generationId, wrong]; }); -// rebuild.sdp.md fnInterruptGeneration:130; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnInterruptGeneration; generation-registry.sdp.md +// tableGenerations, tableGenerationProgress. test("compiled: example interruptGeneration arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -144,7 +146,8 @@ test("compiled: example interruptGeneration arguments, visibility and return", ( void [generationId, wrong]; }); -// rebuild.sdp.md fnResumeGeneration:132; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnResumeGeneration; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: example resumeGeneration arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -178,7 +181,8 @@ test("compiled: example resumeGeneration arguments, visibility and return", () = void [generationId, wrong]; }); -// rebuild.sdp.md fnSwitchGeneration:133; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnSwitchGeneration; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: example switchGeneration arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -221,7 +225,8 @@ test("compiled: example switchGeneration arguments, visibility and return", () = void [generationId, wrong]; }); -// rebuild.sdp.md fnRollbackGeneration:134; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnRollbackGeneration; generation-registry.sdp.md +// tableGenerations, tableGenerationProgress. test("compiled: example rollbackGeneration arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -254,7 +259,8 @@ test("compiled: example rollbackGeneration arguments, visibility and return", () void [generationId, wrong]; }); -// rebuild.sdp.md fnAbortGeneration:135; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnAbortGeneration; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: example abortGeneration arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -292,7 +298,8 @@ test("compiled: example abortGeneration arguments, visibility and return", () => void [generationId, wrong]; }); -// rebuild.sdp.md fnPurgeGeneration:136; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnPurgeGeneration; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: example purgeGeneration arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -326,7 +333,8 @@ test("compiled: example purgeGeneration arguments, visibility and return", () => void [generationId, wrong]; }); -// rebuild.sdp.md fnBackfillBatch:117; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnBackfillBatch; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: example backfillBatch arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -358,7 +366,8 @@ test("compiled: example backfillBatch arguments, visibility and return", () => { void [generationId, wrong]; }); -// rebuild.sdp.md fnVerifyBatch:118; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnVerifyBatch; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: example verifyBatch arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -384,7 +393,8 @@ test("compiled: example verifyBatch arguments, visibility and return", () => { void [generationId, wrong]; }); -// rebuild.sdp.md fnPurgeBatch:119; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnPurgeBatch; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: example purgeBatch arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -414,7 +424,8 @@ test("compiled: example purgeBatch arguments, visibility and return", () => { void [generationId, wrong]; }); -// rebuild.sdp.md fnFillTenants:139; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnFillTenants; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: example fillTenants arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -435,7 +446,8 @@ test("compiled: example fillTenants arguments, visibility and return", () => { void [generationId, wrong]; }); -// rebuild.sdp.md fnFillTenantsBatch:140; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnFillTenantsBatch; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: example fillTenantsBatch arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -460,7 +472,8 @@ test("compiled: example fillTenantsBatch arguments, visibility and return", () = void [generationId, wrong]; }); -// rebuild.sdp.md fnGetGenerations:137; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnGetGenerations; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: example getGenerations arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -491,7 +504,8 @@ test("compiled: example getGenerations arguments, visibility and return", () => void [generationId, wrong]; }); -// rebuild.sdp.md fnStartGeneration:115; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnStartGeneration; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: fixture startGeneration arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -537,7 +551,8 @@ test("compiled: fixture startGeneration arguments, visibility and return", () => void [generationId, wrong]; }); -// rebuild.sdp.md fnInterruptGeneration:130; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnInterruptGeneration; generation-registry.sdp.md +// tableGenerations, tableGenerationProgress. test("compiled: fixture interruptGeneration arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -567,7 +582,8 @@ test("compiled: fixture interruptGeneration arguments, visibility and return", ( void [generationId, wrong]; }); -// rebuild.sdp.md fnResumeGeneration:132; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnResumeGeneration; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: fixture resumeGeneration arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -601,7 +617,8 @@ test("compiled: fixture resumeGeneration arguments, visibility and return", () = void [generationId, wrong]; }); -// rebuild.sdp.md fnSwitchGeneration:133; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnSwitchGeneration; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: fixture switchGeneration arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -644,7 +661,8 @@ test("compiled: fixture switchGeneration arguments, visibility and return", () = void [generationId, wrong]; }); -// rebuild.sdp.md fnRollbackGeneration:134; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnRollbackGeneration; generation-registry.sdp.md +// tableGenerations, tableGenerationProgress. test("compiled: fixture rollbackGeneration arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -677,7 +695,8 @@ test("compiled: fixture rollbackGeneration arguments, visibility and return", () void [generationId, wrong]; }); -// rebuild.sdp.md fnAbortGeneration:135; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnAbortGeneration; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: fixture abortGeneration arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -715,7 +734,8 @@ test("compiled: fixture abortGeneration arguments, visibility and return", () => void [generationId, wrong]; }); -// rebuild.sdp.md fnPurgeGeneration:136; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnPurgeGeneration; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: fixture purgeGeneration arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -749,7 +769,8 @@ test("compiled: fixture purgeGeneration arguments, visibility and return", () => void [generationId, wrong]; }); -// rebuild.sdp.md fnBackfillBatch:117; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnBackfillBatch; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: fixture backfillBatch arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -781,7 +802,8 @@ test("compiled: fixture backfillBatch arguments, visibility and return", () => { void [generationId, wrong]; }); -// rebuild.sdp.md fnVerifyBatch:118; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnVerifyBatch; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: fixture verifyBatch arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -807,7 +829,8 @@ test("compiled: fixture verifyBatch arguments, visibility and return", () => { void [generationId, wrong]; }); -// rebuild.sdp.md fnPurgeBatch:119; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnPurgeBatch; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: fixture purgeBatch arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -837,7 +860,8 @@ test("compiled: fixture purgeBatch arguments, visibility and return", () => { void [generationId, wrong]; }); -// rebuild.sdp.md fnFillTenants:139; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnFillTenants; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: fixture fillTenants arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -858,7 +882,8 @@ test("compiled: fixture fillTenants arguments, visibility and return", () => { void [generationId, wrong]; }); -// rebuild.sdp.md fnFillTenantsBatch:140; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnFillTenantsBatch; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: fixture fillTenantsBatch arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -883,7 +908,8 @@ test("compiled: fixture fillTenantsBatch arguments, visibility and return", () = void [generationId, wrong]; }); -// rebuild.sdp.md fnGetGenerations:137; generation-registry.sdp.md tableGenerations, tableGenerationProgress. +// spec:application.rebuild#design.fnGetGenerations; generation-registry.sdp.md tableGenerations, +// tableGenerationProgress. test("compiled: fixture getGenerations arguments, visibility and return", () => { expectTypeOf< FunctionArgs @@ -914,7 +940,10 @@ test("compiled: fixture getGenerations arguments, visibility and return", () => void [generationId, wrong]; }); -// rebuild.sdp.md typeRebuildTarget:113, typeRebuildConfig:114, fnBatchSizeFor:116, fnResumeChain:131, limitPurgeBatch:147. +// spec:application.rebuild#design.typeRebuildTarget, +// spec:application.rebuild#design.typeRebuildConfig, +// spec:application.rebuild#design.fnBatchSizeFor, spec:application.rebuild#design.fnResumeChain, +// spec:application.rebuild#design.limitPurgeBatch. test("compiled: rebuild configuration and helpers have the pinned signatures", () => { type Dto = { documentId: string }; type Row = { documentId: string }; diff --git a/vendor/libar-dev-software-delivery-protocol-0.0.0-5993da7.tgz b/vendor/libar-dev-software-delivery-protocol-0.0.0-5993da7.tgz deleted file mode 100644 index b2790b0..0000000 Binary files a/vendor/libar-dev-software-delivery-protocol-0.0.0-5993da7.tgz and /dev/null differ diff --git a/vendor/libar-dev-software-delivery-protocol-0.0.0-822aea4.tgz b/vendor/libar-dev-software-delivery-protocol-0.0.0-822aea4.tgz new file mode 100644 index 0000000..930e9fb Binary files /dev/null and b/vendor/libar-dev-software-delivery-protocol-0.0.0-822aea4.tgz differ