Skip to content

feat: prepare first-adopter configuration and package convergence - #1607

Closed
jeremi wants to merge 107 commits into
mainfrom
jeremi/first-adopter-readiness
Closed

jeremi wants to merge 107 commits into
mainfrom
jeremi/first-adopter-readiness

Conversation

@jeremi

@jeremi jeremi commented Sep 27, 2026

Copy link
Copy Markdown
Member

Summary

Draft checkpoint for first-adopter readiness across BReg, Evidence, and Casework, including governed bulk imports. This branch combines the initial operator and lifecycle fixes with shared runtime configuration (WP1) and shared package production/verification (WP2) across BReg, Evidence, Casework, Scheduling, Relay, Render, and Discovery.

The configuration and package changes provide a consistent deployment contract and bind runtime consumers to verified package bytes. Existing product authorization, disclosure, and database boundaries remain product-owned. This draft publishes the reviewed checkpoint for further review; it is not ready to merge or deploy as a completed first-adopter program.

Related: #1408, #1409, #1410, #1418. No issue closure is requested by this draft.

Evidence

Checkpoint: f8194ed70f564f7d30056614b428eb103a256739, rebased onto 0df549ef8c4693921088e69db959f9ff7191ebe5. Independent source and semantic rebase reviews reported no remaining findings.

Focused tests at that exact head passed 148 tests, with no failures or ignored tests:

Command Result and boundary
cargo test --locked -p registry-breg --features runtime --test startup_http 8 passed; HTTP startup behavior
cargo test --locked -p registry-breg --features postgres-test,tooling,schema --test postgres_startup -- --test-threads=1 9 passed against disposable PostGIS; restored-copy refusal/adoption and session/readiness behavior
cargo test --locked -p registry-evidencectl --test production_build 30 passed; package output/path rules and controlled child-process behavior
cargo test --locked -p registry-language-server --test evidence_cards_protocol 12 passed; protocol and bounded process behavior
cargo test --locked -p registry-caseworkctl --lib dev::tests:: -- --test-threads=1 89 passed; package/dev behavior, with 186 unrelated tests filtered

cargo fmt --all -- --check and base-to-head git diff --check passed. Before publication, gitleaks git --redact --config .gitleaks.toml --log-opts='origin/main..HEAD' --no-banner scanned all 107 branch commits and found no leaks.

These focused checks do not establish full-program readiness. The post-rebase Evidence build tests do not boot a service from the resulting package, and the Casework container-backed HTTP lifecycle was not rerun. Earlier WP2 product checks passed on their respective pre-rebase source heads; they are not represented as a fresh full matrix on this checkpoint.

Remaining before ready for review

Notes

This includes pre-1.0 breaking configuration, package, and CLI changes. Follow the changed product documentation rather than mixing old deployment formats with this branch. Database-ledger convergence and the associated signing/activation migration are not implemented yet.

The branch is published unchanged as a draft checkpoint. Keep audit changes minimal while concurrent audit simplification proceeds. Final history folding and upstream integration are intentionally deferred to closeout. Do not merge this draft.

DCO

  • Every commit includes a Signed-off-by trailer.
  • I reviewed the submitted changes and am responsible for the contribution.

A missing socket, a refused request, a key version Transit has not
created or has retired, and a public key that is not the governed one
all reported the same generic signing failure, so an operator could not
tell a proxy to restart from a keyVersion to change.

TransitSigner::initialize now returns a typed, value-free
TransitInitializationError, and Evidence reports the cause after
"runtime signing initialization failed:" in check and serve. The prefix
is unchanged, so evidencectl still classifies the refusal as a
dependency failure.

BREAKING: registry-platform-crypto TransitSigner::initialize returns
TransitInitializationError instead of SigningError.

Refs #1459

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…ts audit lock

`evidencectl doctor --runtime-config` could not pre-check a candidate
while the instance it replaces was serving: the dependency check opens
the audit sink, and the running writer holds its lock, so the check
always refused.

`evidence check --require-runtime-dependencies --without-audit-lock`
and `evidencectl doctor --without-audit-lock` now prove every runtime
dependency the plain form proves except the lock itself. The audit
sink is checked by a new platform `preflight_segmented_audit_sink`,
which validates the directory and any existing active file and lock
companion the way a writer would, and verifies the chain read-only
without holding the lock against the running writer, including the
complete records of the writer's active segment, so a candidate whose
hash secret does not verify the chain is refused even when it never
rotated. Records the writer appends after the check are left to it,
and the lock-free JSON report states that proof boundary separately.
The plain form still refuses a held lock, and its refusal now names
the flag. Startup and the lock-free check share one assembly sequence
and differ only in the audit step, so the two cannot drift apart.

The audit Storage cause now also names a file or directory the
process cannot write, which it already covered.

Refs #1491

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…h a named profile

An issuer that serves its key set under a private certificate authority
could only be reached by pointing the whole process at another trust
store, which also widened every source connection. The bundle's
authentication block may now name a tlsTrustProfile, bound in
runtime.yaml exactly like a source profile. The CA is trusted beside the
system roots for the jwksUri connection alone; hostname verification
stays on and a local HTTP issuer cannot name a profile.

The platform gains ValidatedFetchUrl::immediate_get_with_additional_roots
and JwksFetcher::new_trusting_additional_roots to carry the roots.

Refs #1455

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
… not declare

A registry field rename passed every authoring gate: the source update
installed a fact schema without the old name, the candidate built, and
every request then failed at evaluation with a missing-fact refusal.
The operator learned from production traffic rather than from review.

The authoring library now reads the parsed answer function, without
running it, and names each literal fact read on its first parameter
that the question's source does not declare. evidencectl applies the
check wherever it reads a project, so check, fixtures run, build,
package, source diff, and source update refuse the rename before
anything is installed. The declared set is an inline operation's facts
or the properties of a referenced source's closed fact schema; an open
schema, a computed key, and a read outside answer are left to the
fixtures. The check is source-product neutral.

BREAKING: a project whose derivation reads an undeclared fact is now
refused as evidence.authoring.derivation-fact-undeclared.

Refs #1423

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
`bregctl test --baseline-runtime-config` now rebuilds the predecessor
schema from the verified active package's signed sources on the
disposable test database, requires it to reproduce the recorded
predecessor fingerprint, and runs the successor migration in apply order:
pre-assertions, compiler statements, reviewed steps, deferred constraints
and views, runtime ACL reconciliation, and post-assertions. It requires
the candidate schema fingerprint and rolls everything back in one
transaction. `package` needs the test receipt, so a plan the rehearsal
refuses can no longer be signed and fail only at `apply`.

The rehearsal runs over empty tables: it proves SQL validity, ordering,
and reachability of the target schema, not data-dependent failures.
Field-encryption backfill steps are skipped and reviewed fixture files
are not loaded. The operator docs now describe the reviewed-migration
file layout, the backup binding `apply` checks, and what `test` does and
does not rehearse.

Security: a PostgreSQL error message can quote row values. The
rehearsal failure reports carry only the SQLSTATE, its class, and the
schema, table, column, and constraint names, enforced where
`postgres/rehearsal.rs` maps the database error. The negative test
`real_postgres_rehearsal_refuses_a_reviewed_plan_activation_would_refuse`
plants a canary in a failing step and asserts it never reaches the
report.

Refs #1424

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
`history rebaseline` refused any registry holding more than 1,000 live
rows, so a larger registry could never regain snapshot coverage after an
erasure. The verification now reads each entity's live rows, their
journal heads, and the retained heads in the same record-identifier range
in pages of 1,000 inside the same transaction, proves each entity holds
exactly one retained journal head per live row, and keeps memory and
every per-entity statement flat as the registry grows.

The run is a read and write outage for every entity, as it was under the
old limit, and it now lasts as long as the registry is large. The
migration role lifts forced row security on every entity table before the
first page with ALTER TABLE, which holds each table in ACCESS EXCLUSIVE
mode until the transaction ends, so API reads wait as well as writes. The
migration role deliberately holds no BYPASSRLS authority, which is the
only way to read the rows without that lock, so the operator docs state
the outage and give a sizing and maintenance-window rule instead: time
the command against a restored copy, allow at least twice that, drain or
announce API traffic, and size the statement timeout for one scan of the
revision journal (the unindexed-head check) and the lock timeout for the
in-flight requests at the start.

BREAKING: `HistoryRebaselineError::LiveRowBudgetExceeded` and
`MAX_REBASELINE_LIVE_ROWS` are removed, and bregctl no longer emits
`history.rebaseline.live_rows.budget_exceeded` or
`field_encryption.erase_history.rebaseline.live_rows_budget_exceeded`.

Security: the threat is a baseline that vouches for a live row its
journal does not reproduce, or for a journal head whose live row is
gone. `verify_every_live_row_matches_its_journal_head` in
`history_migration.rs` compares every page's live rows with their heads
and the retained heads in the page's key range, then refuses any head
past the last live row, before the baseline commit is written. The
negative tests `rebaseline_refuses_a_mismatch_on_a_later_page`,
`rebaseline_refuses_a_journal_head_with_no_live_row`,
`rebaseline_refuses_a_journal_head_before_the_first_live_row` and
`rebaseline_refuses_a_journal_head_past_the_last_live_row` prove the
refusals past the first page and at both ends of the key range. Row
security is not weakened: the force is lifted only inside the migration
transaction and restored before it commits.

Refs #1428

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
A reviewed `chunked_backfill` step changed live rows without appending a
history revision, so snapshot and as-of reads kept serving the values the
backfill had replaced. Each chunk now captures the rows it selects,
runs its UPDATE, requires the changed row count to equal the selected
count, and appends one revision per changed row in one history commit,
all inside the chunk's own transaction. The field-encryption backfill
already journaled its chunks this way; both now share the page capture.

`bregctl test` classifies every step `apply` journals with the same
check, so a step the journal would refuse fails the rehearsal as
`migration.rehearsal.history_step_refused` instead of failing `apply`.

BREAKING: `chunkSize` for a reviewed chunked backfill is capped at
1,000 (it was 10,000), the history commit's member budget. Its SQL must
be one UPDATE of the declared entity that writes no record metadata and
uses no refused statement word outside comments and plain string
literals. A plan outside those limits that `test` and `package` accepted
before is now refused.

The lexical first check that journaled steps share sets aside comments
and plain string literals, accepts a line break after UPDATE, and
matches record metadata columns and statement words as whole words, so a
leading licence header or a member named `created_at_source` no longer
refuses a step. It reads a dollar-quoted body, a literal holding a
backslash, or an unterminated comment or literal as written, so a
statement word inside one still refuses the step: the scan never guesses
where such a construct ends. The parsed-statement validator and the
journal's metadata check stay behind it.

Security: the threat is retained history that diverges from the live
rows a reviewed backfill rewrote, so an as-of read or export serves
values the registry no longer holds and the journal cannot account for
the change. `execute_reviewed_chunk` in `postgres/interlock.rs` journals
every chunk through `prepare_reviewed_page_capture` and
`finish_reviewed_page_update`, and `check_reviewed_history_step` in
`history_migration.rs` refuses a step the journal cannot record, both at
apply and in the rehearsal. The negative tests
`reviewed_chunked_backfill_refuses_a_chunk_size_beyond_the_commit_budget`,
`chunked_backfill_refuses_the_statement_shapes_the_journal_cannot_hold`,
the `rank-dollar-quoted` case of
`real_postgres_rehearsal_refuses_a_reviewed_plan_activation_would_refuse`,
and the history assertions in
`real_postgres_backfill_and_destructive_recovery_are_bounded_resumable_and_activation_closed`
cover the refusals and the per-chunk revisions.

Refs #1480

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Removing a field or an entity from a registry package drops the live
column or table, but every revision snapshot recorded before the change
still holds the removed values, and so does every database backup. An
operator reviewing the diff could read the removal as erasure.

`bregctl diff` now reports each removed field or entity as the finding
`diff.history.removed_values_retained`, against the compiled diff, with
a message that names `bregctl history erase` as the only command that
removes the values, whole revisions of one record at a time. The change
and retention guides say the same.

Security: the threat is an operator treating a package removal as data
minimization or erasure while the values stay readable in retained
history and backups. `removed_value_findings` in
`crates/registry-bregctl/src/lib.rs` raises the finding for every
`field_removed` and `entity_removed` change, and
`a_removed_field_is_reported_as_retained_in_history_not_erased` proves
the finding for a removed field and its absence for an unchanged
candidate. The diff is offline, so it does not count the affected
snapshots, and there is still no command that removes one field from
history.

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Raising a `text` field's `maxLength` was a `field_type_changed` change
that needed a reviewed migration, although every stored value already
meets the higher limit. It is now the compatible additive change
`field_length_widened`: the compiler replaces the column's inline length
check under the same lock and row validation a vocabulary code addition
uses, keeping the constraint name a fresh install gives it. A revision
recorded under the lower limit stays readable, and an action that sets
or requires the entity reports `action_target_fields_widened` and stays
additive instead of becoming a reviewed `action_changed`.

A `string` field's `maxLength` is its `varchar` column type, and a
column type change needs the dependent views and row policies rebuilt,
so raising it stays a reviewed `field_type_changed`. Lowering any limit
stays destructive.

Security: the threat is a live check swap that quietly admits values
the successor does not declare, or drops a second check on the column.
`replace_inline_field_check_statement` in `generated_ddl.rs` replaces
only the single unnamed check over the column and now also excludes the
field pattern's named check, and `FieldTypeSource::widens_text_length_of`
in `contract.rs` admits only a `text` to `text` change with a higher
limit. `text_length_widening_replaces_the_length_check_and_keeps_existing_rows`
proves the pattern still refuses and the higher limit still bounds, and
`text_length_widening_is_additive_and_replaces_the_length_check` and
`a_raised_text_limit_on_a_targeted_entity_keeps_the_action_contracts`
prove that narrowing and a `string` limit stay reviewed.

Refs #1427

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The successor rehearsal in `bregctl test` and the rebaseline without a
live-row limit both change what an operator sees, and both remove or
tighten something a pipeline may rely on, so each gets a BREAKING entry
that links the operator guide describing it.

Refs #1424
Refs #1428

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
When PostgreSQL refused a compiler statement or a reviewed step after
`apply` began maintenance, the report said only
`apply.migration.failed`, so an operator had nothing to act on before
reaching for `migration reconcile`. A server refusal now keeps its
SQLSTATE and the table, column, and constraint names PostgreSQL
reported, and `apply` reports it as `apply.migration.statement_failed`
with that SQLSTATE and its class. A lost connection still reports
`apply.migration.failed`, and the target stays pinned either way.

`PostgresFailure` moves out of the rehearsal module so `apply` and the
rehearsal share one value-free error shape; it is now exported from
`registry_breg::postgres` under every feature set. A reviewed chunk's
refusal is classified like a transactional step's, so a chunk that
breaks a field pattern names the field.

BREAKING: a statement refusal after maintenance began reports
`apply.migration.statement_failed`, not `apply.migration.failed`, and
`registry-breg` reports it as the new `MigrationError::StatementFailed`
variant. Automation that matches the old code or matches
`MigrationError` exhaustively must add the new case.

Security: the threat is a PostgreSQL error message, detail, or hint that
echoes a stored or computed row value (a failed cast repeats its input,
a unique violation repeats the key) and would reach the apply report,
logs, or CI output. `PostgresFailure::from_error` in
`postgres/failure.rs` retains only the SQLSTATE and object names, and
`PostgresKernelError::from_statement_error` in `postgres/mod.rs` is the
only path that carries a refusal to `MigrationError::StatementFailed`,
whose Display is built from those fields alone. The negative test
`refused_step_reports_its_sqlstate` in `tests/postgres_migration.rs`
applies a step whose constant PostgreSQL refuses and asserts the
refusal names `SQLSTATE 22P02 (data exception)` and carries neither the
constant nor the managed schema name, and
`apply_reports_a_refused_statement_with_its_sqlstate_and_objects` pins
the report.

Refs #1424

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
A lower `minLength` on a `string` field under the same `maxLength` only
relaxes a column check, but the planner classified it as
`field_type_changed` and demanded a reviewed migration. It is now
`field_length_widened`, applied live by the same constraint swap a raised
`text` `maxLength` uses: the successor replaces the single inline check
over the column with the lower minimum, or drops it when the minimum
falls to 0, since a fresh install of the candidate declares none. The
catalog then matches a fresh install. Revisions recorded under the
higher minimum stay readable, and an action whose only change is the
lower minimum stays additive as `action_target_fields_widened`.

`text` has no `minLength`, so the live lowering applies to `string`.
Changing a `string` field's `maxLength` (its `varchar` column type) and
widening a `decimal` stay reviewed migrations: both alter a column type
that the generated views and row policies depend on, and the reviewed
path is where that dependency is handled.

Security: the threat is a live check swap that loosens a different
constraint than the one the change relaxes, or leaves the catalog
diverged from a fresh install so that the fingerprint no longer proves
the schema. `FieldTypeSource::lowers_string_min_length_of` in
`contract.rs` accepts only the same `maxLength` and a strictly lower
minimum, and `replace_inline_field_check_statement` in
`generated_ddl.rs` selects exactly one single-column check with
`INTO STRICT`, excluding compiler-named, temporal-order and pattern
constraints, and fails closed otherwise. Encrypted fields get no swap.
The negative cases in
`string_minimum_lowering_is_additive_and_replaces_or_drops_the_length_check`
(a raised minimum and a changed maximum stay destructive),
`a_lowered_string_minimum_keeps_recorded_values_readable`,
`a_lowered_string_minimum_on_a_targeted_entity_keeps_the_action_contracts`,
and the refusals and fingerprint equality in
`string_minimum_lowering_replaces_or_drops_the_length_check_and_keeps_existing_rows`
cover it.

Refs #1427

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…ck reconciliation

A reconciliation that failed every pass left /ready green and doctor
reported a generic dependency failure, so an operator could not tell a
wedged source from a missing secret.

Readiness now fails after five consecutive failed passes for a source and
recovers on the next successful pass. Reconciliation and synchronization
continue past a failing subject. doctor names the check that refused
(casework.doctor.check-failed, doctor:/checks/<check>) with a value-free
cause, reports per-source reconciliation health, verifies the retained
audit chain under audit.hashKeyRef, and prints the package digest. An
unmigrated database is named as a schema that is not current.

BREAKING: caseworkctl JSON reports move to v1alpha2; migration 17 must be
applied before serving.

Refs #1448

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
… work

A package that removes a queue, profile or source, edits a pinned review
kind without a new version, or changes what a source read discloses used to
activate silently, leaving in-flight reviews and open work items that no
caller could finish. The runtime now compares the pinned work with the
package before any activation write and refuses by name, with counts,
unless package.acknowledgeStrandedWork names that exact package digest.
caseworkctl doctor reports the same comparison as pinnedWork and fails the
pinnedWork check by name.

BREAKING: casework serve refuses such a package; DoctorReport gains
pinnedWork.

Refs #1416

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Casework and the Base Registry Engine ship in lock-step, but nothing told
an operator when a rolling upgrade left them on different releases: the
adapter kept talking and failures surfaced later as opaque source errors.

BReg now reports its release on GET /v1/registry in the
Registry-Engine-Version header, the BReg client exposes it on the response
metadata, and the Casework adapter treats a different release as the
source being unavailable, naming both versions. caseworkctl doctor reports
the mismatch with the upgrade step to take.

BREAKING: a Casework deployment whose BReg runs another release is now
refused as unavailable until both run the same release.

Refs #1419

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Archiving the earliest sealed segments is an allowed layout, and the
summary already reports the first retained sequence, but verification
still expected the first retained record to open the chain and refused
every archived prefix as a fork. Verification now starts from the first
retained record's own link when the sealed sequence begins after one and
reports that link as start_prev_hash, so the caller can match it against
the head it kept. A renamed or truncated segment still fails as before.

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…metrics

An operator had no offline way to prove the retained Casework audit journal
intact, hand it to an auditor, or scrape the runtime's health. caseworkctl
audit verify walks every retained record's keyed hash and chain link, and
can confirm that a head recorded earlier is still held or continued from.
caseworkctl audit export writes the verified records to a new owner-only
file that is linked into place only after the whole chain verified.

The runtime gains an optional metricsListener: a second, operator-private
socket serving /metrics and /version, refused unless it is loopback or
private and distinct from the API listener. Absent by default, so no new
socket opens. Doctor and the audit reports gain headHash and startPrevHash;
every schema change is additive.

Refs #1422
Refs #980

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…irst

A review authority restored from an older backup or replaced answers an
accepted review's result lookup with an empty 404. BReg left such a review
looking healthy, let it compete with live reviews for every poll batch, and
offered no supported way to submit it again or stop waiting.

The lookup now records result-unknown-to-authority, the poller serves a
review with a pending webhook completion first and an unknown one last, and
a pending completion makes its review due at once. bregctl review-recovery
resubmit submits the exact retained request again under its original
idempotency key, and close stops waiting with operator-closed. Both run
behind the verified operator boundary request retention uses, refuse by a
closed reason that names the state and code, and append one audit record
that carries no raw request identifier.

Refs #1454

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Every runtime drained the audit outbox into its own journal, so two
replicas wrote the same records into two chains, and a shared audit.path
refused the second runtime at startup.

One runtime per Casework schema now holds an audit publication lease, a
PostgreSQL session advisory lock on a connection detached from the pool,
and only that runtime opens and appends to the journal. Other replicas
serve requests as standbys and take the lease once its session ends.
casework_audit_publisher_leader reports the holder, and the lease and
journal-open stages name a failed takeover.

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…tore

The outbox was the only record of what reached the audit file. A database
restored behind its audit file republished every event the backup still
listed as pending, and an audit file restored behind its database silently
continued a chain that no longer held the published records.

The database now keeps the hash of the last audit record it marked
published (migration 0018), written in the same statement as the mark.
Taking the audit publication lease places that hash in the retained audit
file and refuses to start, or to take over as a standby, when either side
is behind; a database without the row adopts the file's tail. Separate
audit files per replica are refused for the same reason.

`caseworkctl audit acknowledge-restore` is the explicit way past the
refusal once every runtime is stopped: it marks pending events the audit
file already holds as published, never republishes, and appends a chained
acknowledgement record naming both heads. Idempotency keys used after the
backup remain reusable after a restore; the operator documentation states
that hazard and how to find the affected operations.

BREAKING: a restore that leaves the database and audit file at different
points, or replicas with separate audit files, now stop the runtime until
acknowledged.

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
A session the server ends mid-step (a terminated backend, a shutdown,
a failover, an idle-in-transaction timeout) arrives as a FATAL
database error, and it was reported as a refused statement with
ReconcileFailedMigration guidance. PostgresKernelError::from_statement_error
now maps SQLSTATE class 08, 57P01 to 57P05, and 25P03 to Connection, so
apply reports apply.migration.failed as it does for a client-side loss.
The verified-DDL path now classifies its errors through the same
function, so a statement timeout is Statement(57014) on both DDL paths.

BREAKING: the changelog entry now names the PostgresKernelError::Statement
variant, which breaks an exhaustive match on that enum as well as on
MigrationError.

Security note: threat is an operator acting on the wrong recovery
guidance after a server-ended session; both classifications keep the
target pinned, so this is a reporting fix, not a safety change.
Enforcement is sqlstate_ends_the_session in
crates/registry-breg/src/postgres/mod.rs. Negative tests are
a_server_ended_session_is_a_connection_failure_not_a_refused_statement
and failed_resume_and_ddl_timeout_are_fail_closed_on_real_postgres.

Refs #1424

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
The lexical check on a journaled reviewed step masked a line comment
through the next line feed only, while PostgreSQL also ends it at a
carriage return, so an assignment after `--\r` was hidden from the
scan. It also read a Unicode-escape identifier such as
U&"created\005Fat" as words other than the column it decodes to.
The mask now ends a line comment at either newline character, and any
U& identifier or string and the UESCAPE keyword are refused anywhere
in the statement.

The runtime metadata-equality check in journal_captured_changes now
also compares created_at and updated_at, captured as text in the same
transaction, so a step that changes either is refused even if a
spelling escaped both the lexical and the syntax-tree checks.

Docs: the reviewed-migration page says metadata may not be named
anywhere in the statement, even in a read; the retention page and the
erase-history lifecycle state that a rebaseline blocks reads as well
as writes, and that a paged verification statement is bounded by one
page of records and their retained revisions rather than a fixed cost.

Security note: threat is a reviewed step that rewrites record metadata
so the history journal no longer matches the live row. Enforcement is
reviewed_update_words and mask_comments_and_literals in
crates/registry-breg/src/history_migration.rs, with
CapturedEntityRow::keeps_record_metadata_of at journal time. Negative
tests are
chunked_backfill_refuses_the_statement_shapes_the_journal_cannot_hold
(carriage-return comment, both U& spellings) and
a_reviewed_step_that_changes_any_record_metadata_is_detected.

Refs #1480
Refs #1428

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Every runtime reads its runtime.yaml under the same file, parsing, and
substitution rules, and spells secret providers, database references, the
JWKS source, the package pin, and the listener the same way. The loader
refuses unsafe files, removed keys with their replacement, a wrong
envelope, and substitution inside secret references; the blocks carry a
canonical JSON Schema the conformance gate compares products against.

Ed25519 and ES256 key generation join ES384 so adopter tooling can mint
the key types the runtimes verify.

BREAKING: reject_deprecated_config_fields is removed; runtimes declare
removed keys on the loader.

Refs #1408

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
… loader

Scheduling now reads its runtime configuration through the shared
registry-platform-config loader and blocks, so its envelope, secret
providers, database, package, listener, and JWKS source share one
implementation and one schema with the rest of the stack.

BREAKING CHANGE: authentication.oidc.jwksUri is removed and refused with a
diagnostic naming jwksSource kind: uri; listener.bind is required; the
runtime configuration and every configured path are refused when they pass
through a symbolic link; an environment expression in the authored policy,
records, or fixtures is refused with the field that holds it.

Adds ${VAR} substitution in runtime.yaml string values outside *Ref fields
and the optional package.expectedDigest pin checked against the verified
package's policy digest.

Refs #1408

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Registry Render loads its runtime file with the platform loader and the
shared package, listener, and secret provider blocks, so the file follows
the same envelope, size, ownership, and substitution rules as the other
runtimes.

BREAKING CHANGE: runtime.yaml must declare
apiVersion registry.registrystack.org/render-runtime/v1alpha1 and kind
RenderRuntimeConfig. server.bind is listener.bind and is required,
server.shutdownGraceSeconds is listener.shutdownGraceSeconds, bundle.path
is package.root, audit.integrityKeyRef is audit.hashKeyRef, and secret
providers are declared under secretProviders with an absolute file root.
Removed keys are refused with their replacement named. serve and
healthcheck require --runtime-config FILE; the default runtime path and
the REGISTRY_RENDER_RUNTIME variable are gone. package.expectedDigest, when
set, must equal the sealed bundle hash reported by /health.

Refs #1408

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
A socket address parser accepts any number of leading zeroes in the
port, so an unbounded listener.bind text is refused before it is parsed.
The shared schema carries the same maxLength, and the Scheduling runtime
schema that embeds the block is regenerated.

Refs #1408

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
…loader

The Discovery runtime loads its runtime file with the platform loader and
the shared listener block, replacing its own YAML reader, so the file
follows the same envelope, path, size, and substitution rules as the other
runtimes. A loader refusal now names the file and field in the startup
error instead of reporting an opaque invalid configuration.

BREAKING CHANGE: runtime.yaml declares apiVersion
registry.registrystack.org/discovery-runtime/v1alpha1 and kind
DiscoveryRuntimeConfig instead of schemaVersion, and listener.address is
listener.bind. Both removed keys are refused with their replacement named.
The discovery binary takes --runtime-config FILE instead of --runtime; the
path must be absolute and free of symbolic links. The container image
passes the new flag in its default command.

Refs #1408

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
A runtime that renders its effective configuration, as Relay's authoring
tooling does, needs the blocks to write back the key names they were read
from. Optional fields are omitted rather than written as null, so the
generated schemas no longer claim a null default for them.

Refs #1408

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Relay now reads its deployment binding through the registry-platform-config
loader and shared blocks, so its envelope, secret providers, package,
listener, and JWKS source share one implementation and one refusal
vocabulary with the rest of the stack. Secrets resolve only through the
providers the file declares, and a file secret no longer resolves beside
whatever directory the runtime file happens to sit in.

BREAKING CHANGE: runtime.yaml opens with apiVersion
registry.registrystack.org/relay-runtime/v1alpha1 and kind
RelayRuntimeConfig. server.bind is listener.bind, packagePath is an
absolute package.root with an optional package.expectedDigest pin checked
against the package revision, audit.integrityKeyRef is audit.hashKeyRef,
and authentication.issuer is authentication.oidc with issuer and a
jwksSource of kind discovery or uri; each removed key is refused with its
replacement. secret:file/ resolves under secretProviders.file.root and
secret:env/ needs secretProviders.environment. relay check and relay serve
take a required absolute --runtime-config; the --runtime flag, the
RELAY_RUNTIME variable, and the default path are gone, and the container
image passes the flag in its default command.

The site changelog also gains the Scheduling entry and uses the file's
BREAKING marker for the Discovery and Render entries.

Refs #1408

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Environment substitution belongs to runtime.yaml. An authored file that
carries a ${VAR} expression would read as a literal and hide the
operator's intent, so a Relay registry.yaml and a Render bundle
manifest.yaml now refuse one with the field that holds it. Relay reports
contract.environment_expression; Render reports a manifest problem.

BREAKING CHANGE: a registry.yaml or bundle manifest.yaml holding a
${...} expression no longer loads.

Refs #1408

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
… schemas

The shared configuration loader changed the BReg, Casework, and
Scheduling runtime schemas and the Relay authoring runtime schema, so
the committed catalog no longer matched its generator. Regenerated with
products/identifiers/scripts/generate.py --write; only source and
artifact sha256 values change, no identifier or publication state.

Refs #1408

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Every runtime serves a package directory from package.root, but each product
proved its package a different way. One format lets an operator check any
package the same way and pin it with one digest: SHA256SUMS lists every
file's SHA-256 in sha256sum format, the package digest is the digest of that
file, and an optional REVISION line is hashed like any other file.

The verifier recomputes every digest, refuses a changed, missing or extra
file by name together with the command that rebuilds the package, refuses
links, special files and names that do not travel, and bounds every read.
PackageDigestMismatch now always carries the digest found, so every runtime
shows both digests in one shape.

Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
caseworkctl package writes SHA256SUMS through the shared package writer,
reports packageDigest, and takes --revision. The runtime verifies the
package at package.root with the shared verifier in every listener mode,
pinned or not, refuses a changed, missing or extra file by name, keeps
its check that the package holds exactly casework.yaml and its source
descriptions, and refuses the retired casework.package.json or a
directory without SHA256SUMS, naming caseworkctl package.
package.expectedPolicyDigest becomes package.expectedDigest and a
mismatch uses the shared refusal shape.

caseworkctl dev packages the authored project on every start and serves
that package, and the init runtime example serves .casework/package.
The doctor pinnedWork verdict development is removed and packageDigest
is always a digest.

Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
A package directory holding only SHA256SUMS was refused as holding no
files, which named no file. Verification now reports each listed file as
missing; only the writer refuses an empty tree.

Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
A Scheduling package is now the shared Registry Stack package: scheduling.yaml
and a SHA256SUMS file listing it, whose SHA-256 digest is the package digest.
The runtime verifies it at every start, in every listener mode and with or
without package.expectedDigest, so an edited, missing, or extra file, an
authored project without SHA256SUMS, and a directory still holding the retired
scheduling.package.json are refused by name before the policy is parsed.

schedulingctl package writes into a new --output directory, plans with
--dry-run, and records --revision in a REVISION file. The manifest and its
byte-exact policy digest are gone; the semantic policy digest that explain,
the store, and hooks use is unchanged. The demo packages its project copy
before it starts the runtime.

Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Relay refusals never carry a configured value, so a package refusal it
reports names the field while keeping the files and the fixing command.

Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
relayctl package writes the shared deployment package: every file listed
in SHA256SUMS, an optional REVISION from --revision, and the package
digest as the sha256 of SHA256SUMS. relay verifies that listing at every
startup, refuses a changed, missing, or extra file by name with the
packaging command and without the directory, re-reads each file it uses
against the verified digest, and still recompiles the Registry and
regenerates every artifact before readiness. package.expectedDigest pins
the package digest.

Artifact exposure is no longer stored: the runtime derives visibility,
operation and access bindings, and media types from the compiled
Registry, and the package report states them for review. A package root
that still holds relay-package.json is refused with the command that
replaces it.

BREAKING: relay-package.json and packageRevision are retired; repackage
with relayctl package and pin the package digest.

Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Wrap the existing signed BReg package in SHA256SUMS, verify the shared closure and optional digest pin before startup, and preserve the signature, trust, environment, database, revision, and sequence checks for the later ledger migration.

Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Replace bundle and runtime revisions with the verified package digest while preserving package-derived requirement revisions. Package production outputs, dev staging, committed fixtures, frozen contracts, and operator guidance now use the shared deterministic envelope.\n\nRefs #1409\nRefs #1352

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Replace manifest sealing with deterministic Registry Stack packages, bind verified captured bytes through rendering, and update Render contracts and examples.

Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Describe maintained examples as immutable packages and restore the last human review dates on Render site pages.

Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Migrate discoveryctl and runtime startup to the shared package envelope,
preserve exact verified index bytes, and update the maintained operator
journey.

Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Bound the verified index capture itself, align package-root schema validation
with the runtime, and give the retired build command explicit package guidance.
Keep discoveryctl's package contract available without enabling server features.

Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Require all seven adopters to prove the common package pin refusal, migrate maintained Evidence and Discovery references, and keep local BReg-Evidence development on generated verified packages.

Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Remove the obsolete runtime digest contract and direct maintained checks through an editable project, deployment target, and installed package.

Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Keep runtime configuration in the deployment target while passing the normalized package-relative fixture name accepted by Evidence.

Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Republish positive test packages after authored mutations, keep deliberate tamper cases negative, and align migrated package CLI expectations.

Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Keep the existing package tamper and pre-database ordering proof while asserting the WP2 SHA256SUMS refusal and exact operator recovery.

Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
Refs #1409

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
@jeremi

jeremi commented Sep 27, 2026

Copy link
Copy Markdown
Member Author

Superseded by #1632, which lands WP1 and WP2 rebased on the #1560 audit work. Follow-ups: #1633 to #1641, plus #1410 and #1418 for WP3.

@jeremi jeremi closed this Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant