Skip to content

Wire the checkpoint invariant predicates - #325

Merged
chris-colinsky merged 2 commits into
mainfrom
chore/wire-checkpoint-invariants
Oct 20, 2026
Merged

chris-colinsky merged 2 commits into
mainfrom
chore/wire-checkpoint-invariants

Conversation

@chris-colinsky

@chris-colinsky chris-colinsky commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

The checkpoint runner never read a case-level invariants: block, and its expected and resume handlers skipped any predicate name they did not handle. So 17 names across 025, 026, 029, 031, 069 and 070 asserted nothing, among them 031's caller correlation_id claims and the resume correlation checks in 025 and 070. The OTel runner's 031 test still covered 031's claim on the span side, but nothing checked it on the saved records.

  • All three positions now reject an unknown predicate name. Each name is checked against what the runs recorded: both runs' saves, the saved record, the resume's events and body trace, and the method each save went through. Boolean predicates accept only true, so a fixture that sets one to false fails instead of switching it off.
  • 057 joins the runner. It passes a caller invocation_id on the first run and another on the resume, which the engine must ignore. No test covered that before.
  • 050's executes_step names and 054's batching scope, which the runner skipped on purpose, are now checked.
  • Two handlers could pass vacuously: the per-instance attempt checks when no recorder saw the instance, and the duplicate checks when the field was not a list. Both now fail instead.
  • The branches that read correlation_id_assertions, invocation_id_differs and correlation_id_preserved are removed. No fixture declares them.
  • POSITION_BLIND_READS gains resume.caller_invocation_id, which the runner reads both at case level and under resume.
  • The module docstring lists every fixture the runner drives. Four docstrings move their spec citations into comments, and the comment on the fixture set maps each fixture to its proposal.

Some predicates leave their meaning open, so these readings are pending a spec ruling. Every fixture that runs today passes under them:

  • last_saved_at_monotonic_across_saves is non-decreasing in save order, since two saves can land in the same clock tick.
  • completed_positions_step_monotonic means steps strictly increase within each save.
  • 029's inner_first_node_not_re_run looks at the first completed position inside the subgraph, and subgraph_re_entry_uses_parent_states means the re-entered step_two receives the saved record's parent_states.
  • 069's rolled-forward slot is found by instance index, which holds while the target field starts empty.
  • 054's batching scope means the fan-out node's own completion save goes through save(), and at least one save goes through the batched path.

Testing:

  • 14 engine mutants, each caught by the predicate it targets:
    • A resume that mints a fresh correlation id fails 025, 070 and both 031 cases. 031's uniform-correlation predicate also catches it on its own.
    • A resume that reuses the first invocation id fails 025, 031's first case, 057 and 070. 031's second case declares no invocation-id predicate, so it passes.
    • A resume that adopts the caller's invocation id fails 057, including when the caller's id is itself a UUIDv4.
    • A first run that drops the caller's correlation id fails 031's first case. A generated correlation id without dashes fails the second.
    • Non-monotonic last_saved_at, out-of-order steps and a non-UUID invocation id each fail 026.
    • Disabling the resume skip gate fails 029 and 070. Changing the outer state at subgraph re-entry fails 029. Replacing the rolled-forward contribution fails 069. Sending the fan-out node's completion save down the batched path fails 054.
  • 12 edited-fixture probes fail with the intended message: wrong expected values, an unknown name at each position, a boolean predicate set to false, case-level invariants with no resume, and per-instance predicates aimed at the wrong or an unseen instance.
  • Known limit: 029's parent_states check cannot tell an engine that restores the state class's defaults from one that restores the saved state, because 029 saves its outer state at the defaults.

Review follow-ups:

  • Every path now rejects an invariant block it never checks. Before this change, resume invariants went unchecked on a resume that expects an error or has no from_first_run, as did expected.invariants on a checkpoint_not_found case and case-level or expected invariants on a seeded-record case. Seven edited-fixture probes that passed before this change now fail with the intended message.

The checkpoint runner never read a case-level invariants block, and
its expected and resume handlers skipped any name they did not know.
So 17 predicate names across 025, 026, 029, 031, 069 and 070 asserted
nothing, among them 031's caller correlation_id claims and the resume
correlation checks in 025 and 070.

All three positions now reject an unknown name, and each name the
runner meets is checked against what the runs recorded: both runs'
saves, the saved record, the resume's events and body trace, and the
route each save took. Boolean predicates accept only true. 050's
executes_step names and 054's batching scope, skipped on purpose
until now, are checked as well.

057 joins the runner. It passes a caller invocation_id on the first
run and another on the resume, which the engine must ignore. Nothing
tested that before.

Two handlers could pass vacuously: the instance attempt checks when
no recorder saw the instance, and the duplicate checks when the field
was not a list. Both now fail instead. The branches that read
correlation_id_assertions, invocation_id_differs and
correlation_id_preserved are gone, since no fixture declares them.
POSITION_BLIND_READS gains resume.caller_invocation_id, which the
runner now reads at both case level and under resume.

The module docstring now lists every fixture the runner drives. Four
docstrings move their spec citations into comments, and the comment on
the fixture set maps each fixture to its proposal without the history.
Copilot AI balanced review requested due to automatic review settings October 20, 2026 04:39

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Predicate interpretations need specification confirmation, and expected-error resumes still bypass invariant validation.

1 open issue
What changed in this PR

Strengthens checkpoint conformance coverage by checking previously ignored predicates against recorded execution evidence.

Changes:

  • Adds invariant validation and checks for checkpoint and resume behavior.
  • Activates fixture 057 and records checkpoint save routes.
  • Registers the nested caller invocation ID directive.
File Description
tests/​conformance/​test_checkpoint.py Implements predicate checks, execution evidence capture, and fixture coverage.
tests/​conformance/​harness/​vocabulary.py Registers resume.caller_invocation_id.

Review effort: Balanced

Comment thread tests/conformance/test_checkpoint.py Resolved
Resume invariant names were checked only at the end of the case
runner, after the resume.expected_error branch and the branch for a
resume with no from_first_run had already returned. A case on either
path could declare an unknown resume predicate, or a valid one, and
nothing checked it.

Both resume blocks are now merged and their names checked up front,
next to the case-level guard. Declaring case-level or resume
invariants on a path that never completes a resume fails. Two other
paths dropped blocks the same way and now reject them: a
checkpoint_not_found case rejects expected.invariants, and a
seeded-record case rejects case-level and expected invariants.

No current fixture declares a block on a path that skips it.
@chris-colinsky
chris-colinsky merged commit 3b246c6 into main Oct 20, 2026
5 checks passed
@chris-colinsky
chris-colinsky deleted the chore/wire-checkpoint-invariants branch October 20, 2026 05:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants