Skip to content

test: drive 402 through real escalation consumers - #56

Merged
thegeorgepu merged 4 commits into
mainfrom
test/40-real-402-consumers
Sep 4, 2026
Merged

test: drive 402 through real escalation consumers#56
thegeorgepu merged 4 commits into
mainfrom
test/40-real-402-consumers

Conversation

@thegeorgepu

@thegeorgepu thegeorgepu commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

What this changes

Replaces the hand-copied 402 control-flow assertion with a deterministic integration test that drives real status: 402 errors through advisor, council, scope, and loopbreak. Each case asserts a terminal throw, exactly one stronger-class attempt, an untouched same-class retry and cheaper fallback, and no unavailable-class or continuation warning. The test is registered in the aggregate Vinci harness.

The test also accepts VINCI_TEST_402_CONSUMER=advisor|council|scope|loopbreak so mutation testing can prove every real consumer fails independently when account errors are misclassified.

Why

Fixes #40. The classifier was pinned, but none of its escalation consumers were; a consumer could silently reintroduce downgrade-on-billing-error while the existing copied test stayed green.

Layer

  • vinci/ - the Vinci layer
  • packages/**/vinci-*.ts - a Vinci change inside an upstream package
  • Something else in packages/ - this probably belongs upstream, see CONTRIBUTING.md
  • Docs, CI, or tooling only

Exact successor

  • Base main: 6e19f1418718e9e33fdfa2d0d6c03054c5768470
  • Head: 240ddebe1948b82772e5b70f199e8e53ce557ce0
  • Tree: 159eff82ed9fae10ac3627c1df6347bf1790401b
  • Head parent: 369804e144232a58dff8c594c991a636425ce9e2
  • Integration merge parents: original PR head e948eb8ead6ea5bbeb7f356196f14dd29557c837, current main 6e19f1418718e9e33fdfa2d0d6c03054c5768470

The successor differs from main in exactly two declared paths:

  • vinci/test/402-escalation-no-downgrade.mjs
  • vinci/test/run.sh

There was no overlap between changes added to main since the original PR base and either PR path, the classifier, or the four consumer extensions.

Checks

  • npm run check is clean - errors, warnings and infos alike
  • ./test.sh passes (not applicable: this is a Vinci-harness-only test change)
  • Added a test for anything that could regress
  • Added a ## [Unreleased] entry (not applicable: test-only, no product behavior changed)
  • No package-lock.json changes
  • No emojis in commits, title, or comments

Focused Node 22 checks passed on the exact successor:

  • node vinci/test/402-escalation-no-downgrade.mjs
  • node vinci/test/402-classification-integration.mjs
  • node vinci/test/no-downgrade-integration.mjs
  • all four individual VINCI_TEST_402_CONSUMER selections

Mutation verification: changing the classifier's account result to transient made every selected consumer independently exit 1 with its own Missing expected rejection assertion (advisor-in-flight, council-in-flight, scope-in-flight, and loopbreak-in-flight). Separately replacing each consumer's preserved cause with a new error carrying the same message and status: 402 made every selected consumer independently fail the strict cause-identity assertion. Restoring production sources made the complete and individually selected matrices pass. No production mutation remains.

The supported local Node 22 offline harness passed all groups through 365/365 units, worker suites, containment broker 42/42, package first-party 30/30, loopbreak 84/84, scope 83/83, unattended 96/96, shell 33/33, coding-agent 101/101, Crew 184/184, resultbudget 15/15, and the first UI suite 33/33. The final 23-case UI group stopped emitting progress on this macOS host and was interrupted after 254 seconds; the harness then completed its native PTY check successfully and exited nonzero only for the interrupted UI group. Fresh exact-head hosted Node 22/24 lanes remain the authoritative full-harness gate. An earlier npx --package node wrapper run is not counted because it contaminated nested npm execution; rerunning directly with the cached Node 22 binary made package first-party pass 30/30.

Packaging evidence:

  • Network-free build passed.
  • Unsigned development package 0.0.51 was created.
  • From an unpacked tarball, all 100 relative extension imports resolved; both launcher dispatch targets and 34 imports resolved; vinci worker reached its usage refusal; packaged CLI reported 0.0.51.

Fresh exact-head hosted checks are all green:

  • Run 33890967535: check job 101082244752, offline (22) job 101082244616, upstream-suite job 101082244278, sandbox-macos job 101082244680, and offline (24) job 101082244851.
  • Run 33890967341: build-check-test job 101082244590.

Required final review: GO for exact head 240ddebe1948b82772e5b70f199e8e53ce557ce0 / tree 159eff82ed9fae10ac3627c1df6347bf1790401b, with no actionable findings. The reviewer independently reproduced the cause-identity mutation failure and the restored positive path.

Anything reviewers should know

The provider stub throws the original error object with status: 402, and the assertion checks that the terminal wrapper retains that error as its cause. Two 402 variants run through all four consumers. A second same-class 402 and a successful cheaper response remain queued as tripwires, proving neither retry nor downgrade occurs.

No billing policy or runtime code changed. Outbox, CFT, provider/model, GPU, and Sentinel boundaries are unchanged.

@thegeorgepu thegeorgepu left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GO at exact head e948eb8ead6ea5bbeb7f356196f14dd29557c837.

I verified both real 402 variants through the production advisor, council, scope, and loop-recovery consumers. Each case terminates with the original status: 402 retained as the cause, never attempts a same-class retry or cheaper fallback, and emits no unavailable-class continuation. The queued second 402 plus successful Forte response is a valid tripwire: forcing the classifier to return transient makes the test fail at advisor-in-flight with Missing expected rejection after that retry/fallback path becomes reachable.

The focused test and adjacent classifier/no-downgrade tests pass; npm run check passes; the aggregate harness locally reached and passed the new group, and all hosted aggregate/check jobs are green. The branch is ahead 1 / behind 0 against main.

No actionable findings.

This review is AI-generated by Codex.

@thegeorgepu
thegeorgepu merged commit 4c22def into main Sep 4, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

No test drives a 402 through a real escalation site — the downgrade guard is unpinned at the consumer

1 participant