Use each model’s configured maximum output budget on every agent request

[m-aebrer]

Closes #519

Ensure ordinary agent requests and bounded retries use each model's configured maximum output budget, while compacting reactively only when exhausted responses may have been constrained by remaining context.

Implementation plan posted as a comment below.

[m-aebrer]

Implementation Plan

Problem analysis

The current code gives maxTokens two conflicting meanings: the model registry exposes it as the model's output maximum, while ordinary provider calls silently start at 32k and treat maxTokens only as a later escalation ceiling. GPT-6 Astra can see that artificial allowance and changes its behavior because of it.

The latest issue clarification establishes one simple invariant: one configured maximum, one meaning. Ordinary requests and bounded retries use the resolved model maxTokens unchanged. Compaction remains based on actual context use; a possible long-response/context collision is handled reactively only after response retries are exhausted.

Deliverables

1. Honor the resolved model maximum on the first request

   • Remove the shared 32k cap from ordinary buildBaseOptions() resolution.
   • Use options.maxTokens only for deliberately bounded call sites; otherwise use model.maxTokens.
   • Preserve the documented 16,384 fallback for custom models that omit maxTokens.
   • Remove DEFAULT_MAX_OUTPUT_TOKENS if nothing legitimate still uses it.

2. Retry truncations without changing the budget

   • Keep bounded lengthRetries behavior, but send the same resolved maximum on every attempt.
   • Remove lengthRetryBudgetMultiplier; it contradicts the fixed-budget rule.
   • Preserve and discard truncated partial responses exactly as today.
   • After retries are exhausted, produce the existing loud terminal failure with accurate wording.

3. Make retry events describe reality

   • Keep the length_retry event name, since the cause remains response-length truncation.
   • Replace escalation-specific previousMaxTokens/nextMaxTokens with one maxTokens field.
   • Update extension types, RPC events, dashboard state, TUI warnings, and tests.
   • Change visible text from “retrying with larger budget” to “retrying at the configured output limit.”

4. Compact only after retries and only when context may be responsible

   • Leave proactive threshold compaction based on actual context and compaction.reserveTokens; do not reserve model.maxTokens in advance.
   • Preserve provider-specific context-exhaustion details when available.
   • After length retries are exhausted, classify the failure as possible context exhaustion only when either:
     • the provider explicitly reports context exhaustion, or
     • measured/estimated input tokens plus configured maxTokens exceed the context window.
   • Reuse the existing one-shot _overflowRecoveryAttempted flow: compact once, retry at the same full maximum, then fail loudly if it still cannot complete.
   • Keep genuine output-limit exhaustion separate: if the response could have fit in remaining context, do not compact.

5. Fit numeric thinking inside the fixed total maximum

   • Refactor adjustMaxTokensForThinking() so it no longer enlarges total maxTokens; it should only choose and safely clamp a thinking allocation within the already-resolved total.
   • Apply the same rule to Anthropic, Bedrock, Google/Vertex, and the duplicated Gemini CLI implementation.
   • Preserve provider invariants, including visible-output headroom and budget_tokens < max_tokens.
   • Leave adaptive-thinking models, including Astra, on effort-level controls rather than inventing a numeric thinking budget.

6. Apply the policy across provider adapters

   • Verify payloads for Anthropic, Bedrock, OpenAI Responses, Azure Responses, OpenAI Completions, OpenAI Codex Responses, Google, Vertex, Gemini CLI, and Mistral.
   • Where a protocol supports an output-limit field, serialize the resolved model maximum.
   • For Codex, verify the live protocol accepts max_output_tokens; add it if supported. If the protocol truly controls the ceiling server-side, make that omission intentional, tested, and loudly reject unsupported user overrides rather than silently ignoring them.

7. Update documentation and release notes

   • Update the root README.md, packages/coding-agent/README.md, packages/coding-agent/docs/models.md, packages/coding-agent/docs/extensions.md, packages/coding-agent/docs/rpc.md, and packages/coding-agent/docs/compaction.md.
   • Update affected package changelogs.
   • Explain that larger limits permit more output but do not consume tokens unless generated.

Files to create or modify

• packages/ai/src/providers/simple-options.ts — resolve ordinary requests from the model maximum and refactor numeric thinking allocation.
• Provider adapters under packages/ai/src/providers/ — serialize or intentionally omit the resolved limit and preserve context-exhaustion details.
• packages/ai/src/utils/overflow.ts — distinguish possible context-constrained truncation from genuine output exhaustion after retries.
• packages/ai/src/index.ts — remove the obsolete shared cap export if unused.
• packages/agent/src/agent-loop.ts — fixed-budget length retries and revised event payload.
• packages/agent/src/agent.ts and packages/agent/src/types.ts — remove multiplier configuration and update public event/config types.
• packages/coding-agent/src/core/agent-session.ts — one-shot reactive context compaction using provider evidence or input-plus-maximum arithmetic.
• packages/coding-agent/src/core/extensions/types.ts — update the public length_retry event shape.
• packages/coding-agent/src/modes/interactive/interactive-mode.ts — accurate fixed-budget retry warning.
• packages/dashboard/src/client/state/reducer.ts and protocol/event tests — accurate fixed-budget retry status.
• Existing provider, agent-loop, session, overflow, registry, TUI, dashboard, and RPC tests — replace 32k escalation assumptions and cover the new behavior.
• Root/package/dedicated documentation and package changelogs listed above.

No new model configuration field is planned: maxTokens remains the single source of truth.

Testing approach

• Assert ordinary first payloads use resolved model maxima below, equal to, and above 32k.
• Assert models.json overrides reach the first outbound provider payload.
• Assert every length retry uses the same maximum and never emits escalation language.
• Assert exhausted retries compact once when explicit provider evidence exists.
• Assert exhausted retries compact once when input + maxTokens > contextWindow.
• Assert exhausted retries fail without compaction when the response could fit.
• Assert a failed post-compaction retry does not loop.
• Assert proactive threshold compaction remains unchanged and does not trigger at half-context merely because output capacity is large.
• Assert explicit summarization and branch-summary budgets remain bounded.
• Assert numeric thinking budgets remain valid and leave output headroom.
• Assert all provider payload field names and Codex's intentional behavior.
• During implementation run focused tests, npm run check, npm test, npm run build, and npm run verify-workspace-links.

Acceptance criteria

• Every ordinary agent request uses the active model's resolved maxTokens, including models.json overrides, from the first attempt.
• Every bounded length retry uses that same value without synthetic 32k/64k escalation.
• Retry events and UI surfaces report fixed-budget retry behavior accurately.
• After retry exhaustion, dreb compacts at most once only when context may have constrained the response; genuine output-limit exhaustion fails loudly without compaction.
• Proactive threshold compaction remains based on actual context and the configured reserve, not theoretical maximum output.
• Explicit bounded internal calls keep their explicit smaller limits.
• Numeric-thinking providers allocate thinking within the fixed total maximum and retain provider safety invariants.
• Every provider path applies the resolved policy intentionally and has payload coverage.
• Documentation and changelogs match the shipped behavior.

Risks and open questions

• lengthRetryBudgetMultiplier and the event field change are public API changes; the plan favors removing misleading semantics rather than retaining silent no-ops.
• Codex protocol support for max_output_tokens must be proven by fixture or live protocol evidence before changing its body.
• Provider token accounting can omit or classify context exhaustion differently, so arithmetic fallback needs measured input usage plus a conservative estimate when usage is missing.
• Larger output permission increases worst-case token use, but eliminates discarded 32k/64k generations and should improve quality for models that react to budget telemetry.

---

Plan created by mach6

[github-actions]

Vitest coverage

Metric	Covered	Total	Coverage
Statements	42061	57844	72.71%
Branches	23005	36713	62.66%
Functions	8987	12147	73.98%
Lines	30333	41568	72.97%

View full coverage run

[m-aebrer]

Progress Update

Implemented the configured-output-budget correction across the provider, agent, session, UI, and documentation layers.

Changes

• Ordinary streamSimple requests now use the active model's resolved maxTokens from the first call; the hidden 32k cap and DEFAULT_MAX_OUTPUT_TOKENS were removed.
• Length retries preserve that same configured limit instead of escalating 32k → 64k → ceiling.
• Removed lengthRetryBudgetMultiplier; simplified length_retry to one truthful maxTokens field and updated TUI/dashboard/extension/RPC surfaces.
• Exhausted length retries now enter one compact-and-retry recovery only when provider evidence or input-plus-configured-output arithmetic indicates possible context pressure. Genuine output-limit exhaustion remains loud and does not trigger threshold compaction.
• Numeric thinking budgets now fit inside the fixed total response maximum rather than enlarging it; Google/Vertex bounded-thinking paths also preserve output headroom.
• OpenAI Responses, Azure, Anthropic, Bedrock, OpenAI Completions, Google, Vertex, Gemini CLI, and Mistral payload paths are covered with the resolved model maximum.
• Live protocol evidence showed ChatGPT's Codex backend rejects max_output_tokens; that route intentionally omits it, and models.json now rejects unsupported openai-codex maxTokens overrides rather than silently ignoring them.
• Updated root, package, model, extension, RPC, and compaction documentation.

Verification

• npm run check
• npm run build
• DREB_SKIP_LIVE_API=1 npm test
• npm run verify-workspace-links
• Commit hook: 6,231 passed, 0 failed, 729 skipped
• Focused provider/retry/overflow/model-registry/UI/dashboard suites passed

Commit: 2624e83

---

Progress tracked by mach6

[m-aebrer]

Unverified Review Candidates — Pending Assessment

Review round: 1
Reviewed commit: 2624e83

These are unverified candidates. Severity reflects reviewer confidence; do not treat any item as a merge blocker until the assessment comment is posted.

Critical

None.

Important

Finding 1 — Fixed-budget length retries may repeat a maximal, already-truncated request (85% confidence)

packages/agent/src/agent-loop.ts now retries stopReason: "length" at the same configured maximum up to lengthRetries. For genuine output-limit exhaustion, the request inputs and budget are unchanged, so the default two retries can discard and bill for as many as three full-ceiling generations before the loud failure. The approved scope allows fixed-budget retries, but this candidate questions whether they have practical value at the model ceiling.

Finding 2 — No end-to-end test proves a registry override reaches an ordinary outbound request (91% confidence)

packages/coding-agent/test/model-registry.test.ts, packages/ai/test/simple-options.test.ts, and provider payload tests cover registry resolution, option construction, and serialization separately. A regression between those layers could violate the central promise that a models.json override reaches the first ordinary provider request. The suggested coverage drives a registry-backed overridden model through the ordinary Agent/provider path and captures the first payload below, at, and above 32k.

Finding 3 — Reactive compact-and-retry is tested with mocked recovery rather than the full request sequence (89% confidence)

packages/coding-agent/test/agent-session-auto-compaction-queue.test.ts invokes the private compaction check using a prebuilt terminal message and mocks _runAutoCompaction(). It does not exercise fixed-budget length retries, real compaction, the resulting agent.continue() request at the unchanged maximum, terminal-message removal, or prevention of a second recovery loop as one integrated sequence.

Suggestions

Finding 4 — Codex rejection does not cover all configuration paths that can be silently ignored (80% confidence)

packages/coding-agent/src/core/model-registry.ts rejects maxTokens only inside modelOverrides when the provider name is exactly openai-codex. Custom model definitions under that provider, and custom providers using API openai-codex-responses, can still accept maxTokens even though openai-codex-responses.ts intentionally omits the wire field. Keying validation to the resolved API and applying it to custom definitions would make the documented loud-rejection policy consistent.

Finding 5 — Bedrock's explicit context-window stop signal lacks response-path coverage (87% confidence)

packages/ai/src/providers/amazon-bedrock.ts newly preserves MODEL_CONTEXT_WINDOW_EXCEEDED as model_context_window_exceeded, but the added Bedrock test only inspects outbound inferenceConfig.maxTokens. A response-stream test would verify the SDK stop value becomes a length result, survives exhausted retries as provider detail, and activates context-overflow classification.

Finding 6 — Explicit summary budgets are not asserted at their call sites (84% confidence)

Compaction and branch-summary code still pass explicit smaller limits, but their tests do not assert those maxTokens arguments. If a call-site limit is later dropped, the new default permits the model's potentially much larger ceiling. Tests should assert the compaction budget and branch-summary 2048-token cap directly.

Finding 7 — Google and Vertex duplicate the same thinking-budget policy helper (93% confidence)

packages/ai/src/providers/google.ts and packages/ai/src/providers/google-vertex.ts contain identical getGoogleBudget() implementations, including model defaults, custom budgets, and the new 1024-token output-headroom clamp. A shared helper would reduce drift risk when this policy changes.

Strengths

• The ordinary option path now uses one clear maxTokens meaning, and retries cannot mutate it.
• Genuine output exhaustion remains loud while reactive context recovery stays one-shot and evidence-based.
• Provider payload coverage is broad, including intentional Codex omission.
• Thinking budgets are constrained within the fixed total while retaining provider headroom invariants.
• Event, TUI, dashboard, extension, RPC, and documentation surfaces were updated consistently.

Agents run: code-reviewer, error-auditor, test-reviewer, completeness-checker, simplifier

---

Reviewed by mach6

[m-aebrer]

Review Assessment

Unverified candidate findings

Classifications

Finding	Classification	Reasoning
Finding 1	discarded observation	Factual: The loop can retry an unchanged maximal request. Scope: The approved plan explicitly retains bounded retries at the same fixed maximum, superseding the original immediate-failure wording. Practical: Length outcomes are not guaranteed deterministic, so a retry can succeed; the alleged inevitability and cost regression are not established strongly enough to override the approved design.
Finding 2	false positive	Factual: model-registry.test.ts already contains carries an overridden maxTokens value into the first provider payload, resolves a 96000-token registry override through streamSimple, and asserts the first max_output_tokens payload. Scope: This directly covers the central override criterion. Practical: The claimed missing boundary does not exist.
Finding 3	useful follow-up	Factual: The new session tests mock _runAutoCompaction() rather than driving the complete retry/compaction/continue sequence. Scope: The implementation behavior is required, but no current code defect was identified; the decision branches and one-shot guard have separate coverage. Practical: An integrated regression test would strengthen confidence, but missing integration coverage alone is not a merge blocker.
Finding 4	useful follow-up	Factual: Codex rejection is limited to modelOverrides under the exact openai-codex provider name; custom model definitions or another provider using openai-codex-responses can accept a limit that the adapter omits. Scope: This is inconsistent with the PR's loud-rejection policy. Practical: The trigger requires an unusual custom Codex configuration, so it does not materially block the documented/common path but should be hardened separately.
Finding 5	useful follow-up	Factual: Bedrock's outbound maximum is tested, but its MODEL_CONTEXT_WINDOW_EXCEEDED response path is not directly exercised. The current code correctly maps it to length, preserves model_context_window_exceeded, and matches overflow classification. Scope: Provider evidence preservation is required. Practical: A stream test would prevent future regressions, but no present runtime defect was found.
Finding 6	useful follow-up	Factual: Summary call sites pass explicit smaller budgets, while their tests do not directly assert those arguments. Scope: Keeping bounded internal calls bounded is an acceptance criterion. Practical: This is useful defense-in-depth against a future regression, not evidence that current calls are unbounded.
Finding 7	nitpick	Factual: Google and Vertex duplicate getGoogleBudget(). Scope: Both copies implement the required policy correctly. Practical: Extraction may reduce drift, but the duplication causes no present user harm or acceptance failure.

Action Plan

No merge-blocking changes required.

---

Assessment by mach6

[m-aebrer]

Progress Update

Addressed all four non-blocking review follow-ups:

• Added integrated coverage for fixed-budget truncation retries, one compact-and-continue recovery, unchanged request maxima, and loop prevention.
• Extended Codex maxTokens rejection to custom model definitions and custom providers selected through the openai-codex-responses API.
• Added Bedrock response-stream coverage for explicit context-window exhaustion and overflow classification.
• Added direct assertions for bounded compaction-summary and branch-summary output budgets.

Verification

• 96 focused tests passed
• npm run check
• npm run build
• DREB_SKIP_LIVE_API=1 npm test
• npm run verify-workspace-links
• Commit hook: 6,235 passed, 0 failed, 729 skipped

Commit: fbf25c9

---

Progress tracked by mach6

[m-aebrer]

Unverified Review Candidates — Pending Assessment

Review round: 2
Reviewed commit: fbf25c9

These are unverified candidates. Severity reflects reviewer confidence; do not treat any item as a merge blocker until the assessment comment is posted.

Critical

None.

Important

Finding 1 — Session recovery lacks coverage when provider usage omits input tokens (93% confidence)

packages/coding-agent/src/core/agent-session.ts now estimates the preceding context and passes estimatedInputTokens into isContextOverflow() as a fallback when an exhausted response records no input usage. Utility-level arithmetic is covered, but the session-level recovery test only drives the recorded-input path. A regression in the session estimate, message slice, or option forwarding could prevent one-shot recovery for providers that omit input usage and incorrectly leave users with the terminal output-limit error.

Finding 2 — Numeric-thinking headroom is not directly tested for Vertex or Gemini CLI (96% confidence)

The PR changes separate reasoning-budget implementations in packages/ai/src/providers/google-vertex.ts and packages/ai/src/providers/google-gemini-cli.ts, but the new bounded-thinking payload assertion covers Google Generative AI only. A regression in either duplicate adapter could consume the full bounded output budget or emit an invalid payload. Direct payload tests should assert that maxTokens: 8192 with high reasoning remains a total of 8192 and clamps numeric thinking to 7168.

Suggestions

Finding 3 — Context estimation runs eagerly on every completed turn (88% confidence)

packages/coding-agent/src/core/agent-session.ts computes estimateContextTokens(this.agent.state.messages.slice(0, -1)) before checking sameModel or whether the response is an exhausted-length candidate. This walks the complete history after every assistant completion even though the estimate is only a fallback when recorded input usage is absent. Error threshold handling can then estimate the history again later in the same method. Computing this fallback lazily would avoid repeated hot-path work without changing behavior.

Finding 4 — Google and Vertex still duplicate the thinking-budget policy helper (93% confidence)

packages/ai/src/providers/google.ts and packages/ai/src/providers/google-vertex.ts retain byte-identical getGoogleBudget() implementations, including model-specific defaults and the new 1024-token visible-output headroom rule. Extracting a shared internal helper would prevent the provider copies from drifting when model budgets or headroom policy change. This repeats the prior round's finding 7, which was classified as a nitpick and was not part of the follow-up fixes.

Strengths

• Ordinary calls and every bounded retry now preserve one resolved output maximum without hidden 32k/64k escalation.
• The latest commit adds meaningful integrated retry → compact → continue coverage, closes all three Codex configuration gaps, covers Bedrock context-exhaustion detail end to end, and directly asserts bounded summary budgets.
• Genuine output exhaustion remains loud, while context-constrained exhaustion receives at most one evidence-based recovery attempt.
• Provider payload coverage, public event/UI updates, and documentation are broad and internally consistent.
• Specialist review found no current runtime error-handling or completeness defect at 80% confidence or above.

Agents run: code-reviewer, error-auditor, test-reviewer, completeness-checker, simplifier

---

Reviewed by mach6

[m-aebrer]

Review Assessment

Unverified candidate findings

Classifications

Finding	Classification	Reasoning
Finding 1	useful follow-up	Factual: Session recovery forwards an estimated-input fallback, while the integrated session test exercises recorded input and the estimate arithmetic is covered only at utility level. Scope: The recovery behavior is required, but no present implementation defect is identified. Practical: A session-level zero-input-usage test would protect providers that omit usage, but the current pass-through and arithmetic are both correct and separately tested.
Finding 2	useful follow-up	Factual: Direct bounded-thinking payload coverage exists for Google Generative AI, not Vertex or Gemini CLI. Scope: Numeric thinking must fit inside the fixed total on every adapter. Practical: Current Vertex logic is byte-identical to the tested Google helper and Gemini CLI's separate clamp correctly produces 7168 within an 8192 total; direct tests would prevent future adapter drift but reveal no current failure.
Finding 3	nitpick	Factual: The session computes the estimated-input fallback before checking whether the same-model overflow path needs it, and an error path may estimate again. Scope: This is a local efficiency concern, not an acceptance requirement. Practical: The estimator generally starts after the latest usage-bearing message rather than rescanning all tokenized content, and its cost once per model turn is not plausibly material beside inference.
Finding 4	nitpick	Factual: Google and Vertex retain identical getGoogleBudget() helpers. Scope: Both implement the required policy correctly; extraction is not required behavior. Practical: Sharing the helper could reduce future drift, but duplication causes no present user harm. This is the same concern already classified as a nitpick in round 1.

Action Plan

No merge-blocking changes required.

---

Assessment by mach6