Repository navigation
Claude 5.x goals survive long thinking with an Anthropic key - #38
Conversation
Claude 5 and later models think by default, and thinking counts toward max_tokens. With a fixed 8192 cap, hard goals (Aider Polyglot Rust forth and bowling on claude-haiku-5-5) spent the whole cap thinking, the reply had no text block, and the goal failed with a dispatch error. - Send max_tokens 64000 to Claude 5+ models; older models keep 8192. - Stream every Anthropic request and rebuild the reply from its events, so a long reply is not cut off; the 600 s total client timeout is now a 600 s read (idle) timeout. - If a reply still stops at max_tokens before any text, ask once more with output_config.effort "low" and count both attempts' tokens.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d6010e3168
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| message.insert("content".into(), Value::Array(content)); | ||
| Ok(Value::Object(message)) |
There was a problem hiding this comment.
If Anthropic or an intermediary closes the SSE response cleanly after one or more text deltas but before the required message_delta/message_stop, this returns a seemingly valid message. Because message_start already supplies usage, call_with_attachments then accepts the partial text and reports the initial output count, silently treating a truncated model response as complete. Track the terminal event and reject incomplete streams so the call can fail or be retried instead.
Useful? React with 👍 / 👎.
…g-max-tokens # Conflicts: # CHANGELOG.md
* Release 0.26.0: Cloud goals run your own checks Bump the version in phonton-cli/Cargo.toml, Cargo.lock, package.json and the README badge, date the Unreleased CHANGELOG entry as 0.26.0, and add release-notes/v0.26.0.md. The release carries #37 (cloud `--check`, `pin_model`, Claude 5.5 prices, linked node_modules, missing test runner stops as unavailable) and #38 (64K reply cap with streaming for Claude 5+, retry of a thinking-only reply at low effort). * Release notes: forth and bowling pass on the release binary
With an Anthropic key, a hard goal on Claude 5.x could end at once with
anthropic response has no text block (stop_reason: max_tokens). Claude 5.5 models think by default and thinking counts towardmax_tokens. The provider sent a fixedmax_tokens: 8192, so on harder tasks the thinking used the whole cap, the reply held only athinkingblock, and the goal failed with a dispatch error and no retry. In the 51-exercise Aider Polyglot benchmark onclaude-haiku-5-5this was 2 of Phonton's 8 failures (Rustforthandbowling;forthalso failed this way in an earlier run).Changes
claude-<family>-<major>…with a major version of 5 or more) getmax_tokens: 64000. Older models, which don't think unless asked, keep 8192. 64K is the starting point Anthropic's docs give for agentic work, is within every Claude 5.x model's 128K output limit, andmax_tokensdoes not count against output-token rate limits."stream": true) and the reply is rebuilt from its events: text deltas,stop_reason, and the cumulative usage; a mid-streamerrorevent (such asoverloaded_error) becomes the error. Anthropic's docs advise against a largemax_tokenswithout streaming because networks can drop idle connections. The client's 600 s total timeout is now a 600 s read (idle) timeout, so a long stream is not cut off.max_tokensbefore any text, the provider asks once more withoutput_config.effort: "low"(the docs' other lever: less thinking) instead of failing the goal. Both attempts' tokens count in the receipt.anthropic_thinking_only_max_tokens_reply_is_retried_at_low_effort: a local stand-in API streams a thinking-only reply that stops atmax_tokens, then a reply with text. The test checks that the request streams with the 64K cap, that the retry asks for low effort, the returned diff, and the summed tokens.claude_5_and_later_think_by_defaultpins the model-id check. Thelive-testssmoke now also coversclaude-haiku-5-5.Verified on Windows
cargo fmt --checkandcargo clippy --workspace --all-targets -D warningspass.cargo test --workspace --no-fail-fast: every suite passes exceptphonton-verifybrowser_check_passes_on_valid_html, which needs the Playwright install that CI performs (Cannot find module 'playwright'locally). The new regression test failed onmainbefore the fix.cargo test -p phonton-providers --features live-tests anthropic_live_smoke):claude-haiku-4-5andclaude-haiku-5-5answered through the streamed path with text and token counts.forthandbowlinggoals.Docs checked: Effort (Claude Haiku 5.5: thinking counts toward
max_tokens, leave room for it), Troubleshooting thinking (amax_tokensstop: raisemax_tokensor lower effort), Errors > Long requests (stream whenmax_tokensis large), Rate limits (max_tokensis not counted toward OTPM).