Skip to content

feat(ai): move chat models to Opus/Sonnet/Haiku 5.5 - #460

Merged
ABB65 merged 2 commits into
mainfrom
feat/models-5-5
Oct 14, 2026
Merged

ABB65 merged 2 commits into
mainfrom
feat/models-5-5

Conversation

@ABB65

@ABB65 ABB65 commented Oct 14, 2026

Copy link
Copy Markdown
Member

What changed

The Studio chat catalog moves to the 5.5 generation. This covers Studio chat and the ee Conversation API: request params, pricing and credits, compatibility for stored IDs, and handling of refusals.

  • shared/utils/ai-models.ts: the catalog, the default model (DEFAULT_CHAT_MODEL = 'claude-sonnet-5-5', :199), the new ModelPricing.longPrompt card (:151) and resolveChatModelId (:220).
  • shared/utils/ai-credits.ts: legacy pricing kept for old IDs (:77-79), ratesForPrompt (:129), and estimateMessageCostUsd (:158), which now prices the long-prompt part on its own card.
  • server/utils/turn-budget.ts: ratesForEstimate (:108); TurnUsageTracker.pricedUsage(model) (:256) classifies each call against the threshold, including the call in flight.
  • server/utils/conversation-engine.ts: the refusal branch (:497). The done event carries longPrompt.
  • server/providers/ai.ts:160,183: 'refusal' stop reason.
  • chat.post.ts:163 settles from tracker.pricedUsage (still in finally). chat.post.ts:246 resolves the requested ID before the plan gate.
  • ee/enterprise/conversation-api.ts:422-483: carries the per-call long split into the settle.
  • server/utils/conversation-keys.ts:73,84 and ee/enterprise/conversation-keys.ts:48,109,187: key list, create and update resolve stored and sent IDs.
  • app/composables/useChat.ts:253-262: a retired ID in the picker's localStorage moves to the current one and is saved again. :759 shows the refusal message.
  • .contentrain/content/system/ui-strings/en.json: the new chat.refused key, edited in the JSON directly with no MCP write.
  • supabase/migrations/051_conversation_key_model_default.sql: new migration (see below). 001_baseline.sql is untouched.

Old → new model IDs

Stored / sent Resolves to
claude-haiku-4-5-20251001, claude-haiku-4-5, any claude-haiku-* claude-haiku-5-5
claude-sonnet-5, claude-sonnet-4-*, claude-3-5-sonnet-*, any claude-sonnet-* claude-sonnet-5-5
claude-opus-5, claude-opus-4-*, any claude-opus-* claude-opus-5-5
anything without a catalog family (gpt-4o, claude-fable-5-1, ...) null, so the caller's existing default fallback applies

Pricing (USD per MTok, from the live pricing page)

Model Input 5m write 1h write (Studio) Cache read Output
Opus 5.5 4 5 8 0.20 20
Sonnet 5.5 2 2.50 4 0.10 (0.05x) 10
Haiku 5.5, prompt ≤ 100K 0.10 0.125 0.20 0.01 0.50
Haiku 5.5, prompt > 100K 0.50 0.625 1 0.05 2.50
  • The 1h write is 2x input on all three models, so CACHE_WRITE_MULTIPLIER = 2 stays.
  • Sonnet 5.5 cache read: the live page (platform.claude.com/docs/en/about-claude/pricing) says $0.10 (0.05x). The founder facts and the migration guide said $0.20. This PR encodes the page's $0.10. A question for ORK is below.
  • Haiku tier: the prompt length counts all input tokens (fresh + cache read + cache write), each request is priced on its own, and the higher rates apply to output too. The code models this per call: the turn tracker keeps a per-call list, pricedUsage splits out the calls over 100K, and the settle, turn budget and Conversation API all use that split. When a summed total arrives without a split, it is classified as a single request. A multi-call turn over 100K in total is then priced on the long card, which is above real cost and never below.
  • Old IDs keep their pricing (claude-sonnet-5 $2/$10, claude-haiku-4-5* $1/$5), so past usage rows still price correctly.

API-breaking changes handled

  • Thinking: every catalog model gets thinking: {type: "adaptive"} + an explicit effort (server/providers/anthropic-ai.ts:254-270, requestThinking). disabled is sent only for IDs outside the catalog. This fixes Sonnet 5.5's 400 on disabled, and Haiku 5.5 never gets budget_tokens.
  • Effort: Haiku low, Sonnet medium, Opus medium, always sent explicitly so the recalibrated defaults (Sonnet high, Haiku medium) never apply.
  • tool_choice / sampling: no request path sends tool_choice, temperature, top_p, top_k, prefill or fallbacks. A unit test asserts this for every catalog model (tests/unit/anthropic-ai.test.ts). The engine is the only request path for both Studio chat and the Conversation API.
  • Preserved thinking: history replays byte-identically (existing invariant). Thinking blocks travel through block_binding: drop_block as before. Persisted rows are not rewritten.
  • Refusal: Haiku 5.5 has no server-side refusal fallback. stop_reason: "refusal" now ends the turn with an error event, code: 'refused' (conversation-engine.ts:497), and runs no tools. The UI shows chat.refused. Credits still settle in finally from real usage.
  • Sonnet 5.5 max output: raised from 16K to 32K, because adaptive thinking counts toward it. Haiku stays at 16K, Opus is unchanged.
  • Prompt cache layout: unchanged. No per-request content went into system.

Stored-ID compatibility

  • Studio chat: chat.post.ts:246 resolves body.model before the plan gate. A retired ID becomes the family's current model. An unknown ID falls back to the default as before.
  • Chat picker: useChat reads the stored ID through resolveChatModelId and saves the new ID straight away.
  • Conversation keys: the list, create and Conversation API runtime use resolveConversationModel, which resolves and then requires a non-premium API model (haiku-5-5 / sonnet-5-5), else falls back to the default. Update writes ai_model only when the sent ID resolves.
  • Migration 051 changes only the column default of conversation_api_keys.ai_model to claude-sonnet-5-5. It rewrites no rows, because read-time resolution covers them. It is one lineage: the next number after 050, applied by both runners.

Credit and gate impact (plan prices and limits unchanged)

  • Haiku 5.5 is about 10x cheaper than Haiku 4.5. Most Haiku turns now settle at the 1-credit floor. Starter economics improve.
  • Sonnet 5.5 has the same input/output price as Sonnet 5 and a cheaper cache read. It now thinks adaptively at medium (Sonnet 5 ran thinking-off), so output tokens per turn will rise.
  • Opus 5.5 is unchanged. The premium gate still closes Opus in a trial on the Studio key.
  • Credit weights are unchanged. The AI_CREDIT_UNIT_USD floor holds: settle prices at or above real cost in every path.

Questions for ORK

  1. Sonnet 5.5 cache read: this PR encodes the live page's $0.10. If a contract or invoice says $0.20, change one number (cacheReadMultiplier: 0.05 → 0.1 in ai-models.ts).
  2. Credit floor on Haiku: the 1-credit-per-message floor now sits well above the real cost of a light Haiku 5.5 turn. That is fine for "never below unit", but you may want to revisit it.
  3. Starter/free limits and the trial premium gate: worth a look now that Haiku is about 10x cheaper. Nothing changed here.
  4. Effort tuning: Haiku low and Sonnet medium were not checked against evals (no live calls in this PR).

Known limits

  • A call cancelled mid-thinking undercounts output, because thinking is not streamed as text. This gap existed before, and now applies to all three models.
  • The history budget estimator undercounts Haiku 5.5's tokenizer (about 30% more tokens). The Haiku history budget stays at 24K.

Tests

Suite Files Tests
unit 200 passed, 1 skipped 2287 passed, 1 skipped
integration 54 passed 517 passed
nuxt 52 passed 358 passed
  • pnpm lint: 0 errors. The 7 v-html/console warnings were already there.
  • pnpm typecheck: clean.
  • Integration needs the generated #contentrain client (contentrain-query generate). Without it, the billing webhook suites fail on import.
  • No live API calls.

ABB65 added 2 commits October 14, 2026 19:38
- Catalog: claude-haiku-5-5 (adaptive, effort low), claude-sonnet-5-5
  (adaptive, effort medium, max output 32K), claude-opus-5-5 unchanged.
  Default chat model is now claude-sonnet-5-5.
- Pricing: Sonnet 5.5 $2/$10 with cache reads at 0.05x ($0.10), Haiku 5.5
  $0.10/$0.50 with a long-prompt card ($0.50/$2.50) for requests whose
  prompt (fresh + cache read + cache write) exceeds 100K tokens. Each
  call is classified on its own in the turn tracker, the turn budget and
  the Conversation API settle; a summed total without a per-call split is
  priced on the long card (never below cost). Legacy IDs keep their
  pricing for past usage rows.
- Compatibility: resolveChatModelId maps retired haiku/sonnet/opus IDs to
  the family's current model in the chat route, the chat picker storage
  and conversation keys; unknown IDs still fall back to the default.
- Refusal: stop_reason "refusal" ends the turn with a "refused" error and
  a dictionary message instead of running tools (Haiku 5.5 has no
  server-side fallback).
- Migration 051 moves the conversation_api_keys.ai_model column default
  to claude-sonnet-5-5; existing rows resolve at read time.
- Move fixtures to the 5.5 IDs; heavy-turn shapes now sit on the Haiku
  5.5 long card so the credit expectations stay the same.
- New cases: per-call long-prompt split, planCall on the long card,
  adaptive thinking + effort per catalog model with no sampling or
  tool_choice params, refusal handling, retired-ID resolution in the
  server and the chat picker storage.
@ABB65
ABB65 merged commit 8aaec5c into main Oct 14, 2026
3 of 4 checks passed
@ABB65
ABB65 deleted the feat/models-5-5 branch October 14, 2026 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant