Skip to content

fix(thinking): generic reasoning_effort for models with no known thinking family (fixes #252) - #253

Merged
ltmoerdani merged 2 commits into
ltmoerdani:mainfrom
ashif4ai:fix/issue252-generic-reasoning-effort
Oct 13, 2026
Merged

ltmoerdani merged 2 commits into
ltmoerdani:mainfrom
ashif4ai:fix/issue252-generic-reasoning-effort

Conversation

@ashif4ai

@ashif4ai ashif4ai commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

Problem

For any model that doesn't match a hardcoded thinking family in src/core/registry.ts (space-bunny-free, big-pickle, hy3, …), the Thinking Effort control appeared in the Copilot model picker — but selecting a level did nothing. The request never carried a reasoning field, so the model always answered at its default effort.

Reproduced on space-bunny-free (Go and Zen): set the per-model Thinking Effort to a non-off level, send a message, and the pick is silently discarded.

Root cause — two independent defects

1. FallbackThinking had no payload path. buildPayload() returned {} for every unknown model, so no reasoning field could ever be emitted — while schema() already used metadata.reasoning to decide whether to show a picker. The UI promised a control the request ignored.

2. buildPayload was never given metadata. All four production call sites called thinkingProviderFor(modelId) with no metadata, so this.metadata was always undefined inside FallbackThinking. Any metadata-driven gate would therefore never have fired, even once #1 was fixed.

This is also why patching a single model name can't work — the model was never the problem, the strategy was.

Fix

models.dev is now the source of truth, with no per-model hardcoding:

  • metadata.reasoning decides whether a model gets a reasoning control at all.
  • Its own reasoning_options decide which levels; the shared off/low/medium/high/xhigh/max ladder only fills the gap — so a model declaring a level this build has never seen works with no code change.
  • The pick is sent as a plain OpenAI-compatible reasoning_effort.
  • New opencodego.thinking.generic setting for a workspace-wide default, surfaced in the Set Thinking Effort… command.

FallbackThinking also serves family-less models on other wire formats (claude-* → Anthropic messages, gemini-* → Google, grok-* → Responses). reasoning_effort is meaningless there, so those are deliberately excluded via the registry's endpointKind and keep today's exact no-op behaviour rather than gaining a picker that silently does nothing. The big-pickle repetition penalty (#207) is preserved and composes with the effort field.

Follow-up bug found while testing

Fixing this exposed a second defect in the same feature: prepareChatRequest called resolveThinkingConfig() before metadata was resolved and without passing it, so FallbackThinking.applyOverride discarded the per-model pick outright. The workspace setting kept working because it flows through buildPayload, which does receive metadata — producing a confusing split where the Copilot dropdown did nothing but the same value in settings worked.

metadata is now a required field on ResolveThinkingConfigInput, so the compiler rejects that call shape and it can't silently return. This also restores stripSchemaDefaultEcho (#226) for generic models — without metadata the picker schema is unknown, no default is stripped, and VS Code's "off" baseline would clobber the user's workspace pick.

Testing

512 unit tests pass (495 before), npm run lint 7/7, npm run compile clean. 14 new cases in src/test/thinking.test.ts cover unknown reasoning-capable models, non-reasoning models, models with no metadata at all, non-chat endpoints, penalty coexistence, and a full known-family regression suite.

Verified live against https://models.dev/api.json to confirm which models take each path:

Model reasoning_options Path
space-bunny-free effort: low,medium,high,xhigh,max metadata levels (the reported bug)
hy4 effort: none,high metadata levels
big-pickle, nemotron-3-ultra-free, ling-3.0-tiny-free (empty) new generic ladder
longcat-2.*, ling-3.1-flash-free toggle pre-existing off/on path (unchanged)

Manually confirmed in the Extension Development Host on space-bunny-free and big-pickle — the latter sends { repetition_penalty: 1.2, reasoning_effort: "high" } at high.

Notes for review

  • Version bump intentionally omitted. Your history does releases as separate chore(release): 0.7.x commits, so I left package.json at 0.7.8 and only added the ## [0.7.9] CHANGELOG heading — happy to fold the bump in if you'd rather ship it here.
  • ARCHITECTURE-MAP.md corrections beyond this fix. The Thinking row had pre-existing drift (glm.ts 53→93, mimo.ts 72→78, provider.ts 78→81) and was missing muse.ts entirely. Since I was updating the row anyway I made it accurate; happy to split that out if you'd prefer a focused diff.
  • Endpoint exclusion is a judgment call. Emitting reasoning_effort to Anthropic/Google/Responses would be meaningless or rejected, so those models keep today's behaviour. Covering Claude properly would need an Anthropic-shaped thinking block — worth a separate issue if you want it.

Full write-up: docs/issues/109-20261004-issue252-generic-reasoning-effort.md

🤖 Generated with Claude Code

ashif4ai and others added 2 commits October 4, 2026 11:07
… family (fixes ltmoerdani#252)

For any model not matched by a hardcoded family in core/registry.ts
(space-bunny-free, big-pickle, hy3, ...), the per-model Thinking Effort
picker rendered but its selection was silently discarded -- the request
never carried a reasoning field, so the model always answered at its
default effort. Two independent defects:

- FallbackThinking had no payload path at all: buildPayload returned {}
  for every unknown model, while schema() already used metadata.reasoning
  to decide whether to show a picker. The UI promised a control the
  request ignored.
- All four buildPayload call sites called thinkingProviderFor(modelId)
  without metadata, so this.metadata was always undefined and any
  metadata-driven gate could never have fired. This is also why patching
  a single model name could not work -- the model was never the problem,
  the strategy was.

models.dev is now the source of truth with no per-model hardcoding:
metadata.reasoning decides whether a model gets a control, its own
reasoning_options decide which levels (falling back to a shared
off/low/medium/high/xhigh/max ladder), and the pick is sent as a plain
OpenAI-compatible reasoning_effort. Family-less models routed to
non-chat endpoints (claude-* -> messages, gemini-* -> google, grok-* ->
responses) are excluded, since reasoning_effort is meaningless on those
wire formats and a picker there would silently do nothing. big-pickle
keeps its repetition penalty (ltmoerdani#207) and now composes with the effort.

Fixing this exposed a second bug in the same feature: prepareChatRequest
resolved thinking config before metadata was available and without passing
it, so the generic strategy dropped the per-model pick outright while the
workspace setting kept working. metadata is now required on
ResolveThinkingConfigInput so the compiler rejects that call shape, which
also restores stripSchemaDefaultEcho (ltmoerdani#226) for generic models.

Adds the opencodego.thinking.generic setting and surfaces it in the
Set Thinking Effort... command.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Documents the generic reasoning path in docs/issues/109 (including the
live models.dev evidence for which models take each path), adds the
0.7.9 changelog entry, corrects the README thinking table, and refreshes
the Thinking row in ARCHITECTURE-MAP.md: fallback.ts 39 -> 93 plus
pre-existing drift in glm/mimo/provider and the missing muse.ts entry.

The version bump is intentionally left to a separate chore(release)
commit, per repo convention.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@ltmoerdani

Copy link
Copy Markdown
Owner

Nice one @ashif4ai, this is a clean fix. The two-defect diagnosis (payload returning {} plus the missing metadata on every thinkingProviderFor call site) explains why patching a single model name could never work, and making metadata required on ResolveThinkingConfigInput so the compiler catches future call shapes is a smart move.

I ran the verification locally before reviewing:

  • compile and the full lint gate (all 7 checks including unit tests) pass on the branch
  • merged against current main (which now includes fix(cache): deterministic Responses ids, stable session anchor, structural Anthropic breakpoints #251's cache-control changes, overlapping file: src/request/anthropic.ts, different hunks), compile and tests still pass, no conflicts
  • regression guard looks good: the tests asserting all 8 known families keep their exact payload shapes, and that the generic setting never leaks into them, are exactly the protection this area needs

One small thing I want to keep an eye on (not a blocker): for models that declare reasoning but no reasoning_options in models.dev, we now send xhigh/max from the shared ladder. If any gateway model only accepts low/medium/high, that could 400. Worth watching after release, and easy to trim the ladder later if a report comes in.

Planning to merge with a merge commit to preserve both of your commits. Thanks for the thorough issue doc and changelog entry too.

@ltmoerdani
ltmoerdani merged commit 64f8d18 into ltmoerdani:main Oct 13, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants