feat: fit runs to the model's real context window; local vision; provider capabilities (Oct 2026); release 0.0.22

[siarheidudko]

Problem

In the demo, a local 13B model on the MCP tab failed with:

WebLLM generation failed: Prompt tokens exceed context window size: number of prompt tokens: 4124; context window size: 4096

WebLLM loads every prebuilt model with a 4096-token window (its default, to save VRAM; "-1k" builds get 1024), whatever the model was trained on. That holds from Qwen3.5 0.8B to Llama 2 13B. The agent, meanwhile, sized compaction from the configured window (8k in the demo; 128k by default). So the tool definitions plus a short conversation overflowed.

Change: the window

• The window in use is the model's real one.
  • webLLMContextWindow(model) reads it from the loaded engine (loadedModelIdToPipeline), else from the app config the model was built with, else from WebLLM's defaults.
  • The agent uses the configured window, but never more than the model's own. agent.contextWindowTokens exposes it.
  • A known window means compacting by tokens even without a compaction config.
• Sizes follow the window. These shrink with it, also when the host set them for a larger window:
  • the compaction threshold (½ of the window);
  • tool-result clearing (¼);
  • the cap on one tool result;
  • the results kept verbatim (1 below 16k).
• Tool search by size. 'auto' tool selection also switches to search once the tool definitions would take ¼ of the window.
• A clear overflow error. When a provider still refuses a prompt for length, the run reports ContextWindowExceededError with the sizes and what to do. This covers WebLLM, OpenAI, Claude, Gemini and llama.cpp.
• A larger window at load.
  • createWebLLMModel(id, { contextWindowTokens }).
  • withWebLLMContextWindow(appConfig, id, tokens) for a host's own factory, through engineConfig.appConfig. @browser-ai/web-llm silently ignores its top-level appConfig.
• Local vision models take images. That is WebLLM's Phi-3.5-vision-instruct and ids naming vision / VL / LLaVA / SmolVLM / Gemma 3n, on the prompted path too. Local models still take no PDFs.

Change: providers and synthesis

• CORS. directBrowserOk is now true for Google, OpenAI, xAI and DeepSeek, which all send CORS headers. This was checked with a preflight from a page origin in October 2026. The docs drop "OpenAI/xAI/DeepSeek need a proxy" and list the other providers a page can call directly: Kimi, Groq, Mistral, OpenRouter and Cerebras.
• Images. Moonshot (Kimi) and Mistral read images, and so does DeepSeek's Flash line. Kimi takes no PDF parts.
• The built-in model's window. contextWindowOf also reads the window of the browser's built-in model (@browser-ai/core).
• No more stock answer. synthesize: false, or a failed synthesizer, used to answer "Done — the changes have been applied.". The answer is now what the steps themselves said; the stock sentence remains only when they said nothing.

New exports: ContextWindowExceededError, contextOverflowOf, contextWindowOf, webLLMContextWindow, withWebLLMContextWindow, fitCompactionToWindow, toolDefinitionTokens, TOOL_SEARCH_WINDOW_SHARE, WEBLLM_DEFAULT_CONTEXT_WINDOW. Docs: docs/capabilities.md → "The model's window is a hard limit", docs/providers.md, the README. Version 0.0.22.

Checks

• typecheck, format:check, build.
• npm test 155/155. That includes 11 new tests in tests/context-window.test.ts:
  • the app-config override;
  • window detection;
  • fitting and clamping;
  • overflow translation;
  • the window for local and cloud agents;
  • tool search by definition size;
  • a run hitting WebLLM's overflow error;
  • local vision;
  • the built-in model's window;
  • provider capabilities;
  • the answer without a synthesizer.
• Real WebLLM. In Chromium (SwiftShader WebGPU, SmolLM2-360M-Instruct-q4f32_1-MLC), a model loaded with defaults reports 4096 from the live engine, and an agent configured for 128k uses 4096.

🤖 Generated with Claude Code

https://claude.ai/code/session_01A7jHa5Pim4ufrdgxg561G5