Web search and fetch for the planner

.changeset/olive-moons-search.md

@@ -0,0 +1,5 @@
+---
+"apollo": minor
+---
+
+global_chat: opt-in web search and fetch for the planner

services/global_chat/PAYLOAD_SPEC.md

@@ -40,7 +40,8 @@ This document defines the input and output payload structure for the Global Agen
   ],
 
   "options": {                            // Runtime options (optional)
-    "stream": false
+    "stream": false,
+    "web_search": false
   },
 
   "api_key": "string (REQUIRED in production, optional in development)"
@@ -82,6 +83,7 @@ This document defines the input and output payload structure for the Global Agen
 
 - **`options`** (object, optional): Runtime options.
   - **`stream`** (boolean): Enable streaming response (default: false).
+  - **`web_search`** (boolean): Let the planner search and fetch pages on the live web for this request (default: `false`). Takes effect **only on planner-routed requests** — the direct `workflow_agent` / `job_code_agent` routes ignore it, and `meta.web_search_requested` records when it was set on a request that never reached the planner. Reachable domains are limited to a server-side allowlist. Requires the caller's own Anthropic key to have web search enabled in their Anthropic Console; searches bill to that key, and clients on a zero-data-retention contract cannot use it. If the key does not have it enabled, the turn still answers — without web results — and sets `meta.web_search_downgraded`.
 
 - **`api_key`** (string, **required in production**, optional in development): API key for the Anthropic API. In production environments this field is required and requests without it will be rejected. In development, the server falls back to the `ANTHROPIC_API_KEY` environment variable if this field is omitted.
 
@@ -131,7 +133,20 @@ This document defines the input and output payload structure for the Global Agen
       { "tool": "call_workflow_agent", "input": { "message": "..." } }
     ],
     "subagent_calls": [],                // Raw sub-agent result dicts (for debugging)
-    "total_tool_calls": 2
+    "total_tool_calls": 2,
+
+    // Only when the planner gave up mid-turn:
+    "truncated": true,
+    "stop_reason": "pause_turn",
+
+    // Only when options.web_search was set:
+    "web_search_requested": true,
+
+    // Only when the planner has web tools on:
+    "web_searches": 2,
+    "web_fetches": 2,
+    "web_domains": ["hl7.org", "docs.openfn.org"],
+    "web_search_downgraded": false
   }
 }
 ```
@@ -168,6 +183,11 @@ Each tool beat streams as: `thinking` spinner → `changes` (if the workflow was
   - **`tool_calls`** (array): List of `{tool, input}` objects for each tool the planner invoked (planner path only).
   - **`subagent_calls`** (array): Raw sub-agent result dicts including `_call_metadata`. On the planner path these are the full results, useful for debugging. On the router's direct job-code path it carries a single entry with just `_call_metadata` and `diff`, so a client can tell on either route whether a code edit actually landed (`diff.patches_applied`).
   - **`total_tool_calls`** (number): Total number of tool calls made by the planner (planner path only).
+  - **`truncated`** (boolean): `true` when the planner spent its `max_pause_continuations` budget while the API still had more of the turn to send `response` is the head of a reply the server split and not a finished answer. Accompanied by **`stop_reason`** (`"pause_turn"`).
+  - **`web_search_requested`** (boolean): Present and `true` only when the request set `options.web_search`.
+  - **`web_searches`** / **`web_fetches`** (number): Server-side web search and web fetch calls the planner made this turn.
+  - **`web_domains`** (array): Hostnames the planner fetched from this turn, deduplicated.
+  - **`web_search_downgraded`** (boolean): `true` when the web tools were dropped mid-turn because the caller's Anthropic key rejected them, and the turn was answered without web results.
 
 ---
 

services/global_chat/config.yaml

@@ -11,3 +11,11 @@ planner:
   model: "claude-opus"
   max_tokens: 24576
   max_tool_calls: 20
+  max_pause_continuations: 5
+  web_search: # Server-side web search/fetch for planner.
+    max_uses: 5
+    max_content_tokens: 10000
+    # Content from these domains steers the agent that edits workflows.
+    allowed_domains:
+      - hl7.org
+      - docs.openfn.org

services/global_chat/global_chat.py

@@ -61,6 +61,10 @@ def get_stream(self) -> bool:
         """Extract stream flag from options."""
         return (self.options or {}).get("stream", False)
 
+    def get_web_search(self) -> bool:
+        """Extract web_search flag from options."""
+        return (self.options or {}).get("web_search", False)
+
 
 @observe(name="global_chat", capture_input=False)
 def main(data_dict: dict) -> dict:
@@ -108,6 +112,7 @@ def main(data_dict: dict) -> dict:
                 attachments=data.attachments or [],
                 user=user_info,
                 metrics_opt_in=data.metrics_opt_in,
+                web_search=data.get_web_search(),
             )
 
             if tracking: