Web search and fetch for the planner

[IZO-Ong]

Short Description

Adds an opt-in server-side web_search / web_fetch to the global_chat planner agent in order to look up external API docs (e.g. FHIR).

Fixes Issue #496

Implementation Details

web_search / web_fetch is off by default. A request turns it on with options.web_search: true, and the flag is plumbed via global_chat.py → router.py → PlannerAgent.

Tool definitions (tools/tool_definitions.py). build_web_tools(config) reads planner.web_search from config.yaml, and returns web_search_20260209 + web_fetch_20260209. Current config ships max_uses: 5 and max_content_tokens: 10000.

Tool loop (planner.py), covering the issue's code-audit list:

• server_tool_use does not consume max_tool_calls as they run on Anthropic's server and are not local tools, as such we track it using max_uses.
• stop_reason: "pause_turn" is handled by continuing the request. Text from the paused round accumulates in paused_text and is prepended to the final answer.
• The assistant turn is appended as response.content.
• A BadRequestError while web tools are active (when caller's key does not have web search enabled) drops the web tools, rebuilds the system prompt, and retries once.

Streaming (streaming_util.py, planner.py). New STATUS_SEARCHING_WEB settled to "Searched the web" on the first web_search_tool_result / web_fetch_tool_result of a round.

Prompt (prompts.yaml). planner_web_tools_prompt is appended only when web tools are active.

Telemetry (PAYLOAD_SPEC.md). meta gains web_searches, web_fetches, web_domains (deduped fetch hostnames), web_search_downgraded, and web_search_requested.

Allowlist

The allowlist currently only ships opendocs.openfn.org and hl7.org (FHIR R4/R5) as the external docs target.

Running a demo

Requires an ANTHROPIC_API_KEY in .env with web search enabled on the key.

Save a payload as demo.json:

{
  "content": "Using the published FHIR R4 specification, which fields does the Patient resource define? Fetch the spec page and confirm from it rather than answering from memory.",
  "options": { "stream": false, "web_search": true }
}

bun py global_chat --input demo.json

The meta should show the web activity:

"web_searches": 2,
"web_fetches": 2,
"web_domains": ["hl7.org"],
"web_search_downgraded": false,
"web_search_requested": true

Testing

Verified:

• pytest services/global_chat/tests/unit services/job_chat/tests/unit services/workflow_chat/tests/unit -q -> 129 passed.
• Ruff: no new findings.

AI Usage

Please disclose whether you've used AI in this work (it's cool, we just want to know!):

• Yes, I have used AI
• No, I have not used AI

You can read more details in our Responsible AI Policy

[elias-ba]

Thanks @IZO-Ong for this, it is a big piece of work and the spinner and settle pattern sits well with what was already there.

Three things I would like changed before it goes in, all in the planner, and I have left them inline. The short version: the 400 handler treats every bad request as a missing web search entitlement, the "Searched the web" status is sent before we know whether the search worked, and a paused turn spends the tool budget and can be returned to the user as a finished answer.

There is also a note inline on the allowlist. Not a change, but it is the only thing standing between fetched page content and the agent that edits someone's workflow, and that is worth saying in the config.

[elias-ba]

This catches every 400, not only the one where the key has no web search. Any other bad request on a web-enabled turn lands here, and an oversized prompt is the easy way to hit it. The user is then told "Web search is unavailable for this account", which is not true, and we pay for a second full call before failing with the original error anyway.

The 400 body carries the reason, so gating on that would keep the fallback and lose the false claim.

[IZO-Ong]

I looked at Anthropic's docs, and to my knowledge there's no documented stable error code for "web search not enabled", as the 400 body only carries type and a free-text message, and Anthropic's own docs say not to string-match on error messages.

As such, I worked around it by reordering the control flow. So now when we hit an error with web search, we first the retry without web tools. If dropping the web tools makes the retry succeed, then we can report that they were the cause. If the retry also 400s, we surface the original error.

However, any request sent with web search enabled that hits an error will go through one extra round-trip, so I am not sure if this fix is the right direction. I hope you can provide some guidance on this!

[elias-ba]

This settles on content_block_start, which arrives before the block's content does.

A refused fetch still produces a web_fetch_tool_result block. I checked against the API: asking for a URL outside allowed_domains comes back end_turn with no exception, and the result block's content is a BetaWebFetchToolResultErrorBlock with error_code: url_not_allowed. So a blocked or unreachable fetch renders as "Searched the web", and since _send_settled also records into response_segments it comes back on reload. The turn then answers from memory while the user has been told we looked it up.

Reading response.content after _call_api returns would let you branch on the error shape.

Smaller thing on the same block: the spinner fires for every server_tool_use but the settle is capped at one per round, so three searches in a round leave two spinners unresolved.

[IZO-Ong]

I moved the settle to fire after the result block is read, so a blocked or failed fetch/search no longer is reported as "Searched the web".

[elias-ba]

I can see the test pinning this, so I take it the budget spend is deliberate. Two things I would still like to talk through.

A paused round made no tool call, so counting it against max_tool_calls means web search can eat the planner's ability to call subagents. With the budget at 10 that bites quickly.

The one I would push on harder is test_paused_text_survives_the_max_tool_calls_exit_without_duplicating. It asserts the answer is "AB" when the loop exits while still paused, which means the user gets the head of a reply the server had split, presented as the finished answer. The empty-output guard below cannot catch it because paused_text is not empty. Could we surface that as a truncation, the way max_tokens is, rather than return it as complete?

[IZO-Ong]

I address these in 2 ways:

1. Pause continuations now have their own budget (max_pause_continuations: 5).
2. A response that got cut off by hitting that pause limit is now flagged as truncated in the payload (meta.truncated / stop_reason).

Hope these resolves both concerns!

[elias-ba]

Worth a line here saying what this list is holding up. Fetched page text reaches the planner's context, and the planner writes the arguments for call_workflow_agent, which edits someone's workflow. The allowlist is the whole of that boundary and it is passed through from config with no validation. The empty-list kill switch is documented, but the reason the list matters is not.

[IZO-Ong]

Added a note on allowed_domains in the build_web_tools docstring and in config.yam!

[elias-ba]

This example says docs.dhis2.org, but the shipped allowlist is hl7.org and docs.openfn.org, so no turn can produce it.

[IZO-Ong]

Changed to include an example of hl7.org and docs.openfn.org!