Port visible-KV checkpoint fixes for tool turns (fixes token-mismatch reprocesses on agentic workloads)

[fradav]

Problem

On agentic workloads (tool-calling, thinking, steering interrupts, "send now" with a new message, or even a mid-thinking stream death), ds4-server frequently logs live kv cache miss ... reason=token-mismatch and then re-prefills the entire context — routinely 140k+ tokens. This happens on every failed tool, every steering interrupt that appends a new user message, and sometimes in the middle of a thinking run with no interruption at all.

Root cause: chat/completions and Anthropic tool-call turns had no live binding to the next request. The sampled tail (hidden reasoning, exact DSML spelling, BPE drift) never token-matches the client's replay, and the byte-exact memory-text path rarely matches either — so every agent turn paid an evict-store (~650 MiB) plus disk-snapshot restore and tail re-prefill.

The Responses API already solves this with a predicted visible-transcript key. This PR reuses that mechanism for chat/Anthropic.

Changes (all in ds4_server.c, server-only)

1. Visible checkpoint for chat/Anthropic tool-call turns — after finish=tool_calls, remember prompt_text + the same canonical suffix canonicalize_tool_checkpoint() builds (byte-equality with the next rendered prompt is already covered by test_tool_checkpoint_suffix_is_future_prompt_canonical). The next request continues from live KV, tokenizing only the new tool results. Hits are labeled tool-visible in logs.

2. Visible checkpoint for tool-context turns that don't call a tool — when the conversation has tools (or tool-using history), the chat template preserves reasoning verbatim in future renders, so a plain turn that finished without calling a tool still renders byte-for-byte into the next request. Previously such turns had no visible checkpoint at all and fell through to raw token matching. New gate should_remember_tool_context_checkpoint() + unit test.

3. Preserve the live frontier on streaming errors — the tool calls were generated and the KV rows written; only HTTP delivery failed. The !response_ok path keeps job_mark_cancelled() and the error accounting but deliberately drops request_live_state_clear() so the freshly recorded frontier survives a retry ("send now").

4. Store KV waypoints through tool calls too — waypoint snapshots are stored on disk in full (compressed KV, indexer, compressor frontiers included), so a snapshot taken mid-tool-call is as safe as any other. Previously waypoints were suppressed for the whole tool-call duration; long agentic turns left no recent restart point, forcing a near-total reprocess when the client stream died mid-tool-call.

Validation

• make ds4_test clean
• ./ds4_test --server → server: OK
• New unit test test_tool_context_checkpoint_remember_gate passes
• Metal, SSD streaming, and distributed paths untouched (server-only change)

These are backported from the fork's b1e3287, 47ee3ce, and ac98ff4, adapted to the slot-threaded upstream structure.

[Copilot]

Pull request overview

This PR ports “visible-KV checkpoint” behavior to chat/completions and Anthropic tool-turns so retries/continuations can bind to an in-memory KV frontier using a predictable visible transcript key, avoiding full-context re-prefills triggered by token mismatches in agentic/tool-heavy workloads.

Changes:

• Adds a “tool-visible” live checkpoint path for chat/completions + Anthropic tool-call turns (and a companion checkpoint for tool-context turns that didn’t call a tool).
• Preserves live continuation state across streaming delivery failures (so retries can continue from the just-generated frontier).
• Enables continued KV waypoint snapshotting during tool calls to improve restart points for long tool-call spans.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Flor1an-B]

Nice — this is broader than #709 (adds Anthropic, waypoints through tool-call turns, and disconnect/error preservation). Flagging where the two meet, since they collide on merge and, more usefully, cover opposite halves of the same axis.

Both remember prompt_text + build_tool_checkpoint_suffix(...) as a visible key for chat/Anthropic tool turns. The one difference that matters is the reasoning argument, because build_tool_checkpoint_suffix() renders reasoning + "</think>":

• This PR passes parsed_reasoning, so the key carries the reasoning verbatim — it matches a client that replays reasoning. Consistent with should_remember_tool_context_checkpoint() gating on prompt_preserves_reasoning.
• server: remember a tools-aware visible KV checkpoint after tool-call turns #709 passes reasoning = NULL, so the key is reasoning-free (<think></think>) — it matches a client that strips reasoning, which is exactly KV cache reuse breaks for OpenAI tools clients that don't replay reasoning_content #691 (chat_history_uses_tool_context() keeps the <think> tag but renders empty reasoning when the client echoes none).

Opposite client modes — and on current main the has_tools + strip-reasoning case is handled by neither: should_remember_thinking_checkpoint() still bails on r->has_tools, and should_remember_tool_context_checkpoint() here requires prompt_preserves_reasoning. So this PR's tool-context path and #709 are complementary, not redundant: this one owns the preserve case (plus Anthropic / waypoints / disconnect), #709 owns the strip case (#691).

The clean unification is to pick the reasoning by the flag rather than hardcoding it — roughly build_tool_checkpoint_suffix(..., r->prompt_preserves_reasoning ? parsed_reasoning : NULL, ...) at both the tool-call and tool-context sites, and dropping the prompt_preserves_reasoning precondition from the tool-context gate so the stripped case is remembered too. That covers both modes in one path and lets #709 fold in as the NULL branch. Happy to do that reconciliation on the #709 side once directions settle.

(Heads-up: #714 also changes thinking_live_remember's signature, so this is a three-way touch on that function plus the post-turn checkpoint block in generate_job_inner.)