server: preserve custom tool kind in Responses output

[carlitose]

What changed

• Preserve a registered Responses custom tool as custom_tool_call instead of changing it into function_call.
• Emit its free-form payload in input, while keeping ordinary functions on JSON arguments.
• Use the same tool-kind decision in final JSON and live SSE output.
• Add focused controls for hosted tools, namespaces, unknown metadata, stable event identity, payload edge cases, and continuation parsing.

Why

DS4 previously forgot the registered tool kind after building the model prompt. A free-form tool such as apply_patch therefore came back in the wrong response shape, and clients rejected it before executing the tool. The request already contains the missing kind, so this change carries that small piece of metadata to the shared response builder.

Before and after

Before: Every generated non-hosted call defaulted to function_call with arguments, even when the request registered a custom free-form tool.

After: Registered custom tools become custom_tool_call items with string input; ordinary functions, hosted tool_search, namespaces, and unknown-tool fallback retain their declared behavior.

flowchart LR
  A[Registered request tool] --> B[Request tool metadata]
  B --> C{Registered kind}
  C -->|custom| D[custom_tool_call and input events]
  C -->|function| E[function_call and argument events]
  C -->|hosted search| F[tool_search_call]

Loading

Code map

• ds4_server.c — records the custom discriminator, unwraps one generic string property into free-form input, shares kind recovery across final/SSE output, and adds server regressions.

How it works

1. Tool parsing records whether a request tool was registered as custom.
2. The shared Responses item builder looks up that metadata for every generated call.
3. Custom calls use custom_tool_call, input, and custom-input SSE delta/done events.
4. A single string-valued internal object is unwrapped generically; ambiguous or malformed payloads are retained verbatim.
5. Missing metadata safely defaults to an ordinary function call, while hosted search requires explicit hosted registration.

Verification

• ✅ make cpu in Docker Linux x86_64 with GCC 14.4.0 — exit 0; all CPU binaries linked.
• ✅ CPU source-provenanced make ds4_test override plus ./ds4_test --server — build exit 0; test exit 0; ds4 tests: ok, server: OK.
• ✅ git diff --check — exit 0, with only Git's Windows LF-to-CRLF warning.
• ⚠️ Exact Linux default make ds4_test — exit 2 because CUDA nvcc is absent; the subsequent binary command was unavailable (127).
• ⚠️ make test — not run because Darwin/Metal and ds4flash.gguf are unavailable.
• ⚠️ Live /v1/responses model/client round trip — not run because a suitable live model/server/client environment is unavailable.

Risks and limits

• Tests inject synthetic internal tool calls after model generation. They cover real request parsing, metadata lookup, final/SSE socket serialization, and continuation parsing, but not model call generation or client routing.
• Release is blocked pending Darwin/Metal model tests and a live Responses custom-tool round trip with ordinary and hosted controls.
• The default Linux CUDA test target was unavailable; only the explicit CPU build/test path passed.

Reviewer checklist

• Confirm custom, ordinary function, hosted search, namespace, and unknown metadata select the expected final and SSE item types.
• Confirm the one-string-property unwrapping rule is appropriately loss-aware and does not special-case patch.
• Run the open Darwin/Metal/model and live Responses gates before any release-readiness claim.