[joelteply]

A relaunch is a pause, not a reload (continuum SERVING-ENGINE.md §3, slice S2, plan line 3b).
POST /reconfigure {"n_ctx", "n_parallel", "cache_type_k", "cache_type_v", "force"}:
admission closes (requests that arrive queue), the slots reach a ubatch boundary, every occupied
slot's target KV, draft KV, speculative tail and checkpoints move to host RAM (S1's page capture),
the draft and target contexts are freed and rebuilt on the SAME llama_model (weights and mmap
stay), the slots are rebuilt for the new n_parallel, the states go back, and serving resumes.

• A request in flight keeps its slot and finishes as if nothing happened (test: identical output).
  A request on a slot past the new n_parallel drains first. A request whose tokens do not fit the
  new window refuses the reconfigure; it is never truncated.
• An idle mind with no seat moves into a free slot, or stays a named RAM page (page_in brings it
  back) with a typed reason; one larger than the new window is likewise kept whole.
• A KV type change is an intentional cold start: refused without "force"; with it, resident
  states, the page store and the prompt cache are dropped, logged loudly and counted.
• A geometry libllama cannot build rolls back to the old one with every state restored.
• Receipt: drain, page out, free, init, page in and total ms; bytes; per-slot fate. /props
  context_geometry and total_slots follow the change (routes read an atomic meta snapshot);
  /metrics reconfigure_total, reconfigure_failed_total. A wake from sleep reloads the new geometry.
• libllama: one sequence's state reads into a cache split into a different number of streams
  (its cells all land in its own stream); the whole-cache read still requires the same layout.
• common: common_init_result::reinit_context/free_context and the speculative init result's
  rebuild the context on the loaded model; the context, sampler and threadpool setup are shared
  with the first build rather than copied. common_kv_cache_type_from_str is public.
• The mechanism is tools/server/server-reconfigure.{h,cpp} behind server_reconfigure_host;
  server-context.cpp implements the host over its slots and wires the task, the route and the
  admission gate. Slot init is one function shared by load_model and the reconfigure.

Measured (M5 Metal, this build):

• Qwen3.5 0.8B hybrid, lane geometry (174,592 ctx, q4_0 KV, 32 checkpoints), two minds of
  25,226 and 17,883 tokens: pause 72-104 ms (page out 6-13, free 3-5, init 57-80, page in 6)
  for 271 MB; median 78 ms over 40 cycles. Each mind's next act prefilled 15 tokens.
• Footprint: 100 grow/shrink cycles (2 -> 3 slots, n_ctx x2 and back) 395 MB -> 327 MB, never
  growing; 20 cycles at lane geometry 1,685 -> 1,617 MB.
• Qwen3.8 27B with the M5 lane's flags (mmproj, MTP draft model, q4_0 KV, 2 x 87,296), same
  two minds: 2 -> 3 slots 826 ms, 3 -> 4 (4 x 59,392) 802 ms, back to 2 626 ms, 2.25 GB paged;
  init (context + MTP context allocation) is 0.5-0.6 s of it. Next acts prefilled 15 tokens
  (6.4 s and 7.2 s wall, against 248 s and 173 s to seat them). CONTENDED: a second 27B
  serving lane started on the node during the run, so these are upper bounds; the 20-cycle
  27B pauses (0.54-4.6 s, median 0.70 s) ran beside it too.

Tests: tools/server/tests/unit/test_reconfigure.py: 5 passed (attention and hybrid), plus the
opt-in 100-cycle footprint validator (LLAMA_SERVER_TEST_RECONFIGURE_CYCLES); with test_slot_page,
test_slot_save, test_speculative: 27 passed. Full unit suite: 395 passed, 185 failed, the same
185 as S1's binary (test_tool_call real models not cached offline, one test_completion).

What S4 (Codex, core side) needs

• Call POST /reconfigure where the core relaunches today to change geometry: serving_daemon/launch.rs:221 and llama_server/reconcile.rs (window growth, seat count). Give the request a long timeout: it answers only after in-flight requests on removed slots drain.
• Read the answer as a typed receipt: rolled_back, cold_start, the per-slot pages[].fate (restored / kept_in_store / dropped) and page names, and the t_*_ms. A kept_in_store page belongs to a mind: page it in (/slots/{id}?action=page_in&page=), spill it, or drop it. Never ignore it, because it is counted against --page-ram-bytes.
• Read /props context_geometry (allocated_tokens, max_tokens_per_sequence, sequence_capacity) and total_slots after a reconfigure, never the launch flags. The slot count can now change under a running engine, so slot ids above the new count are refused (400).
• Never change KV types implicitly. That is a force cold start and should be an explicit, logged decision.
• Launch flags remain startup facts only. A relaunch is for a dead engine, a weights change, or a draft or mmproj change, none of which reconfigure touches.
• The 27B receipt needs a clean re-run in a serving-down window: these numbers ran beside a second live 27B lane.

Stacked on #70 (S1 RAM pages).

🤖 Generated with Claude Code

https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo