Repository navigation
Conversation
… written to disk instead of dropped (card f35cbd9d) The engine's host prompt cache (--cache-ram) dropped its oldest entry when full, so a mind whose saved conversation aged out re-prefilled it whole. Engine fork #78 adds --cache-disk DIR / --cache-disk-mib N: evicted entries are written to DIR and read back on a match. Measured on the M5, 2026-10-27: --cache-ram 11961 holding 2.4-6.6 GiB entries; macOS compressed 13-15 GB of the engine (top CMPRS), swap 11.9/13.3 GB; 40+ "removing oldest entry" evictions and one 3,845 MiB state skipped outright. - llama_server/prompt_cache_disk.rs: the dir (kv-pages/prompt-cache/<model>, under the root the kv-pages disk reporter tracks; the engine owns what is inside: removes another instance's states at start, its own at exit, its oldest at its limit), the limit (headroom to the 95% page-write refusal threshold of paging::pool on that volume, from the disk monitor's sample: no new constant), and the --help probe. - Passed only to an engine whose --help lists --cache-disk (same probe as --train-dir): an older engine would refuse to start. Off, with the reason on serving.prompt_cache.disk_tier, when the engine lacks it, the volume is unsampled or the dir cannot be made. Tests: cache_disk_flags_ride_together_only_when_given, cache_disk_is_accepted_only_when_the_engines_help_lists_it. The engine pin moves to the fork commit in its own PR after #78's review. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Serves the minds by keeping every resident's saved conversation restorable instead of re-prefilled: plan line 3b (minds' memory moved by us, GPU <-> RAM <-> disk) and goal 11. Card f35cbd9d.
Measured (the M5, 2026-10-27, engine-c pid 15919)
--cache-ram 11961; entries 2.4-6.6 GiB;top: engine 22-23G resident + 13-15G compressed; swap 11.9/13.3 GB.removing oldest entry, oneprompt state size 3845 MiB exceeds cache size limit 3137 MiB, skipping.Change
--cache-disk DIR,--cache-disk-mib N; RAM-evicted entries spill to DIR, read back on a match.kv-pages/prompt-cache/<model>(tracked root), limit = headroom to the existing 95% page-write refusal threshold (paging::pool), passed only to an engine whose--helplists the flag (same probe as--train-dir). Every off-reason is onserving.prompt_cache.disk_tier.Tests
cache_disk_flags_ride_together_only_when_given,cache_disk_is_accepted_only_when_the_engines_help_lists_it.Measured after deploy
Owner Fable, at the build window after the pin bump: engine log
spilled ... / restored ... in N ms, engine CMPRS, swap, and per-act re-prefill after an eviction. Card stays open until posted.🤖 Generated with Claude Code
https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo