cache-eviction

Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)

SIEVE cache - lockfree, concurrent cache - faster and simpler than LRU

A comprehensive open-source cache trace dataset

Hardware-level security protocol for mitigating cache-based side-channel attacks.

A prefix-cache advisor for LLM serving infrastructure that recommends KV-cache capacity and eviction policies from your request traces/logs.

A learned, cost-aware eviction policy for semantic LLM caches that replaces heuristic policies like LRU/FIFO with a lightweight offline-trained model, improving regeneration-token savings while maintaining safe LRU fallback.

Trace-driven research harness for KV-cache hierarchy policy evaluation in long-context LLM inference.

Idle but not forgotten: usage-driven agent memory forgets the dormant skills you'll need next quarter; NicheMem's competitive ownership retains them exactly. Mechanism-level benchmark + verified theory (sibling of GAUSE).

CLOCK replacement algorithm, an efficient, compact and concurrent alternative to LRU cache eviction policy.

REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression

Event-driven simulator for prefix KV-cache eviction policies in LLM serving systems

HybridCacheSystem is a modern C++ framework for building high-performance, multi-layered cache systems. It supports both LRU and LFU caching strategies, allows combining them for hybrid setups, and provides a unified interface for cache management and customization.

An implementation of W-TinyLFU policy

Trace-driven simulator for learned KV Cache eviction on synthetic sparse block-level access traces

Educational caching reverse proxy in Go—TTL-based in-memory HTTP cache, size-aware eviction, gzip negotiation, Cache-Control handling, hit/miss headers, and JSON stats.

Deterministic decay-and-eviction policy for bounded agent memory, and the paired-statistics harness that scores it against LRU, LFU, ARC, a branch-aware baseline and a Belady ceiling.

Engineering library for Finite Readout Acceleration (FRA): a quotient framework for deciding when a distinction is safe to stop computing. Companion to Lahtee 2026.

Semantic-cache eviction that refuses to trade a cache miss for a silently wrong answer

Benchmark of KV-cache eviction policies on GPT-2-medium showing two clear regimes: sink-preserving heuristics dominate at tiny budgets, while attention-based eviction becomes near-lossless at moderate budgets. attention_384 matches full-cache quality with ~33% less cache.

FreCoS: freshness- and cost-aware extension to GPTCache (BSc course project)