Token optimization toolkit for AI coding agents.

File-read cache · Evidence ledger · Verification loop · Workflow orchestration
_{Backed by research across 7 state-of-the-art agent systems}

---

🚀 Why This Exists

Your coding agent wastes 30-50% of its tokens doing the same work over and over:

• Re-reading the same unchanged files (read("main.py") × 5 times in one session)
• Re-discovering the same facts across sessions
• Listing every skill every turn, even irrelevant ones
• Running experiments speculatively instead of by necessity
• No verification before declaring success

This package fixes that. Every module is a proven mechanism extracted from the best coding agent systems in the world — and backed by 659 passing tests.

📦 Quick Install

pip install dsh-agent-efficiency

30-Second Quickstart

from dsh_agent_efficiency import ContextManager, ResearchMemory

# 1. Smart file reads (caches automatically)
ctx = ContextManager("/path/to/your/project")
content = await ctx.read_file("main.py")  # cached if unchanged

# 2. Evidence ledger (persists across sessions)
memory = ResearchMemory("/path/to/your/project")
memory.ledger.record(
    claim="PageRank file selection saves 38% tokens",
    evidence="Measured in benchmark (2026-09-10)",
    confidence="high",
)

# 3. Smart file selection
relevant = ctx.select_files("gaussian splatting rendering")
print(relevant)  # ['gaussians.py', 'rendering.py', ...]

CLI

agent-efficiency status /path/to/project      # Show cache + evidence stats
agent-efficiency record /path/to/project "claim" --evidence "..."
agent-efficiency search /path/to/project "file selection"
agent-efficiency report /path/to/project      # Full report

🧩 Modules

Module	What It Does	Estimated Savings
context_manager/	File-read dedup cache + token budgets + file selector	~13K tokens/session
memory/	Evidence ledger + candidate lifecycle + experiment history	Prevents re-discovery
skills/	Verification loop + workflow orchestrator	~18K tokens/session

context_manager/

Component	Mechanism	Key Benefit
ContextCache	SHA256 + mtime tracking — returns cached content if file unchanged	6× speedup, ~2,812 tokens saved per re-read
SummaryCache	Per-file summaries with keyword search	Read summary instead of full file
TokenBudget	4-tier budgets (4K/16K/48K/96K) with category-level allocation	Never exceed context window
FileSelector	Summary search + filename keyword scoring	Smart file relevance ranking

memory/

Component	Mechanism	Key Benefit
EvidenceLedger	Append-only JSONL claim store (hypothesis/verified/refuted)	Evidence > Intuition — no unverified claims
CandidateRegistry	7-status lifecycle (NEW→SCREENING→KEEP→VERIFY→DROP/BLOCKED/MERGED)	Never lose track of what you tried
ExperimentHistory	Hash-based dedup, verdict tracking	Never re-run the same experiment

skills/

Component	Mechanism	Key Benefit
VerificationLoop	Gated phases: inspect→test→verify→conclude	No "modify → declare success"
MinimalExperimentPolicy	Sanity→micro→reduced→full escalation	Never over-experiment
WorkflowOrchestrator	Parallel task dispatch with dependency resolution	Independent work runs concurrently

🧠 Backed by Research

Every mechanism in this package was extracted from real source code of 7 state-of-the-art systems:

System	Key Pattern Borrowed
Codex CLI	Typed context fragments, dual-path compaction, Guardian reviewer
Superpowers SDD	Evidence ledger, file-based artifact passing, 5-round fix loop
OpenHands	Progressive-disclosure skills, two-tier MEMORY.md, resource-locked parallelism
SWE-agent	Tool bundles, history processor chain, cheap-doer + expensive-judge
Roo-Code	Per-task shadow git, non-destructive condense, tree-sitter folded context
ECC	Pre-edit fact-forcing hooks, gated verification loop, confidence-weighted memory
Aider	Architect/editor model split, PageRank repo map, signal-driven reflection

📖 Full 84-mechanism analysis: docs/agent_efficiency_prior_art.md

📊 Measured Performance

Context cache:      6.01x speedup  |  10 hits, 10 misses  |  ~2,812 tokens saved per re-read
Evidence ledger:    100 claims in 0.05s  |  Search in 15μs
Candidate registry: 50 candidates in 0.04s  |  Status update in 0.03s
Experiment history: 30 experiments in 0.02s  |  ✅ Dedup verified
Verification loop:  Cycle in 0.0008s  |  ~0 overhead
Workflow planner:   4-task plan in 0.0001s  |  Dependency resolution in 8μs

Estimated per-session savings: 31,000 tokens ($0.62 at mid-range pricing)

✅ Tests

pip install -e ".[dev]"
pytest tests/ -v

# Output: 659 passed in 2.55s

🔮 Roadmap

• Core modules: context cache, evidence ledger, candidate lifecycle, verification loop
• 659 passing tests, all modules benchmarked
• 7-project prior art analysis (84 mechanisms cross-referenced)
• CLI tool (agent-efficiency)
• DSH extension (auto-loads on DSH startup)
• PyPI package (pip install dsh-agent-efficiency)
• Integration with Claude Code / Aider via hooks
• Token usage dashboard (ECharts + FastAPI)

📄 License

MIT — use it, fork it, ship it.

---

⭐ Star this repo if you build coding agents and hate wasting tokens.