Skip to content

Token economics: see, bound, and prove what a run costs its own context window - #111

Merged
stempeck merged 4 commits into
mainfrom
issues/110
Sep 16, 2026
Merged

stempeck merged 4 commits into
mainfrom
issues/110

Conversation

@stempeck

Copy link
Copy Markdown
Owner

Summary

This change gives the factory an accounting of what each run costs its own context window, and a policy surface that acts on it. Constrained local backends stop wedging under parallel sub-agents, session start stops silently dropping mail and memory, the grader and watchdog stop punishing agents for doing what the harness told them, and there is one verb that can say whether a change actually helped.

Closes #110

What you get

Sub-agents can no longer starve their orchestrator on a shared backend. Declare the backend's shared pool size on a model profile, and a sub-agent launch that would oversubscribe it is refused before it starts, with the arithmetic shown to the agent. Cloud profiles are untouched: with no pool declared, refusal is impossible by construction. A strict one-at-a-time mode holds the slot for the child's whole lifetime, survives crashes and teardowns, and admits exactly one contender when the slot frees.

Efficiency works on every profile, not just cramped ones. A step that historically generates more than it needs gets a reduced effort level on its next session, chosen from history rather than from remaining room, never above what the profile declared, and bounded so a run cannot spend itself relaunching. Self-edits from the improvement loop are checked so a formula cannot buy tokens by deleting a gate.

Session start delivers everything, every time. Formula context, mail, and memory each get their own budget, so a large step body can no longer evict your mail. Identity comes from the agent's own CLAUDE.md, re-rendered at every provisioning site and held byte-equal to its template by a test. Each message reaches a session once; later prompts stop re-injecting the whole mailbox.

You can tell whether a run was normal, and whether a change helped. Every closed step records what it generated (output, thinking, peak, sub-agent spend) beside its timing, with unmeasured figures as null rather than zero. af telemetry band judges each step against learned medians and prints the band it used. af telemetry compare is the one verb allowed to claim an improvement, over two arms of five runs, and voids rather than fails when the arms were not held constant. af telemetry rebuild keeps learned data alive across record rotation. af tokenomics status names every reason the surface is inert. Off is the default; turning it off is operator-only.

The harness stops fighting itself. The fidelity grader is shown every intervention taken during a turn and grades compliance as compliant. Grader sub-processes no longer fire the agent's hooks or take its session id. The watchdog does not recycle an agent that was told to wait. Halted breakers, dark channels, and a dead watchdog show on the pane, in af statusline status, and at af up, whether or not the escalation mail ever arrived.

Recurring work without a triggering issue. dispatch.json accepts a crons list of name, agent, cadence, and vars. Schedules survive restarts, back off with a bound, are validated against the target formula at write time, fire without GitHub access, and appear in af dispatch status.

mergepatrol trusts GitHub, not local git. A PR counts as merged only when GitHub says so. The formula never closes a PR itself or pushes a local landing, and a test fails if those semantics are edited away.

Formulas and skills learned from measured runs. design-plan-impl must justify every phase beyond what the design narrates. design-v7 builds its snapshot by redirection, cites true source lines, and never stashes or checks out in a shared worktree. fable-increment and fable-review cope with shared stash stacks and harnesses that lack run_in_background. improve-agent gains an efficiency review. Two new skills: improve-solution propagates design decisions into derived docs, and perfeval-agent finds an agent's single highest-impact cost from telemetry.

Also in this change

  • New operator guide USING_TOKENOMICS.md; updates to USING_AGENTFACTORY.md, USING_TELEMETRY.md, USING_MODELS.md, and USING_MEMORY.md; a new ADR on the session-start context surface and an amendment to the hooks-never-block ADR enumerating the one gate that may refuse.
  • Hook stdin no longer blocks on a silent pipe or socket, on macOS and Linux.
  • Teardown-call scanner keyed by per-line marker instead of line numbers; exec-capable temp dirs for tests that plant binaries; dark-channel and watchdog-latch tests.

Notes for review

  • The deployed agent workspaces under .agentfactory/agents/ were not regenerated in this commit, so TestDeployedAgentIdentityMatchesEmbeddedTemplate and TestDeployedAgentSettingsMatchEmbeddedTemplate fail for all 26 agents until af install --init is run and the result committed. Every other root and web test passes.
  • USING_TELEMETRY.md and two code comments still say nothing reads the learned digest. af telemetry band and the admission predicate do read it.
  • Only mergepatrol bumped its formula version; the other six edited formulas stay at version 1.
  • After upgrading the binary, re-provision with af up or af install --init so the split SessionStart entries are in place; a stale single-entry settings.json puts three concatenated JSON objects on one stdout.

🤖 Generated with Claude Code

stempeck and others added 4 commits September 15, 2026 22:29
…own context window and act on it. Adds a pre-act sub-agent dispatch gate over declared backend pools, efficiency actuators independent of window room, split SessionStart writers with per-session mail delivered-state and identity parity gates, generation telemetry with band and compare verbs, grader visibility of harness interventions, a watchdog latch and alarm terminus, dispatcher crons, mergepatrol merge authority, and formula, skill, doc, and test hardening. Closes #110.
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ stempeck
❌ agentfactory


agentfactory seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The factory cannot see, bound, or prove what a run costs its own context window

2 participants