agent-shepherd

A supervisor that watches an AI coding agent while it works and injects one short corrective sentence when it drifts — mid-loop, before the next wasted tool call, not in a review afterwards.

Two tiers do that work. Tier 0 is deterministic detectors that run on every event at zero cost: they never call a model and never guess. Tier 1 is an optional LLM step scorer that scores recent steps for on-goal / justified / verified, and only wakes when Tier 0 has seen evidence — so the expensive reviewer is asleep while the agent is healthy. A verdict is admitted through a risk gate fitted from observed outcomes, not through whatever confidence the judge self-reported.

That same gate is also the teacher. Every Tier 0 nudge that reached the agent is labelled by the next Tier 1 review against the gate's current line — reinforce on agreement, demote on disagreement — so the house rules the agent reads at prompt start evolve from Tier 1 scoring instead of from compliance alone.

It supervises QwenPaw, Claude Code and Codex, appends an auditable ledger of everything it saw, and ships an offline counterfactual benchmark so "is it worth its cost?" is a command you can run rather than a claim in a README.

• Quick start
• How it works
• What it catches
• Adapters
• Judge backends
• Configuration
• Audit and replay
• Measuring it
• Running it as a service
• Uninstalling
• Troubleshooting
• Development

Quick start

# the supervisor. not on PyPI — either of these two, both checked:
uv pip install "git+https://github.com/CallMeHFK/agent-shepherd.git"   # or @v0.4.0 to pin
pip install https://github.com/CallMeHFK/agent-shepherd/releases/latest/download/agent_shepherd-0.4.0-py3-none-any.whl  # no git client

shepherd start-bg              # daemon on 127.0.0.1:4890; writes ~/.shepherd/config.yaml on first start
shepherd install qwenpaw       # or: claude | codex — writes hooks, never clobbers them blind
shepherd doctor                # what is configured, what is missing, what to do next

Restart the agent you just hooked. Then check the two things that matter:

shepherd status                # up — and which checkout the code was loaded from
shepherd stats qwenpaw <session-id>   # after a session: what it saw, what it cost

Tier 0 needs no model and no key: with no judge reachable, the detectors still run and the benchmark still scores them (that is exactly what CI does). Adding a judge buys the second tier — see Judge backends.

To take it all off again later: shepherd uninstall all — one command, no hand-editing, see Uninstalling.

How it works

flowchart LR
    A["agent: reasoning + tool calls"] -->|"middleware (QwenPaw)<br/>hooks (Claude, Codex)"| B["daemon :4890"]
    B --> C["Tier 0 detectors<br/>loop · regression · off-spec<br/>binding · context-rot · CUSUM drift"]
    C -->|clean| Z["pass — nothing injected"]
    C -->|needs judgement| D["Tier 1 LLM step scorer<br/>wakes on evidence, not on schedule"]
    C --> E{"admission gate<br/>thresholds fitted by<br/>conformal risk control"}
    D --> E
    E -->|nudge| F["one sentence back into the loop<br/>user turn · additionalContext"]
    E -->|"block / escalate"| G["only for an agent that opted in<br/>block_enabled: true"]
    B --> H[("append-only ledger<br/>~/.shepherd/sessions/&lt;agent&gt;/&lt;session&gt;.jsonl")]
    F -.->|"followed repeatedly"| I["house-rules header<br/>injected at prompt start"]
    E -.->|"each review labels the Tier 0 nudges it covered:<br/>score clearing the fitted line → reinforce,<br/>falling short → demote"| I
    I -.-> A
    A -.->|"daemon unreachable → fail open"| Z

Loading

Four edges in that picture are the design:

• Fail open. An unreachable supervisor is invisible, never fatal. No adapter call to the daemon is allowed to raise, and hooks are built so a crash on the shepherd side cannot break your agent.
• Injection, not logging. A nudge becomes a new user turn (QwenPaw's INTERRUPT_AND_CONTINUE) or hookSpecificOutput.additionalContext (the hook adapters). The agent reads it as its own context, at the point where it still changes what happens next.
• Evidence spends the budget. The judge is woken at natural checkpoints and, mid-iteration, only as the drift statistic climbs toward its alarm line — never on a schedule.
• The tiers supervise each other. The risk gate already decides whether a judge verdict is admitted; that same fitted line now labels every Tier 0 nudge that reached the agent, so the rulebook evolves on Tier 1 agreement instead of on compliance alone. Behaviour answers "was it followed?", Tier 1 answers "was it right?" — a rule only survives both.

What it catches

Tier 0 signals, all deterministic and all free:

Signal	What it actually sees	Default	Knobs
loop	the same tool call ≥ 3× in a sliding window of 8, exact or near-duplicate (token Jaccard ≥ 0.7, same tool name)	on	fixed constants in core/rules/detectors.py
regression	a pytest/lint/build-shaped command that succeeded earlier in this session and now fails	on	—
offspec	an edit outside the session's delegation contract: the path was never named in the goal, matched by an allow-glob, or already seen this session	deny-globs block; nudging on merely-unobserved paths is off	scope_deny_globs, scope_allow_globs, scope_nudge_unobserved
binding	right tool, wrong entity — acting on a near-miss sibling of the file you resolved (similarity judged on the filename, so test_x.py vs x.py is not a confusable pair)	on	binding_enabled, binding_similarity (0.6)
contextrot	long reasoning with no tool activity for ≥ 60 s of wall clock, i.e. spinning rather than streaming	on	min_quiet_seconds (constructor default)
drift	a CUSUM alarm over the outcome stream, with the alarm line calibrated by Monte-Carlo to a false-alarm budget over a session-length horizon and a self-starting per-session baseline	on	drift_enabled, drift_target_fpr, drift_horizon, drift_adaptive_baseline, drift_watch_fraction, drift_unknown_weight

Cross-cutting machinery that is not a detector but changes what you experience:

Mechanism	What it does	Knobs
outcome classification	every tool result is failed / ok / unknown from structured evidence (exit codes, failure hooks) before any text is pattern-matched, so grep error logs/x is not a failure and a lost response is not a success	drift_unknown_weight weights unknown below failed
anti-nagging	repeat nudges from the same detector are suppressed inside a cooldown; blocks and escalations never are	nudge_cooldown_seconds
wake gate	clean iteration boundaries are not reviewed at all, because measured cost was ~23 % of all events waking a judge on healthy sessions	review_clean_iterations
admission	the two score thresholds are priors until enough labelled outcomes accumulate, then conformal risk control replaces them with a finite-sample fit; a key with too little evidence is dropped rather than pinned to a fallback	nudge_threshold, block_threshold, risk_target_fpr, risk_min_samples
rulebook	guidance the agent demonstrably followed is promoted into a short header at prompt start instead of re-nagged mid-run	rulebook_enabled, rulebook_budget_tokens
self-evolution	every Tier 0 nudge that reached the agent is labelled online by the next Tier 1 review — agreed when the judge's drift score clears the risk gate's admission line, disagreed otherwise. Agreement reinforces the rule in the ranking, repeated disagreement demotes it and (past 4 labelled windows under 50 % concordance) vetoes it from promotion	rulebook_judge_feedback

The derivations, the papers behind them and the negative results are in RESEARCH_NOTES.md.

Adapters

Adapter	Install	What lands where	How a verdict gets back to the agent	Worth knowing
QwenPaw	shepherd install qwenpaw	bundle copied to ~/.qwenpaw/plugins/agent-shepherd	an agentscope MiddlewareBase sees every reasoning delta and tool call in-process; a StopGate asks for a verdict at each text/stop boundary and injects guidance as a new user turn (INTERRUPT_AND_CONTINUE) — a verdict that fires mid-tool-call is parked and drained at the next such boundary, because the runner cannot inject there	highest fidelity, and it needs a QwenPaw restart to load. No plugin? The zero-install REST polling client (adapters.qwenpaw.rest_client.QwenPawRestClient) watches 127.0.0.1:19999 and resolves Tool Guard approvals instead
Claude Code	shepherd install claude	PreToolUse / PostToolUse / Stop / UserPromptSubmit in ~/.claude/settings.json	shepherd-hook claude translates the verdict into hookSpecificOutput.additionalContext / decision	one short-lived hook process per event, so the daemon must be up or the hook fails open
Codex CLI	shepherd install codex	the same events in ~/.codex/hooks.json, in Codex's schema: a top-level hooks map of matcher groups with {"type": "command"} handlers	additionalContext, or a top-level decision: "block"	Codex only runs hooks it has persisted as trusted — see below

Codex hook trust. The interactive TUI prompts the first time it sees the new hooks; approve once and later sessions run them. Non-interactive codex exec silently skips untrusted hooks, so automation has to opt out of the persisted trust explicitly:

codex exec --skip-git-repo-check --dangerously-bypass-hook-trust "..."

Installing any adapter is conservative: the previous settings.json / hooks.json is copied to a timestamped .bak first, an unparseable existing file is refused rather than replaced, foreign hook groups are left alone, and a previous QwenPaw bundle is moved to ~/.qwenpaw/plugin-archive/ instead of being deleted. shepherd uninstall keeps the same posture in reverse — see Uninstalling.

Import the bundle from a release instead of a checkout

Every v* release attaches the bundle as a zip named after the plugin, so an agent can install it without this repository anywhere on disk:

qwenpaw plugin install \
  https://github.com/CallMeHFK/agent-shepherd/releases/latest/download/agent-shepherd.zip

That URL is permanent — the bundle is named after the plugin, not the release, because the version the host displays comes from the plugin.json inside it. To install one known release, swap latest for the tag (.../download/v0.4.0/...). A local path works identically (qwenpaw plugin install agent-shepherd.zip), and so do the app's plugin routes: upload the zip, or hand it the URL. Running QwenPaw hot-loads it; a stopped one loads it on the next start. The archive holds exactly what shepherd install qwenpaw writes — including a README.md that says what the bundle does not bring — and a test asserts the two trees cannot drift apart.

Three things worth knowing about that asset:

• A bundle is not a supervisor. The zip is the observer only; it POSTs to 127.0.0.1:4890 and fails open when nothing listens there, so it loads, registers, and supervises nothing. The same release carries the daemon's wheel and sdist for that half — see Quick start.
• Not the source archive. GitHub's auto-generated archive of a tag unpacks to agent-shepherd-<version>/, and the manifest sits at agent_shepherd/adapters/qwenpaw/plugin.json inside it — both installers look in the archive root or one top-level directory, so it is rejected. The release asset is the only installable zip here.
• The version is inside, not in the name. That is what lets releases/latest/download/agent-shepherd.zip be a permanent URL.

Judge backends

The judge is any OpenAI-compatible chat endpoint. The default is Agnes (agnes-3.0-flash via the Agnes AI Hub); a local gateway needs no key at all — leave SHEPHERD_JUDGE_API_KEY unset and no Authorization header is sent, and loopback/LAN endpoints bypass proxy environment:

export SHEPHERD_JUDGE_BASE_URL=http://127.0.0.1:19991/v1
export SHEPHERD_JUDGE_MODEL=<name from that endpoint's /v1/models>

shepherd config judge lists the model names the configured endpoint actually accepts, which beats guessing one.

Configuration

One human-editable file, ~/.shepherd/config.yaml, written on the daemon's first start. You are not meant to hold it in your head: every setting has an owner (file, env:NAME, or default) and the CLI says which.

shepherd doctor                          # entry point: adapter paths, judge reachability, daemon, thresholds
shepherd config show                     # every effective setting and where its value came from
shepherd config set policy.drift_watch_fraction 0.5    # one setting, no editor
shepherd config merge                    # add settings an older config file predates
shepherd config prune                    # drop settings this release no longer reads
shepherd risk                            # which admission thresholds are in force, prior or calibrated

judge.* also reads SHEPHERD_JUDGE_BACKEND / _BASE_URL / _MODEL / _API_KEY / _TIMEOUT, which override the file; ${VAR} inside the file is expanded, and an unset variable resolves to empty rather than to the literal text. Keys retired by a release are reported STALE by doctor instead of being silently ignored, because a dead setting in a config file looks exactly like a live one.

Three more environment variables point the pieces at each other: SHEPHERD_HOME moves the whole runtime directory (config, ledger, rulebook, risk model; default ~/.shepherd), SHEPHERD_PORT overrides the port the daemon binds, and SHEPHERD_DAEMON_URL tells the hooks, the QwenPaw plugin and shepherd status/doctor where the daemon listens when it is not the default http://127.0.0.1:4890.

The starter config, in full

judge:
  backend: agnes
  base_url: https://apihub.agnes-ai.com/v1
  model: agnes-3.0-flash
  api_key: ${SHEPHERD_JUDGE_API_KEY}
  timeout: 30
policy:
  nudge_threshold: 0.6          # admission prior; replaced by a conformal fit once outcomes accumulate
  block_threshold: 0.85
  max_guidance_tokens: 500
  fail_open: true
  nudge_cooldown_seconds: 300
  drift_enabled: true
  drift_target_fpr: 0.05        # budget over drift_horizon re-checks, not one window
  drift_horizon: 120
  drift_adaptive_baseline: true
  drift_watch_fraction: 0.6     # wake the judge this far below the alarm line
  drift_unknown_weight: 0.75    # a lost response counts 75% as much as a failure
  review_clean_iterations: false
  binding_enabled: true
  binding_similarity: 0.6
  scope_nudge_unobserved: false # ~1 false nudge per healthy session, and new test files trip it
  scope_allow_globs: []
  scope_deny_globs: ["*.env", "*.pem", ".git/*", "*.secret*"]
  risk_target_fpr: 0.05
  risk_min_samples: 20
  rulebook_enabled: true
  rulebook_budget_tokens: 160
  rulebook_judge_feedback: true # Tier 1 scores + risk gate evolve the rulebook online
agents:
  qwenpaw: {enabled: true, block_enabled: false}
  claude:  {enabled: true, block_enabled: false}
  codex:   {enabled: true, block_enabled: false}
port: 4890

block_enabled is per agent and defaults to off: guidance is advisory until an agent asks to be stopped.

Audit and replay

Every observation and verdict is appended to ~/.shepherd/sessions/<agent>/<session>.jsonl — append-only, never rewritten.

shepherd replay qwenpaw <session-id>
shepherd tail qwenpaw <session-id>
shepherd stats claude <session-id>     # events, judge calls, judge tokens, nudges, suppressions

stats is what makes the cost question answerable per session instead of per hunch.

The daemon's whole wire surface, for anyone writing a fourth adapter: POST /ingest/<agent> (one normalized event in, one verdict out), GET|POST /pending/<agent>/<session> (drain a verdict that fired mid-event and was parked for the next gate boundary), GET /stats/<agent>/<session> (the live cost counters of one session), and GET|POST /health.

Measuring it

shepherd eval --seed 7 --sessions 8 --steps 60 --fail-under-f1 0.6

eval runs an offline counterfactual benchmark: scripted healthy sessions plus a fault injector that breaks exactly one thing at a known step, scored for per-detector precision and recall, first-detection attribution, detection delay in steps, false-alarm rate and supervision cost. It is deterministic, offline, Tier-1 stubbed, and runs in CI as a gate — a threshold change that makes the supervisor nag healthy sessions fails the build.

Because the benchmark knows by construction whether a session drifted, it is also the only place the admission thresholds can be fitted against a real label:

shepherd eval --judge --fit-risk [path]

Fitting from live sessions uses a weaker label — "did a Tier 0 detector also confirm drift" — which trains the judge to agree with the detectors rather than to be right.

Running it as a service

shepherd start              # foreground
shepherd start-bg           # detached; writes ~/.shepherd/daemon.pid and daemon.log
shepherd status             # up / down — down means supervision is silently failing open
shepherd stop               # works however the daemon was started

For a machine that should always be supervising, a systemd user unit ships with the repo (no root required). It is deliberately not enabled by the installer: turning on a background process that injects text into your agents is your decision, not a side effect of pip install.

install -Dm644 packaging/shepherd.service ~/.config/systemd/user/shepherd.service
systemctl --user daemon-reload && systemctl --user enable --now shepherd

shepherd stop stops whichever daemon is actually serving: the start-bg pidfile when there is one, otherwise it asks /health, which reports the daemon's own pid and who owns it. A unit-managed daemon goes through systemctl --user stop shepherd, because the shipped unit restarts on a bare SIGTERM — signalling it directly would leave the machine supervised again two seconds later.

The first start pays the drift detector's Monte-Carlo threshold fit (a few seconds) before it binds; the result is cached in ~/.shepherd/cusum_thresholds.json, so later starts bind immediately.

Uninstalling

One command, per direction — no hand-editing another tool's JSON, no hunting for the daemon process:

shepherd uninstall claude       # or: codex | qwenpaw | all
shepherd uninstall all --purge  # also deletes ~/.shepherd: config, ledger, rulebook, thresholds
shepherd uninstall all --dry-run  # show what would go, change nothing

Uninstall removes exactly what shepherd install wrote. Foreign hooks, settings and plugins are left alone; every edit is backed up to a timestamped .bak first; and a config file that carried nothing but ours is deleted rather than left as an empty husk. The QwenPaw bundle is moved to ~/.qwenpaw/plugin-archive/ (same place install keeps previous versions), never deleted. When no adapter remains, the daemon is stopped for you — otherwise it would sit there with nothing posting to it, or worse, a systemd unit would keep restarting it.

--purge is the only destructive step, and it is opt-in by name: it deletes ~/.shepherd, which is the session ledger, the learned rulebook, the fitted thresholds and your config. Without it those survive an adapter uninstall.

Troubleshooting

Symptom	Cause and check
Agents behave exactly as before	shepherd status — while the daemon is down every adapter fails open, which is by design and completely silent
A new feature looks broken	shepherd status prints code loaded from:. Two checkouts can both be installed and the winner depends on the working directory
Nudges never arrive even though the plugin loads	QwenPaw was not restarted after the install, or a second copy of the bundle is in the scan path: ls ~/.qwenpaw/plugins/ | grep agent-shepherd must print exactly one line
codex exec shows no supervision	untrusted hooks are skipped without warning; pass --dangerously-bypass-hook-trust
Hook calls raise ImportError about SOCKS	proxy environment leaking into loopback calls; daemon calls must use trust_env=False
A config edit does nothing	shepherd config show — the source column says whether a file key is being overridden by an env: one, and doctor flags keys this release no longer reads
Supervision continues after uninstalling one agent	by design: the daemon stays for the adapters that remain. shepherd uninstall all stops it, and a systemd unit is stopped through systemctl so Restart=always cannot bring it back

Development

uv sync --all-extras && uv run python -m pytest -q && uv run ruff check .

Use python -m pytest, not a bare pytest: if the dev extras are not installed, a bare pytest can resolve from outside this environment and import a stale installed copy of the package, reporting green tests against code that is no longer on disk.

CONTRIBUTING.md covers the development setup, the detector/adapter conventions, commit and PR guidelines, and the strict no-real-secrets rule. Interactions are governed by the Code of Conduct. Release artifacts and their rationale are on the releases page; design decisions and the literature behind them are in RESEARCH_NOTES.md.

License

MIT