Skip to content

feat: autonomous orchestrators p3b — real dispatch (claude --bg workers, stall parking, token budget) #557

Closed

Description

@Jammy2211

Overview

Phase 3b of the autonomous-orchestrators epic: real workers on top of the 3a watcher core. Workers start as claude --bg sessions from one session template, their liveness comes from claude agents --json, a stalled (blocked) worker is parked for a human rather than left waiting, and the brief's token-budget: is consumed from each worker's transcript. Until 3c (one Mind writer) ships, real dispatch is fenced to sandbox: yes briefs.

Plan

  • Session template: claude --bg --name --session-id --permission-mode + a prompt from a template file (start_dev line, end-at-deliverable contract, brief path, .done marker); a test pins that a worker is never bare.
  • Liveness from claude agents --json once per tick: done + marker → released; done without → crashed (3a's re-dispatch-once); blocked past a grace → parked for a human, never killed or re-dispatched.
  • Token budget from each finished worker's local transcript; the brief's token-budget: refuses the next spawn once spent.
  • Worker permission mode: brief key worker-mode:, default auto (human decision 2026-11-03); bypassPermissions always refused.
  • Real dispatch only under a sandbox: yes brief until 3c lands.
  • Witness: fake-mode unit tests + one live sandbox worker (appears in claude agents --json, attachable, lease follows its state, tokens counted, removed after).

Tier: judge — merge mode: human /prm

Detailed implementation plan

Affected Repositories

  • PyAutoBrain (primary)
  • PyAutoMind

Branch Survey

Repository Current Branch Dirty?
./organs/PyAutoBrain main clean
./organs/PyAutoMind main clean

Suggested branch: feature/autonomous-orchestrators-p3b (both repos); worktree root ~/Code/PyAutoLabs-wt/autonomous-orchestrators-p3b/

Implementation Steps

  1. agents/conductors/batch/worker_template.md (new) — the worker prompt with {task_path}, {brief_path}, {lease_dir}, {task} placeholders: /start_dev <path> --auto; end at the deliverable (no timers); read the brief record; never commit to PyAutoMind directly (3c); write <lease_dir>/<task>.done with the PR URLs on finish.
  2. _orchestrator.py — session_command(task, prompt_path, brief_path, lease_dir, mode) → argv (claude --bg --name shift:<task> --session-id <uuid> --permission-mode <mode> <prompt>); RealSpawner.spawn runs it from the workspace root, parses the printed short id, returns agent_id + session_id (no pid).
  3. Liveness backend: one claude agents --json --all call per tick (AgentsView); _reap consults it for real leases (running → heartbeat; blocked → stalled_since, parked after STALL_GRACE_S; done → .done marker decides released vs crashed; absent → gone/crashed), pid liveness kept for fake leases.
  4. Token accounting: transcript_tokens(session_id) sums usage (input, output, cache-creation) from ~/.claude/projects/*/<session_id>.jsonl; counters gain tokens; spawn refused once tokens >= token-budget.
  5. _brief.py — optional worker-mode: ∈ {auto, dontAsk, acceptEdits} (default auto; bypassPermissions refused by name) and sandbox: yes; real spawn refused unless sandbox: yes (lifted by 3c).
  6. Docs: conductor AGENTS.md "The watcher" (real mode, stall parking, token budget); PyAutoMind batches/AGENTS.md documents worker-mode: / sandbox:.
  7. Tests: template/argv never bare; fake AgentsView drives running/blocked/done/gone transitions; token refusal; validator rules. Live witness (opt-in PYAUTO_ORCHESTRATOR_LIVE=1): one sandbox worker end to end, then claude rm.

Key Files

  • PyAutoBrain/agents/conductors/batch/_orchestrator.py, _brief.py, new worker_template.md, AGENTS.md
  • PyAutoBrain/tests/test_orchestrator_dispatch.py (new)
  • PyAutoMind/batches/AGENTS.md

Original Prompt

Click to expand starting prompt

Autonomous orchestrators phase 3b: real dispatch — session template, claude --bg workers, liveness, token budget

Type: feature
Target: PyAutoBrain
Repos:

  • PyAutoBrain
  • PyAutoMind
    Difficulty: medium
    Autonomy: human-required
    Priority: high
    Status: formalised
    Filed: 2026-11-03
    Epic: autonomous-orchestrators
    Phase: 3
    Consequence: judge
    Witness: batch dispatch builds the worker command from the orchestrator's session template (sources, bootstrap, brief path, /start_dev <path> --auto, the end-at-deliverable contract) and a unit test asserts it is never bare; with PYAUTO_ORCHESTRATOR_FAKE=0 on the laptop one claude --bg worker is spawned against a no-op sandbox prompt, appears in claude agents --json, is attachable, and its lease heartbeat follows its liveness; the brief's token-budget: is decremented from the worker's final usage and the N+1th spawn past it is refused; the worker permission mode is declared (decided with the human) and a worker that hits a prompt is reported, not left stalled; pytest PyAutoBrain/tests -q green
    Review-minutes: 20
    Unattended: never
    Lane: local-dev

Parent: draft/feature/pyautobrain/autonomous_orchestrators_epic.md; slice of phase 3 (slicing: complete/2026/11/autonomous-orchestrators-p3a.md, prompt section "Slicing"; 3a shipped as PyAutoBrain#556).

Real workers on top of the 3a watcher core. Decide with the human, at plan time, the worker permission mode (a --bg worker that hits a permission prompt stalls silently: auto mode plus the repo allowlists is the starting proposal). Recheck the CLI facts first (claude --help, claude agents --help; 2.1.316 on 2026-11-03 had --bg, attach, logs, stop, rm, respawn, and agents --json [--all] [--cwd]). Whether --bg sessions show in the desktop app is phase 4's first witness, not this one.

Activity

  1. Jammy2211 commented on Nov 3, 2026

    @Jammy2211
    ContributorAuthor

    Shipped — merged 2026-11-03 via /prm.

    Witness: 17 dispatch + 16 watcher tests; live sandbox worker working → done, 26,510 tokens, removed; full suites Brain 1473 (+1 opt-in live) / Mind 712; CI green first time. CLI findings recorded on the PR (--bg ignores --session-id; state word working; untrusted folders refused fast). Next: 3c one Mind writer (draft/feature/pyautobrain/autonomous_orchestrators_p3c_mind_writer.md), which also lifts the sandbox fence.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions