diff --git a/AGENTS.md b/AGENTS.md index fe3d2d4..8d76838 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -17,7 +17,8 @@ node dist/cli.js run examples/hello.yaml # smoke test, zero cost - **Tests never spawn a real agent CLI.** No test may execute `claude`, `codex`, `opencode`, `enclave`, or `git`, and no test may make a network request. Adapter tests parse fixture strings; engine tests inject stub adapters through the registry argument; `src/handoff/` routes every spawn - including `opencode export` and `enclave push` - through an injected `Exec` seam that tests replace with a fake. - **Never invent cost numbers.** If a CLI does not report a price, record `0`. Do not derive cost from a token count and a price table anywhere in this codebase. - **Checkpoint after every edge crossing.** Not at the end of a batch, not at the end of the run. `CheckpointStore.save` writes a temp file and renames it; never write `state.json` in place. -- **The event log is append-only and unbuffered.** A killed process must leave a readable JSONL log. Do not add buffering or rewrite past lines. +- **The event log is append-only and unbuffered.** This rule governs `.loomgraph/runs//events.jsonl` on the machine that ran the graph. A killed process must leave a readable JSONL log. Do not add buffering or rewrite past lines. +- **The hub's truth is SQLite.** `lg-hub export --jsonl` is a derived, rebuildable artifact. Never make a JSONL file on the hub authoritative, never make the laptop's log a database. - **Graph validation stays loud.** Unknown node ids, cycles, missing budgets and unknown adapters throw with the offending node id in the message. Do not downgrade a validation error to a warning. - **No LLM SDK dependency.** The agent CLIs are the runtime. Adding an API client to `dependencies` is out of scope for this project. - **Adapter output is a contract.** Every adapter returns `{ ok, text, costUsd, raw, error }`. Cost is recorded even when the run failed, because budget accounting depends on it. @@ -36,13 +37,20 @@ src/core/ types, store, events, graph, budget, engine (no CLI concerns) src/adapters/ one file per executor, plus the registry src/commands/ CLI command implementations and pure renderers src/handoff/ the `lg-handoff` bin: session readers, secret scanner, brief renderer +src/hub/ the `lg-hub` bin: HTTP API and SQLite database +src/team/ client-side code for the team hub examples/ graph files that must stay valid (`lg validate`) ``` Nothing under `src/handoff/` may import from `src/core/` or `src/adapters/`. The subtree -owns its own enclave helpers so it stays extractable into a sibling package with a `git mv`. -That is why `buildEnclavePushArgs` exists twice and neither copy should be deduplicated -into a shared module. +owns its own enclave helpers, so it stays extractable into a sibling package with a `git mv`; +once extracted, the hub depends on that sibling package, not on this repo. That is why +`buildEnclavePushArgs` exists twice and neither copy should be deduplicated into a shared +module. + +`src/hub/` and `src/team/` may import from `src/core/` and `src/handoff/scan.ts`; the arrow +never reverses. Do not fork the scanner - the `buildEnclavePushArgs`-exists-twice precedent +covers argv builders, not security gates. Inside `src/handoff/`, one file per job, and the data flows one way: diff --git a/README.md b/README.md index fc94908..5a7612a 100644 --- a/README.md +++ b/README.md @@ -486,18 +486,116 @@ always means "looked at and found nothing". headings. There is no model call, so `pack` works offline and cannot invent a claim. - **`private` visibility only.** A transcript is production data. `--visibility org` and `public` are refused. -- **No signal bus, inbox, or daemon.** That would contradict "Not a workflow server" - below, and an inbox that starts an agent on someone else's laptop is a different product - with a much harder threat model. +- **No inbox.** An inbox that starts an agent on someone else's laptop is a different + product with a much harder threat model, and phase 1 ships no inbox - that is phase 3. + There is a daemon now, and the Hub section below is precise about what it does. Known rough edge: the `enclave share create --json` parser accepts several plausible field names because that stdout shape has not yet been captured from a real invocation. +## The hub + +The third binary is `lg-hub`: an HTTP API in front of a single SQLite database. It +stores what members push and serves reads out of that store - it never runs an agent. +Agents run on the member's own machine, started by the member; the hub has no way to +start one, and that absence is the design. A daemon that can only store and route is a +store, not a scheduler. + +### Set one up + +On the hub host: + +```bash +lg-hub init # create the data dir and hub.db +lg-hub member add alice # prints alice's token once - write it down +lg-hub serve # binds 127.0.0.1:8369 +``` + +On a member machine: + +```bash +lg enroll http://10.0.0.5:8369 # identity lives in ~/.config/loomgraph/hub.json +lg sync --enable # opt this repo in: .loomgraph/hub.json +lg run examples/hello.yaml +lg sync # push one run +lg sync --all # or every run under .loomgraph/runs/ +``` + +The rest of the hub-facing surface: `lg-hub member revoke ` and +`lg-hub member ls` for the roster, `lg-hub export --jsonl` to print the raw stored +lines to stdout for grepping, and `lg-hub export --out ` to write one +`runs///events.jsonl` per run. + +### What the hub receives + +A sync pushes two things: the run's event lines verbatim - the same JSONL that sits +under `.loomgraph/runs//events.jsonl` - and a projection of the run state. The +projection is where content stops. It is built field by field, never as a filtered +copy of the full state, so there is no field a `vars` value or a node `output` could +ride in on: + +- `vars` reach the hub as key names only. +- node `output` never reaches the hub. +- node `error` is path-rewritten, secret-masked, and capped at 200 characters. + +Same error before and after: + +``` +in : command exited with code 1: /home/alice/work/repo/run.sh: token sk-ant-api03-… rejected +out: command exited with code 1: ${REPO_ROOT}/run.sh: token sk-a... rejected +``` + +### Tokens are possession-equals-identity + +Anyone holding a member token is that member to the hub. `member add` prints a token +once and the store keeps only its hash, so it can never be printed again - treat it +like an SSH key, and `member revoke ` is the off switch. + +### `serve` refuses a non-loopback bind + +`lg-hub serve` binds `127.0.0.1` by default and refuses any other host unless +`--behind-tls-proxy` is passed. A bearer token over plaintext non-loopback HTTP is +exactly the credential shape this project's own scanner has a rule for, so a bind that +would put the token on the wire without TLS is an error rather than an option. + +### Two caveats, stated up front + +**Phase 1 does not mask on egress.** The projection is the only gate; whatever does +reach the hub is served back as stored. A repository whose var *key names* are +themselves sensitive should keep sync disabled - redaction-on-read is phase 2. + +**The masking is an allowlist, not a proof.** Error masking reuses `lg-handoff`'s +`SCAN_RULES`, so it catches the shapes those rules know and nothing else. During +development a test canary shaped `AKIA` plus 18 more characters passed through +unmasked, because the rule is `\bAKIA[0-9A-Z]{16}\b` - exactly 20 characters. The +canary was malformed rather than the rule being wrong, but that is exactly the point: a +shape the rules do not cover reaches a team-readable field unmasked. + +### Verified against a live hub + +Both of these were run end to end against the built binary with a live hub: + +- **The export is byte-identical to the local log.** `lg-hub export` reproduces the + ingested lines exactly; it does not re-encode them. +- **A dead hub changes neither the exit code nor the node outcomes.** The hub was + killed mid-exercise; the run was unaffected. + +### What phase 1 does not ship + +- No inbox - that is phase 3. +- No web UI - that is phase 4. +- No briefs on the hub, no encryption at rest, and no redaction on read - all phase 2. + Nothing in phase 1 is encrypted. +- No full transcripts, ever. The handoff refusal stands unchanged: a transcript is a + credential dump, and syncing to the hub does not soften that. + ## What this is not - **Not a model, and not an SDK for one.** loomgraph makes zero API calls of its own and has no LLM SDK dependency. - **Not a replacement for your agent CLI.** It shells out to the CLI you already installed and authenticated. -- **Not a workflow server.** No daemon, no web UI, no cloud, no plugin system in v0.1. +- **Not a workflow server.** A daemon ships in phase 1 - `lg-hub` - but it stores and + routes, and never runs an agent. No web UI (phase 4), no cloud, no plugin system in + v0.1. `lg report --publish` does not change that: it writes a static file and shells out to the `enclave` cli the same way a node shells out to `claude`. If `enclave` is not installed the diff --git a/docs/hub-design.md b/docs/hub-design.md new file mode 100644 index 0000000..ba74fa0 --- /dev/null +++ b/docs/hub-design.md @@ -0,0 +1,1030 @@ +# The team hub — design and threat model + +Status: **proposal, not built.** Nothing in `src/hub/` exists yet. +Author decisions locked before this document was written are in [§1](#1-what-is-locked). +Open questions for the author: [§14](#14-open-questions-for-the-author). + +**Revision 2.** Two questions were put to an adversarial pair of reviews: should the hub +hold a database, and should it hold full conversations? The answers went in opposite +directions and both changed this document. Storage is now SQLite-as-truth +([§6](#6-hub-storage), decision D-3) — revision 1 was wrong, and the specific error is +recorded rather than quietly fixed. Central conversation storage is refused +([§18](#18-decision-record-d-3-and-d-4), decision D-4), on measured evidence from a real +transcript corpus. Three defects the pro-database review found in revision 1's briefs are +fixed: owner-curated brief depth ([§7.2](#72-brief-depth-is-the-owners-choice)), +redaction-on-read ([§7.3](#73-redaction-on-read)), and the admission that this design's +top-ranked threat is uninvestigable with the data it retains +([§9.6](#96-what-this-design-cannot-investigate)). Federated search replaces the central +corpus as the answer to deep search ([§17](#17-federated-search)). + +--- + +## 0. What stops being true + +Three sentences the project can say today, and will not be able to say after this ships: + +| Today | After the hub | +| --- | --- | +| "Not a workflow server. No daemon, no web UI, no cloud." | There is a daemon and a web UI. | +| "No signal bus, inbox, or daemon… an inbox that starts an agent on someone else's laptop is a different product with a much harder threat model." | This is that product. | +| Content leaves a machine only through a print-once, expiring link a human chose to send. | Content leaves on a schedule, to a box, and stays there. | + +That third one is the real change, and the second one is the warning being cashed. The +README already reasoned its way to *not* building this and named exactly why. Building it +is legitimate — it is the author's project and the author's call — but the "much harder +threat model" has to be actually built, not waived by the phrase "approval-gated." Most of +[§9](#9-threat-model) exists because approval-gating answers a question the attack does not ask. + +**What does not change:** loomgraph still makes zero model calls, still shells out to the +agent CLI you installed, and **the hub never runs an agent.** It stores and routes. Agents +run on member machines, under the member's own sandbox, started by the member. + +--- + +## 1. What is locked + +Decided by the author before design; not relitigated here. + +1. **The inbox is approval-gated.** An inbound message never executes anything. It queues. + A human runs `lg inbox accept ` for anything to happen. No auto-dispatch in v1. +2. **The hub ingests run events and distilled extractive briefs only.** No raw transcripts, + under any flag. `src/handoff/scan.ts` is a hard gate on ingest. +3. **One central hub** on a shared server, per-member bearer tokens, each member's CLI + talks to it. + +Locked decision 3 is the single largest risk multiplier in this design: it converts +per-laptop compromise into team-wide compromise, and creates aggregation leaks no +per-brief scanner can see ([§9.4](#94-what-the-scanner-stops-buying-under-retention)). It stays, +so the rest of the architecture compensates: hold as little as possible, never execute on +the hub, and put the real anti-injection defense on the *consuming* machine. + +--- + +## 2. Shape, in one paragraph + +The hub does to the team what `events.jsonl` already does to a run: a server-stamped, +append-only record of what happened, greppable via `lg-hub export --jsonl` and queryable via +SQL. The client's existing event log doubles as the push outbox, so there is no queue and no +spool directory. The existing pure-renderer pattern doubles as the web UI, so there is no +build step and no client JS. `src/handoff/scan.ts` sits in front of everything that ingests +text, and now in front of everything that serves it too ([§7.3](#73-redaction-on-read)). +Only the hub's own storage is a new idiom; the rest is the existing three pointed at a +network. + +--- + +## 3. Component boundaries + +**A third binary, `lg-hub`.** Not folded into `lg` — a long-running daemon would poison +`lg`'s "you run it and it exits" contract. Not folded into `lg-handoff` — that subtree's +import purity is load-bearing. + +``` +src/hub/ the lg-hub bin -> dist/hub/cli.js + cli.ts commander wiring; the only argv reader + server.ts node:http binding; deliberately dumb (see §12 on testing) + handlers.ts (WireRequest, deps) -> WireResponse; where all behavior lives + auth.ts token hashing, member resolution, revocation + storage.ts HubStore: node:sqlite, WAL, one transaction per batch (§6) + inbox.ts message lifecycle transitions + feed.ts feed partitioning and cursor logic + ui/*.ts pure (data) -> html renderers +src/team/ client side + transport.ts the injected Fetch seam (mirrors handoff's Exec seam) + sync.ts cursor logic over events.jsonl + fence.ts untrusted-content fencing <- security-critical, see §8.3 +src/commands/ new thin files: enroll.ts, sync.ts, inbox.ts, wired into src/cli.ts +``` + +**Import rule.** `src/hub/` may import `src/core/` and may import `src/handoff/scan.ts` +one-way. `src/handoff/` still imports nothing outward, so AGENTS.md's rule holds as +written. **Do not copy the scanner.** The `buildEnclavePushArgs`-exists-twice precedent is +for a 20-line argv builder; a security gate must never fork. AGENTS.md needs a line saying +which direction the new arrow points. + +Note what this costs: `src/handoff/types.ts:1-8` keeps the subtree extractable "once a team +fabric exists outside this repo." The fabric now exists *inside* it, so extraction later +means the hub depends on the extracted sibling. Acceptable, but the comment should be +updated rather than left to quietly become false. + +**Exit codes.** `lg`'s team verbs reuse its namespace — sync/inbox failure is `2`, never +`3` or `4`, which stay budget and paused. `lg-hub` gets its own small namespace, per the +`lg-handoff` precedent: `0` clean exit, `1` config or usage, `2` fatal runtime (port bind, +corrupt data dir). + +--- + +## 4. Wire protocol + +HTTP/1.1 + JSON on `node:http`. Client uses global `fetch` (Node ≥ 22) behind the seam. +Zero new dependencies. `Authorization: Bearer ` on everything except `/v1/health`. + +| Endpoint | Method | Notes | +| --- | --- | --- | +| `/v1/health` | GET | unauthenticated; `{ok, version}` | +| `/v1/events` | POST | `{runId, streamId, graphName, state, events[]}` — `state` is a **projection**, see [§4.1](#41-what-a-pushed-state-omits); `events[]` are raw JSONL lines, ordered by `seq` | +| `/v1/briefs` | POST | the four bundle files inline as strings | +| `/v1/feed?after=&limit=50` | GET | newest-first page + `nextCursor`; keyset, see [§6.2](#62-pagination-is-keyset-not-byte-offsets) | +| `/v1/runs/:member/:runId` | GET | stored state + events | +| `/v1/inbox` | POST | send; schema in [§8](#8-the-inbox) | +| `/v1/inbox?state=queued` | GET | addressee is always the authenticated member | +| `/v1/inbox/:id/transition` | POST | `{to, runId?}` | +| — | — | **there is no admin HTTP route.** Member management is CLI-only on the hub host ([§5](#5-identity-and-auth)) | + +### 4.1 What a pushed state omits + +`RunState` is not safe to push as-is, and revision 1 said otherwise by implication. Two of +its fields carry content, not status: + +- `vars: Record` — whatever was passed to `--var`, which is exactly where a + ticket id, an internal URL or a token ends up. +- `nodes[].output: unknown` — the agent CLI's raw stdout, verbatim. + +So [§11](#11-deletion--decision-d-2-revised)'s claim that events "carry status, cost and +timing — not content" is true of the event stream and **false of the checkpoint**. The client +therefore pushes a projection, built on the member's machine before anything leaves it: + +- `vars` becomes `varKeys: string[]` — the key names only. Not a map with the values nulled + out: a nulled slot is somewhere a later change can put a value back, and nothing would + fail when it did. A list of key names has nowhere to put a value at all. The key names are + the useful part for monitoring; the values are the risk, so the shape that can carry them + should not exist. +- `nodes[].output` is dropped entirely. Status, attempts, timing and cost stay; `error` + is kept but **masked and truncated** — see the paragraph below. +- `cwd` is rewritten through the existing `rewritePaths` before it is sent, so an absolute + home path does not become a team-readable field. + +`error` is kept but sanitised, not copied. The adapters build it from exactly the material this +section refuses to publish: `src/adapters/claude.ts:33` embeds 200 characters of raw stdout, +`claude.ts:113` and `codex.ts:120` append stderr, and `command.ts:40` falls back to the prompt +itself. So on projection an error has its paths rewritten, every scanner-known secret shape +replaced by a masked prefix, and its length capped at 200 characters. Dropping it outright would +cost real monitoring value; publishing it raw would undo the rest of this section. + +A batch names its run twice — once at the top level and once inside the projected state — +so the hub **rejects any batch where the two disagree** rather than picking a winner. Left +unchecked, a client could push `runId: "A"` carrying a state describing run `B`, and the hub +would store a row whose status, cost and node table belong to a different run entirely: a +wrong answer that never errors. Identity on the wire is single-sourced by refusal, not by +precedence. + +This is minimization at the source, the same principle as "the readers drop, they do not +carry." A hub that never receives the value cannot leak it, cannot be asked to mask it, and +cannot retain it by accident. If a run's output genuinely needs sharing, that is what a brief +is for — scanned, curated, and revocable. + +**Idempotency uses natural keys, not an `Idempotency-Key` header.** Events already carry +`(runId, seq)` from `src/core/events.ts`. The hub keys them `(member, streamId, runId, seq)` +where `member` comes from the token and **never** from the body. A per-run high-water mark +acks-and-drops anything at or below it. Same seq with different content is `409` plus a +visible feed item — silence about divergence is worse than noise. Briefs are keyed by +`sha256(handoff.md)`. Inbox messages carry a client `crypto.randomUUID()`. + +**Reserve `streamId` in phase 1 even though nothing reads it yet.** It is a random id +minted at `run_started`. Without it, `(runId, seq)` assumes one machine and one history per +run forever; a wiped `.loomgraph`, a copied repo directory, or any future multi-machine +resume produces same-key-different-content and the 409 policy fires noise exactly when the +user is already confused. Reserving the field now is free. Retrofitting it after real data +exists is not. + +**The outbox already exists — do not build one.** `.loomgraph/runs//events.jsonl` is +append-only and unbuffered by hard rule, which is the definition of a durable outbox. Sync +is a cursor over it: + +- `.loomgraph/sync/.cursor` holds the last acked seq, written temp-then-rename. +- `lg run` / `lg resume` hook the **existing** `onEvent` callback already threaded through + `EngineDeps` and used in `src/commands/run.ts`. Batch every 10 events or 5 s, 1500 ms + timeout, and **any failure is one line on stderr and nothing else.** A hub outage cannot + affect a run, its checkpoints, or its exit code. +- `lg sync [runId]` replays from the cursor. **This is the only path that must be correct.** + The live push is best-effort sugar over it. + +The cursor advances only on a 2xx naming `highWaterSeq`, so a cut connection just resends. + +**Ordering is by hub `receivedAt`, never by client `ts`.** Client clocks skew; the feed is +served from the hub's own arrival order. Client timestamps are displayed and labeled as +reported, and no cursor is ever derived from one ([§6.2](#62-pagination-is-keyset-not-byte-offsets)). + +--- + +## 5. Identity and auth + +Enrollment is admin-mediated and print-once, matching the enclave share-link aesthetic the +project already lives with: + +``` +# on the hub host +$ lg-hub member add alice +lgt_a1b2c3d4.<32 bytes base64url> # printed once, never recoverable + +# on alice's machine +$ lg enroll https://hub.internal lgt_a1b2c3d4.xxxx +wrote ~/.config/loomgraph/hub.json (0600) +``` + +The hub stores only `{member, keyId, tokenHash: sha256(secret), scopes, createdAt}` in the +`members` table ([§6.3](#63-schema)). Revision 1 said `members.jsonl`; the table supersedes +it, and there is no members file. `LOOMGRAPH_HUB_URL` / `LOOMGRAPH_HUB_TOKEN` override the file, the same +pattern as `ENCLAVE_TOKEN`. + +**Attribution is server-side, always.** Every stored record gets `member` stamped from the +token's keyId. `HandoffMeta.createdBy` — currently `userInfo().username` in +`src/handoff/commands.ts` — is displayed as *"claims created-by"* at most. Never trust a +client-supplied owner field; that is forgery vector A5. + +**Revocation** sets `members.revoked_at`, and a revoked token resolves to no member on the +next request — no restart, no SIGHUP, no replay. + +**Add a scan rule for the hub token shape before the first token is minted.** This is +non-optional and easy to forget. Members will paste tokens into shells and configs; agents +will read those shells; `lg-handoff pack` will faithfully distil a session quoting one. +The pipeline is *designed* to republish exactly this, and `SCAN_RULES` in +`src/handoff/scan.ts` has no rule for a shape that does not exist yet. Choose the `lgt_` +prefix, add the rule in the same commit. + +**Transport.** `lg-hub serve` refuses to bind a non-loopback address unless +`--behind-tls-proxy` is passed, and says why. Deploy behind Caddy, or on a WireGuard or +Tailscale interface. Bearer tokens over plaintext LAN HTTP are precisely the credential +class `scan.ts`'s `auth-header` rule exists to catch; the project should not ship the +vulnerability its own scanner names. + +--- + +## 6. Hub storage + +**SQLite (WAL, `node:sqlite`) is the hub's truth. JSONL is a derived export.** + +Revision 1 said the opposite, and was wrong in a way worth recording rather than silently +correcting. + +### 6.1 Why revision 1 was wrong + +Revision 1 did not choose JSONL *over* SQLite. It chose **both**: JSONL as truth, plus a +SQLite index, plus `lg-hub reindex`, plus a phase-4 test proving the rebuild was +byte-identical. That is two storage engines, two write paths, and a consistency proof +between them — assembled to avoid one engine that ships inside Node 22. Simplicity was the +stated goal and was not what the design delivered. + +Three specific errors: + +- **"Append-only is a physical property" was false.** Nothing physically prevents `sed -i` + on a `.jsonl` file. Append-only-ness of a file is discipline too. SQLite enforces it + *harder*, because the prohibition can be declared: + `CREATE TRIGGER … BEFORE UPDATE ON events BEGIN SELECT RAISE(ABORT,'append-only'); END;` +- **Tamper-evidence against the hub operator was zero in both designs.** An operator with + root rewrites a JSONL line as easily as a row. The real mechanism is a hash chain, which + revision 1 did not have; it is now a column. +- **`EventLog.read` skipping unparseable lines is correct on a laptop and wrong as server + truth.** `src/core/events.ts` states the intent plainly — "A torn or corrupt line must + not take down the audit trail" — which on a laptop is graceful degradation. As the + server's only copy it means a torn line silently deletes an event from history, and a + rebuild bakes the loss in. Loud corruption beats silent loss. + +**What does not change: the laptop.** `.loomgraph/runs//events.jsonl` stays exactly +as it is, unbuffered and append-only. AGENTS.md's invariant is about the run log, and the +run log is where the greppable-log property actually lives; revision 1 mistakenly read that +rule as binding on a component that did not exist when it was written. The hub is a +different component with a different job. **AGENTS.md needs one added line saying so**, and +saying that the hub's greppable artifact is a derived export, not its truth. + +### 6.2 Pagination is keyset, not byte offsets + +Revision 1's cursor was `base64({day, offset})` — a byte offset into a day-partitioned +file, handed to clients who hold it indefinitely. That is a public API made of the wrong +material: + +- `reindex`, the design's own recovery mechanism, invalidated every outstanding cursor + unless the rebuild was bit-perfect — which is precisely why that test had to exist. The + escape hatch and the pagination scheme were at war. +- Tombstoning ([§11](#11-deletion--decision-d-2-revised)) shifts offsets, and compaction was + rejected, so clients would page through tombstones forever. +- **A wrong byte offset is undetectable.** The client lands mid-line, or skips items, or + repeats them, and nothing errors. + +Cursors are now keyset over `(received_at, rowid)`, which survives rebuilds, retention +purges, schema evolution and reordering. + +### 6.3 Schema + +```sql +PRAGMA journal_mode=WAL; +PRAGMA foreign_keys=ON; +PRAGMA user_version=1; -- bumped by any later column addition; phases 2-4 add tables + +-- the verbatim client line is kept in `json`, so the export in §6.4 is lossless +CREATE TABLE events ( + member TEXT NOT NULL, stream_id TEXT NOT NULL, run_id TEXT NOT NULL, seq INTEGER NOT NULL, + received_at TEXT NOT NULL, kind TEXT NOT NULL, node_id TEXT, + json TEXT NOT NULL CHECK (json_valid(json)), -- the client's line, byte-for-byte + prev_hash BLOB, row_hash BLOB NOT NULL, + UNIQUE (member, stream_id, run_id, seq) +); -- an ordinary rowid table: §6.2's cursor needs rowid to exist +CREATE INDEX events_feed ON events(received_at); -- see the note below on rowid + +-- the hash chain is global and single-writer; the head is updated inside the ingest +-- transaction so it can never disagree with the rows +CREATE TABLE chain_head (id INTEGER PRIMARY KEY CHECK (id = 1), head BLOB NOT NULL); + +CREATE TABLE runs ( + member TEXT NOT NULL, run_id TEXT NOT NULL, stream_id TEXT NOT NULL, + graph_name TEXT, state_json TEXT, high_water_seq INTEGER NOT NULL, updated_at TEXT NOT NULL, + PRIMARY KEY (member, run_id)); + +CREATE TABLE briefs ( + brief_id TEXT PRIMARY KEY, member TEXT NOT NULL, sha256 TEXT UNIQUE NOT NULL, + received_at TEXT NOT NULL, expires_at TEXT, revoked_at TEXT, + key_id TEXT REFERENCES item_keys(key_id), -- null only if encryption is off + handoff_md BLOB, meta_json BLOB, html BLOB); +CREATE TABLE brief_files (brief_id TEXT, path TEXT, PRIMARY KEY (brief_id, path)); +CREATE TABLE brief_shares (brief_id TEXT, grantee TEXT, granted_at TEXT, revoked_at TEXT, + PRIMARY KEY (brief_id, grantee)); + +CREATE TABLE inbox ( + id TEXT PRIMARY KEY, from_member TEXT NOT NULL, to_member TEXT NOT NULL, + subject TEXT, body TEXT, re_json TEXT, proposed_graph TEXT, + state TEXT NOT NULL, created_at TEXT NOT NULL); +CREATE TABLE inbox_history (id TEXT, to_state TEXT, ts TEXT, by TEXT, run_id TEXT); + +CREATE TABLE members ( + key_id TEXT PRIMARY KEY, member TEXT NOT NULL, token_hash TEXT NOT NULL, + scopes TEXT NOT NULL, created_at TEXT NOT NULL, revoked_at TEXT); +CREATE TABLE sessions (sid TEXT PRIMARY KEY, member TEXT, expires_at TEXT); +CREATE TABLE read_marks (member TEXT, kind TEXT, ref TEXT, read_at TEXT, + PRIMARY KEY (member, kind, ref)); +CREATE TABLE access_log (ts TEXT, member TEXT, action TEXT, ref TEXT); +CREATE TABLE item_keys (key_id TEXT PRIMARY KEY, wrapped_key BLOB NOT NULL); + +CREATE VIRTUAL TABLE search USING fts5(member, kind, ref, text); + +CREATE TRIGGER events_no_update BEFORE UPDATE ON events + BEGIN SELECT RAISE(ABORT, 'events is append-only'); END; +CREATE TRIGGER events_no_delete BEFORE DELETE ON events + BEGIN SELECT RAISE(ABORT, 'events is append-only'); END; +``` + +Everything revision 1 hand-rolled becomes a declared constraint: the high-water mark is a +primary key, 409-on-divergence is `INSERT OR IGNORE` plus a compare-on-conflict, the feed +is an index, receipts are columns, the members-file replay is a table, the §11 tombstone is +`revoked_at`. One ingest batch is one transaction — it happened or it did not — replacing +revision 1's four-file interleaving that had to be reasoned about by hand. + +`row_hash = sha256(prev_hash || json)` where `prev_hash` is the value in `chain_head`, +genesis being 32 zero bytes; the head advances in the same transaction as the insert. The +head is published to members periodically. This is the tamper-evidence revision 1 claimed +from file semantics and did not actually have. + +**`events` is written with `INSERT … ON CONFLICT DO NOTHING`, never `INSERT OR REPLACE`.** +`OR REPLACE` is a delete followed by an insert, so it fires the append-only delete trigger +and aborts the transaction. The shortest path to a green test at that point is deleting the +trigger, which is why this is written down here rather than left to be discovered. + +`WITHOUT ROWID` was in revision 1 and is removed: SQLite gives such tables no `rowid` +column, which the keyset cursor in [§6.2](#62-pagination-is-keyset-not-byte-offsets) +requires. Verified against this machine's `node:sqlite` — `SELECT rowid` on a +`WITHOUT ROWID` table fails with `no such column: rowid`. + +The index on `events` is `(received_at)` alone, for the same reason and verified the same way: +`rowid` is not referenceable inside an index expression, so +`CREATE INDEX ... ON events(received_at, rowid)` fails with `no such column: rowid` on SQLite +3.51.3. It is also unnecessary - every SQLite index on a rowid table implicitly ends in `rowid`, +so `(received_at)` already gives the `(received_at, rowid)` ordering +[§6.2](#62-pagination-is-keyset-not-byte-offsets) needs. Two forms of the same mistake: do not +write `rowid` into a `WITHOUT ROWID` table, and do not write it into an index. + +### 6.4 The greppable artifact survives as an export + +`lg-hub export --jsonl` emits exactly revision 1's directory layout — `events.jsonl` per +run, one JSON object per line — reconstructed from the `json` column, which holds the +verbatim client line. The grep audience loses nothing; the query audience gains +`lg metrics`, full-text search, unread state, threading, revocable share grants and +multi-day range queries, none of which are reachable by walking files. + +The export has two modes and neither re-encodes a line. `--out ` writes the layout above, +`runs///events.jsonl`, so identity lives in the path rather than in an envelope. +`--jsonl` writes the raw lines to stdout for grepping, where identity is simply not +representable - a flat stream cannot carry it without corrupting the line, and corrupting the +line is the one thing this export exists not to do. + +**The law for AGENTS.md, inverted from revision 1:** the hub's database is truth; JSONL is +a rebuildable export. The laptop's `events.jsonl` is untouched and remains append-only. + +### 6.5 Operations + +- **Backup** is `VACUUM INTO 'snap.db'` — one statement, consistent. Revision 1's + "`rsync` the data directory" was a live copy of dozens of files mid-write. +- **Recovery** at 2am: `PRAGMA integrity_check`, then restore the last snapshot and replay + from members' local cursors, which are the real durable outbox ([§4](#4-wire-protocol)) + and are unaffected by hub state. +- **Migrations** are cheap because the event payload is an opaque verbatim `json` column; + new client fields need no `ALTER TABLE`. Only hub-side projections migrate. +- **`engines.node` must be `>=22.13`.** `node:sqlite` is behind `--experimental-sqlite` + before 22.13.0, so the current `>=22` would let `lg-hub` crash at import on 22.0–22.12. + It also emits an `ExperimentalWarning` on import at every version tested; the `lg-hub` + bin filters that one warning before importing the store, which requires the import to be + dynamic. Stdout is unaffected either way — the warning goes to stderr. +- **Postgres is not warranted.** One process, ten members at most, embedded synchronous + access, zero-dependency ethos. Revisit at multiple hub nodes or roughly fifty members; + arguing for it now would only discredit the SQLite case. + +## 7. Visibility — decision D-1 + +**The two memos disagreed here, and this is the resolution.** + +The architecture memo said every member reads everything; right-sized for a small team. +The threat memo said private-by-default with explicit scoped sharing, because +`lg-handoff` is private-only and *refuses* `--visibility org`, and because a flat pool +means one leaked token or one XSS drains the whole team's briefs. + +**Resolution: the push is the sharing decision.** + +- Nothing reaches the hub that a member did not push. Sync is **opt-in per repository** + (`lg sync --enable` writes `.loomgraph/hub.json`), never on by default, never global. + A member who never enables sync is invisible to the hub, and that must stay true. +- **Run events, once pushed, are team-readable.** This is the monitoring feature the author + asked for, and enabling sync on a repo is the consent act. Making pushed runs private + would make the feature pointless. +- **Briefs are private to the sender until explicitly shared** to named members, revocably. + A brief is quoted session content — a different asset class from a status table. +- **An inbox message is readable only by its sender and its addressee.** No broadcast. + +So the threat memo wins on briefs and inboxes; the architecture memo wins on run events; +and the granularity that makes both defensible is per-repo opt-in rather than per-item +prompting. What was given up: a member cannot enable sync on a repo and then hide one +embarrassing run in it. Retraction ([§11](#11-deletion--decision-d-2-revised)) is the answer to +that, not per-run visibility flags. + +**Token scopes** (`ingest` / `read` / `admin`) are separate from this and should land by +phase 3. A CI token that pushes events should not be able to read everyone's briefs or send +inbox messages. + +### 7.1 What "conversations" means here — decision D-4 + +The author asked whether the hub should hold a database to share **all conversations** +between members. The database half is [§6](#6-hub-storage); this half is refused, and the +evidence is in [§18](#18-decision-record-d-3-and-d-4). The short form: on the author's own +machine, 61% of real agent transcripts contain a credential shape the existing scanner +already recognises, which is a floor rather than an estimate. Centralising transcripts +means roughly six in ten uploads carrying a known credential shape, permanently, on one +shared box, readable by everyone with a token. + +The counter-design — per-session opt-in, encryption at rest, short retention, access +logging, redaction-on-read — was argued well and defeated by its own requirement: server-side +search, redaction and rendering all need the hub to hold decryptable plaintext, so it +mitigates every threat except A4 while materially raising A4's payoff. Its author's summary +of the position was "I chose the honeypot." On a shared server, that is the wrong choice. + +**What is kept from that argument is [§7.2](#72-brief-depth-is-the-owners-choice), +[§7.3](#73-redaction-on-read), [§9.6](#96-what-this-design-cannot-investigate) and +[§17](#17-federated-search)** — because the objection that briefs are too thin was correct +even though the proposed remedy was not. + +### 7.2 Brief depth is the owner's choice + +Revision 1 inherited `lg-handoff`'s fixed extraction: `firstTurn(user)`, `lastTurn(assistant)`, +`lastTurn(user)`, plus a file list. A sixty-turn session becomes three quoted turns, and the +load-bearing one — Done — is **the agent's summary of its own work**, which the README's own +failure-mode section teaches you to distrust: Claude Code returns `subtype: "success"` with +`is_error: true` on a lapsed session, and a sandboxed verifier can report PASS having read +nothing. The brief keeps the claim and discards the evidence, then tells the reader to +verify every claim against the repo. + +That makes a fixed-shape brief the worst point on the curve: most of the retention risk, +little of the value. The fix is not more content by default — it is letting the person who +was there choose: + +```bash +lg-handoff pack claude --turns 12-31,44 --include-tool-result 27 --session-file +``` + +Still extractive, still no model call, still scanned, still owner-curated. What changes is +that the dead end at turn 23 — "we tried patching `disburse.ts` first and it broke +reconciliation" — can be carried, because that sentence is worth more to the next person +than the summary is. `--turns` without an explicit list keeps today's default. + +The turns and tool-result blocks a reader can request are bounded by what the reader +already extracts; **this does not loosen "the readers drop, they do not carry."** Adding a +field to `DistilledSession` still means deciding it is safe to publish. + +### 7.3 Redaction-on-read + +Scanning only at ingest means a rule added later protects nothing already stored. The +`lgt_` token rule from [§5](#5-identity-and-auth) is the worked example: any token that +leaked before that rule existed is exposed for as long as the store keeps it. + +So stored content is also served through `scanText` + `rewritePaths` masking **at egress**, +on every read path — API, web UI and export. Consequences worth stating: every rule added +in future retroactively protects all history; a finding at read time is logged and surfaced +to the owner rather than silently masked; and the ingest gate stays exactly as it is, since +egress masking is a second layer and not a replacement for refusing to store a secret. + +Cost: reads are no longer a straight file copy. At this data volume that is not a +performance question. + +--- + +## 8. The inbox + +### 8.1 Message schema + +``` +{ v: 1, id: uuid, from: , to: { member: "bob" }, + subject: string, body: string, + re: { member, runId } | { briefId } | null, + proposedGraph: { source: , vars: {...} } | null, + createdAt, state, history: [{to, ts, by, runId?}] } +``` + +**Addressing is person-only.** A repo has no owner who can approve; a run has no inbox. +Both exist only as the optional `re:` context reference. Repo-addressing is the feature +that quietly turns this into a dispatch system, so it is deliberately absent. + +**Ingest gates**, in this order, fail-closed, mirroring `pushCommand`: `scanText` over +`subject`, `body`, `proposedGraph.source` and every var value — reject with masked findings +on any hit; then `parseGraph` on any `proposedGraph`, rejecting invalid graphs at send time +so an acceptor never receives an unrunnable request. Validation stays loud, per AGENTS.md. + +**Lifecycle:** `queued → seen → accepted | declined | expired`, then +`accepted → done | failed`, reported by the acceptor's own sync. Only the addressee's token +may transition its own messages. + +### 8.2 What `accept` actually does — decision D-1b + +The memos disagreed here too. The architecture memo had `accept` write the sender's graph +via `saveGraphSource` and enter the normal `runCommand` path. The threat memo said a +message must never be able to name the task, because that is attack A2 with a green light. + +**Resolution: the acceptor names the graph. The message is only ever data inside it.** + +``` +$ lg inbox show 7f2a # mandatory reading step; see §8.3 +$ lg inbox accept 7f2a --graph ./graphs/triage.yaml +``` + +`--graph` points at a **local file the acceptor already has and trusts.** The message body +is exposed to that graph only as `{{inbox.body}}`, which is materialized pre-fenced +([§8.3](#83-fencing-is-the-load-bearing-control)). The sender's `proposedGraph` is inert by +default; running it requires `--use-proposed-graph`, which prints the full YAML plus +`renderPlan` and requires typing the message id to confirm. There is no flag that skips +`show`, and **there must never be an `--auto-accept`, a trusted-sender bypass, or an +accept triggered by an event.** + +`--cwd` is always the acceptor's. A message may name a repo *remote* as a suggestion and +can never name a local path. + +Inbox-sourced runs default to the most restricted sandbox available and never inherit +`workspace-write` or `bypass`. A message cannot name its own execution mode — sender-supplied +capability is the whole attack with permission attached. + +Progress flows back to the sender through ordinary event sync. No new mechanism. + +### 8.3 Fencing is the load-bearing control + +**This is the most important section in this document.** + +An inbox message is untrusted input authored by someone else's agent, which may itself have +been steered by a web page, a dependency README, or a PR body it read. Approval-gating is a +boolean on *ingestion*; the exploit is in *interpretation*. A human clicking accept is +saying "this looks like real work from a colleague," not auditing an instruction set they +were never shown as an instruction set. Habits decay into muscle memory within a week. + +So `src/team/fence.ts` wraps every inbox-sourced value before any agent CLI sees it: + +- an explicit, un-spoofable delimiter, with delimiter-lookalikes in the body neutralized; +- a preamble stating the content is untrusted third-party data and instructions inside it + are not to be followed; +- every line prefixed, reusing the discipline in `src/handoff/render.ts` — whose `quote()` + exists so "no line of transcript can break out of the quote," and which deliberately + ships **no markdown engine** because an inline-link parser is a way to smuggle + `javascript:` into a page. The same reasoning applies verbatim to inbox content. + +`lg inbox show` renders with the same fence: sender, source run, timestamp, and the entire +body — untruncated — inside a visible quarantine frame labeled *"untrusted message from +<member>; loomgraph did not write this and cannot vouch for it."* No link activation, +no markdown, escape everything. + +**Say plainly, in the README and in `show`'s own output, that fencing is mitigation and not +proof.** Nothing at the prompt layer is a hard boundary against a determined injection. +Fencing lowers the odds, the restricted sandbox bounds the blast radius, and the human is +the last check — the same posture the scanner section already takes. + +--- + +## 9. Threat model + +### 9.1 New trust boundaries + +Today there are two: transcript → readers (the narrowing boundary in +`src/handoff/readers/*.ts`), and bundle → enclave (scan, then constraints, then spawn, in +`pushCommand`). The hub adds: + +- **B1 member → hub.** Every run now has a network side effect, on a schedule, not per + human decision. +- **B2 hub → member.** Entirely new direction. `lg-handoff` has "No pull" as a design + point; this deletes it. +- **B3 member ↔ member, transitively.** Any teammate can author input to my machine. Since + teammates run agents, this is really: **anything any teammate's agent ever read** can + author input to my machine. +- **B4 browser ↔ hub.** A client class with cookies, a DOM, and adversarial text to render. +- **B5 storage at rest.** Aggregated team content, long-lived, on one box. A new asset class. +- **B6 hub operator and co-tenants.** Rooting my laptop gets you my sessions. Rooting the + hub gets you the team's. +- **B7 token custody.** A new secret on N machines — and one the handoff pipeline is built + to accidentally republish ([§5](#5-identity-and-auth)). + +### 9.2 Attack paths, ranked + +**A1 — prompt-injected teammate → my inbox → my agent. (High × Critical; not close.)** +Teammate's agent reads a poisoned page, is instructed to send a hub message, the message +queues, I accept because it reads like plausible colleague work, my agent consumes it as +instructions. Sender authenticated, transport intact, human approved: **every planned +control passes and the attack still lands.** Mitigated only by [§8.3](#83-fencing-is-the-load-bearing-control) +fencing + acceptor-named graph + restricted sandbox. This is what the README's warning was about. + +**A2 — malicious accept. (High × Critical.)** A1's mechanism restated, because +approval-gating is designed as the defense against it and does not defend against it. +Accept gates whether a message enters the workflow; it says nothing about what the message +says once in. Approval authorizes the topic; the payload is in the details. + +**A3 — compromised member token. (Medium × High.)** Possession equals identity, replayable +until noticed. Grants reads per [§7](#7-visibility--decision-d-1) plus forged messages to +every other member — feeding A1 from an authenticated sender, which clears reputation +checks. Uniquely here, the token can leak *through loomgraph's own handoff pipeline*. + +**A4 — compromised hub. (Low × Catastrophic.)** Read everything ingested, impersonate +anyone, inject into every inbox with no injection needed, rewrite the log. The mitigation is +not "trust the hub" — it is that the hub holds as little as possible and cannot itself +execute, which is why A1's real defense lives on the consuming machine. + +**A5 — replay or forgery of events. (Medium × Medium-High.)** Forged `run_finished`, +resurrected states, spoofed run ids. Corrupts the shared record and anything keyed off it. +Countered by server-side attribution and the natural-key high-water mark. + +**A6 — XSS from brief content. (Medium-High × High.)** Briefs are arbitrary quoted model +and user text by construction, rendered in an authenticated origin holding the team's data. + +**A7 — scanner miss, retained forever.** See below. + +### 9.3 Why the scanner still earns its place + +Keep it as a mandatory, **server-side, non-bypassable** ingest gate. It genuinely catches +URL-embedded credentials, `Authorization` headers, vendor-prefixed keys, JWTs, and +`TOKEN=`-style assignments, and — the part that matters most — `scanBundleDir` **fails +closed** via `UNREADABLE_RULE`, so "clean" means "looked and found nothing." Re-run it on +the server even when the client claims clean. + +### 9.4 What the scanner stops buying under retention + +Retention inverts the cost of a false negative. Under handoff a miss sat behind a link that +expired in 7 days and could be revoked. On the hub it sits indefinitely, readable by +everyone [§7](#7-visibility--decision-d-1) admits. Every named gap — AWS secret access +keys, header-less PEM bodies, hex client secrets, non-home absolute paths — becomes +permanent team-wide exposure instead of a week-long single-recipient one. + +And aggregation creates leak shapes that are in no single brief, which a line-oriented +single-file scanner structurally cannot see: + +- **The hub token itself**, until the rule from [§5](#5-identity-and-auth) exists. +- **Cross-brief correlation** — a hostname here, a username there, a ticket scheme in a + third. Individually beneath notice; together, a map of the team's infrastructure. +- **The org graph.** `meta.json` carries `createdBy`, `createdAt`, and + `repo.remote/sha/branch`. Across a team that is an accurate timestamped record of who + touched what in which private repo. No rule flags it, and it is exactly what a departing + employee or an attacker wants. +- **`files.txt` as a source-tree map.** The union of `filesTouched` sketches private + codebases' structure. + +Mitigation is minimization, not more rules: ingest the least identity that works, make +`repo.remote` and `files.txt` opt-in, and be able to actually delete. + +### 9.5 Web UI surface + +Requirements, all v1: contextual escaping on every interpolated value (the existing +`escapeHtml` in both renderers); **no markdown engine**, matching +`src/handoff/render.ts`'s stated rationale; no `innerHTML` on any brief-derived value; +strict CSP `default-src 'none'` with no inline script, so a missed escape cannot execute; +``, already present in the handoff page; +`X-Frame-Options: DENY` and `frame-ancestors 'none'` against clickjacking. + +Note `svg` is in `ENCLAVE_ALLOWED_EXTENSIONS` and SVG is an XSS vector (inline `