feat(harness): jam agent, a verifier-gated coding agent harness - #14
Merged
Conversation
Specs the first slice of the CodeHarness PRD in ideas/1-spec.md: journal, tool pipeline, execution world, trusted kernel boundary, session and turn model, agent loop, verification engine and evidence ledger. Key decisions recorded: evolve jam-cli in TypeScript rather than a new Rust repo; generic harness before the cross-language impact wedge; authority is not pluggable while everything else is; semantic and telemetry event streams are separate; compaction never mutates the journal. This is the AI surface the v0.12 pivot archived rather than deleted, rebuilt around the authority boundary instead of assistant commands.
TDD task breakdown for sub-project 1. Each task carries its own test cycle and ends with an independently testable deliverable. Tasks 7, 16 and 18 include mutation checks: break each guard, confirm a test fails, revert. A security test that passes against a disabled guard is not a test.
…ects better-sqlite3's native binding cannot load (built for Node 20 ABI 115, running Node 26 needs 147) and cannot be rebuilt without network. The harness now uses the built-in node:sqlite; the package keeps engines >=20 and jam agent fails fast below Node 22.5. Pre-flight scan of the plan also caught: - Verification commands were split on whitespace, so a quoted command like node -e "process.exit(1)" made node evaluate a string literal and exit 0. A failing check would have reported success. Now runs via the shell. - CheckpointStore was built but never wired, leaving checkpointId always empty. The loop now checkpoints before each mutating batch. - Tool gains a mutates flag so the loop knows which batches need one.
Review of Task 1 found the plan's own reference implementation used raw Date.now(), so a backward clock step (NTP, VM resume) reset the counter and emitted a smaller timestamp than the previous id, breaking the ordering the event journal depends on. Clamp with Math.max(Date.now(), lastMs), and add the two boundary tests whose absence let it through.
- Clamp clock to never go backward using Math.max(Date.now(), lastMs) - Update spin-wait to compare against clamped value for correct exit condition - Add test for clock regression: ids still sort despite backward clock step - Add test for counter overflow: handles multiple exhaustions with advancing time
Re-review found the clock clamp introduced a stall: lastMs never decays, so accumulated backward-clock debt makes the counter-exhaustion spin-wait burn CPU for the entire debt duration. Borrowing a millisecond instead removes the stall class rather than bounding it, and drops the recursion. Also replaces the counter-overflow test, which never reached the branch it was named for (peak counter 999 against a 4096 threshold) and passed identically against the broken implementation.
- Replace spin-wait with borrow mechanism: when counter exhausts, advance lastMs by 1ms and reset counter - Removes stall risk from accumulated clock debt; uses future timestamp instead - Write timestamp from lastMs (not now) so borrowed millisecond is reflected in id - Add resetUuidv7State() test seam for proper test isolation - Fix counter overflow test to actually test the borrow branch: * Freeze Date.now() to single constant value * Generate 5000 ids (well past 4096 counter limit) * Assert timestamp advances via borrow by comparing first and last id timestamps
Nothing asserted the 48-bit timestamp, so its byte order and offset were unverified: swapping writeUIntBE for writeUIntLE left the whole suite green. Consecutive millisecond increments almost never cross a 256 boundary, so little-endian survives the sort test even though the big-endian field is the entire basis of cross-millisecond ordering. Parse the field back out of the rendered id and bound it by the wall clock either side of the call. Watched fail against the writeUIntLE swap, green against writeUIntBE.
vitest 1.6.1's vite-node strips the node: prefix from every builtin except node:test, then fails to resolve bare sqlite, so a static import breaks every test touching storage. resolve.alias, server.deps.external and ssr.external were all tried and cannot work, because the strip happens before config applies. src/harness/sqlite.ts loads the driver via createRequire and becomes the single place the driver is obtained. Also drops setState's TerminalState | string, which collapses to string and trips no-redundant-type-constituents.
Append-only session event journal: RuntimeEvent union, JournalEvent, and Journal (append/replay/createSession/setState/listSessions) backed by node:sqlite. Events are ordered by a per-session LogicalClock plus UUIDv7 ids, never a positional sequence integer. DatabaseSync is loaded through a small createRequire shim (src/harness/sqlite.ts) instead of a direct node:sqlite import: the installed vitest 1.6.1's bundled vite-node strips the node: prefix from every builtin except node:test before resolution, which breaks node:sqlite specifically (it has no legacy unprefixed alias, unlike fs/path/crypto). The shim is a workaround for that test-tooling gap, not a runtime behavior change.
… test Review proved two defects in the plan's own code. preview() capped retained error lines at 20 with no marker, so a stack trace with 25 assertions lost 5 without saying so - the exact guarantee it was written to uphold. It now reports how many it omitted. The dedup test compared two digests, which are sha256(content) computed without touching storage, so it passed with PRIMARY KEY dropped and INSERT OR IGNORE weakened to INSERT. It now asserts the stored row count.
…orld
Review found two real defects in the plan's own reference code.
addEventListener('abort') never fires on an already-aborted signal, so a
caller that aborts before invoking run() waited out the entire timeout and was
told aborted: false. Measured at 5007ms against a 5000ms timeout. Now
short-circuits before spawning.
exitCode -1 meant both 'binary could not start' and 'process was killed with
no exit code'. Task 15's verifier keys 'requirement not executable' off that,
so a timed-out verification command would have been reported as
COMPLETED_UNVERIFIED instead of COMPLETED_PARTIAL. ProcResult now carries
spawnFailed and the verifier uses it.
…ilure from killed process
…catch Review verified toJsonSchema silently typed z.object, z.enum and z.union as 'string' - the exact schema/validator drift that generating from zod is meant to prevent. It now throws on a shape it does not model, and arrays carry items. safePath swallowed every realpath error, not just ENOENT, so a symlink loop or an EACCES on an intermediate directory returned success. A boundary guard that fails open is not a boundary guard. Only ENOENT now passes. The schema test registered a single tool shape, so a hardcoded toJsonSchema passed it. Replaced with one covering six field kinds plus an unsupported one.
…ath errors Address code review on task 6: toJsonSchema now uses a jsonTypeOf helper that maps each zod field to its real JSON Schema type (including enum values and array items) and throws on any shape it does not model, instead of silently defaulting unknown shapes to string. safePath now treats every realpath error other than ENOENT as a refusal instead of letting it fall through as allowed. Added tests: multi-field schema derivation, refusal on an unmodeled nested shape, and a symlink loop inside the workspace.
Verified hole in the single most important rule in the design. apply_patch touching .jam/ is denied outright, but run_command with sh -c 'echo ... > .jam/config.yaml' reached only approval_required, so the one categorical rule degraded to a prompt the model can talk its way past. Two causes, both fixed. run_command was absent from the mutating-tool set even though tools/types.ts documents it as workspace-mutating. And the guard scanned Object.values for strings, while run_command's args is an array, so it never looked at the payload at all. The scan now recurses into arrays and nested objects, the segment match is separator-normalised and anchored so .jamfile is unaffected, and reading .jam through read_file stays allowed.
run_command was absent from the mutating-tool set even though tools/types.ts documents it as workspace-mutating, and the guard's string scan only looked at Object.values(input), which never sees strings inside run_command's args array. Together these let a shell command reach .jam/config.yaml with only an approval prompt instead of the categorical deny the design requires. Widen the mutating set to include run_command, recurse into arrays and nested objects when collecting strings to check, and normalize backslashes before matching so .jam\config.yaml is caught alongside .jam/config.yaml. run_command referencing .jam/ at all is now denied outright, read or write, since telling the two apart needs real command parsing; read_file still covers legitimate reads.
Verified bypass. apply_patch with a header naming .JAM/config.yaml returned unconditional allow - not even an approval prompt, because apply_patch is hardcoded R1. On macOS and Windows the filesystem is case-insensitive, so git apply then modified the real tracked .jam/config.yaml. One character defeated the one categorical rule in the design. The guard now lower-cases before matching.
The guard was case-sensitive while the filesystem is not: apply_patch naming .JAM/config.yaml or .Jam/config.yaml reached unconditional allow instead of deny, since apply_patch is hardcoded risk R1 and R1 allows once the guard fails to match. Confirmed end-to-end that git apply on a patch naming .JAM/config.yaml modifies the tracked .jam/config.yaml on a case-insensitive filesystem (the macOS and Windows default). This predates the run_command fix and is more severe, since it reaches full allow rather than a prompt. Lower-case each candidate string, after backslash normalization, before testing against PROTECTED_SEGMENT.
Review found read_file and list_dir throw an uncaught EACCES when a path exists but is unreadable, violating the never-throw-for-expected-failure constraint. Dispatch's catch-all would turn that into 'internal, recoverable: false', strictly less actionable than telling the model it hit a permission wall. Added a shared fsError errno mapping. git_diff had no tests at all: removing its artifact storage entirely, so the whole diff returns inline, left all 64 tests passing. That is exactly the failure preview() exists to prevent.
read_file and list_dir threw an uncaught EACCES/EPERM when a path exists but is unreadable, since the stat guard only checks existence. Added a shared fsError errno mapper (types.ts) so permission and I/O failures come back as structured sandbox.denied / internal results instead of throwing, matching the never-throw-for-expected-failure constraint. git_diff had no test coverage at all. Removing its artifact storage left every other test passing, which is exactly the failure preview() exists to prevent. Added tests for the no-repo error path and for the artifact-plus-preview behavior; verified by temporarily stripping the artifact call and confirming the new test fails, then restoring it.
…ed, refuse to record outcomes after cancellation Threading the abort signal into verification opened a path to a false-positive COMPLETED_VERIFIED: cancelling cleanly between two requirements left a partial results array whose entries all passed, and satisfied never checked that every declared requirement had actually run. Require results.length to equal the declared requirement count before satisfied or runnable can be true. Also recheck the signal immediately after evaluate() returns so a cancelled session never gets a terminal state, belt and braces alongside the verifier fix.
The test I specified pre-aborted before evaluate() ran, so results stayed empty and the pre-existing length>0 check already forced satisfied:false. It proved nothing about the completeness fix - it exercised 'abort before verification starts', not the disaster window of an abort BETWEEN requirements after the first has passed. It now wraps subprocess.run to abort after the first requirement resolves, so the array really does hold one passing entry out of two declared.
…of pre-aborting The prior test aborted before evaluate() was ever called, so the break guard fired on the first requirement and results stayed empty -- the pre-existing results.length > 0 check already covered that case regardless of the completeness fix. Replace it with a test that aborts strictly between two requirements, after the first has passed, which is the actual window a partial-but-passing results array becomes possible.
Wires the harness (journal, verifier, loop, tools) into a runnable `jam agent [task]` command and adapts jam's existing provider layer to the harness's ModelProvider seam. Adds --task-file, --verify, --json, --max-tool-calls and --timeout flags, and maps terminal session states to process exit codes (0 verified, 1 partial/failed, 3 unverified, 4 cancelled). loadRequirements is now wrapped so a malformed .jam/config.yaml fails with a clear message and exit 1 instead of an unhandled rejection. AgentOptions.dbPath lets tests redirect the journal/artifact store away from the real ~/.jam/harness.db.
runAgent fell back to CANCELLED whenever no terminal event existed, so a session that ran out of tool calls, tokens or wall clock was reported exactly as if someone had pressed Ctrl-C. Writing no terminal event is correct for both - the session stays resumable either way - but runTurn already returns the StopReason saying which, and runAgent was discarding it. The report now names the cause and tells the user how to resume.
runTurn already returns the StopReason for a budget-exhausted turn (max tool calls, max tokens, or wall-clock deadline), but runAgent discarded it and fell back to the same CANCELLED label used for a real Ctrl-C. Both cases correctly write no terminal event so the session stays resumable, but a user whose run hit its budget was being told they cancelled it. Capture the StopReason and surface it via describeStop() in the human-readable report, with a resume hint shown only for a stopped (not finished) session. Exit code is unchanged: both cancellation and budget exhaustion still map to exit 4, since exitCodeFor only knows about terminal states, not stop reasons.
My own line rendered 'CANCELLED - budget exhausted (max_turn_requests)', which still tells the user they pressed Ctrl-C. state is only the hardcoded fallback when no terminal event exists, so when a cause is known it should replace the placeholder rather than prefix it.
renderReport was prefixing the stop cause onto the CANCELLED literal
("CANCELLED - budget exhausted (max_turn_requests)"), which still told
the user someone pressed Ctrl-C. state is only ever a hardcoded
fallback when no terminal event exists, so the cause should replace
it, not sit next to it.
Added a genuine-cancellation integration test through runAgent (an
abort-aware provider plus a simulated SIGINT, no fixed-delay race) to
confirm the real Ctrl-C path reads "cancelled by user" with no
CANCELLED literal anywhere, and strengthened the COMPLETED_VERIFIED
report test to check the exact terminal-state line and the absence of
a resume hint.
…race Verified against the real binary: an unknown provider, a real provider that lacks tool calling, and Node below 22.5 all crashed with a raw Node stack trace. Only loadRequirements had a guard. The Node version check exists specifically to print an actionable message and did the opposite. One try/catch now covers the version guard, config loading and provider construction - everything that can throw before a session exists. Also stops the stop-report naming --resume, a flag that does not exist.
A real-binary review found that an unusable --provider (an unknown name, or one that structurally lacks tool calling) and a pre-22.5 Node runtime all crashed runAgentCommand with a raw stack trace, exiting 1 only because that is Node's default for an unhandled rejection, not because exitCodeFor decided anything. The Node version guard in particular exists specifically to print an actionable message, and this gap defeated it. Wrap config loading and provider construction in runAgentCommand in one try/catch (return await, so a rejection cannot escape it) and print the same clean "cannot start" message loadRequirements already used. Also replace the stop report's reference to a --resume flag that does not exist in index.ts with the session id, and add an integration test that drives a mutating tool through a real git repo to cover the checkpoint-per-mutating-batch wiring, which previously had no test that would fail if it were silently dropped.
The boundary added last round covered the provider and the version guard, but the task-file read sat above it, so jam agent --task-file /nonexistent still crashed with a stack trace. A mistyped path is as ordinary a mistake as a mistyped provider name and deserves the same one-line message.
The previous startup-boundary fix left readFile(taskFile) and the empty-task check above the try block, so a mistyped --task-file path still crashed with a raw ENOENT stack trace. A mistyped path is at least as common as a mistyped provider name, so it gets the same guard: move both inside the try, print the same clean "cannot start" message on failure. The blank/missing-task early return stays a clean return, not a throw, so it does not go through the catch.
Covers every attack that was once live during this build: the goalpost attack via patch and via shell (including case variants like .JAM/), lookalikes that must not be denied, workspace escape via traversal and symlinks, indirect prompt injection projected as untrusted tool output, authority that cannot be escalated (R4 denied outright, fail-closed with no approver), a full audit trail with no gaps, and a snapshot that governs verification even if .jam/config.yaml is rewritten out of band. All 27 tests pass against current production code, no defects found. Each of the four named guards was manually disabled and confirmed to break its regression test, then restored. One gap found and fixed along the way: the no-approver test taken from the brief (AutoDenyApprovalHost) does not isolate applyFailClosed, since that host also denies via request() independently. Added a dedicated test using a host that is unavailable but would rubber-stamp anything if asked, which does isolate the fail-closed guard.
Verified live against unmodified production code: run_command never calls
safePath, and cat/head/tail/grep/find are R0, so
run_command({command:'cat', args:['/etc/passwd']}) returned ok:true with the
real contents and no approval prompt. A file outside the workspace leaked the
same way. The workspace boundary that stops read_file reaching ~/.ssh simply
did not apply to the shell tool.
Full confinement is the sandbox's job in sub-project 2 and stays deferred, but
R0 auto-allow for a path that leaves the workspace is a classification choice
made here, and the kernel knows workspaceRoot. Such calls now require approval
rather than running silently.
run_command never called safePath, and cat/head/tail/grep/find are R0, so cat /etc/passwd (or any path outside the workspace) was auto-allowed with no policy check and no approval prompt at all. The boundary that stops read_file reaching outside the workspace never applied to the shell tool. DefaultPolicy now escalates any MUTATION_CAPABLE call (apply_patch, write_file, run_command) whose arguments reference an absolute path outside workspaceRoot, or walk out via .., to approval_required. It does not deny outright: full confinement stays the sandbox's job, but a path that leaves the workspace must at least reach a human first, and the existing fail-closed behavior still denies it when nobody is available to ask. Extends the adversarial security suite with end-to-end dispatch coverage for this path, a symlink-loop test for safePath's previously-uncovered non-ENOENT branch, and a strengthened absolute-path traversal case. Every new and existing guard was manually disabled and confirmed to break its matching test, then restored; the full suite passes again afterward.
Review demonstrated both end to end with real leaked content.
node -e "require('fs').readFileSync('/etc/passwd')" was R1 auto-allow: the
path lives inside the code string, so no argument-level path check can ever see
it. Interpreters given an inline-code flag are now R2, so a human looks. Running
a script FILE stays R1 - that is ordinary work.
escapesWorkspace now resolves each argument against the workspace root instead
of pattern-matching. That handles absolute paths, .. walks and Windows drive
letters uniformly, and stops src/../src/index.ts - which never leaves - from
prompting.
Still open and now stated plainly: a workspace-local symlink pointing outside.
The policy layer is pure and cannot stat the filesystem, so that one is the
sandbox's job in sub-project 2.
…resolution Two more auto-allowed leaks found by review, both demonstrated end to end with real content returned. node -e "require(fs).readFileSync(/etc/passwd)" (and python3 -c, ruby -e, and similar) classified as R1: the path lives inside the code string, so no argument-level check, including the run_command escape guard added in the previous fix, can ever see it. classifyRisk now treats an interpreter given an eval-style flag as R2, ahead of its normal tier, while leaving a plain script-file invocation such as node scripts/build.js at R1. Separately, the escape guard from the previous fix pattern-matched for a leading slash or a literal .. segment instead of resolving the path. That missed a Windows drive-letter path entirely (a real bypass, since verify already branches on win32) and falsely flagged src/../src/index.ts, which never leaves the workspace, forcing a needless prompt on ordinary work. Rewritten to resolve every path-shaped argument against the workspace root uniformly, with an explicit check for a drive letter, which a posix workspace root can never contain. Extends the security suite with matching coverage for both, confirms ordinary work (npm test, npm run build, running a script file, git diff, in-workspace relative paths, and-prefixed apply_patch diffs) still runs unprompted, and re-runs every guard in the suite to confirm none lost coverage. Also corrects an earlier report claim that one existing safePath branch had no test coverage; it already did, predating this task. A workspace-local symlink pointing outside, read with no .. in the argument at all, still leaks through run_command at R0. The policy layer has no filesystem access to see it. Recorded as a known gap for the sandboxing work in a later sub-project rather than half-fixed here.
Drives a real fixture repo through search, read, a failing test run, apply_patch, a passing test run, and stop, then asserts the session reaches COMPLETED_VERIFIED only because the Verifier ran the declared requirement and it passed. Also asserts a fresh NaiveContext rebuilds identical model-visible history from the journal alone.
The vertical slice test creates a real git repo per test via mkdtemp and never removed it, leaving jam-e2e-* directories in the OS temp dir after every run.
demo-full.sh, demo-raw.gif and an unpublished blog draft were pre-existing untracked files with no connection to the harness. My first commit on this branch used 'git add -A docs/' and took them along. demo-full.sh printf's hardcoded fake 'jam trace --impact' output. A script that fabricates tool output does not belong in a branch whose entire claim is that completion must be backed by real evidence.
…ries A NaN budget silently disables the cap because every >= comparison against NaN is false, so --max-tool-calls oops or --timeout oops let a run go unbounded. positiveIntOr validates both flags and throws inside the existing startup boundary instead. loadRequirements accepted verification.required: ["npm test"], the most natural YAML a user would write, because it parses to bare strings which pass the array check. Each entry then has neither command nor gitDiffCheck, so the Verifier silently skips it and the session reports COMPLETED_UNVERIFIED with no error at all. Each entry is now validated to be an object with a non-empty command or gitDiffCheck: true, naming the offending index when it is not.
apply_patch's input is a single opaque unified-diff blob. stringsIn returned the whole blob as one string, and resolve(root, wholePatch) split it on slashes, so a benign patch containing a deep relative import anywhere in a diff line tripped the workspace-escape check and returned approval_required, which applyFailClosed turns into a hard deny in CI. The escape check now applies only to run_command, whose args are real path-like arguments; the .jam/ protection is unaffected and still covers apply_patch unconditionally. Also drop write_file from MUTATION_CAPABLE. No such tool is registered, and listing it read as coverage that does not exist.
Budget.check() returned max_turn_requests for both the tool-call cap and the wall-clock deadline, so a 15s --timeout run and a --max-tool-calls 0 run printed the identical "budget exhausted (max_turn_requests)" in the human-readable report. Added a distinct 'deadline' StopReason, returned only from the wall-clock branch, and describeStop now renders it as "time limit reached".
…on abort AdaptedProvider.generate() checked signal.aborted once, then awaited chatWithTools with no way to cancel it, since jam's ProviderAdapter interface takes no AbortSignal. So Ctrl-C could not interrupt the single longest operation in the loop, the in-flight model call. generate() now races the real call against the abort signal so it resolves promptly on abort. This does not cancel the underlying request: the real chatWithTools call keeps running in the background regardless of which side wins, and its eventual result or rejection is discarded once abort has already been reported (the losing promise gets its own no-op catch so a later rejection never surfaces as an unhandled rejection). AdaptedProvider is now exported so it can be constructed directly against a fake ProviderAdapter, and provider-factory.ts had no test file at all until now. New coverage: the abort race (and that the background call is provably still running, not cancelled), the 'tool' -> 'user' role remap, the tool-call id fallback to array index, and the tool-support guard in createHarnessProvider rejecting a provider without chatWithTools or with supportsTools: false. Also fixed two no-unnecessary-type-assertion lint errors introduced in loadRequirements (src/harness/verify.ts) by an earlier fix in this branch: the forEach callback is now typed unknown, since the entries are only claimed to be Requirement by an unsafe cast on js-yaml's output and a bare string really does reach it as a string at runtime.
…on verified runs restore(id) looked its ref up through an in-memory `meta` Map, so it could never work across processes even though the checkpoint id is readable from the journal alone. It now derives the ref path directly from the id (refs/jam/checkpoints/<id>), verifies the ref actually exists first, and throws a clear "Unknown checkpoint" error if not. meta is kept as-is for list(). create() also wrote a permanent refs/jam/checkpoints/<uuid> on every mutating batch, a dozen refs from one run, immune to git gc. Added prune(), which deletes every ref this store created and reports how many. runAgent calls it from its finally block only when the session reached COMPLETED_VERIFIED, since that is the only outcome with nothing left that could need rolling back; every other terminal state leaves the checkpoints in place, and the human-readable report now says how many were kept so a permanent ref is never left silently.
The CHANGELOG said COMPLETED_VERIFIED means "every declared verification requirement ran and passed" -- true but misleading. The verifier snapshots a requirement's command as TEXT at session start, not what that text resolves to, so a model that rewrites what the command resolves to (for example package.json's scripts.test) can still make the frozen command report success. Amended the entry to say so plainly. This is the ordinary reward-hacking failure mode, not an exotic attack. Added a `jam agent` section to README.md: what it does, a config example, the Node 22.5 requirement, and the same verification limitation. Exit codes are documented from the actual exitCodeFor switch in src/commands/agent.ts (0 verified, 1 partial/failed, 3 unverified, 4 stopped) -- there is no separate "policy violation" exit code in the current implementation; a denied or escalated tool call is fed back to the model as a recoverable tool result rather than ending the session on its own, so it does not need one. Also added a regression test locking in that the human-readable report prints "Changed:" above the terminal-state line for every outcome (already true in code; this closes the gap in coverage).
82 rulings taken on the maintainer's behalf across 19 tasks, each with what it costs if wrong, plus every parked and deferred item. The working ledger was gitignored scratch; these decisions should outlive it.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Moved to docs/superpowers/, which .gitignore already covers. The design doc, implementation plan and decision log are working artifacts, not deliverables.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds
jam agent, a coding agent harness whose completion is decided by a deterministic verifier rather than by the model.The model stops calling tools. The harness then runs the verification commands declared in
.jam/config.yamland decides the outcome. A session reachesCOMPLETED_VERIFIEDonly when every declared requirement ran and passed. If none are declared,COMPLETED_VERIFIEDis unreachable by construction.This is sub-project 1 of 5 from
docs/specs/2026-08-29-harness-core-design.md. It is the AI surface the v0.12 pivot archived rather than deleted, rebuilt around the authority boundary instead of assistant commands.Fixes #
Type of Change
Changes Made
node:sqlite, UUIDv7 ids, with a separate disposable telemetry stream so streamed tokens can never bloat durable historyExecutionWorldseam for all filesystem and subprocess work, so a sandbox can be swapped in later without touching any tool. Kills by process groupread_file,list_dir,search_text,git_diff,apply_patch,run_command.apply_patchis the only mutation primitive.jam/mutation, risk classification R0 to R4jam agentCLI with headless--jsonand documented exit codesnode:sqliteis used instead ofbetter-sqlite3because the latter's native binding is compiled per Node ABI and cannot be rebuilt offline. This is whyjam agentneeds Node 22.5+ while the package keepsengines: >=20. Reasoning is in the design doc, section 14.1.Screenshots / Terminal Output
Nothing in the Verification block is generated text. Each line renders from a
VerificationResultwith a real exit code and digest.Testing
npm test— all tests passnpm run typecheck— no type errorsnpm run lint— no lint errorsOn the two unchecked boxes.
npm testreports 601 passed, 30 failed. All 30 failures are pre-existing and unrelated:src/trace/*andtrace-smokefail becausebetter-sqlite3's native binding was built for Node 20 (ABI 115) and this machine runs Node 26 (needs 147). This branch touches zero files in those areas, andmainfails the same way.npm rebuild better-sqlite3fixes it.npm run lintlikewise carries 41 pre-existing errors in 5 files this branch never touches.New code is green on its own: 221 tests across 23 files, all passing.
Test Instructions
Checklist
Additional Notes
Two limitations worth knowing before merge
The verification snapshot freezes the command text, not what it resolves to. Requirements are captured at session start and the model cannot edit
.jam/config.yaml. But for anpm testrequirement it can rewritepackage.json'sscripts.testtoexit 0, and the verifier will faithfully run the frozen string and get 0. This is the ordinary reward-hacking failure mode, not an exotic attack. The changed-files list prints above the verdict so a human reading the report sees it, but--jsonand the exit code do not carry it. The CHANGELOG and README say this plainly.run_commandis not workspace-confined. A workspace-local symlink pointing outside still leaks at R0. Absolute paths and..escapes now require approval, and interpreters given inline code (node -e,python3 -c) are no longer auto-allowed, but real confinement is the sandbox in sub-project 2.Known test-integrity gaps
Two tests in the final fix wave do not fail when their target regresses, and are recorded rather than papered over: the workspace-escape regression fixture's
../sequences cancel out so it passes against the broken code, and the checkpoint prune guard has no test asserting the git ref actually survives a non-verified run. Both are correct in source and verified by hand.Decision log
docs/specs/2026-08-29-harness-core-decision-log.mdrecords all 82 decisions taken during the build, each with what it costs if wrong, plus every parked and deferred item. Read it before picking up sub-project 2.🤖 Generated with Claude Code