fix: three silent-success bugs, plus lg report - #2
Merged
Merged
Conversation
execa 10 declares `engines: {"node": ">=22"}` and calls
`Set.prototype.union`, which does not exist before node 22. The ci matrix
still ran node 20, so every adapter test file crashed at import with
`TypeError: TEXT_ENCODINGS.union is not a function`.
Node 20 reached end of life in april 2026, so the fix is to drop it rather
than pin execa back.
- package.json engines: >=20 -> >=22
- ci matrix: [20, 22] -> [22, 24]
- tsup target: node20 -> node22
- readme and contributing state node >= 22
claude -p --output-format json now emits an array of message objects instead of a single result. the parser read subtype off the array, failed every agent node with "subtype unknown", and recorded 0 usd for a run that had really spent money - so the maxUsd ceiling went blind. take the last type=result element, keep the legacy single-object form, and carry total_cost_usd onto every return path including failures.
a command that exits 0 having done nothing was recorded as succeeded. npm run lint --if-present in a repo with no lint script is the case: it passed a fan-in gate having verified nothing.
pure argv builder and json parser, tested against output captured from a real enclave push. the binary is spawned without a shell so a title cannot be reinterpreted as shell syntax.
lg report <runId> writes the checkpoint and event log to one html file with no external requests. every interpolated value is escaped - a node whose output contains a closing tag would otherwise inject markup. --publish shells out to enclave; a missing binary still writes the file.
the ceiling was a pre-dispatch gate only, so the final batch could overshoot without limit: a real run spent 0.5371 usd against a maxUsd of 0.30, finished succeeded, exited 0, and emitted no budget_exceeded event. completed nodes keep their results - the run fails, the work does not unwind.
the reference regex class excluded '-', so {{nodes.my-node.output}} never
matched and was left in the prompt verbatim. the node reported succeeded
and the run exited 0, contradicting the readme's promise that an
unresolvable reference is an error.
a dot collides with the reference syntax, which splits on '.', so a dotted id is unrepresentable in a template. reject it at validate rather than let it mean nothing at run time. the END check keeps its own message and still runs first.
the model was whatever each cli's own resolution produced. with no -m, opencode ignored the configured model and fell through to a provider with no credentials, so the adapter could not run at all. OPENCODE_MODEL is ignored - the flag is the only way in. a model on a command or human node is a validation error: zod strips unknown keys, so it would otherwise vanish silently.
… cost opencode reports a price per step under --format json; the default format prints prose and no price. loomgraph used the default, so every opencode node recorded 0.0000 usd and was a permanent blind spot inside maxUsd.
datj9
force-pushed
the
fix/claude-array-json-and-report
branch
from
August 16, 2026 23:17
0e567ed to
ff3903f
Compare
datj9
force-pushed
the
fix/node-22-floor
branch
from
August 16, 2026 23:17
75bdfd5 to
0f17c7c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four bugs and one feature. Three of the four bugs are the same shape: loomgraph reported success while doing the wrong thing. Every one was found by running the tool, not by reading it — the unit suite was green throughout.
1. The claude adapter was broken against every real run
claude -p --output-format jsonnow emits a JSON array of messages, not a single result object:parseClaudeJsonreadsubtypeoff the array, got nothing, and failed every agent node withclaude run ended with subtype unknown— recording $0.0000 for a run that had really spent $0.26.The cost side is the dangerous half:
maxUsdaccounting went blind, so a retry loop could burn real money against a ceiling that believed it had spent nothing. This violated theAGENTS.mdrule "Cost is recorded even when the run failed, because budget accounting depends on it."Fix: take the last
type: "result"element, keep the legacy single-object form working, and carrytotal_cost_usdonto every return path including failures. Fixtures are captured from a real invocation.Worth knowing:
claude --helpstill documents the old single-result shape. The contract and the behaviour disagree with no version note, which is exactly whyAGENTS.mdrequires fixtures from a real binary rather than from docs.2. Command nodes passed having run nothing
npm run lint --if-presentexits 0 in a repo with no lint script. That node was recordedsucceededand satisfied awhen: all_succeededfan-in gate having verified nothing.Adds two opt-in assertions to
commandnodes:3. A run could report success having blown its budget ceiling
checkBudgetran only before each dispatch batch, so the final batch could overshoot without limit. A real run:179% of the ceiling, zero
budget_exceededevents. The README promised "Hitting a ceiling stops the run with status failed... and exit code 3" — that never fired when the overshoot happened during the last batch rather than before one.Fix: re-check all three ceilings before a run may finish
succeeded. Completed nodes keep their results — the run fails, the work does not unwind:4. Hyphenated node references passed through as literal text
The reference regex class was
[A-Za-z0-9_.]— no hyphen. Node ids were unconstrained, somy-nodewas legal but its reference could not be tokenized:The node succeeded, the run exited 0.
README.mdpromised "An unresolvable reference is an error, not an empty string" — it was neither. An agent node would have been billed to reason about a literal template placeholder.Hyphenated ids are the natural style for this tool:
run-tests,type-check,fix-bug.Fix, in two parts: widen the class to accept
-, and constrain node ids to[A-Za-z0-9_-]{1,64}. A dot is excluded because references split on., making a dotted id structurally unrepresentable — now rejected atlg validateinstead of meaning nothing at run time.Together these give a property the README can now state: every template reference reachable from a valid graph either resolves or throws. No third outcome.
Feature:
lg reportRenders a run's checkpoint and event log to one self-contained HTML file — zero external requests, every interpolated value escaped, deterministic output.
--publishshells out to theenclaveCLI to host it, the same way a node shells out toclaude.A missing
enclavebinary is not a failed report — the file is still written, exit 0. This gives loomgraph a web view without a daemon, a server, or a cloud dependency, which keeps the "What this is not" section honest.Verification
lg report --publishwith a real URL · budget gate at zero cost · hyphen fix with the original probeEach of the 12 commits has a green
npm run typecheck && npm testbehind it. Escaping is tested against a hostile graph name and a node error containing a closing tag — a node's output reaches the HTML, so that was an injection path.Notes for review
{ ok, text, costUsd, raw, error }unchanged.src/adapters/types.tsuntouched.enclaveis never spawned in a test.0.[A-Za-z0-9_-]characters are affected; every id inexamples/is unaffected.5. OpenCode was a permanent $0 blind spot inside every budget
The adapter used OpenCode's default output format, which prints prose to stdout, the model banner to stderr, and no price anywhere. So every opencode node recorded
0.0000and could never move amaxUsdceiling.[RUN] Under
--format json, opencode 1.18.17 emits JSONL and reports a real price:{"type":"step_finish", "part":{"type":"step-finish", "tokens":{"total":58887,...}, "cost":0.003991708}}The adapter now walks that stream — concatenating
textevents, summingpart.costfrom everystep_finish, and keeping that cost on the failure paths too, because budget accounting depends on it. Fixture captured by hand from a real invocation.This also corrects the README and the
lg statusfooter, both of which claimed OpenCode reports no price.6. You could not choose the model
AdapterInputgains an optionalmodel, threaded to all three CLIs —--modelfor claude and codex,-mfor opencode:Without it, each CLI's own resolution decides — and that is not always what the config says. [RUN] With no
-m, opencode ignored the configuredmodelinopencode.jsonand fell through toclaude-sonnet-4-5, which failed withNo active credentials for provider: anthropic.OPENCODE_MODELis ignored; the flag is the only way in.For a tool that sells budgeted orchestration, this matters more than it looks: model choice is cost control.
A
modelon acommandorhumannode is a validation error rather than a silently ignored key — zod strips unknown keys, so it would otherwise vanish without a word. That is the same silent-success family as bugs 1–4.The OpenCode adapter is no longer experimental
It had never been executed against a real binary. It has now been, twice:
The first of those is worth keeping in mind during review: the adapter's error handling was already correct — it surfaced the real CLI failure verbatim rather than swallowing it. What was missing was the ability to pass a model at all.
Verification, updated
lg report --publish(real URL) · budget gate (zero cost) · hyphen fix · opencode adapter with model + cost