Repository navigation
Prompts that say why commands run from the root, and no longer ban cd nor git -C - #102
Merged
Merged
Conversation
… nor git -C The prompts of the three commands, of the executor and of the reviewer held one rule in two parts: every command runs from the root of the repository, with paths from there, and never `cd` nor `git -C`, "since [...] a `cd` followed by git stops for an approval" and the permission rules do not read `git -C` as the git command it runs. #87 took the rest of how an agent writes a command out of the prompts, and left this ban. With a ban, a session has a breach to report. The last sentence of the last message of a run of `05-fine-cap`, in the fourth campaign of the evaluations: "The slice 2 executor ran one `cd` into the repository root, which the instructions forbid. It made no edits through it." The developer can do nothing with it (#67). The developer decided on 2026-10-07: the prompts keep what the rule is for, and lose the ban and its reason of permissions. - `skills/surface-plan/SKILL.md`, `skills/surface-execute/SKILL.md`: "Every command runs from the root of the repository, with paths from there, since the shell is shared with the agents." - `skills/surface-status/SKILL.md`, which launches no agent: "since one shell serves the whole session." - `agents/surface-executor.md`: the line leaves what the executor never does, and the rule stands where the state script is named, "since you share the shell with the dispatcher". - `agents/surface-reviewer.md`: the same, "with the dispatching command". - `ARCHITECTURE.md` and ADR 0032: the invariant and the sentence of the decision say what the prompts hold now, and why they ban nothing: a ban gave a session a breach to report to a developer who could do nothing with it. Left unsaid, someone would put it back. - `tests/prompts/test_agent_mandates.py`: every prompt that runs commands states the rule with its reason, the shared shell, and none names `cd` or `git -C`, bans a change of directory in other words, nor gives a reason of permissions. The commands the prompts spell themselves still change no directory. What it may cost: an agent that changes directory moves the shell for the commands after it, the dispatcher's included, which call the state script by its path from the root. The rule and its reason say so, and nothing checks it. A developer on the default mode may be asked to approve a command the ban steered an agent away from: their mode's call, as for every other command (ADR 0032). Not measured: no session was played. What to read in the next runs played whole is whether a message still reports a `cd`, and whether a command fails for running from elsewhere.
PierreMardon
added a commit
that referenced
this pull request
Oct 7, 2026
Kept for the next campaign to be compared with: `evals/reports/2026-10-07-9d5de55f57a9/`. It measures the chain on `main` with the nine pull requests of 2026-10-07 that did the ten decisions of the developer (#97 to #105), none of which had been played. The six cases are played twice on one version for the first time: `02-reservations`, `04-suspension`, `05-fine-cap` and `06-borrow-limit` whole, `01-overdue-list` and `03-overdue-reminders` stopped at the hand over. It is also the first campaign judged by the rubric of #104 and counted by the method as it stands. 200 sessions, 30.89 USD at list price, the check of the judge included. `report.md` and `summary.json` are as `report --against <summary> --keep` wrote them, by the harness of #106, #108 and #109, against the fourth campaign. `evals/README.md` says what this campaign is worth to compare with, that it was judged before the two lines the rubric draws for the hand back at conformity, and that its ledger counted 6 points of the weekly gauge where its streams show 3: its runs were played before #106. What it shows: - The eight runs played whole reached `conformant` and passed all of their hidden acceptance tests. - The hand back at conformity is one to three sentences in the eight runs, names the files of the critical zones and ends on the developer's step (#103). `messages.next-step` goes from 3 to 4.4. - No final message reports a `cd` (#102), and one names an agent, in 12 runs (#105). `messages.clear` goes from 2 to 4.4, by a rubric that changed with the prompts. - Every page that holds a rule the plan settled on its own marks it as an assumption that approving confirms (#100). - `05-fine-cap` draws no diagram in its two runs, where it drew one in each (#101), and `blueprint.diagrams` is 5 on the twelve pages. - The check of the judge: the lean page scores 4 and 4 on `blueprint.no-padding`, its padded version 2 and 2, where one point parted them. On the pages of the runs the score is 2 on ten and 3 on two. - What did not move: `following.no-noise`, 2 to 3, on the steps a hand over still tells, checked, pushed, refreshed. - The judge faults the hand back at conformity for not telling what was built, which is what the developer asked of it: the rubric says so in a pull request of its own. - The proof of conformity lists a file of a critical zone under a blueprint that said none in 3 runs of 8 (#107), which `zones_announced` tells since #108. - No blueprint holds a title inside a section, and nine of twelve have a section of more than two paragraphs without one (#63, measured since #109). - Planning costs 0.98 to 2.60 USD a run, and a run played whole 1.59 to 3.75. The conclusions, issue by issue, are in the issues #77 lists.
PierreMardon
added a commit
that referenced
this pull request
Oct 7, 2026
Kept for the next campaign to be compared with: `evals/reports/2026-10-07-9d5de55f57a9/`. It measures the chain on `main` with the nine pull requests of 2026-10-07 that did the ten decisions of the developer (#97 to #105), none of which had been played. The six cases are played twice on one version for the first time: `02-reservations`, `04-suspension`, `05-fine-cap` and `06-borrow-limit` whole, `01-overdue-list` and `03-overdue-reminders` stopped at the hand over. It is also the first campaign judged by the rubric of #104 and counted by the method as it stands. 200 sessions, 30.89 USD at list price, the check of the judge included. `report.md` and `summary.json` are as `report --against <summary> --keep` wrote them, by the harness of #106, #108 and #109, against the fourth campaign. `evals/README.md` says what this campaign is worth to compare with, that it was judged before the two lines the rubric draws for the hand back at conformity, and that its ledger counted 6 points of the weekly gauge where its streams show 3: its runs were played before #106. What it shows: - The eight runs played whole reached `conformant` and passed all of their hidden acceptance tests. - The hand back at conformity is one to three sentences in the eight runs, names the files of the critical zones and ends on the developer's step (#103). `messages.next-step` goes from 3 to 4.4. - No final message reports a `cd` (#102), and one names an agent, in 12 runs (#105). `messages.clear` goes from 2 to 4.4, by a rubric that changed with the prompts. - Every page that holds a rule the plan settled on its own marks it as an assumption that approving confirms (#100). - `05-fine-cap` draws no diagram in its two runs, where it drew one in each (#101), and `blueprint.diagrams` is 5 on the twelve pages. - The check of the judge: the lean page scores 4 and 4 on `blueprint.no-padding`, its padded version 2 and 2, where one point parted them. On the pages of the runs the score is 2 on ten and 3 on two. - What did not move: `following.no-noise`, 2 to 3, on the steps a hand over still tells, checked, pushed, refreshed. - The judge faults the hand back at conformity for not telling what was built, which is what the developer asked of it: the rubric says so in a pull request of its own. - The proof of conformity lists a file of a critical zone under a blueprint that said none in 3 runs of 8 (#107), which `zones_announced` tells since #108. - No blueprint holds a title inside a section, and nine of twelve have a section of more than two paragraphs without one (#63, measured since #109). - Planning costs 0.98 to 2.60 USD a run, and a run played whole 1.59 to 3.75. The conclusions, issue by issue, are in the issues #77 lists.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The prompts of the three commands, of the executor and of the reviewer held one rule in two parts: every command runs from the root of the repository, with paths from there, and never
cdnorgit -C, "since [...] acdfollowed by git stops for an approval" and the permission rules do not readgit -Cas the git command it runs. #87 took the rest of how an agent writes a command out of the prompts, and left this ban.With a ban, a session has a breach to report. The last sentence of the last message of a run of
05-fine-cap, in the fourth campaign of the evaluations: "The slice 2 executor ran onecdinto the repository root, which the instructions forbid. It made no edits through it." The developer can do nothing with it (#67).The developer decided on 2026-10-07, in #67: the prompts keep what the rule is for, and lose the ban and its reason of permissions. This is the tenth of the decisions #77 lists.
What changes
skills/surface-plan/SKILL.md,skills/surface-execute/SKILL.md: "Every command runs from the root of the repository, with paths from there, since the shell is shared with the agents."skills/surface-status/SKILL.md, which launches no agent: "since one shell serves the whole session."agents/surface-executor.md: the line leaves what the executor never does, and the rule stands where the state script is named, "since you share the shell with the dispatcher".agents/surface-reviewer.md: the same, "with the dispatching command".ARCHITECTURE.mdand ADR 0032: the invariant and the sentence of the decision say what the prompts hold now, and why they ban nothing: "a ban gave a session a breach to report to a developer who could do nothing with it". Left unsaid, someone would put the ban back.Tests
tests/prompts/test_agent_mandates.py: every prompt that runs commands states the rule with its reason, the shared shell, and no prompt namescdorgit -C, bans a change of directory in other words, nor gives a reason of permissions. The commands the prompts spell themselves still change no directory, which the existing test holds.What it may cost
Read by an independent reviewer before the commit: the reason written in
ARCHITECTURE.mdand the ADR, and the check of a ban said in prose, are theirs.Not measured
No session was played. What to read in the next runs played whole: whether a message still reports a
cd, and whether a command fails for running from elsewhere.Refs #67