feat: how to build the gate that keeps a rule true - #5
Merged
Merged
Conversation
The audit workflow ends by arguing for a mechanical gate and says nothing about writing one. Building the first such gate surfaced failures that were not obvious from the rule it enforced, and every one of them is generic. Three ways a gate is too quiet. It matches one case when the same token arrives lowercase through a branch name and uppercase through a citation. It ends matches on `\b`, which cannot fire before `_`, so every snake_case and SCREAMING_CASE spelling is invisible. It scopes by an extension allowlist, which fails open on the file types nobody listed — and machine-read configuration is exactly where names live. One way it is too loud: a rule written as a shape rather than a roster. The shape that catches what you mean also catches fixtures, identifiers, and standard names that merely resemble it, and a gate that cries wolf gets suppressed everywhere it is inconvenient. Also records what makes an exemption reviewable rather than a blanket skip: scope it as narrowly as its reason allows, make any approved count measure what the exemption actually admits, prefer a carve-out the repository itself validates over one somebody has to maintain, and pin the set of files the check exempts itself. The naming rule gains the test that makes it decidable at all: a label is a reference if a reader holding only the repository or a public specification can resolve it, and shape decides nothing — seat identifiers, invoices, and cipher names all look like a ticket.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
cmk:rule's audit workflow ends by arguing for a mechanical gate and then says nothing about how to write one. Building the first real gate against a large tree surfaced failure modes that were not obvious from the rule being enforced, and none of them are specific to that rule or that repository.What this adds
A
## Workflow: Gatesection inskills/rule/SKILL.md, covering the ways a first-draft gate is simultaneously too loud and too quiet:Too quiet — three boundary failures, each of which had live instances that the gate reported clean:
\b, which cannot fire before_, hiding every snake_case and SCREAMING_CASE spellingToo loud — a rule written as a shape rather than a roster. The shape that catches the intended thing also catches fixtures, identifiers, and standard names that merely resemble it, and a gate that cries wolf gets suppressed wherever it is inconvenient.
Exemption design — what keeps a carve-out reviewable instead of a blanket skip: scope it as narrowly as its reason allows, make any approved count measure what the exemption actually admits rather than a proxy for it, prefer a carve-out the repository itself validates over one somebody has to maintain, and pin the set of files the check exempts itself.
skills/agent-instructions/references/rules-naming.mdgains the resolvability test that makes the coordinate rule decidable: a label is a reference if a reader holding only the repository or a public specification can resolve it. Shape decides nothing on its own — seat identifiers, invoice fixtures, and cipher names all look like a ticket.Versions
cmk:rule0.3.0 → 0.4.0 (new workflow)cmk:agent-instructions0.2.1 → 0.2.2 (rule refinement)Testing
scripts/skill-lint.sh— OK acrossskills/(frontmatter, size, references, citations, paths, eval.json)skills/rule/SKILL.mdis 72 lines against the 150-line budgetProvenance
Every item is a corrected defect from building a real gate, not a hypothetical. The boundary failures each had live instances in the tree that the gate reported clean until the boundary was fixed.