test: cover hook decisions with 65 behavioural tests - #17
Merged
Conversation
Both hook bugs this project has shipped were logic errors, not shell errors: a PreToolUse condition that silently disabled two hooks, and a regex boundary that let `rm -rf /` through while blocking `rm -rf .git`. Neither shellcheck nor the config validator can see that class of fault. Each test runs the real script in a subprocess against a crafted payload and asserts on the decision it emits. Validated by mutation: restoring the old \b boundary in protect-main.sh turns 8 tests red in both directions, matching the original bug exactly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The hooks are the only enforced guardrails in this setup — everything else is advice Claude may or may not follow. Both hook bugs this project has shipped were logic errors, not shell errors:
PreToolUseifcondition packed multiple patterns into one string, so it never matched andenforce-uvandprotect-mainsilently stopped firing — taking the force-push andrm -rfguards with them.\bword boundary that letrm -rf /,rm -rf .,rm -rf ~through while wrongly blockingrm -rf .gitandrm -rf ~/tmp-dir.Neither
shellchecknorscripts/validate_config.pycan see that class of fault. Both scripts were syntactically perfect and structurally wired; they just decided the wrong thing.Approach
Each test runs the real script in a subprocess against a crafted payload and asserts on the decision it emits. Nothing is mocked — "the hook silently stopped firing" is precisely the regression worth catching, and a mock cannot fail that way.
tests/hook_harness.pywraps the output in aHookResultexposingpermission_decision,updated_command,decision,reasonandis_silent, so tests read as statements about behaviour rather than JSON spelunking.test_protect_main_hook.pytest_enforce_uv_hook.pyuv run ..., 6 denials, 7 pass-throughstest_guard_secrets_hook.pytest_session_start_hook.py.venv/staleuv.lockreported, healthy project silenttest_verify_hook.pystop_hook_activeloop guard, non-Python-project exittest_auto_lint_hook.py65 tests, ~10s.
Proof they work
A test suite that passes proves nothing on its own, so I mutated the hook back to the #11 bug — restoring the
\bboundary — and re-ran:Failures in both directions, matching the original bug's profile exactly. The hook was then restored.
Gaps I chose to leave, and why
Stated in the README rather than papered over:
verify.sh's ruff/pytest body is untested. Running it from inside the suite would invoke pytest recursively; provisioning a throwaway uv project would need a network install on every CI run. Only its guard clauses are covered.auto-lint.sh's formatting body is untested, because whether ruff acts on a file depends on the surrounding project'sincludeconfig — an assertion here would test that config, not the hook.settings.jsonis still unverified. Nothing checks that amatcherandifcondition route the events you expect. That is exactly what broke in fix(hooks): correct PreToolUse if-condition syntax and add explicit settings defaults #10, and it remains verifiable only by running Claude Code. Worth knowing this PR does not close that hole.Note on the test credentials
test_guard_secrets_hook.pyneeds credential-shaped strings. They are fake — one is AWS's own published documentation example — and assembled at runtime from fragments so the repository never contains a contiguous string matching a real credential format, which would otherwise trip push protection and secret scanners.Also
pyteststep.includeandknown-first-partyextended to covertests/.🤖 Generated with Claude Code