test: cover hook decisions with 65 behavioural tests

.github/workflows/ci.yml

@@ -46,3 +46,8 @@ jobs:
       # have shipped before. Warning severity skips style nitpicks.
       - name: Shellcheck hooks
         run: shellcheck --severity=warning .claude/hooks/*.sh
+
+      # Runs each hook script for real against crafted payloads. shellcheck
+      # catches shell mistakes; these catch wrong decisions.
+      - name: Test hook behaviour
+        run: uv run pytest

CHANGELOG.md

@@ -8,6 +8,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ### Added
 
+- **Tests** (`tests/`): 65 tests covering the six hook scripts. Each runs the real script in a subprocess against a crafted payload and asserts on the decision it emits — nothing is mocked, since "the hook silently stopped firing" is the regression worth catching. Both hook bugs this project has shipped were **logic** errors, not shell errors, and neither `shellcheck` nor `validate_config.py` can see that class of fault. Validated by mutation: reintroducing the pre-v1.3.0 `\b` boundary in `protect-main.sh` turns 8 tests red in both directions — `rm -rf /`, `rm -rf .`, `rm -rf ~` slipping through, and `rm -rf .git`, `rm -rf ~/tmp-dir` wrongly blocked. Two paths are deliberately uncovered and documented as such: `verify.sh`'s ruff/pytest body (running it from inside pytest would recurse) and `auto-lint.sh`'s formatting body (its outcome depends on the surrounding project's ruff `include`)
 - **CI** (`.github/workflows/ci.yml`): a GitHub Actions workflow on pull requests and pushes to `main`. Until now nothing verified this repository at all — every PR merged without a single automated check. It runs ruff (lint + format), `shellcheck --severity=warning` over the hook scripts, and a new configuration validator
 - **Scripts** (`scripts/validate_config.py`): validates that the configuration this template ships is internally coherent — `settings.json` and the MCP configs parse; hook scripts referenced by `settings.json` exist on disk; `CLAUDE.md`'s `@`-imports resolve; every skill's `SKILL.md` still declares `name` and `description`. All but the first fail **silently** at runtime: a hook whose script was renamed just stops firing, a broken `@`-import drops that rule from Claude's context, and a skill missing frontmatter becomes undiscoverable — none of which surfaces an error. Verified by injecting each fault and confirming a non-zero exit
 

README.md

@@ -105,8 +105,8 @@ claude-code-python-setup/
 │   │   ├── protect-main.sh   # Block force push, direct push to main, broad rm -rf
 │   │   ├── auto-lint.sh      # Auto-format Python files with ruff after edits
 │   │   └── verify.sh         # Run ruff + pytest on Stop; block until green
-│   ├── rules/               # Modular coding standards
-│   │   ├── api-patterns.md  # FastAPI/Pydantic (path-scoped)
+│   ├── rules/                # Modular coding standards
+│   │   ├── api-patterns.md   # FastAPI/Pydantic (path-scoped)
 │   │   ├── architecture.md
 │   │   ├── compaction.md
 │   │   ├── documentation.md
@@ -118,7 +118,7 @@ claude-code-python-setup/
 │   │   └── testing.md
 │   ├── settings.json          # Project-level hooks, permissions, status line
 │   ├── statusline.py          # Status line script (Python, cross-platform)
-│   └── skills/              # Reference docs and scripts
+│   └── skills/                # Reference docs and scripts
 │       ├── api-design/
 │       ├── claude-api/
 │       ├── claude-automation-recommender/
@@ -149,6 +149,10 @@ claude-code-python-setup/
 │   └── windows.mcp.json     # MCP server config (Windows)
 ├── scripts/
 │   └── validate_config.py   # Checks this template's own config is coherent
+├── tests/
+│   ├── conftest.py          # Fixtures that run a hook against a payload
+│   ├── hook_harness.py      # HookResult and payload builders
+│   └── unit/                # One file per hook, asserting its decisions
 ├── .env.example             # Environment variables template
 ├── .gitattributes           # Force *.sh to LF so hooks run on Windows
 ├── .gitignore
@@ -550,15 +554,36 @@ The script is written in Python and works cross-platform: Windows, macOS, and Li
 |-------|---------|
 | `ruff check` / `ruff format --check` | Lint and formatting on `.claude/statusline.py` and `scripts/` (`src/**/*.py` covers projects built from this template) |
 | `scripts/validate_config.py` | Unparseable `settings.json` or MCP config; hooks pointing at scripts that no longer exist; `CLAUDE.md` `@`-imports that don't resolve; skills whose `SKILL.md` lost its `name`/`description` frontmatter |
-| `shellcheck --severity=warning` | Quoting and logic bugs in the hook scripts |
+| `shellcheck --severity=warning` | Shell quoting and syntax bugs in the hook scripts |
+| `pytest` | Hooks reaching the **wrong decision** — see below |
 
-The middle three failures are all silent at runtime: a hook whose script was renamed simply stops firing, and a broken `@`-import drops that rule from Claude's context without an error. Run the same checks locally with:
+The `validate_config.py` failures are all silent at runtime: a hook whose script was renamed simply stops firing, and a broken `@`-import drops that rule from Claude's context without an error. Run everything locally with:
 
 ```bash
-uv run ruff check . && uv run ruff format --check . && uv run python scripts/validate_config.py
+uv run ruff check . && uv run ruff format --check . && uv run python scripts/validate_config.py && uv run pytest
 ```
 
-There is no test job: this repository ships configuration, not application code, so `pyproject.toml`'s pytest setup is there for the projects you build from it.
+### Hook tests (`tests/`)
+
+The hooks are the only enforced guardrails in this setup, and every past bug in them has been a **logic** error rather than a shell error — a condition that silently disabled a hook, a regex boundary that let `rm -rf /` through while blocking `rm -rf .git`. shellcheck cannot see either.
+
+So each test runs the real script in a subprocess with a crafted payload and asserts on the decision it emits. Nothing is mocked, because "the hook stopped firing" is precisely the regression worth catching.
+
+| File | Asserts |
+|------|---------|
+| `test_protect_main_hook.py` | Force pushes, pushes to main, `reset --hard` and broad `rm -rf` are denied — while `--force-with-lease`, `rm -rf .git` and `rm -rf ~/tmp-dir` still pass |
+| `test_enforce_uv_hook.py` | Bare `python`/`pytest`/`ruff` are rewritten to `uv run ...`; `pip` and compound commands are denied; anything already using uv is left alone |
+| `test_guard_secrets_hook.py` | Credential-shaped prompts are blocked, prose *about* credentials is not |
+| `test_session_start_hook.py` | Missing `.venv` or a stale `uv.lock` is reported; a healthy project stays silent |
+| `test_verify_hook.py` | The `stop_hook_active` loop guard and the non-Python-project exit |
+| `test_auto_lint_hook.py` | Non-Python files, deleted files and payloads without a path are ignored |
+
+Two gaps are deliberate and worth knowing:
+
+- **`verify.sh`'s ruff/pytest execution path is not covered.** Running it from inside the suite would invoke pytest recursively, and a throwaway uv project would need a network install on every CI run. Only its guard clauses are tested.
+- **`auto-lint.sh`'s formatting path is not covered**, because whether ruff acts on a given file depends on the surrounding project's `include` configuration. Only the conditions under which the hook must do nothing are tested.
+
+Neither the tests nor `validate_config.py` check the **hook wiring** in `settings.json` — that a hook's `matcher` and `if` condition actually route the events you expect. That wiring has broken before, and it remains verifiable only by running Claude Code.
 
 ## Contributing
 

pyproject.toml

@@ -56,15 +56,21 @@ line-length = 99
 src = ["src"]
 # src/** is for projects built from this template; the other two are the Python
 # this repo actually ships. Vendored .claude/skills/ scripts stay out of scope.
-include = ["pyproject.toml", "src/**/*.py", "scripts/**/*.py", ".claude/statusline.py"]
+include = [
+    "pyproject.toml",
+    "src/**/*.py",
+    "tests/**/*.py",
+    "scripts/**/*.py",
+    ".claude/statusline.py",
+]
 
 [tool.ruff.lint]
 extend-select = ["I"]  # Add import sorting
 select = ["E", "F", "W", "I", "UP"]
 ignore = []
 
 [tool.ruff.lint.isort]
-known-first-party = ["src"]
+known-first-party = ["src", "tests"]
 force-sort-within-sections = true
 
 [tool.ruff.format]

tests/__init__.py

@@ -0,0 +1 @@
+"""Test suite for the .claude configuration."""

tests/conftest.py

@@ -0,0 +1,73 @@
+"""Fixtures for running the hook scripts against crafted payloads."""
+
+from __future__ import annotations
+
+import json
+from pathlib import Path
+import shutil
+import subprocess
+from typing import Any
+
+import pytest
+
+from tests.hook_harness import HOOK_TIMEOUT_SECONDS, HOOKS_DIR, REPO_ROOT, HookResult, RunHook
+
+
+def _require(executable: str) -> str:
+    """Return the path to `executable`, skipping the test when it is absent."""
+    resolved = shutil.which(executable)
+    if resolved is None:
+        pytest.skip(f"{executable} is required to run the hook scripts")
+    return resolved
+
+
+@pytest.fixture(scope="session")
+def bash_executable() -> str:
+    """Path to bash, which interprets every hook script."""
+    return _require("bash")
+
+
+@pytest.fixture(scope="session")
+def jq_executable() -> str:
+    """Path to jq, which every hook uses to read its payload and build output."""
+    return _require("jq")
+
+
+@pytest.fixture(scope="session")
+def uv_executable() -> str:
+    """Path to uv, needed by the hooks that inspect or drive the toolchain."""
+    return _require("uv")
+
+
+@pytest.fixture
+def run_hook(bash_executable: str, jq_executable: str) -> RunHook:
+    """Return a callable that runs a hook script against a payload.
+
+    Returns
+    -------
+    RunHook
+        `run_hook(hook_name, payload, cwd=None)` -> `HookResult`. `hook_name` is
+        the script's stem, for example `"protect-main"`. `cwd` defaults to the
+        repository root; pass a temp directory for hooks that inspect the
+        working directory.
+    """
+
+    def _run(
+        hook_name: str,
+        payload: dict[str, Any],
+        cwd: Path | None = None,
+    ) -> HookResult:
+        script_path = HOOKS_DIR / f"{hook_name}.sh"
+        assert script_path.is_file(), f"hook script not found: {script_path}"
+
+        completed = subprocess.run(
+            [bash_executable, str(script_path)],
+            input=json.dumps(payload),
+            capture_output=True,
+            text=True,
+            cwd=str(cwd or REPO_ROOT),
+            timeout=HOOK_TIMEOUT_SECONDS,
+        )
+        return HookResult(exit_code=completed.returncode, stdout=completed.stdout)
+
+    return _run

tests/hook_harness.py

@@ -0,0 +1,110 @@
+"""Helpers for driving the hook scripts in `.claude/hooks/`.
+
+The hooks are shell scripts that read a JSON payload on stdin and answer on
+stdout, so the tests run each real script in a subprocess and assert on the
+decision it emits. Nothing is mocked: a hook that silently stops firing is
+exactly the regression these tests exist to catch.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Callable
+from dataclasses import dataclass
+import json
+from pathlib import Path
+from typing import Any
+
+REPO_ROOT = Path(__file__).resolve().parent.parent
+HOOKS_DIR = REPO_ROOT / ".claude" / "hooks"
+HOOK_TIMEOUT_SECONDS = 60
+
+
+@dataclass(frozen=True)
+class HookResult:
+    """Outcome of running a hook script.
+
+    Attributes
+    ----------
+    exit_code
+        Process exit status. Hooks exit 0 even when they deny an action: the
+        decision travels in the JSON payload, not in the status code.
+    stdout
+        Raw text the hook wrote to stdout.
+    """
+
+    exit_code: int
+    stdout: str
+
+    @property
+    def payload(self) -> dict[str, Any]:
+        """Parsed hook output, or an empty dict when the hook stayed silent."""
+        if not self.stdout.strip():
+            return {}
+        return json.loads(self.stdout)
+
+    @property
+    def is_silent(self) -> bool:
+        """True when the hook emitted nothing, meaning it took no position."""
+        return not self.stdout.strip()
+
+    @property
+    def permission_decision(self) -> str | None:
+        """`permissionDecision` from a `PreToolUse` hook, if it emitted one."""
+        return self.payload.get("hookSpecificOutput", {}).get("permissionDecision")
+
+    @property
+    def updated_command(self) -> str | None:
+        """Command a `PreToolUse` hook rewrote the call to, if it rewrote one."""
+        hook_output = self.payload.get("hookSpecificOutput", {})
+        return hook_output.get("updatedInput", {}).get("command")
+
+    @property
+    def decision(self) -> str | None:
+        """Top-level `decision`, used by `UserPromptSubmit` and `Stop` hooks."""
+        return self.payload.get("decision")
+
+    @property
+    def reason(self) -> str:
+        """Explanation the hook gave, from whichever field carries it."""
+        hook_output = self.payload.get("hookSpecificOutput", {})
+        return hook_output.get("permissionDecisionReason") or self.payload.get("reason", "")
+
+    @property
+    def additional_context(self) -> str:
+        """Context a `SessionStart` hook injected into the session."""
+        return self.payload.get("hookSpecificOutput", {}).get("additionalContext", "")
+
+
+RunHook = Callable[..., HookResult]
+
+
+def bash_payload(command: str) -> dict[str, Any]:
+    """Build the `PreToolUse` payload Claude Code sends for a Bash call.
+
+    Parameters
+    ----------
+    command
+        The shell command Claude proposed to run.
+
+    Returns
+    -------
+    dict[str, Any]
+        Payload shaped like the real hook input.
+    """
+    return {"tool_name": "Bash", "tool_input": {"command": command}}
+
+
+def edit_payload(file_path: str) -> dict[str, Any]:
+    """Build the `PostToolUse` payload Claude Code sends after an Edit or Write.
+
+    Parameters
+    ----------
+    file_path
+        Path of the file the tool touched.
+
+    Returns
+    -------
+    dict[str, Any]
+        Payload shaped like the real hook input.
+    """
+    return {"tool_name": "Edit", "tool_input": {"file_path": file_path}}

tests/unit/__init__.py

@@ -0,0 +1 @@
+"""Unit tests for the hook scripts."""

tests/unit/test_auto_lint_hook.py

@@ -0,0 +1,37 @@
+"""Guard clauses of the `auto-lint` PostToolUse hook.
+
+The formatting path itself runs `uv run ruff` against the edited file, which
+depends on the surrounding project's ruff configuration deciding whether that
+path is in scope. That coupling makes it unsuitable for an assertion here, so
+these tests cover the conditions under which the hook must do nothing at all —
+the cases where running ruff would be wrong or impossible.
+"""
+
+from pathlib import Path
+
+from tests.hook_harness import RunHook, edit_payload
+
+
+def test_auto_lint_ignores_non_python_file(run_hook: RunHook, tmp_path: Path) -> None:
+    readme = tmp_path / "README.md"
+    readme.write_text("# Title\n", encoding="utf-8")
+
+    result = run_hook("auto-lint", edit_payload(str(readme)))
+
+    assert result.is_silent
+    assert result.exit_code == 0
+
+
+def test_auto_lint_ignores_deleted_file(run_hook: RunHook, tmp_path: Path) -> None:
+    """A file edited then removed must not make the hook fail the tool call."""
+    result = run_hook("auto-lint", edit_payload(str(tmp_path / "gone.py")))
+
+    assert result.is_silent
+    assert result.exit_code == 0
+
+
+def test_auto_lint_ignores_payload_without_file_path(run_hook: RunHook) -> None:
+    result = run_hook("auto-lint", {"tool_name": "Edit", "tool_input": {}})
+
+    assert result.is_silent
+    assert result.exit_code == 0