Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .changeset/crap-gate-test-hygiene.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@
"tech-stack": patch
---

Testing practice: CRAP gate design (per-clause scoring, whole-codebase gate, no exemptions, hand-run mutation testing), test declutter rules, and test-database hygiene (dirty-start detection, validator name patterns).
Testing practice: hand-run mutation testing, test declutter rules, and test-database hygiene (dirty-start detection, validator name patterns).
7 changes: 7 additions & 0 deletions .changeset/remove-crap-gate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"tech-stack": patch
---

Remove the CRAP gate in favour of the complexity limit; coverage is not gated.

Require tools to enforce dependency direction into the domain core, and prefer contract-tested fakes behind ports over mocking internal modules.
2 changes: 1 addition & 1 deletion skills/tech-stack/references/practices/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
- The server owns authorization; browser and mobile clients never do.
- Contracts first: wire contracts in `contracts/`, versioned apart from the app.
- Domain modules own behavior and state; the web/transport layer only parses, admits and projects.
- Module boundaries are enforced by a tool (Boundary, Spring Modulith, an import rule), not by convention.
- Module boundaries — required: tools enforce direction too; the domain core imports no adapter, framework or UI module: Elixir [Boundary](https://hexdocs.pm/boundary/Boundary.html) (`deps`/`exports` per boundary; `type: :strict` checks external apps), Spring [ArchUnit layer rules](https://www.archunit.org/userguide/html/000_Index.html#_layer_checks) beside [Spring Modulith](https://docs.spring.io/spring-modulith/reference/verification.html), TypeScript [Oxlint `no-restricted-imports`](https://oxc.rs/docs/guide/usage/linter/rules/eslint/no-restricted-imports) scoped to core files (cover aliases and relative paths; dependency-cruiser rejected: reuse the stack's linter), Rust [separate crates](https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#specifying-path-dependencies) with no outward dependencies in the core manifest; verified: 2026-10-21.
- Safety-critical and costly features sit behind server-side switches that default off and fail closed.

## Lessons (each from a real defect)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ verified: 2026-10-18 · https://vitest.dev/guide/cli#changed · https://pnpm.io/

### Elixir: full run in shards — default
setting: `mix test --partitions N` as N separate jobs (`MIX_TEST_PARTITION` set per job)
why: Elixir has no CI-safe "changed since ref"; `mix test --stale` compares against the last local run. Shards finish sooner and a failure reruns alone. With a coverage gate, export coverage per shard and gate once on the merged data.
why: Elixir has no CI-safe "changed since ref"; `mix test --stale` compares against the last local run. Shards finish sooner and a failure reruns alone.
verified: 2026-10-18 · https://hexdocs.pm/mix/Mix.Tasks.Test.html

### Graph tools (Turborepo, Nx, Moon, Bazel) — optional
Expand Down
15 changes: 4 additions & 11 deletions skills/tech-stack/references/practices/testing.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,8 @@
# Tests, complexity and baselines

## Coverage
- No coverage threshold (decided 2026-10-06). Use a CRAP limit of 30 on changed functions: CRAP = complexity² × (1 − coverage)³ + complexity.
- Complexity per function: warn above 10, fail above 15 on new or changed code (Credo, PMD, detekt, Oxlint `complexity`).

## CRAP gate
- Limit 30, the Crap4j original; tighten it later once the codebase sits well under it.
- Score each clause of a multi-clause function (Elixir, Erlang) on its own complexity and coverage, and report the worst clause. Summing per function name makes a dispatch function with many trivial clauses fail forever.
- Count only executable lines and AST nodes, never comments or formatting, so a comment-only edit is not a change.
- When the whole codebase already passes, gate all of it rather than only changed functions: it is simpler (no base-diff logic) and also catches a coverage drop from a deleted test. Use a changed-only gate while old code still fails.
- No exemption list. The fix is a split or a test.
- A failure line names the way out: the coverage the clause needs to pass, or "split it" when even 100% coverage cannot (complexity over about 30).
## Coverage and complexity
- No coverage threshold (decided 2026-10-06).
- Complexity per function: warn above 10, fail above 15 on new or changed code (Credo, PMD, detekt, Oxlint `complexity`); fix failures by separating pure decision logic from I/O, not by chopping code into arbitrary tiny functions.
- Mutation testing is a hand run on a few risky modules (auth, admission), not a CI step: it is slow, and a 2-core runner cannot afford it.

## Baselines for old code
Expand All @@ -20,6 +12,7 @@
- For project-wide compiler warnings (for example Godot `project.godot`), lock per-file exemptions to the content hash. Any edit removes the exemption; inline suppressions of strict rules fail the gate.

## Tests
- Mocking your own modules — rejected: fakes behind a port, checked by the same contract tests as the real adapter, instead; tests coupled to internals fail on refactors, not regressions.
- A test asserts the reason for a failure, not only the status, so a crash cannot pass as a denial.
- Lock-wait and race behavior needs a database interleaving drill, because a sandboxed test cannot race; plain authorization rules are ordinary tests. Drill each distinct race that ordinary tests cannot exercise, and merge cases that would catch the same failure.
- Clone ordinary drill databases from one migrated template (`CREATE DATABASE … TEMPLATE`) instead of migrating from zero per test or module; migration and bootstrap tests start from their real starting state.
Expand Down
Loading