From 8ac4e014dce5d4262e25ab8edd88e7c64647792b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Olu=CC=81was=CC=A3e=CC=80yi=CC=81=20S=CC=A3o=CC=81bande?= =?UTF-8?q?=CC=81?= Date: Tue, 20 Oct 2026 18:19:37 +0100 Subject: [PATCH] fix(testing): require 5 interleaved benchmark runs per side --- .changeset/benchmark-run-count.md | 5 +++++ skills/tech-stack/references/practices/testing.md | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) create mode 100644 .changeset/benchmark-run-count.md diff --git a/.changeset/benchmark-run-count.md b/.changeset/benchmark-run-count.md new file mode 100644 index 0000000..9fe31e1 --- /dev/null +++ b/.changeset/benchmark-run-count.md @@ -0,0 +1,5 @@ +--- +"tech-stack": patch +--- + +Benchmark comparisons need at least 5 interleaved runs per side, up from 3. diff --git a/skills/tech-stack/references/practices/testing.md b/skills/tech-stack/references/practices/testing.md index f4e2a0f..a4e3a4a 100644 --- a/skills/tech-stack/references/practices/testing.md +++ b/skills/tech-stack/references/practices/testing.md @@ -35,4 +35,4 @@ - Record an engine-exit diagnostic that cannot be fixed in an exact-match allowlist: message, resource path, count. Every other ERROR/WARNING fails. ## Performance comparisons -- Interleave baseline and candidate runs on the same machine, at least 3 runs each (Alárinà requires 5). Report median and p95 with ranges; serial batches can turn time-dependent noise into a false regression. +- Interleave baseline and candidate runs on the same machine, at least 5 runs per side. Report median and p95 with ranges; serial batches can turn time-dependent noise into a false regression.