choiyounggi · choiyounggi · Sep 10, 2026
@@ -104,3 +104,4 @@ Append-only. Format: `## [YYYY-MM-DD] <ingest|revise|lint|gap|contradiction|drif
 ## [2026-08-30] revise | frontend/design/design-canvas-workflow — agent-gate Check 3 remediation (PR #164). The gate's fact finding was half right: its "no skill named `design` exists" conclusion is refuted by ground truth (the skill is listed in the authoring session's available-skills roster as `design: Create a design canvas…` and its bundled payload exists on the authoring machine's disk; the reviewer's CI environment lacks the preview, and it conflated the skill with the unrelated `/design-sync` component-upload tool) — but the verifiability objection stands: the skill is an early research preview absent from public docs, so a single un-fetchable in-session source cannot carry `confidence: verified`. Fixed per AGENTS.md: confidence downgraded to field-tested with the experience context stated in the body; two live-fetched public sources added (anthropic.com/news/claude-design-anthropic-labs — official Claude Design product announcement, research preview, Claude Code handoff; explainx.ai 2026 post — /design ships in Claude Code as a research-preview command producing editable artboards via the Artifacts runtime); the mandatory-routing directive and the frontend/index.md load-when line conditioned on the skill appearing in the session's available-skills list, with the existing no-skill edge case as the explicit fallback; the unverifiable "design-review skill" name-drop in the audit edge case generalized to session-provided audit tooling.
 ## [2026-09-03] ingest | databases/selection — new category: datastore selection by workload (4 pages: choosing-a-datastore-by-workload, relational-jsonb-vs-document-store, vector-search-engine-selection, graph-workloads-relational-vs-graph-db); brave-search research, sources cited per page
 ## [2026-09-04] ingest | infrastructure/agent-orchestration/code-graph-as-orientation-layer (field-tested — 1 new page). A locally built code knowledge graph (graphify) is a freshness-gated, lead-not-evidence orientation layer for planning and parallel task decomposition: gate on graph mtime vs git log, symbol-anchored explain/path only, CLI delivery with bounded output, graph-derived assumptions named in reports. Sources: Developers Digest, AQ Score, Autonoma, Tiare Balbi, graphify README + measurements on graphifyy 0.4.23.
+## [2026-09-10] ingest | knowledge-flush of the harvested insight queue — 6 new pages, 0 merges, 0 drops, 5 domain indexes updated (auto-flush headless run `20260910-161926-53458`, 6 claimed rows from 3 session files). New: infrastructure/agent-orchestration/gate-evidence-exit-code-class (verified — a gate-check EVIDENCE `exit=127`/`126` names the checker's own invocation failure, e.g. `${CLAUDE_PLUGIN_ROOT}` unset in a plain shell expanding to `/skills/...`; classify by exit code before editing the plan), platforms/tools/jq-dot-rebinding-in-predicates (verified — `$arr | index(.)` inside a generator rebinds `.` to `$arr`, a self-subsequence match that is unconditionally true for non-empty `$arr`; bind with `. as $x` or use `IN($arr[])`; local repro on jq 1.7.1; field site dev-loop graph-drop.sh:67), infrastructure/agent-orchestration/worker-reported-plan-contradiction (field-tested — a worker reporting that the design doc and step file disagree is a hypothesis to test by running the disputed change and the suite, then patch the wrong document; sqlite3 exception hierarchy example), debugging/performance/attributing-a-benchmark-speedup (verified — read the baseline arm's harness config and trace the changed knob to its read site before crediting a change; Gregg active benchmarking / benchmarking checklist, Heiser benchmarking crimes), testing/quality/sequential-dispatch-assumption-under-concurrency (verified — an exact-count "stop after item N" test is a false-pass once dispatch becomes a thread pool with cap ≥ item count because `submit()` is non-blocking; run at the real cap, not cap=1; CPython 3.14.6 repro), testing/quality/proving-a-critical-section-is-lock-protected (verified — raw stress passed 20/20 with the lock removed; a `__setitem__`-sleeping dict double made the missing lock fail 10/10, restored lock 10/10; validate by mutation). Related links added both ways on 16 existing pages; four back-links deferred until open PRs land (#181 checkable-claims-in-an-adopted-plan, #182 vendor-benchmark-claims-for-an-llm-tool, #183 gate-parsing-vs-command-execution, and tests-that-cannot-fail which three open PRs already edit). Lint: structure 282 pages / 13 indexes / 0 findings; prohibitions directives 75 / violations 0 (bats pin unchanged).
@@ -8,7 +8,7 @@ sources:
   - https://github.com/brettwooldridge/HikariCP/wiki/About-Pool-Sizing
   - https://docs.oracle.com/javase/tutorial/essential/concurrency/sync.html
 last_verified: 2026-07-10
-related: [databases-transactions-isolation-level-selection, backend-common-caching-invalidation-and-stampede, backend-common-reliability-timeouts-and-retries, databases-query-optimization-n-plus-one-queries, databases-indexing-index-selection]
+related: [databases-transactions-isolation-level-selection, backend-common-caching-invalidation-and-stampede, backend-common-reliability-timeouts-and-retries, databases-query-optimization-n-plus-one-queries, databases-indexing-index-selection, testing-quality-sequential-dispatch-assumption-under-concurrency]
 ---
 
 # Shared In-Process State and Pool Sizing under Concurrent Requests

@@ -11,7 +11,7 @@ sources:
   - https://docs.python.org/3/library/multiprocessing.html
   - https://docs.python.org/3/howto/free-threading-python.html
 last_verified: 2026-07-10
-related: [backend-common-concurrency-shared-state-and-pools, backend-common-jobs-idempotent-handlers, backend-python-serving-app-servers-and-workers]
+related: [backend-common-concurrency-shared-state-and-pools, backend-common-jobs-idempotent-handlers, backend-python-serving-app-servers-and-workers, testing-quality-sequential-dispatch-assumption-under-concurrency, testing-quality-proving-a-critical-section-is-lock-protected]
 ---
 
 # The GIL and Choosing Threads vs asyncio vs Processes

@@ -8,7 +8,7 @@ sources:
   - https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html
   - https://testing.googleblog.com/2017/04/where-do-our-flaky-tests-come-from.html
 last_verified: 2026-07-10
-related: [debugging-methodology-isolate-by-bisection, debugging-methodology-reproduce-first, testing-flaky-diagnosing-flaky-tests]
+related: [debugging-methodology-isolate-by-bisection, debugging-methodology-reproduce-first, testing-flaky-diagnosing-flaky-tests, testing-quality-proving-a-critical-section-is-lock-protected]
 ---
 
 # Making an Intermittent Failure Reproducible

@@ -28,6 +28,7 @@ Match your situation to a "load when" line; load only matching pages.
 | Page | Load when |
 |------|-----------|
 | [profile-before-optimizing](performance/profile-before-optimizing.md) | Something is slow (endpoint, job, test suite, page) and you are about to optimize; choosing CPU profiling vs wall-clock tracing; cold vs warm measurement; verifying a speedup (a single slow SQL statement → databases/query-optimization/reading-execution-plans) |
+| [attributing-a-benchmark-speedup](performance/attributing-a-benchmark-speedup.md) | A benchmark reports a speedup/slowdown and you are about to attribute it to a specific code change in a commit message, PR, or report; reviewing a PR that cites a benchmark number as evidence a specific change helped; a claimed removed-cost change (deleted sleep, removed retry) shows no measured effect because the harness already zeroed that knob for the baseline arm |
 
 ## concurrency
 

@@ -8,7 +8,7 @@ sources:
   - https://www.debuggingbook.org/html/Intro_Debugging.html
   - https://sre.google/sre-book/effective-troubleshooting/
 last_verified: 2026-08-06
-related: [debugging-methodology-reproduce-first, debugging-methodology-isolate-by-bisection, qa-deliverables-exclusivity-and-absence-claims, debugging-methodology-probe-path-vs-operation-path]
+related: [debugging-methodology-reproduce-first, debugging-methodology-isolate-by-bisection, qa-deliverables-exclusivity-and-absence-claims, debugging-methodology-probe-path-vs-operation-path, infrastructure-agent-orchestration-worker-reported-plan-contradiction, debugging-performance-attributing-a-benchmark-speedup]
 ---
 
 # Testing a Suspected Cause Before Changing Code

@@ -0,0 +1,81 @@
+---
+id: debugging-performance-attributing-a-benchmark-speedup
+domain: debugging
+category: performance
+applies_to: [general]
+confidence: verified
+sources:
+  - https://www.brendangregg.com/activebenchmarking.html
+  - https://www.brendangregg.com/blog/2018-06-30/benchmarking-checklist.html
+  - https://gernot-heiser.org/benchmarking-crimes.html
+last_verified: 2026-09-10
+related: [debugging-performance-profile-before-optimizing, debugging-methodology-hypothesis-testing, testing-quality-harness-reverse-controls, qa-deliverables-quantitative-claims-in-a-published-document]
+---
+
+# Attributing a Benchmark Speedup to a Specific Code Change
+
+## When this applies
+
+A benchmark run (yours or someone else's) reports a speedup or slowdown between
+a "before" and "after" arm, and you are about to name which code change caused
+it — in a commit message, PR description, README, or report. Also applies when
+reviewing a PR that cites a benchmark number as evidence a specific change helped.
+
+## Do this
+
+1. **Find the exact construction of the baseline ("before") arm in the harness
+   before naming a contributor.** Read the harness code, not the plausible
+   story — locate the line that builds the baseline's config/pipeline/client and
+   list every flag, feature toggle, and mocked/short-circuited step it sets.
+2. **For each candidate contributing change, confirm its code path actually ran
+   differently between the two arms.** A change contributes zero to the
+   measured delta if the harness's baseline arm never reaches the code the
+   change touches, or reaches it with a value that makes the change's effect
+   moot (a timeout set to 0, a call mocked out, a cache pre-warmed only on one
+   side).
+3. **Trace the value, not the name, to its read site.** A knob the harness
+   passes (`delay_seconds=0.0`) can be shadowed by a different default read
+   deeper in the call stack (`default_config.DELAY_SECONDS`) if the component
+   under test does not consult the harness's config for that value — grep for
+   where the changed code actually reads its input, not where the harness sets
+   it.
+4. **When several changes landed in the same "after" arm, isolate each one's
+   contribution before crediting all of them.** Re-run the benchmark with only
+   the candidate change applied against the same baseline config; the
+   plausible story ("we removed a sleep, so it's faster") is not evidence on
+   its own — an untested co-change can be doing all the work.
+5. **Explain the limiting factor before publishing the number.** State what
+   made the after-arm's result the value it is (CPU-bound loop, network round
+   trip removed, cache hit) — a number without a named mechanism is not yet an
+   attribution, it is an observation.
+
+| Check before attributing | Fails when |
+|---|---|
+| Baseline arm's harness-level config for the changed knob | The harness zeroes/mocks/short-circuits the exact setting the change touches, so the "before" cost was already absent in the measurement |
+| Where the changed code reads that setting | The component reads a package-level/module default instead of the harness's injected config, so the harness's knob never reached the code path |
+| Whether the change's code path executed at all in the baseline arm | A feature flag, early return, or stub in the baseline arm skips the code entirely — the delta then measures something else that also changed |
+| Attribution when multiple changes landed together | Only one change was benchmarked in isolation; the others are credited by narrative, not measurement |
+
+## Edge cases
+
+| Case | Then |
+|------|------|
+| The knob the harness sets has a same-named default elsewhere in the codebase (env var, module constant, class default) that the exercised code path actually reads | Confirm which value wins by adding a temporary print/log at the read site during a harness run, not by reading the harness's construction call alone — construction and consumption can disagree |
+| The "before" and "after" arms were run as separate benchmark invocations, not from the same harness config builder | Diff the two arms' effective config (dump it rather than assuming it), because a config default can differ between an old and a new invocation of the same script |
+| The benchmark predates the change under review (already published, cited in a PR) | Re-run it with the suspected knob deliberately un-zeroed in the baseline arm and confirm the delta shrinks or disappears — a benchmark that cannot reproduce its own claimed mechanism should not be cited as attributing to it |
+| No single change explains the whole delta after isolation | Report the unattributed remainder rather than folding it into the last change measured — an unexplained delta is a gap, not evidence for whichever change was tested last |
+
+## Instead of
+
+| If you are about to | Do this instead | Why |
+|---------------------|-----------------|-----|
+| Credit a change because the headline number moved and the story is plausible | Read the baseline arm's harness config for the exact knob the change touches, and trace it to its read site | A knob the harness sets to zero for both arms makes the corresponding change contribute nothing to the delta, no matter how real the change is elsewhere — the story and the number can both be true while the attribution is backwards |
+| Cite a benchmark number without naming what limited the "before" time | State the limiting factor (what made it slow) before publishing the delta | A number with no named mechanism cannot be checked by a reader, and often turns out to explain a different arm than intended |
+| Attribute a multi-change PR's speedup to "the changes" collectively | Re-run the baseline with each candidate change applied alone | Collective credit hides that one change (or an unrelated harness detail) did all the work |
+
+## Sources
+
+- https://www.brendangregg.com/activebenchmarking.html — "casual benchmarking: you benchmark A, but actually measure B, and conclude you've measured C"; verify what is being exercised while the benchmark runs rather than trusting the intended target
+- https://www.brendangregg.com/blog/2018-06-30/benchmarking-checklist.html — "Can they explain why the benchmark result was X, and not 2X (twice as fast)? ie, what is the limiting factor?"; a misconfiguration (e.g. a firewall silently blocking traffic) can make a benchmark client believe it measured something it never ran
+- https://gernot-heiser.org/benchmarking-crimes.html — "it does not at all follow that" a measured throughput delta equals the overhead of the change believed to cause it; comparisons must be made against the real, correctly configured baseline, not an assumed one
+- Field evidence (a Python web-scraping pipeline, 2026-09-09): its `scripts/bench_pipeline.py:354` constructed the "before" arm with `Pipeline({"delay_seconds": 0.0, ...})`; `src/local_scraper.py:126` read `default_config.DELAY_SECONDS` (3.0) rather than the pipeline's injected value. The benchmark's headline speedup was attributed to a removed 3-second sleep, whose cost was already zero in the baseline arm's measurement; the actual driver was a real ~3s-per-host cost added by the sleep's replacement
@@ -8,7 +8,7 @@ sources:
   - https://www.brendangregg.com/flamegraphs.html
   - https://sre.google/sre-book/effective-troubleshooting/
 last_verified: 2026-07-10
-related: [debugging-methodology-hypothesis-testing, databases-query-optimization-reading-execution-plans]
+related: [debugging-methodology-hypothesis-testing, databases-query-optimization-reading-execution-plans, debugging-performance-attributing-a-benchmark-speedup]
 ---
 
 # Locating Where Time Goes Before Optimizing

@@ -8,7 +8,7 @@ sources:
   - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Errors
   - https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html
 last_verified: 2026-07-10
-related: [debugging-signals-stack-traces, debugging-methodology-hypothesis-testing, backend-common-errors-diagnostics-from-a-shared-code-path]
+related: [debugging-signals-stack-traces, debugging-methodology-hypothesis-testing, backend-common-errors-diagnostics-from-a-shared-code-path, infrastructure-agent-orchestration-gate-evidence-exit-code-class]
 ---
 
 # Reading an Error Message Before Acting on It

@@ -7,7 +7,7 @@ confidence: field-tested
 sources:
   - https://github.com/obra/superpowers
 last_verified: 2026-08-22
-related: [infrastructure-agent-orchestration-unattended-worker-questions, infrastructure-agent-orchestration-shared-run-state, infrastructure-agent-orchestration-control-signals-vs-primary-artifacts, security-agent-exposure-authorization-scope-persistence]
+related: [infrastructure-agent-orchestration-unattended-worker-questions, infrastructure-agent-orchestration-shared-run-state, infrastructure-agent-orchestration-control-signals-vs-primary-artifacts, security-agent-exposure-authorization-scope-persistence, infrastructure-agent-orchestration-worker-reported-plan-contradiction]
 ---
 
 # Deciding Without a Human During an Unattended Run