Skip to content

Detectors allocate a findings slice only when they find something - #69

Merged
SUSTAPLE117 merged 1 commit into
mainfrom
perf/lazy-findings
Oct 5, 2026
Merged

SUSTAPLE117 merged 1 commit into
mainfrom
perf/lazy-findings

Conversation

@SUSTAPLE117

Copy link
Copy Markdown
Contributor

Summary

Registry.DetectAll runs once per scanned line, and it started every call with a findings slice sized to the whole registry (16 slots in the default registry). The Stripe detector did the same with room for four findings. Almost every line finds nothing, so nearly all of that was allocated for nothing.

Both now start from a nil slice and append only when something matches. Returning nil for "no findings" is what callers already handle: every one ranges over the result, appends it, or checks len. Findings, their order and their fields don't change.

Detector audit (default registry)

Detector Before Change
Registry.DetectAll make(0, len(r.detectors)) per call now lazy
stripe-key make(0, 4) per call now lazy
github-token var findings none needed
npm-token var findings none needed
ai-service var findings none needed
http-authentication var findings none needed
cloud-credentials var findings none needed
vault-token var findings none needed
generic-api-key var findings none needed
jwt var findings none needed
ssh-private-key make(0, len(matches)) none needed: zero matches means zero capacity, which doesn't allocate
pypi-token make(0, len(matches)) none needed (same reason)
wireguard-key make(0, len(matches)) none needed (same reason)
splunk-token make(0, len(matches)) none needed (same reason)
database-connection-string make(0, len(matches)) none needed (same reason)
slack-token make(0, len(ws)+len(app)) none needed (same reason)
twilio-key make(0, len(matches)) none needed (same reason)

I also measured it: testing.AllocsPerRun over each detector's Detect on finding-free lines showed only stripe-key allocating (1 alloc per call). After the change, every detector and DetectAll allocate nothing on those lines.

Tests

  • TestRegistry_DetectAllCleanLineAllocatesNothing: DetectAll on a set of finding-free lines (including near-miss prefixes such as sk_, ghp_, AKIA) allocates 0 times. It is skipped under -race, because race instrumentation changes escape analysis.
  • TestRegistry_DetectAllKeepsRegistrationOrder: findings from several detectors still come back in registration order, with probe and line metadata set.
  • No existing detector test or fixture changed.

Benchmark

BenchmarkRegistry_DetectAllCleanLines (default registry, 5 finding-free lines per op, Apple M1 Pro, n=6):

                                │   before    │                after                 │
                                │    B/op     │    B/op     vs base                  │
Registry_DetectAllCleanLines-10   20.64Ki ± 0%   0.008Ki ± ?  -99.96% (p=0.002 n=6)
                                │  allocs/op  │  allocs/op  vs base                  │
Registry_DetectAllCleanLines-10     10.00 ± 0%    0.00 ± 0%  -100.00% (p=0.002 n=6)

Time per op is unchanged within noise (91.7µs vs 92.6µs; regex matching dominates).

Checks

go test ./... -cover, go test -race -count=1 ./... and golangci-lint run (0 issues) all pass.

🤖 Generated with Claude Code

Registry.DetectAll preallocated a slice sized to the whole registry on every
call, and the Stripe detector preallocated room for four findings, even though
almost every scanned line finds nothing. Both now start from a nil slice and
append on a match. Findings, order and fields are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@SUSTAPLE117
SUSTAPLE117 added this pull request to stack #71 October 2, 2026 20:52
@SUSTAPLE117
SUSTAPLE117 marked this pull request as ready for review October 2, 2026 20:54
Copilot AI balanced review requested due to automatic review settings October 2, 2026 20:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The allocation optimization preserves behavior and is covered by focused regression tests.

Review effort: Balanced
Findings: None

What changed in this PR

Reduces detector allocations on finding-free input without changing finding order or metadata.

Changes:

  • Lazily allocate registry and Stripe finding slices.
  • Add allocation, ordering, and benchmark coverage.
  • Handle allocation tests correctly under race instrumentation.
File Description
pkg/​detector/​detector.go Lazily initializes aggregate findings.
pkg/​detector/​stripe_key.go Lazily initializes Stripe findings.
pkg/​detector/​detector_test.go Tests allocations and ordering; adds benchmark.
pkg/​detector/​race_on_test.go Identifies race-enabled tests.
pkg/​detector/​race_off_test.go Identifies non-race tests.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@SUSTAPLE117
SUSTAPLE117 requested review from a team October 5, 2026 14:37
@SUSTAPLE117
SUSTAPLE117 merged commit ca07a1a into main Oct 5, 2026
7 checks passed
@SUSTAPLE117
SUSTAPLE117 deleted the perf/lazy-findings branch October 5, 2026 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants