Line scanner reuses its buffer across files

[SUSTAPLE117]

Stacked on #69.

Summary

Every line scan (scanFileLines, scanReaderLines, and the shell history probe) allocated a fresh 64 KiB bufio.Scanner buffer. Most scanned files are small configs and dotfiles that use only a fraction of it.

Line scans now take that buffer from a sync.Pool through newLineScanner and give it back when the scan finishes:

• Line size limits are unchanged. The scanner still starts with 64 KiB and grows up to the same maxLineSize. A line over the limit still stops the scan with ErrTooLong, which is logged as before.
• No buffer is shared while in use. Each scan holds its own pooled buffer until release runs, which is deferred to the end of the scan. Every line is copied out with scanner.Text(), so nothing still points into the buffer once it returns to the pool.

Tests

• TestScanFileLines_SmallFilesDoNotEachAllocateALineBuffer: scanning 200 small files allocates under 16 KiB per file (before: about 67 KiB per file). It is skipped under -race.
• TestScanReaderLines_MaxLineSize: a line just under the limit is read whole and lines after it are still scanned. A line over the limit stops the scan, keeping the findings before it. This test passed before the change and still passes, which pins the existing behaviour.
• TestScanFileLines_ConcurrentScansKeepTheirOwnLines: 128 concurrent scans over 32 files. Each scan must return only its own finding, with the right path and line number. Run under -race.
• Existing probe tests are unchanged and pass.

Benchmark

BenchmarkScanFileLines_SmallFiles (200 three-line config files per op, default registry, Apple M1 Pro, n=6):

                            │    before     │                after                │
                            │    sec/op     │   sec/op     vs base                │
ScanFileLines_SmallFiles-10   9.856m ± 14%   7.859m ± 19%  -20.26% (p=0.004 n=6)
                            │     B/op      │     B/op     vs base                │
ScanFileLines_SmallFiles-10   13.29Mi ± 0%   0.16Mi ± 6%   -98.78% (p=0.002 n=6)
                            │   allocs/op   │  allocs/op   vs base                │
ScanFileLines_SmallFiles-10    2.479k ± 0%    2.406k ± 0%   -2.94% (p=0.002 n=6)

Checks

go test ./... -cover, go test -race -count=1 ./... and golangci-lint run (0 issues) all pass.

🤖 Generated with Claude Code

[Copilot]

Copilot review overview

🟢 Approval recommended

Deferred buffer release preserves scan ownership and existing limits, with regression coverage and no unresolved findings.

Review effort: Balanced
Findings: None

What changed in this PR

Reduces allocation overhead in Bagel’s line-based probes by reusing scanner buffers, complementing the detector allocation improvements in #69.

Changes:

• Adds pooled 64 KiB buffers while preserving line-size limits and per-scan ownership.
• Adds allocation, line-limit, and concurrency tests plus a small-file benchmark.

File	Description
pkg/​probe/​shell_history.go	Uses the pooled scanner for history files.
pkg/​probe/​race_on_test.go	Enables skipping allocation checks under race detection.
pkg/​probe/​race_off_test.go	Enables allocation checks in ordinary test runs.
pkg/​probe/​probe.go	Adds the buffer pool and shared scanner helper.
pkg/​probe/​probe_test.go	Adds regression tests and an allocation benchmark.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.