Skip to content

Fix storage correctness across acknowledgements and recovery - #4

Merged
JohanLindvall merged 1 commit into
mainfrom
fix/storage-correctness-and-recovery
Oct 24, 2026
Merged

JohanLindvall merged 1 commit into
mainfrom
fix/storage-correctness-and-recovery

Conversation

@JohanLindvall

Copy link
Copy Markdown
Owner

What changes?

Corruption retirement could commit another worker's outstanding reservation, failed commit writes could make retries silently succeed, and interrupted recovery could lose the evidence needed to replay later segments. Partial header writes could also resurrect an append that had returned an error.

  • Keep damaged spans in the reservation ledger until earlier work is acknowledged, preserve commit targets, and roll back unpublished commit state while retaining successfully published progress.
  • Restore partial header overwrites from the last complete header image; latch a durability error when restoration cannot be guaranteed.
  • Persist recovery repairs from the tail backward, preserve replay after empty torn segments, propagate recovery read/write failures, and close handles on failed recovery.
  • Propagate durability failures through concurrent publication, eviction, and acknowledgements; flush directory metadata on explicit Sync and Close; preserve descriptor limits and skip reclaimed files during off-lock flushes.
  • Wake blocked readers and producers after corruption, rollback, flush failure, or a batch codec panic. Preserve delivery metrics on decode failure and reject segment-number exhaustion or existing-file collisions.

Add 28 focused regression tests covering reopened storage, injected failures, concurrent waiters and flushes, and real short header writes. Update package comments and the guide to match the corrected behavior.

How was this checked?

All passed locally:

  • make check: formatting, vet, pinned lint, normal and fault-injection suites with the race detector, and runnable examples.
  • 32-bit normal and fault-injection test suites (GOARCH=386).
  • A 30-second store fuzz run.
  • Cross-compilation for Windows/amd64, Darwin/arm64, and Plan 9/amd64.
  • Allocation checks for BenchmarkAddTake, BenchmarkAddTakeCommit, BenchmarkRead, and BenchmarkBatchCommit: all retain 0 allocs/op.
  • Documentation links and git diff --check.

Compatibility

No public API, on-disk format, dependency, or minimum Go-version changes. Opening a queue now fails if required cursor/count repairs cannot be persisted, and those repairs are flushed even under NoSync. Explicit Sync and Close also flush directory metadata under NoSync. Failed header writes remain retryable only when the previous published image is intact or restored; otherwise they report a latched ErrIO.

@JohanLindvall
JohanLindvall merged commit b473ff3 into main Oct 24, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant