Skip to content

Fix multipart integrity and reduce encrypted read overhead - #156

Open
ServerSideHannes wants to merge 6 commits into
mainfrom
fix/integrity-and-streaming-performance
Open

Fix multipart integrity and reduce encrypted read overhead#156
ServerSideHannes wants to merge 6 commits into
mainfrom
fix/integrity-and-streaming-performance

Conversation

@ServerSideHannes

@ServerSideHannes ServerSideHannes commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Multipart replacements could reuse GCM nonces or overwrite accepted bytes before signature validation. Completion could report success for the wrong upload, and stale or unavailable sidecars could select the wrong read path. This change binds publication and reads to an object generation and keeps each verified part attempt immutable.

  • Stage and validate parts before publishing state, freeze the upload key atomically, assemble in client order, retain state on failed completion, and renew completion leases. Compatible whole-object copies preserve native ciphertext copying; later writers cannot change their key.
  • Publish required manifests before ciphertext, bind completion receipts to bucket/key/upload ID, and use consistent client ETags through GET/HEAD/LIST/conditions. Preserve user metadata and pass atomic write preconditions to the backend.
  • Verify buffered and streaming payloads, including AWS's published chunk-signature vectors; reject malformed/truncated bodies and unsupported trailer modes.
  • Share object resolution and contiguous frame reads, coalesce listing lookups, pool credential-isolated S3 clients, and tie cleanup/metrics to stream lifetime. Preallocated frame buffers remove the fragmentation found in load testing; large legacy seals authenticate through bounded spooling.

Validation: the CI unit selection passed 779 tests locally (including 110 mock integration tests); the separate full unit-directory run passed 670 tests including its slow case. 27 real MinIO/HTTP compatibility tests passed (including the optional Redis scenario), 11 additional copy/concurrency tests passed, and all nine native-copy tests passed, including 1,280 MiB objects and concurrent copies. Ruff lint and format checks pass. Tests cover hash tampering below/at/above 8 MiB, rejected replacement, uncertain state publication, retry after failed completion, key selection, out-of-order assembly, resource cleanup, range recovery, and real backend preconditions.

A local ten-sample 32 MiB GET comparison against d45732d measured median latency of 317.12 → 289.26 ms and sampled RSS of 179.23 → 175.06 MiB. A separate 50-GET run stayed below 176 MiB sampled RSS. These are local measurements, not production forecasts; staging adds storage/copy work to ordinary multipart writes.

Deployment: this is a new write format. Drain legacy active uploads and switch the fleet together; old readers cannot read v3 generations. Persistent Redis is required to resume active uploads across restarts. Configure orphan-attempt lifecycle expiry longer than the supported upload/retry window. Generation manifests are retained because copies and versions may reference them. Unsupported checksum-trailer formats fail explicitly. No deployment or bucket-policy changes are included.

See docs/GENERATION_FORMAT.md for the commit protocol, migration requirements, cleanup policy, limitations and benchmark results; docs/CODE_REVIEW.md retains the original findings.

Final CI status: all 11 pull-request checks passed for commit 2c68c39672badc849ac621ae82c4c8fe7e378cec, including both unit jobs, all seven Linux integration shards, Ruff and Helm lint. Test run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants