Salvage what a damaged stream can give, keep it out of the field callers paint from, and say what went wrong - #16
Merged
Conversation
A stream whose filter chain cannot be run to the end used to yield nothing
at all: Decode returned an error and the caller lost the stream. qpdf's rule
is the opposite, and pdf.js's too — a filter that cannot be applied ends the
chain, and what the filters before it produced is still usually usable. The
one thing neither of them does is pass the salvage off as a clean decode.
This reader did exactly that in one place. flateDecode returned the prefix a
damaged stream managed to inflate together with a nil error, so a truncated
stream was indistinguishable from a whole one. Measured over the corpora, that
is 33 streams in 18 of the 1633 real form documents under /Users/Shared/pdfforms
and 4 streams in 2 of the 635 vendor fixtures: 37 streams the reader called
clean while handing back a fragment.
The other direction was measured too. Over the same 1633 real documents, 263
streams in 212 files (13.0%) fail to decode and yielded no bytes whatever; over
2015 arXiv PDFs from /Users/Shared/axc, 2 streams in 1 file.
So: DecodeRecovering never fails and reports Recovered, Cause and Filter;
Decode is the strict reading and refuses a chain it cannot finish, so a caller
that must not act on damaged data says so by which function it calls. Every
filter now returns the prefix it produced along with the reason it stopped, and
a Flate or LZW prefix still goes through the predictor, which undoes one row at
a time and so applies to a buffer that stops mid-row. The salvage is always as
far down the chain as the filters got — a damaged Flate stream yields what it
inflated, never the compressed bytes. Only a filter that produced nothing falls
back to what went into it.
Object streams and page content use the recovering reading, because an object
stream that stops short still holds whole objects and a page whose content will
not decode is still a page; PageContentDecoded reports whether any salvaging
happened. Cross-reference streams stay strict: a table that has to be guessed
at is worse than no table, and repair() is the answer to one.
Measured, base vs this commit:
1633 real forms pages 9362 -> 9362, content bytes 224709710 -> 224709710,
per-file content hashes identical, open failures 0 -> 0,
streams the strict Decode refuses 263 -> 296 (+33, exactly
the silently truncated ones)
635 vendor fx pages 1236 -> 1236, content bytes 11608189 -> 11608189,
hashes identical, strict refusals 2 -> 6 (+4, likewise)
1184 damaged 1184 real files cut to 92% of their length: pages
3780 -> 3780, extracted content 126.5 MB -> 128.8 MB
(+2 387 301 bytes, +1.89%), open failures 136 -> 136.
No file lost content.
Cost, 409 real forms read into memory first, three interleaved runs each:
1474/1153/848 ms before, 1335/1617/844 ms after — inside the noise of this
machine; peak heap 335-343 MB before, 343-347 MB after; total allocation
563 MB in both.
Fuzz targets for the two contracts the salvage rests on: that Decode and
DecodeRecovering agree on every clean decode and disagree on nothing else,
and that no byte string makes Open hand back a document it cannot walk.
This is a second map-order non-determinism, distinct from the inline-image one fixed in 62002d5: that one was in InlineImage.Expanded, this one is in indexObjectStreams, and PageContent never calls Expanded, so the fix for the first does not touch the second. Measured on 1184 real files cut to 92% of their length, extracting the content of every page and hashing it: v0.4.2, with 62002d5 in 5 runs, 5 different answers 126460833 / 126474891 / 126466402 / 126453047 / 126469752 bytes v0.4.2 + this commit only 5 runs, 1 answer: 126464943 bytes, hash bef99b751ed33040 indexObjectStreams walked d.xref in map order looking for object streams. Two object streams in a damaged file may both claim the same object number — a partial rewrite leaves the old stream in place beside the new one — and the first stream walked wins, because the loop below it declines to overwrite an entry that is already there. So which definition of a shared object the reader took was whichever order the map handed out that run, and the same file gave different pages on different runs. The numbers are now sorted before they are walked, so the lowest-numbered object stream defines a contested object. That is arbitrary but it is a property of the file, which is the point. The test writes the higher-numbered stream first, so agreeing with the file's own layout would not be enough to pass it; it fails on the unsorted walk at run 0.
Not one of the three defects this branch set out to fix, but the biggest real-world decode gap the measurement that looked for them turned up, and the one that decides whether the salvage in dbb2558 tells the truth. Measured over the 1633 real form documents in /Users/Shared/pdfforms — the real ones, not the 635 vendor fixtures — 263 streams in 212 files fail to decode. 209 of those failures, in 209 distinct files (12.8% of the corpus), are the one message "reader: unsupported filter /Crypt". Every one of the 209 files is encrypted, carries /EncryptMetadata false, and puts /Filter[/Crypt] on its /Type/Metadata stream. /Crypt is not a transformation. It names the crypt filter a stream's bytes were encrypted with, and /Identity — which is also what /Crypt with no /Name means — says they were not encrypted at all. So two things were wrong, and either alone leaves the reader worse off than fixing both: - the filter was unimplemented, so the chain failed; - the decryptor decrypted the stream anyway, because it decrypts every stream that is not an /XRef one, so the bytes were noise before the filter ever saw them. Implementing only the filter is measurably the worst of the three states. With the filter in but the stream still decrypted, all 209 streams decode "cleanly" and 0 of 209 are readable XMP: the reader hands back noise and calls it clean. With both halves, 209 of 209 decode cleanly and 209 of 209 are readable XMP, beginning `<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>`. A /Crypt filter that names something other than /Identity is reported rather than waved through: this reader cannot re-apply a named crypt filter at the filter layer, and pretending otherwise would hand back ciphertext that looks like data. There are no such streams in either corpus — 0 of 240 292 streams — so the branch is a guard, not a feature. Measured, base vs this commit: 1633 real forms streams the strict Decode refuses 263 -> 87. That is -209 /Crypt and +33 damaged Flate prefixes that dbb2558 stopped calling clean. Pages 9362 -> 9362, page content bytes 224709710 -> 224709710, per-file content hashes identical, open failures 0 -> 0. 635 vendor fx strict refusals 2 -> 4 (-2 /Crypt, +4 prefixes). Pages, bytes and hashes identical. Page content does not move, and should not: the streams this recovers are metadata, not content. What moves is that a caller asking a form for its XMP now gets its XMP.
Two faults in OpenWithPassword, one of them the masked error this branch went
looking for.
The masking. When the cross-reference tables cannot be read, Open rebuilds the
file by scanning it; when that rebuild also fails, the error returned was the
tables' error and the rebuild's was dropped on the floor:
if rerr := d.repair(); rerr != nil {
return nil, err
}
The tables' error only says that the tables could not be read. The rebuild's
says what the file turned out to be, having read every object header in it.
Measured over the 118 843 arXiv PDFs in /Users/Shared/axc, 30 files fail to
open, and all 30 of them — 100% — reported the wrong thing:
reported reader: no startxref in the last 4096 bytes 30/30
actually found reader: the file holds no indirect objects 28/30
reader: no document catalogue found 2/30
The 28 hold no PDF body at all: one is PostScript, the rest are truncated
before their first object. "No startxref" sends a reader looking for a tail
that was never the problem. The catalogue lookup's own error was thrown away
the same way, replaced by a flat "the trailer does not lead to a catalog", so
"/Root is a null" and "the catalogue has no page tree" arrived as one message.
Both are kept now. The rebuild's diagnosis leads, since it is the one about the
file as it is, and the tables' error follows in parentheses because it says why
there was a rebuild at all. Wrapped with %w, so errors.Is still reaches
ErrWrongPassword and the rest.
The ordering. repair() reads object streams, and in an encrypted file it cannot
read one until the file key is known — but the key was established after the
rebuild, not before. So a file whose catalogue lives in an encrypted object
stream was rebuilt from bytes nobody could read, found no catalogue, and was
refused with the tables' error: "no startxref", for a file that needed a key.
That is a file failing for a decryptable reason and reporting something else.
repair() now establishes the key itself, before it indexes object streams, from
any trailer a scan can see. That needs one more source than scanTrailers: a
file written with cross-reference streams has no trailer keyword anywhere in it,
and its /Encrypt then exists only in a /Type /XRef stream's dictionary — which
is the one stream a PDF never encrypts, precisely so it can be read before the
key is known. The bytes "/Encrypt" not appearing in the file at all short-
circuits the whole thing, since no trailer is ever compressed.
The document remembers its password so that a rebuild triggered later, by a
bad offset mid-read, can do the same.
Measured. The ordering fault bites no file in either corpus: it needs an
encrypted file whose tables are unreadable, and none of the 118 843 arXiv PDFs
or 2 268 forms is one. Shipped anyway, because the file that shows it is three
lines of the package's own writer — NewPackedWriter, Encrypt, drop the
startxref — and on v0.4.2 it reports "no startxref in the last 4096 bytes"
where the answer is the password. The new tests fail on v0.4.2 and pass here.
118 843 arXiv open failures 30 -> 30, pages and per-file content hashes
unchanged; all 30 messages now name what the rebuild found.
Brotli-Prototype-FileA.pdf still cannot be opened — there is
no Brotli in the standard library and this module takes no
dependencies — but it now says "no document catalogue found
(the cross-reference information was unusable too: reader:
unsupported filter /BrotliDecode)" instead of the second
half alone.
2 268 forms open failures unchanged, pages and hashes unchanged.
The post-rebuild key setup this replaces is deleted rather than left dead.
/StmF and /StrF name the crypt filters that protect a file's streams and strings, and both default to /Identity, which leaves bytes alone. A file where both are /Identity — or name a filter its own /CF does not define, which comes to the same thing — has nothing encrypted in its body. /Encrypt is there for /EFF, the embedded files, and for the permission bits. This reader demanded a valid password for such a file anyway, and refused it. Nothing was gained by refusing: the key it was asking for would have been used to decrypt nothing, because both methods resolve to cryptNone either way. The file's every byte is already plain text. Measured over 12 172 files — the 1633 real forms, the 635 vendor fixtures and 9904 arXiv PDFs — 495 declare /Encrypt with readable tables, and their crypt filters divide up like this: /StmF and /StrF both /StdCF 441 both absent, and /V below 4 (no /StmF) 47 both /DefaultCryptFilter 4 both /Identity, /V 5 3 Only the last three are affected, so the blast radius is exactly three files, all openpdf fixtures, all /V 5 /R 6 with /EFF /StdCF and no user password this reader can guess. Before: "reader: the password does not open this file". Now they open, with 30, 1 and 30 pages — which is what poppler's pdfinfo reports for the same three files, and it calls them "Encrypted: no" for this reason. The 47 files with no /StmF are untouched: below /V 4 there are no crypt filters and RC4 protects everything, which readMethods already knows. Nothing is authenticated on this path, and the reader says so rather than implying otherwise: Encrypted() still reports true, Protection() reports the revision and the /P permissions, Method is "none" — the method that really applies to the content — and Owner is false, because no password was checked. This unblocks the fixture that made the case for dbb2558. openpdf's issue375_unfilterable-with-crypt.pdf carries a stream whose /Filter is [/Crypt /ZlateDecode], a filter name invented to be unimplementable. It could not be reached at all before, because the file could not be opened; it opens now, all 30 pages, and the invented filter is salvage rather than a lost page.
19dc306 made the rebuild deterministic by giving a contested object number to the lowest-numbered object stream. Sorting by object number was the wrong tie-break, and the corpus said so. Comparing 1184 real files cut to 92% of their length against v0.4.2, 120 files extracted more page content and one extracted less. That one is worth spelling out, because "got worse" is not quite what happened: on v0.4.2 the file has no single answer at all. Six runs of the same binary on the same bytes: 89170 / 85910 / 88937 / 99927 / 99961 / 102097 bytes 19dc306 gave it one answer, 84229, which is below all six samples. So it was not a regression against a baseline — there was no baseline — but it was still a worse choice than the file could support. The rule now matches the one the header scan a few lines above already applies to objects written directly: a later definition wins, an incremental update having appended it. Object streams are walked latest in the file first, so the newer of two streams claiming the same object defines it. 1184 truncated files v0.4.2 branch pages returning nothing 591 0 pages 3780 3780 content bytes 126452734 137250792 (+8.54%) streams the strict Decode refuses 12371 22 open failures 136 136 files extracting more — 120 files extracting less — 0 Deterministic across three runs: 137250792 bytes, hash abdff1d485f8dd38, every time. Of those bytes, 137209833 are clean decodes and 36185 across 4 pages are flagged salvage — so the gain is decoded content, not compressed bytes counted as if they were content. The test now writes the higher-numbered stream first, which a rule going by object number would fail.
The filters list gains /Crypt, which the encryption paragraph below already claimed. Two paragraphs say what a reader has to be told: which of Decode and DecodeRecovering it wants, and that a rebuild is now deterministic, can reach an encrypted object stream, and reports what it found.
A flag is not a guarantee. dbb2558 returned the bytes a failed chain could not get past in Decoded.Data with Recovered set beside them, which asks every caller to check a field before believing another one. render has just been found painting 273 encoded image masks as one-bit samples on 51 first pages of real forms — noise, unnoticed, because a page with noise on it looks like a page with something on it. That is this shape of bug, and a fallback that can feed it is not finished. So the two kinds of bytes go in different fields: Decoded.Data what the chain decoded. Fewer bytes than the stream meant to carry when Recovered is set, but bytes of that kind. Decoded.Undecoded what the chain could not get past, still in the encoding Filter names. Set instead of Data, never beside it. A caller reading Data cannot be handed a compressed stream by accident, and one that wants the stream as it lies asks for it by a name that says what it is. Two holes showed up the moment the guarantee was written down as a test, both of them the exact failure described above: - cryptFilter returned the bytes it could not decrypt alongside its error, so ciphertext arrived as Data. It returns nothing now. - v0.5.0 decodes /CCITTFaxDecode in applyFilter rather than stopping at it, so a fax whose /Columns is absurd, or whose pixel count runs past the limit, now fails inside the chain. Its still-encoded bytes were landing in Data — a fax handed to a stencil path as though it were samples. TestUndecodedBytesNeverArriveAsData asserts it over six shapes of failure (unimplemented filter, Flate that is not Flate, two bad faxes, an unreadable /Filter, an unapplicable crypt filter), and FuzzDecodeRecovering asserts that Data and Undecoded are never both set and that Undecoded never appears on a clean decode. Page content follows: a page whose content stream cannot be decoded at all now yields no bytes rather than compressed ones, and PageContentDecoded says why. Object streams likewise parse only Data, so a rebuild cannot take compressed bytes for objects. Measured, v0.5.0 vs this branch: 1633 real forms pages 9362 -> 9362, content bytes 224709710 unchanged, every per-file content hash identical, open failures 0. Streams the strict Decode refuses 263 -> 87. 635 vendor fx open failures 31 -> 28, pages 1236 -> 1297, 3 files change. 1184 truncated open failures 136 -> 136, pages 3780 -> 3780, content 126466671 -> 137214633 bytes (+8.50%), pages returning nothing 591 -> 0, strict refusals 12373 -> 22. Of the branch's 137 214 633 bytes, 137 214 607 are clean decodes and 26 across 4 pages are flagged salvage; a further 29 320 bytes sit in Undecoded on 95 pages and are counted nowhere near the content total, which is the point. The byte figure went down by 36 159 against the pre-split design and is worth more for it: what it counts is now decoded content only. One of the 1184 extracts fewer bytes than the v0.5.0 sample in that table (1227119 -> 1213608). v0.5.0 has no single answer for that file: six runs give 1213608, 1218311, 1221792, 1223638 twice, and 1227119. The branch's answer is one of them, arrived at the same way every time. The two openpdf fixtures, re-measured after the rebase: unfilterable-with-crypt.pdf opens, 30 pages — poppler's number. 38 of its 40 streams decode cleanly; the 2 whose /Filter is [/Crypt /ZlateDecode] are Undecoded, so the invented filter cannot masquerade as anything. Brotli-Prototype-FileA.pdf still refused, and rightly: its cross-reference stream is Brotli, there is no Brotli in the standard library, and a table that has to be guessed at is the one place salvage is wrong.
tannevaled
force-pushed
the
robustness-corpus-fixes
branch
from
August 27, 2026 13:31
f897ae7 to
c89f94c
Compare
tannevaled
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebased on v0.5.0; every number below is re-measured against it.
A bibliography pass over qpdf, pdf.js, MuPDF and pdfium named three defects.
Two were real, one does not exist, and looking for the third turned up three
more that were. Eight commits: five fixes, one refinement the corpus asked for,
one guarantee the render bug asked for, one README.
Measured over 118 843 arXiv PDFs in
/Users/Shared/axc, the 1 633 realdocuments of
/Users/Shared/pdfforms(its 635 vendor fixtures countedseparately, never folded into a prevalence figure), and 1 184 of those real
files cut to 92% of their length — the damage the recovery paths exist for.
1. A stream whose filter fails should fall back to its raw bytes — real
dbb2558, hardened byc89f94c. Half was already there and lying about it:flateDecodereturned the prefix a damaged stream managed to inflate with anilerror, so 37 streams (33 in 18 real forms, 4 in 2 fixtures) werereported as clean decodes of a fragment. The other half was missing: 263
streams in 212 of the 1 633 real forms (13.0%) failed and yielded nothing.
DecodeRecoveringnever fails and reportsRecovered,Cause,Filter;Decoderefuses a chain it cannot finish. A caller says which it wants by whichone it calls.
On the masquerade you warned about. A flag is not a guarantee, so the two
kinds of bytes are in different fields:
Decoded.DataRecoveredis set, but bytes of that kindDecoded.UndecodedFilternames — set instead ofData, never beside itA caller reading
Datacannot be handed a compressed stream by accident, andone that wants the stream as it lies asks for it by a name that says what it is.
PageContentreturnsDataonly, so a content scanner can never be fedcompressed bytes; object streams parse
Dataonly, so a rebuild cannot takecompressed bytes for objects.
Writing that guarantee down as a test found two holes immediately, both of them
your bug's shape:
cryptFilterreturned the bytes it could not decrypt alongside its error, sociphertext arrived as
Data./CCITTFaxDecodenow decoded inapplyFilterrather than stopping the chain, a fax with an absurd
/Columnsor a pixelcount past the limit fails inside the chain — and its still-encoded bytes
were landing in
Data. A fax handed to a stencil path as though it weresamples, which is exactly what
renderwas doing.TestUndecodedBytesNeverArriveAsDataasserts it over six failure shapes;FuzzDecodeRecoveringassertsDataandUndecodedare never both set.2.
/Lengthdisagreeing with the cross-reference table — does not existNo commit. The cross-check is already implemented, in
parse.go readStream:/Lengthis trusted only whenendstreamreally does follow at that offset,and otherwise the extent is recovered by scanning. I went looking for the
residual gap qpdf's next-object bound would close, over 344 367 streams in
4 283 files:
/Lengthnot corroborated byendstream, so scanned forendstreamcandidate before the next object headerN G objheader/Lengththe reader trusted/Lengthis wrong often — 149 of the 1 633 real forms write it as an indirectreference, one file has 44 streams whose
/Lengthputsendstreamnowhere —but the existing scan recovers every one, unambiguously. Measured, does not
occur here; no fix, because there is nothing to fix.
3. A masked error in
OpenWithPassword— real, and worse than described2ea1bfe. Two faults.The masking. When the tables cannot be read and the rebuild also fails, the
error returned was the tables' and the rebuild's was dropped. Over the full
arXiv corpus, 30 files fail to open and all 30 reported the wrong cause:
no startxref in the last 4096 bytesthe file holds no indirect objectsno startxref in the last 4096 bytesno document catalogue foundThe 28 hold no PDF body at all — one is PostScript, the rest truncated before
their first object. The catalogue lookup's error was flattened the same way, so
"
/Rootis a null" and "the catalogue has no page tree" arrived as one message.Both are kept now, rebuild first, wrapped with
%wsoerrors.Isstill reachesErrWrongPassword.The ordering.
repair()reads object streams, and in an encrypted file itcannot read one until the key is known — but the key was established after the
rebuild. So a file whose catalogue lives in an encrypted object stream was
rebuilt from bytes nobody could read and refused with "no startxref", for a file
that needed a password.
repair()establishes the key itself now, taking/Encryptfrom a cross-reference stream's dictionary when the file has notrailer keyword to name it in — which is every file a modern producer writes.
This one bites no file in either corpus: it needs an encrypted file with
unreadable tables, and none of the 121 111 files is one. Shipped anyway, because
the file that shows it is three lines of the package's own writer, and on v0.5.0
it reports "no startxref" where the answer is the password.
What looking for those three turned up
38b34df— the/Cryptfilter, and the streams that carry it. The biggestreal-world decode gap in the corpus: of the 263 failing streams in the real
forms, 209 — in 209 files, 12.8% of the corpus — are one message,
unsupported filter /Crypt. All 209 are encrypted, carry/EncryptMetadata false, and put/Filter[/Crypt]on their metadata stream.Fixing either half alone is worse than fixing both:
The middle row is the trap: the reader hands back noise and calls it clean.
19dc306+062161b— a second map-order non-determinism, distinct from theInlineImage.Expandedone fixed in #14: this one is inindexObjectStreams,and
PageContentnever callsExpanded. v0.4.2 with that fix in still gavefive different answers in five runs over the truncated corpus.
19dc306made it one; the corpus then showed the tie-break was wrong, so
062161bchanged it to the rule the header scan already applies to direct objects — the
later definition in the file wins.
035190c— a file whose crypt filters say nothing in it is encrypted. Of495 files declaring
/Encryptacross 12 172 examined, 3 are/V 5with/StmFand/StrFboth/Identity: their bodies are plain text, the keyexists for
/EFFalone, and demanding a password gained nothing because bothmethods resolve to
cryptNoneeither way. They now open with 30, 1 and 30 pages— exactly what poppler's
pdfinforeports, calling themEncrypted: no.Nothing is authenticated there and the reader says so:
Encrypted()still true,Protection().Method"none",Ownerfalse. The 47 files with no/StmFareuntouched, being below
/V 4where RC4 protects everything.The two files you flagged, re-measured after the rebase
issue375_unfilterable-with-crypt.pdf— opens, 30 pages, poppler's number.It needed
035190c(/V 5 /R 6, both crypt filters/Identity,/EFF /StdCF,a user password nobody has). Behind that gate it is the ideal witness for defect
1: 38 of its 40 streams decode cleanly, and the 2 whose
/Filteris[/Crypt /ZlateDecode]— a filter name invented to be unimplementable — comeback as
Undecoded, so the invented filter cannot masquerade as anything.Brotli-Prototype-FileA.pdf— still refused, and rightly. Itscross-reference stream is Brotli-encoded; there is no Brotli in the standard
library and this module takes no dependencies. A table that has to be guessed at
is the one place salvage is the wrong answer. What changed is the diagnosis:
no document catalogue found (the cross-reference information was unusable too: reader: unsupported filter /BrotliDecode)instead of the second half alone.What changed, measured on the output
118 843 arXiv PDFs — the healthy case:
Two files differ, by one byte each, and it is accounted for: both have a
/Contentsarray holding an empty stream, for which v0.5.0 emitted a redundantthird newline separator. Operation count and operator-sequence hash are
identical on both pages (3293 ops /
1fb979ca71f7e2b3, 1806 ops /8393f0e7ff0968b4), so nothing about the content changed.The four extra rebuilds are files whose cross-reference stream is a truncated
Flate stream: v0.5.0 opened them from a half-inflated table, the branch refuses
the table and scans every object header instead. Same output, sounder route.
1 633 real forms: pages 9362 → 9362, content bytes 224 709 710 unchanged,
every per-file content hash identical, open failures 0 → 0, streams the
strict
Decoderefuses 263 → 87 (−209/Crypt, +33 damaged prefixes itstopped calling clean).
635 vendor fixtures: open failures 31 → 28, pages 1236 → 1297, 3 files
change — the three
/Identityfiles above.1 184 real files cut to 92% — the damaged case:
DecoderefusesOf the branch's 137 214 633 bytes, 137 214 607 are clean decodes and 26
across 4 pages are flagged salvage. A further 29 320 bytes on 95 pages sit in
Undecodedand are counted nowhere near the content total — which is the pointof the split, and why this figure is 36 159 bytes lower than the pre-split
design's and worth more.
One of the 1 184 extracts fewer bytes than the v0.5.0 sample in that table
(1 227 119 → 1 213 608). v0.5.0 has no single answer for that file: six runs
give 1 213 608, 1 218 311, 1 221 792, 1 223 638 twice, 1 227 119. The branch's
answer is one of them, arrived at the same way every time.
Cost
Interleaved runs of one harness against both trees, files read into memory first
so the timing is the reader's:
Flat on healthy files. On the damaged path, recovering 8.5% more content costs
about 12% more allocation and nothing in wall clock beyond this machine's noise.
Gates
go vetclean,gofmtclean,-raceclean, exactly 100.0% statementcoverage. Cross-compiles 9/9 for
linux/{amd64,arm64,riscv64,loong64,ppc64le,s390x}, js/wasm, darwin/arm64,
windows/amd64 — built one target per line rather than through a word-split
loop, with bogus
GOOSandGOARCHas controls to prove the environment washonoured.
Three fuzz targets where there were none:
FuzzDecodeRecovering(thatDecodeand
DecodeRecoveringagree on every clean decode, thatRecoveredandCausenever disagree, and that
DataandUndecodedare never both set) andFuzzOpen(that no byte string makesOpenhand back a document it cannotwalk).
Every new test was checked against the parent commit: the ones that assert a fix
fail there and pass here.