Read a page back as text somebody can actually search

[tannevaled]

Found by a robustness campaign over mozilla's pdf.js adversarial corpus (977 files), 2 268 public forms from 18 issuers (including the deliberately broken suites of veraPDF, qpdf, pdfbox, pypdf, pdfcpu and openpdf), and 33 628 arXiv figure PDFs.

Four independent findings, one theme: what it costs to read a page, and whether what comes back is worth having.

---

1. Word breaks that were not there

A run's position came through the text matrix and the page's transform together. Its width, its size and the width of a space in it came through the page's transform alone — the text matrix's own scale was left out.

A document may put the whole of its scale there — /F1 1 Tf followed by a text matrix of ten, rather than /F1 10 Tf — and TeX puts it there in every PDF it has ever produced. Such a document reported one-point text with half-point letters, while saying quite correctly where on the page each run began.

Those are the two numbers a word break is decided by. A gap measured in points was weighed against a space width ten times too small, so every ordinary kern between two letters looked like a space. IRS Form 1040:

before:  F o r t h e y e a r J a n . 1 – D e c . 3 1 , 2 0 2 5
after:   For the year Jan. 1–Dec. 31, 2025

before:  S p o u s e ’ s s o c i a l s e c u r i t y n u m b e r
after:   Spouse’s social security number

Nothing failed and nothing was slow. The page read back as fully readable and was unusable, and no panic or timing measurement would ever have said so.

Measured

Share of extracted words that are one or two letters long — on a page of prose that number should be small:

corpus	before	after
2 268 government forms (2 079 with text, 7 630 pages, 19.4 M characters)	52.42%	25.81%
4 000 arXiv figures (mostly axis labels and single-letter maths, so a high floor)	57.26%	38.43%

On the forms, 1 036 400 fragments became parts of words again and 1.23 million spurious spaces went away.

The threshold was not the problem

wantsSpace was already relative to the font's own space width, not an absolute displacement. Sweeping the factor across an order of magnitude, with a two-sided measure so that running words together would show up too:

factor	1–2 letter words	20+ letter words (run together)
0.10	30.62%	0.027%
0.20	29.73%	0.029%
0.30 (unchanged)	29.32%	0.033%
0.50	29.09%	0.032%
1.00	29.03%	0.036%

A flat plateau: the factor is worth 1.6 points across a tenfold range, against the 26.6 the units were worth. Leaving it at 0.3 is the measured answer, not the untouched one.

Every test in this package wrote its size into Tf, which is why none of them saw this. The new ones draw the same page both ways and require the same answer.

2. Reading the text unpacked every picture

Runs, Text and Images all walk the page through the same walk, and the walk undid every picture's filters whatever the caller had asked for.

2608.05837/figures/torus/teaser.pdf is a 4.96 MB arXiv figure holding 378 MB of image once decompressed. Asking that page what it said cost 1 094 MB and 1.33 seconds to produce 53 text runs — 98.93% of the allocation under noteImage then reader.Decode.

After: 1.110 ms, 5 MB. 1 200× faster, 219× less memory. Images is unchanged and still unpacks them. Where a picture lands is still worked out either way; only the bytes are left alone.

3. Every operation on the page was materialised first

The walk only ever goes forwards, so nothing needs the list reader.Operations builds. 2608.06617/neglected_curvature.pdf holds one page of 84.8 MB of content stream and 4 579 973 operations:

extract.Runs   2180 MB  ->  948 MB

The scanner is the same code the list was built from — Operations is a loop over it — so the operations are the same in the same order.

4. A stale dependency still shipping a known denial of service

reader v0.4.1 stopped counting from zero to the largest object number a file names. This package was still asking for v0.4.0.

bug1980958.pdf is 219 bytes, has no trailer and no startxref, and its last object is numbered 2 147 483 647:

reader.Open on 219 bytes   21.2 s, 0.0 MB allocated  ->  under 1 ms

Not one byte allocated, which is why no memory limit caught it. Measured with GOMAXPROCS(1) — a goroutine waiting for a core cannot tell waiting from working.

Found by the fuzz target this PR adds: it was the 205th seed, and it did not fail the run so much as stop it — the coordinator gave up on the worker as hung.

---

The regression tests, each shown failing first

TestKerningIsNotAWordBreak and TestTheTextMatrixScalesTheRunItself, against the parent commit:

--- FAIL: TestKerningIsNotAWordBreak/scale_in_the_text_matrix
    the page says "O r i g i n a l", and it was drawn as "Original":
    kerning between letters was read as a word break
--- FAIL: TestTheTextMatrixScalesTheRunItself
    size is 1 with the scale in the text matrix and 10 with it in the font size
    a space is 0.5 wide with the scale in the text matrix and 5 with it in the font size

The scale in the font size subtests pass in both, so the fix is aimed at the broken case and leaves the working one alone. TestARealSpaceIsStillAWordBreak is the other half — a fix that never breaks a word would score perfectly on the measure that found this and be useless.

TestTextDoesNotUnpackThePictures asserts on memory rather than time, because memory does not depend on how busy the machine is; reverting only the behaviour fails it with reading the text allocated 16884792 bytes. It also requires Images to still return all 8 000 000 bytes, so it cannot be satisfied by never decoding anything.

TestATinyFileWithAHugeObjectNumber builds the 219-byte file rather than committing somebody else's, and fails in 15.5 s against the stale pin.

What was verified not to change

Before the word-break commit, the text and image output was checked to be byte-identical: every page's runs with their Text, X, Y, Width, Size, Font, Space, Invisible and Unreadable, the assembled text, and every image's dimensions, placement, filter, length and content hash, over 21 311 files.

• Streaming vs materialised operations: identical across all 21 311.
• Two full runs of the stack: identical across all 21 311.
• Against the original: one file differed, and it differed from itself — issue14256.pdf gave four different answers in five runs of the unmodified code. Fixed separately in content: an inline image that spells an entry twice must not be read twice reader#13.

The word-break commit changes text output deliberately, and is measured by the tables above rather than by a hash.

Fuzz target

There was none in this repository. FuzzExtract reads a page back three ways and asserts a time budget as well as absence of panic. 2 438 962 executions over 10 minutes: clean (after finding #4 above). Point PDF_SEEDS at a corpus to seed from it.

What else was looked for and not found

• 2 268 forms and 33 628 arXiv figures through Runs, Text and Images: no panics, no non-termination — re-run after the word-break change with the same result.
• 100.0% statement coverage held throughout.

Gates

• go test ./... green, 100.0% of statements
• go vet clean, gofmt clean, CGO_ENABLED=0
• Builds for linux amd64/arm64/riscv64/loong64/ppc64le/s390x, js/wasm, darwin/arm64, windows/amd64

🤖 Generated with Claude Code