ops

The verbs of go-pdfkit: what people actually do to a PDF they already have. Merge two files, pull out pages three to seven, turn a page the right way up, drop the metadata, split a report into chapters.

Built on reader, which takes a file apart and puts one back together; nothing outside the Go standard library is used, so this builds for GOOS=js/wasm and every architecture the fleet targets.

The model

A document here is an ordered list of pages, each borrowed from a source file, plus the document-level pieces. Every operation rearranges that list or annotates its entries; nothing is applied until the document is written out. An operation therefore costs nothing until it has to, several files can be mixed freely, and — the reason the model is shaped this way — the list is a plain value that two people can edit at the same time.

What it does

go install github.com/go-pdfkit/ops/cmd/pdfops@latest

pdfops merge whole.pdf part1.pdf part2.pdf
pdfops select -pages 3-7,last report.pdf extract.pdf
pdfops delete -pages even scan.pdf fronts.pdf
pdfops rotate -pages all -by 90 sideways.pdf upright.pdf
pdfops crop -box 20,20,575,820 wide.pdf trimmed.pdf
pdfops split -every 10 book.pdf chapters/
pdfops reverse back-to-front.pdf right-way-round.pdf
pdfops nup -n 4 slides.pdf handout.pdf
pdfops booklet chapter.pdf to-fold.pdf
pdfops overlay -with letterhead.pdf plain.pdf headed.pdf
pdfops blank -before 3 report.pdf report-with-a-gap.pdf
pdfops watermark -text CONFIDENTIAL contract.pdf marked.pdf
pdfops number -format "{page} of {pages}" report.pdf numbered.pdf
pdfops bates -prefix ACME- -start 1 exhibits.pdf stamped.pdf
pdfops stamp -text "seen 25 August" -at top-right -bold notes.pdf seen.pdf
pdfops sanitize downloaded.pdf safe.pdf
pdfops flatten filled-form.pdf final.pdf
pdfops strip -annotations -bookmarks private.pdf clean.pdf
pdfops compress fat.pdf lean.pdf
pdfops encrypt -user letmein -allow print,copy plain.pdf locked.pdf
pdfops -password letmein decrypt locked.pdf plain.pdf
pdfops -password letmein permissions locked.pdf
pdfops text paper.pdf
pdfops text -layout -pages 1 paper.pdf
pdfops images paper.pdf pictures/
pdfops info file.pdf

A page range is written 1-3,7,10- and may say all, even, odd or last. It keeps its own order and its own repeats, so select -pages 3-1 reverses three pages and select -pages 1,1 gives you two copies.

-password opens an encrypted file.

text reads the page back as words, and -layout says where every piece of it sits — a page, a place, a size, and what it says. A piece the document gave no way to read comes back marked rather than guessed at, so you can tell a page that says nothing from one that could not be read. images writes out the pictures a page places, a JPEG as a JPEG.

Links and bookmarks are carried over and pointed at the pages they became here, so extracting three pages of a book leaves the links between those three working and drops the ones that led out of them. A link or a bookmark that already led nowhere in the source keeps its place without a destination, because a file with broken bookmarks should keep its shape rather than lose it.

The catalogue is always written from nothing, so a document-level script, an action that runs on opening, or a tree of embedded files never survives any operation at all. sanitize deals with what travels attached to a page: its own actions, an annotation's actions, JavaScript, launching, form submission, the annotation types that exist to play or embed something, and files associated with a page. A link to the web is not executable and stays.

What a screen reader follows

A tagged document says which run of marks on which page is a heading, a paragraph, a table cell, the label of a form field. That is the reading order a screen reader follows, and for a government form it is often what the law requires — RGAA in France, Section 508 in the United States, EN 301 549 in the EU. It cannot be copied across a rebuild: every part of it points into the document, and the number tree that indexes it is keyed by a number each page carries. So it is rebuilt, element by element, around the pages that survived, and those numbers are handed out afresh. A page that kept the number it had in a file it is no longer part of is a page a reader would look up and be told, with every confidence, about somebody else's.

An element whose every page has gone is removed and its parent pruned; one with some pages gone keeps the children that are left. An empty table cell is kept, because the shape of a table is part of what it says. A page written twice carries the structure once, on the first copy, since an element names one page. Merging two files carries no tree at all: two files have two role maps in which the same name may mean two different things, and there is no honest way to choose between them.

Of 1 633 real government forms from eighteen issuers, 1 021 carry a structure tree. After a rotate 1 014 still do — the seven that do not are the seven whose tree was empty in the file we were given — and 991 of the 1 021 are identical down to the last element, mark and annotation reference. The thirty that differ are accounted for one by one in the commit that added this. What cannot be carried is named in catalogue.go with the reason, rather than disappearing quietly.

Text is drawn in the four faces every viewer already has — Helvetica, Helvetica-Bold, Courier and Courier-Bold — so nothing is embedded and a watermark costs about a kilobyte. Stamp text may say {page}, {pages} and {n}, the last being a counter for serial numbering.

Verified against real files

Every operation is checked on the same corpus of 118 863 real PDFs the reader is measured against — Matplotlib, cairo, pdfTeX, Ghostscript, Adobe, R, Apache FOP, PDF 1.3 through 1.7. For each of the 118 833 that open, four properties have to hold, compared on the bytes of every page's content stream rather than on an exit status:

• writing the document unchanged reproduces every page;
• reversing twice is the identity;
• merging a document with itself doubles it exactly;
• selecting the last page yields exactly that page.

All four hold on all 118 833 files.

Laying pages out is checked the same way: for each of the 1 959 multi-page files in that corpus, two-up produces the right number of sheets and every form drawn on them carries exactly the bytes of the page it stands for, blanks included. All 1 959 hold.

Testing

go test -covermode=set ./...

CI gates on exact 100% statement coverage, go vet, and a cross-compile across linux/{amd64,arm64,riscv64,loong64,ppc64le,s390x}, js/wasm, darwin/arm64 and windows/amd64.

The rest of the family

reader reads and writes the format · ops is the verbs and the pdfops command · render turns a page into pixels · pdffont is what a document says about a font · extract reads a page back · coedit is a PDF several people edit at once · app is the browser workbench.

Documentation for all of it: https://go-pdfkit.github.io/docs/

License

BSD-3-Clause — see LICENSE. Copyright the go-pdfkit/ops authors.

Forms

pdfops fields lists what a form asks for and what it holds; pdfops fill fills it in and saves it.

$ pdfops fields fw9.pdf
note: the file also carries an XFA form, which is not read; the standard one is.
topmostSubform[0].Page1[0].f1_01[0]                       text      ""
topmostSubform[0].Page1[0].Boxes3a-b_ReadOrder[0].c1_1[0] checkbox  "Off"
                                                            buttons [1]

$ pdfops fill -set 'topmostSubform[0].Page1[0].f1_01[0]=Wolfgang Amadeus Mozart' \
              -set 'topmostSubform[0].Page1[0].Boxes3a-b_ReadOrder[0].c1_1[0]=1' \
              fw9.pdf filled.pdf

A filled form is written as an incremental update: the original file, byte for byte, with the objects that changed appended after it and a new cross-reference section pointing back at the old one. That is how every program that saves a form saves one, and it is the safest thing a program can do to somebody's document — nothing already there is rewritten, so whatever this does not understand survives untouched, and if the update is wrong the original is still the first part of the file.

The update says where its objects went the same way the file already does. A file whose cross-reference is a stream cannot be pointed back at by a plain table: a reader following /Prev would find an object where it expected the word xref. That is not a nicety — macOS's own renderer draws nothing at all for such a file, which is how the mistake was found.

Two things it will not do. A document that had to be repaired to be read has no cross-reference section worth pointing back at, so it is refused rather than added to. A document that is encrypted has every string and stream in it written through a key, and this does not yet write into one.

Measured on a real form: fw9.pdf filled in every field, written out, read back with every value in place, and rendered by macOS — which drew what we drew, in the same places, comb cells and ticks included.

⚠ Filling a form is the only verb that keeps one. Every other verb here takes the pages apart and builds a new document round them, and a form is tied into a document by object number in a dozen places at once — so merging, splitting or rotating a form's pages loses the form. Use fill on the file itself.