Walk the objects a file has, not every number up to the largest of them - #12
Merged
Conversation
Repairing a damaged file means listing its objects in order, and that was done by counting from nought to the highest object number and asking the table about each. A file may name any object number it likes. One of 219 bytes in the wild — mozilla/pdf.js's bug1980958.pdf — declares object 2147483647 beside its three real ones. Counting up to that is two thousand million map lookups, done twice, for three objects: twenty-five seconds of somebody's afternoon for a file that fits in a tweet, and a denial of service anybody could post. The numbers that exist are sorted and walked instead, which is what the order was wanted for. Twenty-five seconds becomes none. Found by battle-testing the forms library against 1 046 files: 1 029 opened, 14 580 fields read and filled, no panics, and one file that took 12.87 of the run's 13.1 seconds.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Repairing a damaged file means listing its objects in order, and
objectsOfTypedid that by counting from nought to the highest object number and asking the cross-reference table about each one.A file may name any object number it likes. mozilla/pdf.js's
bug1980958.pdfis 219 bytes and declares object 2147483647 beside its three real ones. Counting up to that is two thousand million map lookups, done twice, for three objects.219 bytes in, Open took 25.39s219 bytes in, Open took 0sNo allocation either way — it was a loop, not a memory bomb, which is why nothing caught it.
The numbers that exist are collected, sorted and walked, which is what the order was wanted for in the first place.
Two tests: the 219-byte file itself, with a five-second deadline so it fails loudly rather than hanging the suite; and that the listing is still in object-number order when the numbers are scattered rather than consecutive.
Found by battle-testing
go-pdfkit/formsagainst 1 046 files — the 69 public forms plus mozilla/pdf.js's whole test corpus. 1 029 opened, 14 580 fields read and filled, no panics, and this one file took 12.87 of the run's 13.1 seconds.100% statement coverage,
go vetclean.