Repository navigation
test: add never-panic harness for lexer and parser - #93
Open
shayyz-code wants to merge 2 commits into
Open
shayyz-code wants to merge 2 commits into
shayyz-code wants to merge 2 commits into
Conversation
Feed Parser::parse_checked every prefix of the example and conformance files, random edits of them, random token sequences, random Unicode, and pathological inputs (deep nesting, 1 MiB of whitespace or comments, 30-digit literals, a 100k-char identifier, unterminated literals). Each input runs under catch_unwind on a worker thread with an 8 MiB stack. Unknown panics fail the test with a minimized input; known stack overflows and panics are listed with their issue in tests/never_panics/known_crashers.txt. A deterministic xorshift64* PRNG is seeded from POO_FUZZ_SEED and sized by POO_FUZZ_ITERS. The default run takes about 2 s in a debug build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
tests/never_panics.rs, a zero-dependency harness that checks the lexer and parser return errors instead of panicking. It drivesParser::parse_checked, which runs the lexer. The lexer's own checked API is crate-private, and execution is out of scope until the interpreter has budgets.Inputs:
examples/*.pooandtests/conformance/**/*.poo(the conformance directory is optional, so this works before test: add conformance corpus harness #79 lands)<:,>>,..and non-ASCII text (မ,é, emoji, zero-width space, BOM)(/[/{/if/not/-nesting, a 10k-term binary chain, 1 MiB of whitespace, 1 MiB of// //comments, a 1 MiB comment body, 30-digit integer, range and float literals, a 100k-char identifier, a 1 MiB string, unterminated string/escape/char/comment at EOF, non-ASCII identifiers and strings, NUL and control characters, a BOM prefix, and empty inputHow it works:
POO_FUZZ_SEED(decimal or0xhex).POO_FUZZ_ITERSsets the number of random inputs per generator (default 5000). Each generator gets its own stream, so skipping one does not change the inputs of another.catch_unwind. A quiet panic hook records worker panics so they don't flood the output. Panics on other threads still print, and the previous hook is restored before reporting. If a parse doesn't finish within 30 s, the test fails as a hang instead of blocking.tests/never_panics/known_crashers.txtlists known crashers with their issue. Ageneratorentry skips that generator, which is needed for stack overflows because they abort the process. Apanicentry turns a caught panic whose message contains the key into a "known" result.POO_FUZZ_INCLUDE_KNOWN=1also runs the skipped generators, andPOO_FUZZ_ONLY=<generator>runs just one. Together they confirm a fix.Issue
Closes #80
Verification
just check: fmt, check, clippy-D warnings,cargo test, mdBook build and test,dist generate --check,cargo deny check. All pass.cargo test --test never_panics -- --nocaptureon this machine (debug build): about 2 s for the whole run (1.6 to 2.0 s across runs). Summary:never_panics: seed=0xb00cafe5eed00080 iters=5000 corpus_files=2 inputs_tried=16273 known_skipped=9 known_panics=72 failures=01and0xdeadbeef.#48and#46entries makes the test fail as expected, printing minimized inputs"9999999999999999999","12345678901234567890.."and"poof f(a int s str)>>int{return x;}".Known crashers found:
i64::MAX, e.g.9999999999999999999or…..(lexer.rslines 170 and 184)// //comments: the lexer recurses once per comment (same pattern as #49)(,[,if true {,not,-(balanced or not): the parser overflows an 8 MiB debug stack at 1,000 to 2,000 levelsUndefined variable:in parse-time type inference, e.g.poof f(a int)>>int{return x;}Undefined function:in parse-time type inference, e.g.poof g()>>void{return n();}Unsupported binary operation:, e.g.poof f(s str){return "">'';}Unexpected expression:(listed from the #47 audit report; the default seed does not hit it)#45, #46, #47 and #50 each cover only specific programs. #96 tracks making parse-time type inference return errors instead of panicking in general. The second commit,
test: link known crashers to tracking issues, puts these issue numbers intoknown_crashers.txt.No hangs were found, and no panics outside these families.
just checkDocumentation and Visual Impact
None. This only adds a test, and user-visible behaviour does not change. Usage is documented in the module comment of
tests/never_panics.rsand in the header ofknown_crashers.txt.🤖 Generated with Claude Code