Skip to content

perf: reduce await context parser duplication - #73

Merged
exKAZUu merged 2 commits into
mainfrom
perf/reduce-await-parser-cost
Oct 6, 2026
Merged

exKAZUu merged 2 commits into
mainfrom
perf/reduce-await-parser-cost

Conversation

@exKAZUu

@exKAZUu exKAZUu commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Close #59

Why

The await grammar permits three independent optional contexts before the same canonical operand. Consolidating them reduces generated parser duplication while preserving the emitted operand/postfix boundaries needed for incremental reuse. With the existing 1.0.16 generator, C shrinks 154,840 bytes (3.08%), Wasm 26,320 bytes (3.31%) and states 60 (1.72%) against current main.

Requirements

  • required: Improve the requested grammar repositories, check existing PRs to prevent duplicates, and create a separate PR for each eligible repair.
  • required: Reduce the generated parser cost from the await boundary repair in fix: keep await operands across line breaks #53 while retaining its contracts.
  • required: Preserve the named await operand, public expression and primary_expression query compilation and captures, comment nodes and source ranges, nested arrow grouping, and incremental restoration.
  • required: Retain rejection of optional-chain and unary assignment targets.
  • required: Compare generated C bytes, Wasm bytes, state counts and parse time against fix: keep await operands across line breaks #53 and main 61650b1 with identical build options; demonstrate a smaller artifact and no slower representative parsing.
  • required: Pass native/Wasm corpus, incremental fuzzing, public query membership and body-range checks, including postfix updates followed by newline suffix statements and comments.
  • required: Run review-booster and complete-pr simultaneously, then release after the review and readiness gates.
  • required: Preserve the exported JavaScript grammar's TypeScript/TSX caller contracts, existing public node metadata, external token order and serialized scanner state.
  • chosen: Consolidate the three optional leading await contexts into one optional choice, retaining the yield-start/context pair, named expression operand and mandatory emitted operand end. All 199 corpus and 2,552 focused full trees/captures remain exact on the existing 1.0.16 and independent 1.2.2 runtimes; C/Wasm shrink 3.08%/3.31% with existing generation, without changing the scanner. Retain the local rationale: independent optional prefixes recreate 60 states and larger tables without repairing any of the measured callers; size-mapping tests would only restate generator output.
  • chosen: Retain emitted postfix/await end guards and pending-ASI serialization: reused update subtrees bypass never-emitted boundary flags, and both original-toolchain seeds 1/56 plus 38 real edits preserve comment/suffix restoration.
  • chosen: Keep current generator, runtime and dependency pins. Source-only consolidation works on 1.0.16; generator-only 1.2.2 savings are measured separately rather than attributed to this grammar repair.
  • chosen: Keep canonical expression/primary_expression operands. A restricted canonical-alias prototype saves only 8,439 C bytes while adding 476 Wasm bytes and 13 states under 1.2.2; it is weaker than the simpler no-alias consolidation.
  • chosen: Retain the measured partial reduction rather than claim restoration of the old artifact budget: candidate Wasm 768,881 remains 2.51% above original fix: keep await operands across line breaks #53 750,024 and 82.4% above main 616 (421,624 bytes); original-toolchain medians match/improve current main on the two pinned fixtures, while current main remains slower than the historical parsers. These limits are explicit in Notes.

Customer Summary

Smaller JavaScript parser artifacts with the same observed await operands, query captures, comment boundaries and incremental trees.

Technical Summary

grammar.js replaces three optional leading await contexts with one optional choice. The yield identifier's optional context stays paired with its emitted start; expression remains the named operand and _await_operand_end remains mandatory. Scanner source, token ordering, two-byte serialization, node schema, dependencies and queries are unchanged.

The rejected restricted-supertype alias experiment increased Wasm and states. The chosen consolidation also generates under the released 1.2.2 toolchain, but this proposal keeps the existing 1.0.16 pins and separates generation-only changes.

Testing

The await-prefix rationale is documented beside the rule. Lint/type checking passed after this comment-only change; generated JSON/C, scanner/schema and Wasm bytes remain exact.

Ignored, immutable-source proof: original 1.0.16 and official 1.2.2 native/Wasm 199-case corpus pass; both seed 1 and seed 56 fuzz 1,000 iterations × 10 edits pass under each generator. Strict scanner C compile passes. Full fields, flags, API indices/points and all seven query sets are identical across 199 corpus and 2,552 focused inputs on both runtimes; 38 length-changing edit/undo operations per artifact match fresh parsing.

Derived TypeScript main 2214f32919e4b2df59d0d59916a34041c9c0e69b source generates TS/TSX before/after the JavaScript substitution. All 2,748 corpus/focused trees/captures and 38 real edits per dialect are exact, including all four shipped TS queries. Node schemas match. Tracked bun run build-wasm reproduces the frozen 768,881-byte candidate exactly. mise exec rust@1.98.1 -- bun wb verify --full with corpus, queries, bodyRanges, incremental, performance and partialComments passes lint, types and six files/60 tests in 24.7 seconds. Matching-head PR CI and merge-readiness checks pass (1791314554-7717d3ad985d1f70, no new review events). Independent review 5a8d7829 completed at 1ea5545 with successful final-head coverage and five successful reviewer routes; both Claude routes were unavailable. Its one minor documentation finding is fixed. Review verification limits are retained in the exported run notes; CI passed independently. Existing corpus/query/incremental checks cover this behavior; no test of fixed byte counts is proposed.

Native cc -O1 timing uses the same pinned jquery.js and text-editor-component.js, one warmup then nine rotating runs/artifact, with all parse exits 0. Same-batch existing-generator medians current/candidate are 17.64/17.64 ms and 10.55/10.53 ms. Historical main 616/#53 in that batch are 14.75/14.91 ms and 8.79/8.89 ms. These two fixtures do not establish performance on all inputs.

Notes

This is a partial reduction against current main, not restoration of its historical artifact budget. Existing-generator candidate has 3,420 states, 4,877,484 C bytes and 768,881 Wasm bytes. Original #53 has 3,538 states, 5,190,599 C bytes and 750,024 Wasm bytes; main 616 has 1,860 states, 2,829,051 C bytes and 421,624 Wasm bytes. Candidate Wasm is still 2.51% above original #53. Current-main native parsing is slower than the two historical artifacts in this batch; consolidation preserves that inherited timing level rather than fixing it.

The independently regenerated 1.2.2 comparison yields current/candidate C 4,942,073/4,786,918 and Wasm 765,823/739,503, but those extra toolchain savings are excluded from this source-only proposal. Fresh 1.2.2 generators change historical tables too; original tables are separately retained to reproduce the issue's published Wasm sizes exactly.

TypeScript's current declared JavaScript dependency is 2.0.1. The derived proof holds actual TS main source fixed and substitutes the proposed JavaScript DSL on both sides; it does not claim an upgrade of TS dependencies or equality to its published generated parser. The extra JavaScript-only highlights-params fragment is unsupported by TS required_parameter and fails identically before/after; all actual shipped TS queries compile.

The existing #55/#57/#58 ambiguity and invalid-constructor limitations remain outside this cost repair. No compiler-validity or universal performance claim is made for the malformed focused recovery inputs.

Consolidate leading await contexts while retaining canonical operands and
emitted postfix boundaries. Keep the existing generator and runtime pins.

Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
@exKAZUu exKAZUu self-assigned this Oct 6, 2026
Co-authored-by: WillBooster (Codex CLI) <agent@willbooster.com>
@exKAZUu
exKAZUu merged commit c32cd36 into main Oct 6, 2026
11 checks passed
@exKAZUu
exKAZUu deleted the perf/reduce-await-parser-cost branch October 6, 2026 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf: reduce await boundary parser artifact size

1 participant