perf: chunked module building, and component $ref fixes - #30
Merged
Merged
Conversation
maxholman
Bot
force-pushed
the
feat/event-stream-output
branch
from
September 27, 2026 11:14
124102c to
3c90444
Compare
ts-morph re-parses the whole source file on every insert, so emitting types.ts, commands.ts and valibot.ts one statement at a time was quadratic in their size. Their statements now go into scratch files of ten and join each module once it is complete, and each command class trims its default output argument while it is still small. A body alias declared inside the arguments of the alias that unions them could land in the next chunk, after it, so those are declared first. Output is byte-identical: openai generates in 8s rather than 59s, docker in 17s rather than 48s. Co-Authored-By: LLM <noreply@block65.dev>
getDependents collected every $ref anywhere under a schema, including inside example, default, const and enum values. A $ref-shaped key in sample data became a dependency, and could close a cycle that failed the component sort. It now walks only the keywords that hold subschemas, so a property named default or example still counts. Co-Authored-By: LLM <noreply@block65.dev>
A component $ref to a schema the document does not define failed later as "ref used before available", the message of an ordering bug. The component sort now rejects it first, naming the schema and the ref. Co-Authored-By: LLM <noreply@block65.dev>
Schemas registered under `#/components/schemas/<name>` as written, so a $ref that escaped the name as a JSON pointer does (`~1` for `/`, `~0` for `~`), percent-encoded it, or pointed into part of a schema found nothing and failed as "ref used before available". The document's local $refs are now normalized before generation. A ref is percent-decoded, a component registers under its escaped name, and a ref into part of a component is replaced by that subschema, keeping any keywords beside the $ref. A ref into its own schema or to a missing part is refused by name. Such names also cased into invalid identifiers (`Pet~tag`, `inputPet/kindSchema`), so casing now splits on every character an identifier cannot hold, and valibot names use the same helper. Co-Authored-By: LLM <noreply@block65.dev>
Only refs between component schemas were checked, so an operation's body, response or parameter that referred to a missing schema still failed as "ref used before available". Every component schema $ref in the document is now checked as the refs are normalized, and the error gives the JSON pointer of the ref. Co-Authored-By: LLM <noreply@block65.dev>
A schema that contains itself, or schemas that refer to each other, failed the component sort with "Cyclic dependency". No order can satisfy a cycle, so the sort now drops the ref that closes each loop, and generation no longer needs the order for such refs. A type alias may name one declared later, so a type ref to a component not registered yet uses the name that component will register under. Validators are named for every component before any is emitted, and a ref to one not emitted yet reads it with v.lazy. The lazy getter is annotated with the generated type, imported from the types module, which stops TypeScript inferring the schema from itself. Only the output type is given, since a wire schema coerces its input. Acyclic documents generate byte-identical output. Co-Authored-By: LLM <noreply@block65.dev>
Codegen used the ref parser only to look up refs inside the document it was given, but resolve() also followed refs to other files and URLs, reading and fetching them at codegen time. A ref is now looked up in the in-memory document with the same JSON pointer decoding the schema registry uses, and a ref outside it is refused by name. Optional: this commit stands alone and can be dropped. Co-Authored-By: LLM <noreply@block65.dev>
A misspelled keyword now fails typecheck instead of silently never matching. getDependents reads each keyword through the schema's own types. additionalItems and definitions are dropped: OAS 3.2 does not define them. Co-Authored-By: LLM <noreply@maxholman.dev>
maxholman
Bot
force-pushed
the
perf/batch-statements
branch
from
September 27, 2026 12:28
7c21f7d to
c746e50
Compare
maxholman
Bot
changed the base branch from
feat/event-stream-output
to
master
September 27, 2026 12:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes fixture generation roughly 4–8× faster and fixes the component
$reffailures ("ref used before available", "Cyclic dependency"). Output for existing documents is byte-identical.Speed
9352976buildstypes.ts,commands.tsandvalibot.tsin 10-statement scratch files and joins them once, instead of inserting into the whole module. ts-morph re-parses the entire file on every insert, so generation was quadratic in module size.$reffixesf58b248: dependencies come from subschemas only. A$refinsideexample/default/const/enumdata no longer creates a fake edge or cycle; a property nameddefaultstill counts.7bd100f,483058c: a$refto an undefined schema is rejected up front with the JSON pointer of the ref (component or operation), instead of "ref used before available".29e493d: escaped (pet~1kind), percent-encoded (Pet%20Kind) and partial (…/Address/properties/postcode,…/$defs/X) refs resolve. It also fixes schema names with/or~casing into invalid identifiers (Pet~tag,inputPet/kindSchema).eca5933: recursive schemas (self and mutual) generate. Types name the later alias, and valibot reads it withv.lazy((): v.GenericSchema<unknown, T> => …). Input schemas useUndefinedOnPartialDeep<T>. The generated output typechecks under strictest (exactOptionalPropertyTypes), including coerced wire types through a cycle.Optional
7c21f7ddrops@apidevtools/json-schema-ref-parserfor a local RFC 6901 resolver over the in-memory document. Itsresolve()read other files and URLs at codegen time; non-local refs are now refused by name. The commit stands alone and can be dropped.Verification
124102c, warnings included. The chunk ordering was also stress-tested at chunk size 1.__tests__/refs.test.ts: 12 tests with whole-file snapshots orrejects.toThrow.tscover the generated refs snapshots passes under@block65/tsconfig/nodejs26.test1.test.tsfails as on the base, because it needs rest-client 16 (EventStreamCommand).