stream: cut per-chunk overhead in web streams pipelines

[anonrig]

Reduces per-chunk overhead across common web streams pipelines (fetch body decompression/upload, TextDecoderStream/TextEncoderStream transcoding), targeting workloads that move large payloads in small (e.g. 4 KiB) chunks.

Changes

stream: process CompressionStream chunks without threadpool round trips

CompressionStream/DecompressionStream wrapped a zlib stream.Duplex in the web streams adapters, so every written chunk was dispatched to the threadpool and its completion observed on a later event loop turn. For a 64 MiB body written in 4 KiB chunks that is 16384 threadpool round trips plus the Transform and adapter machinery around them. The classes are now built on a TransformStream that drives the raw zlib/brotli handle synchronously, mirroring the write loop of the zlib streams (same handle setup, same error mapping, same trailing-garbage rejection). Inputs larger than 64 KiB are processed in slices with an event-loop turn in between, so a huge chunk cannot block the loop for its full duration. Output is emitted in up to 64 KiB chunks, either as zero-copy views or as right-sized copies so tiny chunks do not retain large buffers.

stream: fast-path TextEncoderStream chunk encoding

The encode-and-enqueue algorithm was a literal transcription of the spec's per-code-unit loop: it extracted a single-character string, called charCodeAt(), and appended to an accumulator string for every code unit of every chunk (~67 million temporary strings for a 64 MiB payload). The loop's only observable effects are the surrogate hand-off at chunk boundaries and U+FFFD replacement of unpaired surrogates, which TextEncoder.encode() already performs; the transform now handles the chunk boundary explicitly and encodes the rest of the chunk with a single encode() call.

stream: remove unused sync-error destroy plumbing from adapters

The kValidateChunk/kDestroyOnSyncError hooks existed only for the previous Duplex-based compression implementation.

Benchmarks

64 MiB of log-like text, 4 KiB chunks, best of 5, idle Linux x64. Baseline is v27.0.0-nightly20260819 (current main); "this PR" is a from-source build of this branch.

End-to-end pipelines

Pipeline	main	this PR	speedup
Download (fetch → gunzip → decode → for await)	154 ms · 417 MB/s	96 ms · 668 MB/s	1.6x
Upload (fs → gzip → fetch POST)	1154 ms · 56 MB/s	750 ms · 85 MB/s	1.5x
Transcode (fs → decode → encode → fs)	771 ms · 83 MB/s	204 ms · 315 MB/s	3.8x
Subprocess (fetch → cat → for await)	67–81 ms · ~800–950 MB/s	77–83 ms · ~780–830 MB/s	~1.0x (already at the pipe/syscall floor)

Isolated stages (64 MiB through just the stage, 4 KiB chunks)

Stage	main	this PR	speedup
TextEncoderStream	612 ms · 105 MB/s	44 ms · 1464 MB/s	13.9x
CompressionStream('gzip')	306 ms · 209 MB/s	136 ms · 471 MB/s	2.3x
DecompressionStream('gzip')	79 ms · 807 MB/s	36 ms · 1796 MB/s	2.2x

End-to-end wall time is capped by physical floors on this machine (gzipSync of this payload is ~620 ms of raw CPU; gunzipSync is ~139 ms; UTF-8 decode+encode of 64 MiB is ~84 ms). After this PR the remaining non-codec overhead on upload is ~130 ms (~8 µs/chunk) and on download is well under the gunzip floor. A literal 10x on wall time is not available for upload/download/subprocess; the one place a full 10x+ existed was the encoder stage.

Adds benchmark/webstreams/compression.js and benchmark/webstreams/encoding.js.

Behavior notes

• Compression/decompression now runs on the JS thread. For small chunks this strictly removes latency (the deflate of a 4 KiB chunk costs far less than a threadpool dispatch); for oversized chunks the input is sliced with event-loop yields to bound blocking.
• Decompressed/compressed output chunks may now be emitted as Uint8Array views over a larger ArrayBuffer (when at least half the buffer is filled) rather than always exact-sized copies. Those views never share memory with one another: the backing buffer is retired after a view is emitted.
• Node.js-specific behaviors are preserved and covered by tests: string chunks are accepted, null rejects with ERR_STREAM_NULL_VALUES, non-BufferSource chunks reject with ERR_INVALID_ARG_TYPE, trailing garbage rejects with ERR_TRAILING_JUNK_AFTER_STREAM_END (all TypeErrors), and 'brotli' remains supported.

Testing

Local out/Release/node (this branch): 30 targeted tests, all passing, including test-whatwg-webstreams-compression, test-webstreams-compression-bad-chunks, test-webstreams-decompression-reject-trailing, test-webstreams-compression-buffer-source, test-compression-decompression-stream, test-zlib-type-error, test-whatwg-webstreams-encoding, the Readable.toWeb / adapters tests, and WPT compression, encoding, and streams.

[nodejs-github-bot]

Review requested:

• @nodejs/performance

[jasnell]

AI agents cannot used Signed-off-by

[anonrig]

AI agents cannot used Signed-off-by

Sorry for the frustration. My AI keeps making the same mistake. Fixing it now.

Resolved

[codecov]

Codecov Report

❌ Patch coverage is 91.22807% with 35 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.11%. Comparing base (f83e7df) to head (e3f6bd2).
⚠️ Report is 115 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/webstreams/compression.js	90.38%	35 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #65437      +/-   ##
==========================================
- Coverage   90.31%   90.11%   -0.20%     
==========================================
  Files         751      752       +1     
  Lines      249956   252618    +2662     
  Branches    47204    47503     +299     
==========================================
+ Hits       225745   227649    +1904     
- Misses      15612    16252     +640     
- Partials     8599     8717     +118     

Files with missing lines	Coverage Δ
lib/internal/webstreams/adapters.js	85.08% <100.00%> (-1.62%)	⬇️
lib/internal/webstreams/encoding.js	100.00% <100.00%> (ø)
lib/internal/webstreams/compression.js	93.39% <90.38%> (-6.61%)	⬇️

... and 106 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[mcollina]

lgtm

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/76108/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/76137/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/76167/