http2: fix write deadlock exposed by larger window sizes - #65440
Open
pimterry wants to merge 2 commits into
Open
http2: fix write deadlock exposed by larger window sizes#65440pimterry wants to merge 2 commits into
pimterry wants to merge 2 commits into
Conversation
This removes a guard (no reads while write pending) that creates this deadlock, which was added as a security mechanism. This guard is redundant given then other existing mechanisms, and a test is added to demonstrate that. Signed-off-by: Tim Perry <pimterry@gmail.com>
Collaborator
|
Review requested:
|
pimterry
force-pushed
the
fix-h2-write-deadlock
branch
from
August 20, 2026 18:18
e2733d2 to
a46d64f
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #65440 +/- ##
==========================================
- Coverage 90.12% 90.09% -0.03%
==========================================
Files 752 752
Lines 252325 252280 -45
Branches 47456 47435 -21
==========================================
- Hits 227407 227301 -106
- Misses 16217 16268 +51
- Partials 8701 8711 +10
🚀 New features to boost your workflow:
|
jasnell
approved these changes
Aug 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This completes the fix from #65079. That PR resolved one test flake (test-worker-terminate-http2-respond-with-file), and reduced the second (test-stream-pipeline-http2) but left an underlying issue there which is still causing flakes.
This took more work, as it's a bit complicated. The issue is a real bug which can result in deadlocks between two Node HTTP/2 peers, not a test issue. This was preexisting though very hard to hit, but is exposed in some cases recently by the new window size update (#64623).
This PR fixes this by dropping a security guard (no reading while writing) completely. I've put it in two commits: the first does the tiny fix to drop the guard & tests it, the second removes various code which is now unreachable without the guard.
Dropping this guard needs careful review, but I think that dropping this guard is safe due to the various other mechanisms in place. As long as we're happy that this is safe, it has a lot of upsides: it fixes the flake, resolves a real deadlock, simply deletes some code, and provides some performance boosts.
An example deadlock flow looks roughly like this:
This PR fixes that deadlock, by removing the "no reading while writing" guard completely, in its two forms.
These were added as part of a larger set of many HTTP/2 DoS mitigations in 2019 by @addaleax in #29122, so this needs careful review please!
I do think it's safe though: the key thing it's protecting against (one peer forcing the other to buffer large amounts of outgoing data through manipulation of flow control & window updates) is covered by other existing mechanisms here (most notably
maxSessionMemory). I can't find any attacks that depend on this guard alone. I've added an additional test which covers CVE-2019-9517 directly, and confirms that maxSessionMemory blocks the key attack scenario regardless.Removing this guard has notable performance benefits. @mcollina previously looked at this same issue briefly with #63009 testing small focused tweaks on the same guard. AFAICT tweaks alone had minimal benefit, but the guard is still a real perf problem, because it blocks all reads on the whole H2 session while writing any frames anywhere. That means during e.g. a large read (server large upload/client large download), every sent window update to read more data from any stream pauses reading on all streams until the write completes.
Many different dimensions where removing this entirely should improve things, but I've included one example in the benchmark here: this saturates TLS bidirectional streaming, and with this change you get roughly 25% throughput boost. This is using default settings except it shrinks the stream window size back to the standard H2 default - without that you can't benchmark against previous versions, because they deadlock.