Skip to content

fix(test): stop the npm absence cancellation carrier from losing its signal - #237

Open
iperev wants to merge 1 commit into
chain/d1-fast-urifrom
chain/rp-npm-absence-race
Open

iperev wants to merge 1 commit into
chain/d1-fast-urifrom
chain/rp-npm-absence-race

Conversation

@iperev

@iperev iperev commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

fix(test): stop the npm absence cancellation carrier from losing its signal

Part of the UI-DESIGN-01 chain, stacked on it; the change itself is a test fix, not UI work. Base branch: chain/d1-fast-uri (#236, "fix(deps): lock fast-uri 3.1.9 to clear four advisories"), which sits on #235; merge it after #236. It is one commit and changes two test files in internal/tools/releasepreflight.

Why

A local full gate on a candidate of the parent change, which did not touch this package, failed once in go:test:

--- FAIL: TestReleaseWorkflowExecutesNPMAbsenceBranches/carrier/removed_cleanup_wait_counterexample (3.04s)
    npm_absence_branch_test.go:182: removed-wait rejection=classifier exited without completing admission
        write |1: broken pipe, want live-classifier cause

Since the gate runs Go tests with -count=1, this test runs on every gate instead of replaying a cached pass.

Rate before the change (macOS /bin/bash 3.2.57, 8 CPUs, go test -count=N -run 'TestReleaseWorkflowExecutesNPMAbsenceBranches' ./internal/tools/releasepreflight):

Load Runs Failed Rate
host, 1-min load 20 to 36 200 1 0.5 %
host plus eight busy loops, 1-min load 14 to 48 (400 of the runs with failure-message diagnostics only) 800 9 1.1 %

Every failing subtest took 3.02 to 3.05 s; passing ones took at most 0.20 s. The three other cancellation subtests failed the same way; in 3 of the 10 failures Bash 3.2 instead crashed with SIGSEGV in the same wait.

Cause. The cancellation carrier is test code. Bash runs the extracted release fragment with a go function that starts the classifier (this test binary, -test.timeout=3s) in the background. The classifier blocks on a release pipe. The function writes P and runs wait "$child". The controller reads P and sends one SIGTERM, whose trap writes cancelled and C and exits; the EXIT trap writes W, waits for the classifier (the counterexample's mutant removes this wait) and writes D. The controller releases the classifier after W; in the mutant, only after D and after a nonblocking read that must still find the classifier alive.

Bash 3.2 checks for pending traps when a command starts, and wait_builtin does not check again before it blocks (builtins/wait.def:92–134). Where that one SIGTERM lands decides the outcome:

  • Before wait sets interrupt_immediately (wait.def:93): trap_handler only records the trap (trap.c:363–364), and wait blocks until the awaited child exits, here only at the classifier's own 3-second test timeout. Then the trap runs, and:

    • the exit oracle reads EOF instead of a live classifier: "classifier exited without completing admission";
    • the release write hits a closed pipe: write |1: broken pipe.

    That is the gate's message, and 17 of the 24 unforced failures across the measured and two diagnostic runs.

  • After wait sets interrupt_immediately but while its setjmp (wait_intr_buf) (wait.def:103) still runs: trap_handler longjmps to that incomplete buffer (trap.c:366–369), and Bash faults with SIGSEGV. Because the script traps EXIT, Bash 3.2 installed its terminating-signal handler, which only records the fault, so Bash lives on until SIGCHLD from the classifier's timeout. Its terminating-signal path then runs the EXIT trap, which first runs the recorded TERM trap; that trap's exit 124 ends the EXIT trap before cleanup writes W. The trace holds cancelled, and the controller reads C, then EOF: carrier marker 'W' unavailable: EOF. That is the other 7 of the 24, each with a macOS crash report 3.03 to 3.07 s after launch. In all seven, the backtrace attributes the interrupted frame to the shim's wait setting up its interruption (/bin/bash disassembled with its function starts).

Forcing the signal into the first window (it arrives while Bash expands wait's operand) makes 120 of 120 cancellation subtests fail after 3 s, and every counterexample fails with the gate's exact message. A standalone Bash 3.2 script with one trapped SIGTERM, an EXIT trap and wait reproduces both outcomes: forced, the trap waits for the child in 10 of 10 runs; unforced, 3000 runs gave 12 deferred traps and 1 SIGSEGV that wrote the TERM trap's marker but not the EXIT trap's. That crash report has the same frames as the measured runs' reports from the signal handler down to wait's setjmp. Ubuntu 24.04's Bash 5.2, the shell of the hosted source job, shows neither: it checks for pending traps after wait's setjmp (wait.def:193), and longjmps only while it is blocked waiting for a child (trap.c:553). The same script ran the trap at once in 10 of 10 forced runs, with no deferral and no crash in 3000 unforced runs.

Not a product defect. The extracted .github/workflows/release.yml bytes contain no trap, wait, background job or signal handling. The trap, cleanup, shell functions, wait and signal belong to the carrier in npm_absence_branch_test.go. The counterexample's verdict depended on that timing, so this is an oracle defect, and the release workflow is unchanged.

No hosted runner reaches it. Linux CI and the release's npm run check run this test on ubuntu-24.04 with Bash 5.2. The macOS platform-smoke jobs do not run this package. Only local macOS gates with /bin/bash 3.2 do.

What changes

  • npm_absence_branch_test.go, the controlled shim: after P it no longer waits for the classifier. It reads descriptor 7, the read end of a new signal-sent pipe, and then runs return 97. The pipe reaches EOF only after the signal was sent. Wherever the signal lands, the trap runs before return 97:

    • before the read starts, at the read's start-of-command check;
    • during the read, by interrupting it;
    • between the check and the blocking read, after the EOF: that EOF exists only after the signal was sent, so the signal is pending when the read returns, and Bash runs the pending trap before the next command.

    The classifier therefore stays held until the controller releases it, and the shim no longer enters wait while the signal can arrive. If the trap ever failed to run first, the controller would read W where it requires C, or C without the W that must follow; both fail. The classifier's spawn closes descriptor 7. The uncontrolled shim, used by the branch cases, still waits for the classifier.

  • New schedule, "signal pending when the carrier blocks": the read's redirection is expanded by a command substitution that writes E and then waits for the EOF, and the controller signals only after E. So the signal arrives after Bash's last pending-trap check before the read, deterministically. Both the blocked-entrypoint cancellation test and the removed-cleanup-wait counterexample run under it; the counterexample also keeps its immediate schedule.

  • npm_absence_carrier_test.go: the control protocol owns the new pipe and the E event.

    • cmd.Cancel sends its SIGTERM only through signalCarrier. That function calls the signal sender and closes the pipe only after the sender returned, so the order "signal, then close" lives in one function.
    • The new subtest carrier/signal precedes its pipe close gives signalCarrier a sender that reads the pipe without blocking. It requires the pipe still open at the signal, the sender's result returned, and EOF afterwards. A swapped order fails it on every run, not only when Bash happens to read the EOF first.

Unchanged: the counterexample's required cause "carrier cleanup did not wait for classifier exit", its protected trace view absent cancelled reaped and exit 124, the exit oracle, the mutant's substitution, every timeout and deadline. No retry is added.

Every site of the mechanism. This carrier's TERM trap is the only shell signal trap in the repository (the two traps in the workflows are EXIT traps), and all four cancellation subtests share it and its one signal. workflowsmoke's tests run /bin/sh without traps, and its process cleanup sends only SIGKILL, which no handler sees, before processgroup's absence probe. Every other signalled process installs a Go, Python or Node handler, not a shell trap.

Compatibility

Not breaking. Test code only: no shipped byte, workflow, requirement, binding, contract or expected value changes. The counterexample's subtests are now named …/removed_cleanup_wait_counterexample/immediate and …/signal_pending_when_the_carrier_blocks, and the new subtest is …/carrier/signal_precedes_its_pipe_close; no tracked file names subtests of this test.

Proof

Proof: A local npm run check sequence on darwin/arm64 passed (exit 0) on tree 7f6336a38096 with Node 26.11.1, npm 12.2.0, CPython 3.15.0 and Go 1.27.3, with no Go test verdict replayed from the test cache: 561/561 runtime browser cases in Chromium, Firefox and WebKit with zero retries. One recorded deviation: the host firewall blocks Go-built binaries from the network, so go:vulncheck ran against a vulnerability-database snapshot fetched with curl (modified 2026-10-21T00:27:29Z): no vulnerabilities.

Narrow evidence on macOS with Bash 3.2.57 and Go 1.27.3, on a shared host, on this tree unless stated:

  • Rate after the change, same command:

    Shell and load Runs Failed
    Bash 3.2, host, 1-min load 14 to 26 (5-min load up to 74) 1000 0
    Bash 5.2.21 in ubuntu:24.04 200 0
    Bash 3.2, host plus eight busy loops, earlier version of this change (same carrier, same signal-then-close order written inline) 4038 0

    Every run on this tree passed both subtests of the new schedule and the new order subtest; the longest cancellation subtest took 0.92 s. Before, the rates were 1 in 200 and 9 in 800. No Bash crash report appeared during these runs.

  • Old wait caught: with the shim's old wait restored in control mode, both subtests of the new schedule fail in 3 of 3 runs on Bash 3.2. The counterexample fails with the gate's exact message, the cancellation test with write |1: broken pipe; the immediate schedules pass.

  • Swapped order refused: with signalCarrier closing the pipe before it calls the sender, carrier/signal precedes its pipe close fails in 20 of 20 runs ("signal-sent pipe closed before the signal"). The six cancellation subtests caught the same swap in 1 of 120 subtest runs, by chance.

  • The counterexample still rejects its mutant for the stated reason: both of its subtests pass in every run above, so each run required the cause "carrier cleanup did not wait for classifier exit", the trace view absent cancelled reaped and exit 124. With its mutation disabled (the cleanup wait kept, the release still withheld until D), both fail in 3 of 3 runs with "classifier exited without completing admission": that cause needs the removed wait.

  • go test -race -count=5 -run TestReleaseWorkflowExecutesNPMAbsenceBranches ./internal/tools/releasepreflight passes.

  • go test -p 2 -count=1 ./internal/tools/releasepreflight/... ./scripts passes.

  • go vet and staticcheck on the package pass, and gofmt -l is empty.

  • git diff --check is clean, and so is the committed change against its parent.

Non-claims

  • The instruction-level failure is not decoded. Why the longjmp to the incomplete wait_intr_buf fails pointer authentication was not established at instruction level. One of the seven crash reports was interrupted just after that setjmp returned rather than inside it.
  • Fault bounds remain. A host stall longer than the classifier's 3-second timeout between its start and the controller's D observation still fails the test. The longest cancellation subtest in the after-fix runs took 0.92 s.
  • Linux does not discriminate the old primitive. On Bash 5.2 the new schedule passes even with the old wait, because Bash 5.2 handles the pending signal at once. It is a falsifier only with Bash 3.2, which is where the defect exists.
  • The order subtest pins signalCarrier, not its call site. That cmd.Cancel keeps sending its signal through signalCarrier rests on the code, not on a test.
  • No hosted run was made for this commit; nothing was pushed, and npm run check was not run here.

…signal

TestReleaseWorkflowExecutesNPMAbsenceBranches failed once in a local
full gate on a candidate of the parent change, which did not touch this
package: the removed-cleanup-wait counterexample reported "classifier
exited without completing admission" and "write |1: broken pipe" instead
of "carrier cleanup did not wait for classifier exit". With the macOS
/bin/bash 3.2 it failed in 1 of 200 runs at a 1-minute load of 20 to 36
on 8 CPUs, and in 9 of 800 runs with eight added busy loops (400 of them
with failure-message diagnostics only). Every failing subtest took 3.02
to 3.05 s. The three other cancellation subtests failed the same way; in
3 of the 10 failures Bash 3.2 instead crashed with SIGSEGV in the same
wait.

The carrier cancels with one SIGTERM after the shim reports P, its last
step before it waits for the classifier. The classifier blocks on a
release pipe until the controller releases it after the cleanup has
started. Bash 3.2 checks for pending traps when a command starts, and
wait does not check again before it blocks. Where that SIGTERM lands
decides the outcome:

- Before wait sets interrupt_immediately, the trap handler only records
  the trap, and wait blocks until the awaited child exits, here only at
  the classifier's own 3-second test timeout. Then the trap runs, the
  exit oracle reads EOF instead of a live classifier, and the release
  write fails with EPIPE.
- After wait sets interrupt_immediately but while its setjmp of
  wait_intr_buf still runs, the trap handler longjmps to that incomplete
  buffer, and Bash faults with SIGSEGV. Because the script traps EXIT,
  Bash 3.2 only records the fault and lives on until SIGCHLD from the
  classifier's timeout. Its terminating-signal path then runs the
  recorded TERM trap, whose exit 124 ends the EXIT trap before cleanup
  writes W: the trace holds cancelled, and the controller reads C, then
  EOF.

Sending the signal while Bash expands wait's operand reproduces the
first outcome in 120 of 120 cancellation subtests. A standalone Bash 3.2
script with one trapped SIGTERM and an EXIT trap reproduces both, the
second with the crash-report frames of the measured runs from the signal
handler down to wait's setjmp; Bash 5.2 shows neither. The extracted
release workflow bytes contain no trap, wait or signal handling; the
defect is in the test carrier.

- npm_absence_branch_test.go: after P, the controlled shim reads
  descriptor 7, the read end of a new signal-sent pipe, instead of
  waiting for the classifier, and then returns 97. The pipe reaches EOF
  only after the signal was sent, so the read ends at the latest at that
  EOF and the trap runs before the next command while the classifier is
  still held. If the trap did not run first, the controller would read W
  where it requires C, or C without the W that must follow; both fail.
  The classifier's spawn closes descriptor 7. The uncontrolled shim
  still waits for the classifier.
- A new schedule, "signal pending when the carrier blocks", expands the
  read's redirection with a command substitution that reports E and then
  waits for that EOF; the controller signals only after E. The
  blocked-entrypoint cancellation test runs it, and the
  removed-cleanup-wait counterexample runs it besides its immediate
  schedule. With the shim's old wait restored, both fail on every run
  under Bash 3.2, the counterexample with the gate's message.
- npm_absence_carrier_test.go: the control protocol owns the new pipe.
  cmd.Cancel sends its SIGTERM only through signalCarrier, which calls
  the signal sender and closes the pipe only after the sender returned.
  The new subtest "signal precedes its pipe close" gives signalCarrier a
  sender that reads the pipe without blocking; it requires the pipe open
  at the signal and at EOF afterwards, so a swapped order fails on every
  run.

The counterexample still requires its cause, the trace view, absent,
cancelled, reaped and exit 124. No assertion, timeout or deadline
changes, and no retry is added. On this tree 0 of 1000 runs failed at
host load under Bash 3.2 and 0 of 200 under Ubuntu 24.04's Bash 5.2; an
earlier version of this change, with the same carrier and the same
signal-then-close order written inline, passed 4038 of 4038 runs with
eight busy loops. This carrier's TERM trap is the only shell signal trap
in the repository. Every other signalled process installs a Go, Python
or Node handler, not a shell trap.

Proof on this tree:
A local npm run check sequence on darwin/arm64 passed (exit 0) on tree
7f6336a with Node 26.11.1, npm 12.2.0, CPython 3.15.0 and Go 1.27.3,
with no Go test verdict replayed from the test cache: 561/561 runtime
browser cases in Chromium, Firefox and WebKit with zero retries. One
recorded deviation: the host firewall blocks Go-built binaries from the
network, so go:vulncheck ran against a vulnerability-database snapshot
fetched with curl (modified 2026-10-21T00:27:29Z): no vulnerabilities.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant