Skip to content

fix(js): unblock npm release by sizing the host-callback timeout for Rosetta - #2780

Merged
chaliy merged 4 commits into
mainfrom
eve/clever-mccarthy-n5iz6i
Oct 26, 2026
Merged

chaliy merged 4 commits into
mainfrom
eve/clever-mccarthy-n5iz6i

Conversation

@chaliy

@chaliy chaliy commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

What changed

The four timeout aborts host work and permits reuse cases in
crates/bashkit-js/__test__/runtime-functions.spec.ts get a timeout budget
sized for the slowest lane we ship instead of 2s, and assert their two failure
modes apart. js.yml now runs the runtime-callback suite on the Rosetta
x86_64-apple-darwin lane, which previously ran only at release time.

No library behaviour changes — test and CI coverage only.

Why

Publish JS for tag v0.20.1 failed, and release-js is gated on its test
jobs, so both npm packages are still on 0.19.0 while crates.io and PyPI are
on 0.20.1. v0.20.1 was cut specifically to unblock npm after #2776; this is what
is still holding it.

Two of fifteen test jobs failed, both on x86_64-apple-darwin (node 22.23.4 and
24.22.1), three cases each:

runtime-functions › Bash: typescript timeout aborts host work and permits reuse
runtime-functions › BashTool: python timeout aborts host work and permits reuse
runtime-functions › BashTool: typescript timeout aborts host work and permits reuse

  __test__/runtime-functions.spec.ts:191
   190:       await delay(0);
   191:       t.true(aborted);
  Value is not `true`:
  false

publish-js.yml builds x86_64-apple-darwin but runs it on macos-latest,
which is arm64. x64 Node, the x64 binding and the embedded Wasm runtimes
therefore all execute translated under Rosetta 2 — the only translated lane
in the matrix, and the only failing one. Every native lane passed.

The budget covers more than the hang. timeoutMs starts at execute, so it
also covers interpreter start-up on first use in the process and scheduling
latency while the rest of the suite runs concurrently. Only the remainder is the
hang these cases mean to cancel. When it expires first, the run is cancelled
before the host callback is ever dispatched, the handler never registers an
abort listener, and aborted stays false. The run still exits non-zero, so
the preceding assertion passes and the failure surfaces one line later as a bare
Value is not `true` .

The same job's timings show 2s was simply too small there:

case (same job, same binding) wall time
Bash: python host functions — first embedded-runtime call 3.7s
BashTool: python host functions 3.8s
Bash: python timeout aborts… — the one variant that passed 4.7s for a 2s timeout

The first runtime call alone cost 3.7s, already past the entire 2s budget.

Before / After

Both directions were verified on the new lane rather than argued.

At 2s (budget temporarily reverted on this branch, commit since reverted),
the new lane reproduces the release failure — and the new assertion names the
cause outright instead of leaving a bare false:

✘ [fail]: Bash: typescript timeout aborts host work and permits reuse
          host callback never ran; the timeout fired first
✘ [fail]: BashTool: typescript timeout aborts host work and permits reuse
          host callback never ran; the timeout fired first
✔ Bash: python timeout aborts host work and permits reuse (3s)
✔ BashTool: python timeout aborts host work and permits reuse (3s)

2 tests failed

entered is false, so the run really is cancelled before the interpreter
dispatches the callback — the root cause, not an abort-delivery race. Note the
same split as the release failure: some variants pass, some lose the race.

At 20s, the whole JS workflow is green, Rosetta lane included:

success  Runtime callbacks (macos-latest x86_64 (Rosetta))
success  Runtime callbacks (macos-latest)
success  Runtime callbacks (windows-latest)
success  node 20 / 22.23.4 / 24.22.1 / latest, bun latest+canary, deno 2.x+canary
success  JS Check

Locally runtime-functions.spec.ts is 75/75 green; the file goes from ~5s to
~22s wall, because the cases wait out the full timeout by design. ava runs them
concurrently, so the cost is one timeout rather than four.

One limit worth stating: the failure does not reproduce on Linux. A first
embedded-runtime call costs 1–3ms natively versus 3.7s translated, so the
budget is never close to exhausted here, and probing down to timeoutMs: 1
still entered and aborted the handler. The Rosetta lane is what actually
exercises this, which is the main reason for adding it.

Risk

  • Low. Test and CI configuration only; no change to shipped library code, to
    the abort contract, or to any published artifact.
  • The suite gains a macOS job, which adds CI time; it runs concurrently with the
    existing matrix entries. It earns that cost: it is demonstrably the only lane
    in CI that catches this.
  • The timeout path keeps its coverage, and the split assertions mean a future
    regression cannot pass vacuously.
  • The lane's first run died in the cross-build with clang: error: unable to execute command: Segmentation fault: 11 linking libc's build script. The
    Rust cache is now scoped per target so a restored arm64 target/ cannot reach
    the x64 build, and MACOSX_DEPLOYMENT_TARGET matches publish-js.yml; it has
    built cleanly on all three runs since.

Not fixed here

Publish Web for v0.20.1 failed separately and is not addressed by this PR:

npm error 404 The requested resource '@everruns/bashkit-wasm@0.20.1'
could not be found or you do not have permission to access it.

@everruns/bashkit-wasm exists and is published through 0.19.0, and
scripts/publish_npm.py already passes --access public, so this is npm
masking an authorization failure as a 404 — the NPM_TOKEN secret needs
rotating. That requires repository-secret access and cannot be fixed in code.
Both publish workflows read the same secrets.NPM_TOKEN, so getting this PR
green is necessary but probably not sufficient to publish npm
; the token
likely needs rotating before the next release tag.

Checklist

  • Tests added or updated
  • Backward compatibility considered

`macos-latest` is arm64, so the shipped x86_64-apple-darwin binding only ever
runs translated under Rosetta 2. Embedded Python/TypeScript runtime start-up
costs several times native there, and publish-js.yml was the only workflow
running that lane, so a timing regression on it could not surface until a
release tag.

That is what happened to v0.20.1: three `timeout aborts host work and permits
reuse` cases failed on x86_64-apple-darwin while every native lane stayed
green. `release-js` is gated on those test jobs, so both npm packages are
still stuck on 0.19.0.

This commit only adds the lane, so it reproduces the release failure in CI.
The fix follows.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
bashkit b4d9e70 Commit Preview URL

Branch Preview URL
Oct 26 2026, 10:11 AM

The four `timeout aborts host work and permits reuse` cases gave the whole
execution 2s. That clock starts at `execute`, so it covers interpreter
start-up on first use in the process and scheduling latency while the rest of
the suite runs concurrently; only the remainder is the hang the cases mean to
cancel. When it expires first the run is cancelled before the host callback is
ever dispatched, so the handler never registers an abort listener and the case
reports a missing abort rather than a slow start.

On x86_64-apple-darwin, where the shipped Intel binding runs translated under
Rosetta 2, 2s was not enough. In the v0.20.1 `Publish JS` run the first
`python host functions` call alone took 3.7s, already past the entire budget,
and three of the four variants failed there. The fourth passed only because it
ran later against a warm runtime, and still took 4.7s of wall time for its 2s
timeout.

Raise the budget to 20s, well clear of both figures, and assert the two
outcomes apart so a run cancelled before the callback starts can no longer
read as an abort regression.

The cases now wait out the full timeout by design, but ava runs them
concurrently: runtime-functions.spec.ts goes from ~5s to ~22s locally, 75/75
passing, and the cost is one timeout rather than four.

Also scope the Rosetta lane's Rust cache by target and match publish-js.yml's
MACOSX_DEPLOYMENT_TARGET, so the cross build cannot pick up the arm64 lane's
`target/`.
Scratch commit. Reverted before merge. Verifies that the new Rosetta lane
actually reproduces the v0.20.1 failure rather than passing for unrelated
reasons: this lane runs only runtime-functions.spec.ts, while publish-js.yml
runs the whole suite under far heavier concurrency.
…s regression

Reverts the scratch commit. With HOST_TIMEOUT_MS back at 2s the new Rosetta
lane failed exactly as the v0.20.1 release did, and the new assertion named
the cause outright:

  x [fail]: Bash: typescript timeout aborts host work and permits reuse
            host callback never ran; the timeout fired first
  x [fail]: BashTool: typescript timeout aborts host work and permits reuse
            host callback never ran; the timeout fired first
  / Bash: python timeout aborts host work and permits reuse (3s)
  / BashTool: python timeout aborts host work and permits reuse (3s)

So `entered` is false: the run is cancelled before the interpreter dispatches
the host callback, which is the root cause rather than any abort-delivery
race. Same split as the release failure, where some variants pass and some
lose the race. At 20s all four pass on that lane.
@chaliy
chaliy marked this pull request as ready for review October 26, 2026 10:11
@chaliy
chaliy merged commit 1dc7911 into main Oct 26, 2026
38 checks passed
@chaliy
chaliy deleted the eve/clever-mccarthy-n5iz6i branch October 26, 2026 10:41
chaliy added a commit that referenced this pull request Oct 26, 2026
…2788)

The x86_64-apple-darwin lane added in #2780 is flaky, and because `js-check`
gates on `callback-platforms` it blocks unrelated PRs. It already did: #2787,
a GitHub Actions bump touching none of this, went red on it and passed on
re-run.

Cross-linking to x86_64 on the arm64 runner intermittently dies with

  error: linking with `cc` failed: exit status: 1
  clang: error: unable to execute command: Segmentation fault: 11

while linking a build script. Build scripts are host binaries, so no
cross-compilation flag of ours is involved, and crates/bashkit-js/.cargo/
config.toml only sets rustflags for x86_64-pc-windows-msvc. It hit `libc` on
one run and `getrandom` on another, roughly two runs in five, and recurred
after #2780 scoped the Rust cache per target and again under a different
Swatinem/rust-cache revision on #2787 -- so it is the toolchain crashing, not
our cache or configuration.

Retry the cross build up to three times, logging a ::warning:: per retry so
the flake stays visible. cargo keeps whatever already compiled, so a retry
resumes rather than rebuilds.

This narrows rather than widens what passes: a genuine compile error fails
all three attempts and still fails the job, and the test step is outside the
loop, so the regression the lane exists to catch still fails it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant