Skip to content

test(setup-matrix): give each case its own home directory - #142

Open
John-David Dalton (jdalton) wants to merge 1 commit into
mainfrom
fix/isolate-home-in-setup-matrix
Open

test(setup-matrix): give each case its own home directory#142
John-David Dalton (jdalton) wants to merge 1 commit into
mainfrom
fix/isolate-home-in-setup-matrix

Conversation

@jdalton

@jdalton John-David Dalton (jdalton) commented Aug 1, 2026

Copy link
Copy Markdown

Running the setup matrix on your own machine currently installs packages into your real home directory. scripts/setup-matrix.sh run --host drives real installs across nine ecosystems, and each one writes to the cache, store, or credentials location it finds in $HOME. After a run, your own npm cache, cargo registry, Go module cache, Maven repository and NuGet packages all contain whatever the test cases pulled down.

This change gives every case its own home inside the temporary work directory it already creates, so a host run leaves your machine exactly as it found it.

It also makes the matrix more trustworthy. Right now a case can pass because a package was already sitting in your cache from something unrelated — the kind of pass that disappears on a colleague's laptop or a fresh CI runner.

The evidence — the script proves this against itself

run-case.sh isolated exactly one thing, the pypi hook's stamp, via XDG_CACHE_HOME. Nothing pointed HOME anywhere, so every package manager fell back to the real one.

You can see it in the script's own verification helpers, which go looking for the installed artifacts in the caller's home:

find "${CARGO_HOME:-$HOME/.cargo}/registry/src" …
find "$HOME/.m2/repository" -name "$base" …
find "${NUGET_PACKAGES:-$HOME/.nuget/packages}"

Those lookups succeed today. That is the proof the artifacts really are landing there.

The change — one home per case, plus the vars that outrank it

HOME now points at $WORKDIR/home, which is already unique per case.

Setting HOME alone would not have worked. These tools read their own variable first and ignore HOME when it is set, so each is pointed inside the new home:

Variable Without it, this leaks
CARGO_HOME cargo registry and credentials
GOPATH + GOMODCACHE Go module cache
GOCACHE Go build cache — separate from the module cache, so GOPATH alone is not enough
PNPM_HOME pnpm content store
COREPACK_HOME corepack shims
NUGET_PACKAGES NuGet packages
XDG_DATA_HOME, XDG_CONFIG_HOME, XDG_STATE_HOME anything XDG-aware

The verification helpers needed no changes — they read through these same variables, so they follow the new location automatically.

What I checked — script-level, not a full matrix run

Ran:

  • bash -n on the modified script — passes.
  • Sourced the new block with a temporary WORKDIR — every variable resolves inside it, and the home directory is created.

Did not run: the full matrix end to end. It needs Docker images per ecosystem and performs real installs, so this is verified at the script level rather than by a complete run.

CI is unaffected. The setup-matrix job runs each case inside a per-ecosystem container, where $HOME is the container's and thrown away. This problem is specific to host mode.

Trade-off: a host run now starts from an empty cache, so the first run of each case re-downloads its dependencies. That cost is also what makes the result reproducible.

Left for a follow-up — the Rust tests have a similar gap

Not touched here, to keep this small:

  • Several tests under crates/socket-patch-cli/tests/ run real installs during a plain cargo test without redirecting caches.
  • e2e_vendor_pypi_build.rs passes an empty environment to pip, while the uv half of that same file sets one up properly.

Happy to follow up if you would like these handled the same way.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is ON. A cloud agent has been kicked off to fix the reported issue.

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit a83e84c. Configure here.

Comment thread tests/setup_matrix/run-case.sh
run-case.sh isolated only XDG_CACHE_HOME, so every package manager it
drives resolved its cache, store and credentials from the caller's real
home. Under `setup-matrix.sh run --host` that is the developer's own
home directory.

Each case now gets HOME inside its per-case WORKDIR, along with the
variables that outrank HOME for their tools (CARGO_HOME, GOPATH,
GOMODCACHE, GOCACHE, PNPM_HOME, COREPACK_HOME, NUGET_PACKAGES and the
remaining XDG dirs). The verification helpers read through those same
variables, so they keep resolving without changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant