Skip to content

test: migrate to xUnit v3 and run the Execution suite in CI - #69

Merged
StuartMeeks merged 1 commit into
mainfrom
chore/xunit-v3-and-ci-gap
Aug 19, 2026
Merged

test: migrate to xUnit v3 and run the Execution suite in CI#69
StuartMeeks merged 1 commit into
mainfrom
chore/xunit-v3-and-ci-gap

Conversation

@StuartMeeks

Copy link
Copy Markdown
Owner

Two changes, both in the test stack. The shipped app is unaffected.

xUnit v3

The xunit package ID is the feature-frozen v2 line — 2.9.3 is its latest. v3 ships as xunit.v3, currently 4.0.0. v3 self-hosts on Microsoft Testing Platform instead of VSTest, so the whole VSTest stack comes out:

Out In
xunit 2.9.3 xunit.v3 4.0.0
Microsoft.NET.Test.Sdk 18.9.0 — (v3 hosts itself)
xunit.runner.visualstudio 4.0.0 — (MTP replaces the adapter)
coverlet.collector 10.0.1 Microsoft.Testing.Extensions.CodeCoverage 18.10.0

Knock-on changes:

  • Test projects are now executables (<OutputType>Exe</OutputType>) — a v3 test project hosts its own runner.
  • A repo-root global.json opts dotnet test into the MTP runner. That runner takes the project as --project <path> rather than a bare argument, so every invocation in ci.yml, release.yml, README.md and CONTRIBUTING.md was updated.
  • TRX reports come from --report-xunit-trx --report-xunit-trx-filename <name>; v3 4.0 namespaced its report switches, and --logger "trx;..." is a VSTest concept that no longer applies.

Test-code changes

The v3 analyzers (2.0.0) added two rules that TreatWarningsAsErrors turns into hard build failures. Both were fixed rather than suppressed:

  • xUnit1051 (208 sites) — calls taking a CancellationToken now pass TestContext.Current.CancellationToken, so a hung await is actually interruptible by a timeout or a cancelled run. This matters most in the Execution suite, which does real process and PTY I/O. Applied mechanically via the analyzer's own code fix (dotnet format analyzers), not by hand.
  • xUnit2033 (12 sites) — use the value Assert.Single returns instead of re-deriving it with [0]. Hand-applied; the fixer doesn't support fix-all. This matches the idiom already used elsewhere in these suites.

CI gap

Snipdeck.Execution.Tests was built but never run by any job — its 61 tests only ever executed on a developer machine. It now runs:

  • on ubuntu, alongside Core (the job is renamed accordingly);
  • on Windows too, because Execution wraps ConPTY and process launching — the ubuntu run only proves it is portable, not that it works on the platform Snipdeck ships to.

Also

Acknowledgements in README.md and the in-app Settings list swap Coverlet for Microsoft Testing Platform, matching what is actually referenced now.

Verification

  • 372 tests pass on both platforms — Linux (Core 245, Execution 61, importer 66) and Windows (Core 245, Execution 61), the latter run with the exact switches CI now uses, confirming TRX artefacts are produced.
  • Full solution, WinUI head included, builds clean: 0 warnings, 0 errors on a Windows 11 host.

🤖 Generated with Claude Code

Two changes that both land in the test stack.

xUnit v3. The old `xunit` package ID is the feature-frozen v2 line (2.9.3 is its
latest); v3 ships as `xunit.v3`, currently 4.0.0. v3 self-hosts on Microsoft
Testing Platform rather than VSTest, so the whole VSTest stack comes out:
Microsoft.NET.Test.Sdk, xunit.runner.visualstudio, and coverlet.collector (a
VSTest data collector, inert under MTP) are replaced by xunit.v3 plus
Microsoft.Testing.Extensions.CodeCoverage. Test projects are now executables,
`dotnet test` opts into the MTP runner via a repo-root global.json (hence
`--project <path>`), and TRX comes from `--report-xunit-trx`, not `--logger trx`.

The v3 analyzers (2.0.0) added two rules that TreatWarningsAsErrors turns into
build failures, so the test code moved with them rather than suppressing:

  xUnit1051 (208 sites) — CancellationToken-accepting calls now pass
    TestContext.Current.CancellationToken, so a hung await is actually
    interruptible. Applied via the analyzer's own code fix.
  xUnit2033 (12 sites)  — use the value Assert.Single returns instead of
    re-deriving it with [0]. Hand-applied; the fixer has no fix-all.

CI gap. Snipdeck.Execution.Tests was built but never run by any job, so its 61
tests only ever executed locally. It now runs alongside Core on ubuntu, and also
on Windows — Execution wraps ConPTY and process launching, so Windows is the
platform that actually matters for it.

Acknowledgements (README and the in-app list) swap Coverlet for Microsoft
Testing Platform to match what is now referenced.

All 372 tests pass on both Linux and Windows, and the full solution builds clean
with no warnings on the Windows head.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@StuartMeeks
StuartMeeks merged commit dd508dd into main Aug 19, 2026
6 checks passed
@StuartMeeks
StuartMeeks deleted the chore/xunit-v3-and-ci-gap branch August 19, 2026 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant