test: migrate to xUnit v3 and run the Execution suite in CI - #69
Merged
Conversation
Two changes that both land in the test stack.
xUnit v3. The old `xunit` package ID is the feature-frozen v2 line (2.9.3 is its
latest); v3 ships as `xunit.v3`, currently 4.0.0. v3 self-hosts on Microsoft
Testing Platform rather than VSTest, so the whole VSTest stack comes out:
Microsoft.NET.Test.Sdk, xunit.runner.visualstudio, and coverlet.collector (a
VSTest data collector, inert under MTP) are replaced by xunit.v3 plus
Microsoft.Testing.Extensions.CodeCoverage. Test projects are now executables,
`dotnet test` opts into the MTP runner via a repo-root global.json (hence
`--project <path>`), and TRX comes from `--report-xunit-trx`, not `--logger trx`.
The v3 analyzers (2.0.0) added two rules that TreatWarningsAsErrors turns into
build failures, so the test code moved with them rather than suppressing:
xUnit1051 (208 sites) — CancellationToken-accepting calls now pass
TestContext.Current.CancellationToken, so a hung await is actually
interruptible. Applied via the analyzer's own code fix.
xUnit2033 (12 sites) — use the value Assert.Single returns instead of
re-deriving it with [0]. Hand-applied; the fixer has no fix-all.
CI gap. Snipdeck.Execution.Tests was built but never run by any job, so its 61
tests only ever executed locally. It now runs alongside Core on ubuntu, and also
on Windows — Execution wraps ConPTY and process launching, so Windows is the
platform that actually matters for it.
Acknowledgements (README and the in-app list) swap Coverlet for Microsoft
Testing Platform to match what is now referenced.
All 372 tests pass on both Linux and Windows, and the full solution builds clean
with no warnings on the Windows head.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two changes, both in the test stack. The shipped app is unaffected.
xUnit v3
The
xunitpackage ID is the feature-frozen v2 line — 2.9.3 is its latest. v3 ships asxunit.v3, currently 4.0.0. v3 self-hosts on Microsoft Testing Platform instead of VSTest, so the whole VSTest stack comes out:xunit2.9.3xunit.v34.0.0Microsoft.NET.Test.Sdk18.9.0xunit.runner.visualstudio4.0.0coverlet.collector10.0.1Microsoft.Testing.Extensions.CodeCoverage18.10.0Knock-on changes:
<OutputType>Exe</OutputType>) — a v3 test project hosts its own runner.global.jsonoptsdotnet testinto the MTP runner. That runner takes the project as--project <path>rather than a bare argument, so every invocation inci.yml,release.yml,README.mdandCONTRIBUTING.mdwas updated.--report-xunit-trx --report-xunit-trx-filename <name>; v3 4.0 namespaced its report switches, and--logger "trx;..."is a VSTest concept that no longer applies.Test-code changes
The v3 analyzers (2.0.0) added two rules that
TreatWarningsAsErrorsturns into hard build failures. Both were fixed rather than suppressed:CancellationTokennow passTestContext.Current.CancellationToken, so a hungawaitis actually interruptible by a timeout or a cancelled run. This matters most in the Execution suite, which does real process and PTY I/O. Applied mechanically via the analyzer's own code fix (dotnet format analyzers), not by hand.Assert.Singlereturns instead of re-deriving it with[0]. Hand-applied; the fixer doesn't support fix-all. This matches the idiom already used elsewhere in these suites.CI gap
Snipdeck.Execution.Testswas built but never run by any job — its 61 tests only ever executed on a developer machine. It now runs:Also
Acknowledgements in
README.mdand the in-app Settings list swap Coverlet for Microsoft Testing Platform, matching what is actually referenced now.Verification
🤖 Generated with Claude Code