ci: report Rust and web test coverage on pull requests - #49
Merged
Merged
Conversation
Nothing in CI showed how much of the code the tests exercise, so a PR could drop coverage without anyone noticing. Rust coverage runs in its own job with cargo-llvm-cov and nextest. The instrumented build is slower, and a separate runner keeps the partitioned test jobs unchanged. Test code is excluded from the figures so they describe the code under test. A failing test does not fail this job, because the partitions already report that; the numbers are still written. Web coverage comes from Vitest's v8 provider, which the frontend job now runs in place of the plain test command. The provider is pinned to the locked Vitest version so adding it does not move the test runner. Coverage `include` covers all of src, so a module no test imports counts against the total instead of being invisible. A report job collates both into the job summary and a single pull request comment that is edited in place, the same way the API contract report is. It only reports and never fails the run. The summary is rendered by a small script that tolerates a missing input, so a broken test job shows up as "no report" rather than a second red check. The lcov files are kept as artifacts for 14 days. There is no external coverage service, so no history is kept across commits.
Deploying codex with
|
| Latest commit: |
78daf0c
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://60f5f2a4.codex-asm.pages.dev |
| Branch Preview URL: | https://ci-coverage-report.codex-asm.pages.dev |
API contract changesCompared against Report |
CoverageBackend (Rust)
Frontend (web)
Line-level reports (lcov) are attached to the workflow run as the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
Coverage (Rust)):cargo llvm-cov nextest --workspace --features raron its own runner, writing an lcov file and a JSON summary. Test code (tests/) is excluded so the figures describe the code under test.npm run test:coverage(Vitest's v8 provider, now actually installed).coverage.includecovers all ofsrc, so a module no test imports counts against the total instead of being invisible.coverage-rustandcoverage-web(lcov plus summaries), kept for 14 days.Design choices
@vitest/coverage-v8pinned to the locked Vitest (4.0.18) so adding it does not move the test runner. The lockfile otherwise only adds the provider's dependencies and patch-bumps a few@babel/*parser packages it needs.main. Codecov or similar can be added later if trends become worth the third-party upload.Cost
One extra instrumented run of the backend suite per PR, in parallel with the existing jobs.
Verified locally
npm run test:coverage: all tests pass;coverage-summary.jsonandlcov.infoproduced (web totals: lines 64.0%, statements 63.4%, functions 52.0%, branches 58.2%).npm run lintclean,npm run buildsucceeds.cargo llvm-covoncodex-utilswith the same report flags as CI, then.github/scripts/coverage-summary.shon the real Rust and web outputs: renders the per-crate and web tables. The script was also checked with missing inputs.