test(ci): pin required status contexts to the jobs that report them - #1923
test(ci): pin required status contexts to the jobs that report them#1923seonghobae wants to merge 6 commits into
Conversation
.github is excluded from the organization required-workflow ruleset, so its default branch runs classic branch protection with twelve named required status contexts. GitHub matches those on the check-run name, which comes from a job's name: when it has one and from the job id otherwise. Renaming or folding away such a job is therefore a repository-wide outage rather than a local edit: protection keeps waiting for a context nothing will report, so every pull request stays blocked with no failing check to point at. Nothing in the suite pinned these names, and the two identifiers can drift apart -- they already have, since job id opencode-review-target reports the context opencode-review. The hazard is live because .github/workflows/ is under active consolidation (21 consolidation/coalescing commits between 2026-09-01 and 2026-09-05), and folding jobs together is exactly the edit that renames them. Verified the guard fails for both spellings: renaming an explicit name: and renaming a job id each trip it, and it is green on unmodified main. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Warning Review limit reachedNext included review available in 4 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…names A sweep of branch protection across all 76 organization repositories found 13 using classic protection, several of which pin the job names these central workflows declare: opencode-review and coverage-evidence are each required by 7 repositories, strix by 5, scan-pr-queue by 4, required-workflow-bootstrap by 3, and coverage-source-tree by 2. strix and coverage-source-tree are required by siblings but not by .github, so pinning only this repository's own contexts left them unguarded -- renaming either blocks every pull request in those repositories. admit-current-head is required by none, so it stays out. The check also had a false negative. It accepted either a `name:` match or a job id match, so renaming a job's `name:` passed whenever the id still matched, even though GitHub names the check run after `name:` when one is present. It now resolves each job's effective check-run name instead. Verified by mutation: renaming a job id, and renaming a `name:` on a job that also has an id, are both caught -- the latter was not caught before this change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Live proof, found the same day this PR was opened. Sweeping branch protection across all 76 organization repositories turned up two that already hit exactly this failure, both from consolidation commits merged on 2026-09-04:
Neither repository has a failing check to point at. Both look like queue congestion from the outside. This is the failure this test exists to make loud. The sweep also drove the two additions in the second commit here: |
…s-contexts # Conflicts: # CHANGELOG.md
|
Merged Gates on the merged head: 2896 passed, 1 skipped, 21 subtests; |
…s-contexts # Conflicts: # CHANGELOG.md
|
Merged Gates on |
…s-contexts # Conflicts: # CHANGELOG.md
|
Merged Gates on One note for anyone reading the PR state: GitHub reported |
|
BandScope now provides a live consumer-repository instance of the same required-context drift class, but across the organization required-workflow boundary rather than inside Fresh 2026-09-06 evidence: protected I updated BandScope #1172 with the exact repair contract: preserve CodeQL coverage, migrate the two stale required contexts to For this guard lane, please treat required-workflow producer names plus consumer branch/ruleset required-context names as one rollout invariant. A central workflow can remain perfectly self-consistent while consumer repositories are permanently blocked on retired contexts. |
…s-contexts # Conflicts: # CHANGELOG.md
|
Merged Gates on GitHub reported |
1 participant
.githubis excluded from the organization required-workflow ruleset (repository_name.excludelists it), so its default branch is guarded by classic branch protection with twelve named required status contexts. GitHub matches a context on the check-run name — a job'sname:when it has one, and the job id otherwise.That makes renaming such a job a repository-wide outage rather than a local edit. Branch protection keeps waiting for a context nothing will ever report, so every pull request stays blocked with no failing check to point at. Nothing in this suite pinned these names.
Why now
.github/workflows/is under active consolidation — 21 consolidation/coalescing commits between 2026-09-01 and 2026-09-05 (#1826OSV/Scorecard, Gitleaks, Noema/OpenCode/Strix bootups, SBOM attestation, empty-PR cleanup, head coalescing folded into the scheduler). Folding jobs together is precisely the edit that renames or removes them.The drift this guards against is not hypothetical — it already exists: job id
opencode-review-targetreports the contextopencode-review. Renaming only thename:breaks protection while the job id still looks correct.What it checks
Each of the twelve live contexts is pinned to the workflow whose job reports it, accepting either spelling GitHub accepts:
opencode-review,coverage-evidence,required-workflow-bootstrap,noema-review,Detect CodeQL languagesname:trivy-fs,scorecard,dependency-review,scan-pr-queue,osv-scanname:declared)CodeQL compatibility analysis (actions|python)(${{ matrix.language }})That split is worth noting: four of these jobs declare no
name:at all, so their context is the YAML key. A guard that only looked forname:would pass while leaving them unprotected.Verification
The test failed on unmodified
mainon the first attempt — my initial mapping assumed every context came from aname:line, and the four id-derived ones exposed that. Fixed, then verified in both directions:mainname:is renamed (opencode-review→opencode-review-consolidated)trivy-fs→trivy-filesystem)Both negative cases matter, because a guard that passes today but wouldn't catch the rename it exists to catch is worse than none. Full suite: 2886 passed, 1 skipped, 21 subtests.
interrogate: 100%.If a context is ever deliberately retired, update branch protection first, then this test — changing the test alone re-arms the outage. The docstring says so.
🤖 Generated with Claude Code