Skip to content

feat: add a ci-result check and stop repeating check names - #65

Open
aryanjasala wants to merge 4 commits into
fix/wp-ci-known-bugsfrom
feat/ci-result
Open

aryanjasala wants to merge 4 commits into
fix/wp-ci-known-bugsfrom
feat/ci-result

Conversation

@aryanjasala

@aryanjasala aryanjasala commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

What this PR does

Gives branch protection one stable check to require, <caller job> / CI result, and makes the other check names readable. Today a repository has to require individual leaf checks, whose names and count change with project-type, skip, the PHP matrix and what a PR touched, and the names repeat themselves (CI / Lint PHP / Lint PHP, Test PHP (8.3, ) / PHPUnit).

Closes

N/A

Changes

  • New ci-result job in wp-ci.yml, shown as <caller job> / CI result. It needs every job, runs on always(), writes a table of every job's result to the job summary, and fails when any job failed, was cancelled or never started. A job skipped by its own gate (nothing it checks changed, skip, an opt-in left off) passes.

  • The test-php matrix job is named Test PHP <php> or Test PHP <php>, WP <wp> instead of GitHub's default Test PHP (8.3, ).

  • Leaf jobs that repeated their caller's name now say what they run: ci-lint-php.yml is PHPCS (or PHPCS + PHPStan with enable-phpstan), ci-build.yml is Production build, ci-detect-changes.yml is Bucket changed files, ci-build-artifact-gate.yml is Check gated paths. So CI / Lint PHP / Lint PHP reads CI / Lint PHP / PHPCS + PHPStan.

  • README: a Branch protection section saying to require CI result and nothing else.

  • Docs for the renames (37a6a6b): the README's Branch protection section gains a "Renamed checks" table (old and new name per workflow) and a note that a calling job with its own if: reports no CI result when skipped. AGENTS.md's rule now says a renamed job name is feat: when a code search finds no caller requiring the old check (with that evidence here), and feat!: otherwise, which is the call this PR makes.

How I verified

$ actionlint .github/workflows/*.yml examples/*.yml
$ bin/check-workflows.sh
Workflow/example contract check passed.
$ pipx run yamllint .   # only the existing warning on main

The ci-result step run on its own with two needs payloads:

{a: success, b: skipped}                 -> "Every job passed or was skipped by its own gate." exit 0
{a: success, b: failure, c: cancelled}   -> ::error::CI did not pass: b (failure), c (cancelled).  exit 1

I also checked who requires the current names: on 2026-10-29 none of icimod, theme-elementary, plugin-elementary, wp-primitives, wp-tooling or humanetics had a required check coming from wp-ci.yml, so the renames break no branch protection today. Rechecked on 2026-11-06 across every caller code search finds (icimod, stepwise, theme-elementary, plugin-elementary, Anupkankale/imet-theme): none requires a wp-ci check; stepwise requires only its own Validate PR title, and imet-theme's protection isn't readable from outside its org. That is why this is feat, not feat!: a ! would make it v2, which @v1 callers would never get.

Acceptance criteria

Runtime behavior

  • CI result fails on any failed, cancelled or never-started job
  • CI result passes when the remaining jobs were skipped by their own gates
  • Matrix and leaf check names no longer repeat
  • Seen on a real PR in a consumer repository

Code quality

  • Lint: zero errors
  • Static analysis: zero errors
  • Tests pass (result script run against both payloads)

Housekeeping

  • No CHANGELOG.md edit: release-please writes it from the PR title
  • examples/<workflow>.yml added or updated to match any input change (no input changed)
  • yamllint, actionlint and bin/check-workflows.sh pass locally

Reviewer notes

  • always(), not !cancelled(), on purpose. GitHub counts a skipped required check as passing. With !cancelled(), someone cancelling the run on a PR's head commit would leave CI result skipped and the PR mergeable with nothing checked. With always() a cancelled run fails the check.
  • The renames are a check-name change for anyone who already requires a leaf check. Nobody does today (see above), and after this PR the documented answer is to require CI result instead.
  • A monorepo caller with a matrix gets one CI result per unit (plugins/acme-blocks / CI result), which the README example shows.

wp-ci.yml gains a ci-result job, shown as "<caller job> / CI result":
one stable check to require in branch protection. It needs every other
job, runs even when they fail or are skipped, and fails if any of them
failed or was cancelled, so a skipped job still counts as passing but a
broken one does not.

The leaves' job names no longer repeat the caller's: the test-php
matrix job is "Test PHP <php>" or "Test PHP <php>, WP <wp>", and the
build, artifact-gate, detect-changes and lint-php jobs drop their
duplicated prefixes. No known caller requires a wp-ci check by name
today; ci-result is the one to require from now on.
AGENTS.md now matches what this change does: a renamed job name is feat
when a code search finds no caller requiring the old check, with that
evidence in the PR body, and feat! otherwise. The README lists each old
and new check name, and says that a calling job with its own if: reports
no CI result when it is skipped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant