ci(a11y): add PR checks with a shrink-only axe baseline - #38
Conversation
There was no CI on pull requests. Gate unit tests, the build, and an axe scan in both color schemes on today's known violations so the remediation wave cannot silently regress. axe still cannot see A1. Co-authored-by: Cursor <cursoragent@cursor.com>
📝 WalkthroughWalkthroughAdds a pull-request workflow that installs dependencies, runs tests, builds the frontend, starts the application, and executes light and dark accessibility scans. Adds deterministic CI API stubs and limits CI audits to supported states. Adds a checked-in axe baseline with shrink-only comparison logic, regression detection, missing-scan checks, update support, and unit tests. Documents CI limitations, including live-region behavior not detected by axe. Possibly related issues
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/pr.yml:
- Around line 17-23: Update the check job in the workflow to grant only contents
read permission, and configure the actions/checkout@v4 step with
persist-credentials set to false. Keep the existing checkout behavior unchanged
otherwise.
In `@a11y-audits/tools/README.md`:
- Around line 49-53: Update the workflow command documentation in the README to
state that .github/workflows/pr.yml runs npm run audit, matching the workflow’s
explicit CI variable configuration; alternatively, change the workflow command
to npm run audit:ci and keep the documentation consistent.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 9d841ef2-e47c-4529-894b-61482119485a
📒 Files selected for processing (8)
.github/workflows/pr.ymla11y-audits/tools/.gitignorea11y-audits/tools/README.mda11y-audits/tools/audit.mjsa11y-audits/tools/baseline.jsona11y-audits/tools/baseline.mjsa11y-audits/tools/baseline.test.mjsa11y-audits/tools/package.json
| jobs: | ||
| check: | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Workflow token defaults:"
gh api "repos/${GITHUB_REPOSITORY}/actions/permissions/workflow" \
| jq '{default_workflow_permissions, can_approve_pull_request_reviews}'
echo "Checkout and permissions configuration:"
sed -n '12,35p' .github/workflows/pr.ymlRepository: CodeSignal/learn_cosmo-chat
Length of output: 240
🏁 Script executed:
set -euo pipefail
echo '--- workflow ---'
cat -n .github/workflows/pr.yml
echo '--- package manifests ---'
for f in package.json npm-shrinkwrap.json package-lock.json; do
if [ -f "$f" ]; then
echo "--- $f ---"
cat -n "$f" | sed -n '1,180p'
fi
done
echo '--- workflow references ---'
rg -n 'npm (ci|test|run|install)|yarn|pnpm|checkout|permissions|pull_request|persist-credentials' .github/workflows/pr.ymlRepository: CodeSignal/learn_cosmo-chat
Length of output: 12994
Restrict the workflow token and disable checkout credential persistence.
The workflow runs pull-request-controlled scripts after checkout. Set permissions: contents: read and persist-credentials: false for actions/checkout@v4.
🧰 Tools
🪛 zizmor (1.29.0)
[warning] 22-23: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/pr.yml around lines 17 - 23, Update the check job in the
workflow to grant only contents read permission, and configure the
actions/checkout@v4 step with persist-credentials set to false. Keep the
existing checkout behavior unchanged otherwise.
Source: Linters/SAST tools
| `.github/workflows/pr.yml` runs: | ||
|
|
||
| ```bash | ||
| npm run audit:ci | ||
| ``` |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Correct the documented workflow command.
This section states that .github/workflows/pr.yml runs npm run audit:ci. The workflow runs npm run audit and sets the CI variables explicitly at .github/workflows/pr.yml Line 76 through Line 82. State the actual command, or change the workflow to call npm run audit:ci.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@a11y-audits/tools/README.md` around lines 49 - 53, Update the workflow
command documentation in the README to state that .github/workflows/pr.yml runs
npm run audit, matching the workflow’s explicit CI variable configuration;
alternatively, change the workflow command to npm run audit:ci and keep the
documentation consistent.
Co-authored-by: Cursor <cursoragent@cursor.com>
1 participant
Closes #36. Second Wave 0 foundation item from the accessibility remediation plan.
Why
There is currently no CI on pull requests.
release.ymlonly fires onrelease:created, so nothing runsnpm testwhen a PR opens, and nothing checks accessibility at all. Without a gate, the 16 accessibility fixes will erode.What's here
.github/workflows/pr.ymlonpull_request:npm cinpm test+npm run builda11y-audits/tools/adaptations for CI:A11Y_CI=1stubs/api/*(no Octavus credentials in Actions), skips the live-agent send flow, and scans empty / settings / settings-with-dropdown in light and darkbaseline.json+baseline.mjs— fail on new rules or higher node counts; counts may only shrinknpm run audit:ci/audit:update-baseline/testfor the compare helperThe baseline matches the issue's known table (contrast, nested-interactive, landmark, region, aria-input-field-name). Streaming/with-messages are skipped in CI — those states matched the empty-state shell in the original audit; re-run the full local audit when a fix needs them.
Important limitation
axe cannot detect A1. Stated in the workflow file and in the baseline
$comment. A green badge is a floor, not conformance.Test plan
npm test(app) — 173 passed | 2 expected faila11y-audits/toolsbaseline helper tests — 5 passedA11Y_CI=1audit against a running server — gate passesReference
a11y-audits/8-5-26/resolution-plan.md→ Wave 0 → P0-2Made with Cursor