Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 97 additions & 0 deletions .github/workflows/pr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Pull-request checks for ChatCPT.
#
# Includes an axe-core accessibility gate against a checked-in shrink-only
# baseline (a11y-audits/tools/baseline.json). The baseline may only ever
# shrink — new rules or higher node counts fail the build.
#
# IMPORTANT: axe cannot detect live-region over-announcement (finding A1 in
# a11y-audits/8-5-26/audit.md). A green badge here is a floor, not WCAG
# conformance. Manual VoiceOver checks remain mandatory for live-region and
# focus work.

name: PR

on:
pull_request:

jobs:
check:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v4
Comment on lines +17 to +23

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "Workflow token defaults:"
gh api "repos/${GITHUB_REPOSITORY}/actions/permissions/workflow" \
  | jq '{default_workflow_permissions, can_approve_pull_request_reviews}'

echo "Checkout and permissions configuration:"
sed -n '12,35p' .github/workflows/pr.yml

Repository: CodeSignal/learn_cosmo-chat

Length of output: 240


🏁 Script executed:

set -euo pipefail

echo '--- workflow ---'
cat -n .github/workflows/pr.yml

echo '--- package manifests ---'
for f in package.json npm-shrinkwrap.json package-lock.json; do
  if [ -f "$f" ]; then
    echo "--- $f ---"
    cat -n "$f" | sed -n '1,180p'
  fi
done

echo '--- workflow references ---'
rg -n 'npm (ci|test|run|install)|yarn|pnpm|checkout|permissions|pull_request|persist-credentials' .github/workflows/pr.yml

Repository: CodeSignal/learn_cosmo-chat

Length of output: 12994


Restrict the workflow token and disable checkout credential persistence.

The workflow runs pull-request-controlled scripts after checkout. Set permissions: contents: read and persist-credentials: false for actions/checkout@v4.

🧰 Tools
🪛 zizmor (1.29.0)

[warning] 22-23: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false

(artipacked)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/pr.yml around lines 17 - 23, Update the check job in the
workflow to grant only contents read permission, and configure the
actions/checkout@v4 step with persist-credentials set to false. Keep the
existing checkout behavior unchanged otherwise.

Source: Linters/SAST tools


- name: Populate design system submodule
run: git submodule update --init

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'

- name: Install dependencies
run: npm ci

- name: Unit tests
run: npm test

- name: Build frontend bundle
run: npm run build

- name: Install accessibility audit tools
working-directory: a11y-audits/tools
run: npm ci

- name: Baseline helper unit tests
working-directory: a11y-audits/tools
run: npm test

- name: Install Playwright Chromium
working-directory: a11y-audits/tools
run: npx playwright install chromium --with-deps

- name: Start app server
# No Octavus credentials in CI — audit.mjs stubs /api/* when A11Y_CI=1.
# A placeholder AGENT_ID keeps the server from warning; the stub never
# reaches the create-session path.
run: |
PORT=3100 AGENT_TARGET=dev OCTAVUS_AGENT_ID=ci-placeholder \
node server.js > /tmp/chatcpt-ci-server.log 2>&1 &
echo $! > /tmp/chatcpt-ci-server.pid
for i in $(seq 1 30); do
if curl -sf http://127.0.0.1:3100/ >/dev/null; then
echo "Server ready"
exit 0
fi
sleep 1
done
echo "Server failed to start" >&2
cat /tmp/chatcpt-ci-server.log >&2
exit 1

- name: Axe scan (light + dark) against shrink-only baseline
working-directory: a11y-audits/tools
env:
A11Y_CI: '1'
A11Y_BROWSER_CHANNEL: bundled
A11Y_BASE_URL: http://127.0.0.1:3100
A11Y_OUT: a11y-out-ci
A11Y_BASELINE: ${{ github.workspace }}/a11y-audits/tools/baseline.json
run: npm run audit

- name: Upload axe report
if: always()
uses: actions/upload-artifact@v4
with:
name: axe-report
path: a11y-audits/tools/a11y-out-ci/
if-no-files-found: ignore

- name: Stop app server
if: always()
run: |
if [ -f /tmp/chatcpt-ci-server.pid ]; then
kill "$(cat /tmp/chatcpt-ci-server.pid)" || true
fi
1 change: 1 addition & 0 deletions a11y-audits/tools/.gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
node_modules/
a11y-out/
a11y-out-ci/
29 changes: 28 additions & 1 deletion a11y-audits/tools/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

The Playwright + axe-core scripts that produced the evidence in
[`../8-5-26/audit.md`](../8-5-26/audit.md). Kept in the repository so findings can be
re-measured after each fix, and because `audit.mjs` is the intended starting point for the
re-measured after each fix, and because `audit.mjs` drives the
CI accessibility gate ([#36](https://github.com/CodeSignal/learn_cosmo-chat/issues/36)).

These have their own dependency tree. They are **not** part of the application build and are
Expand Down Expand Up @@ -37,10 +37,37 @@ npm run audit
| `A11Y_BASE_URL` | `http://localhost:3100` | Where the app is running |
| `A11Y_OUT` | `a11y-out` | Directory for screenshots and `report.json` |
| `A11Y_BROWSER_CHANNEL` | `chrome` | Set to `bundled` to use Playwright's own Chromium |
| `A11Y_CI` | unset | `1` stubs `/api/*`, skips the live-agent send flow, and enables the baseline gate |
| `A11Y_BASELINE` | `./baseline.json` when `A11Y_CI=1` | Path to the shrink-only axe baseline |
| `A11Y_UPDATE_BASELINE` | unset | `1` rewrites the baseline from the current run |

`A11Y_BROWSER_CHANNEL` defaults to system Chrome because Playwright's bundled Chromium was
missing on the audit machine. In CI, run `playwright install chromium` and set it to `bundled`.

### CI gate

`.github/workflows/pr.yml` runs:

```bash
npm run audit:ci
```
Comment on lines +49 to +53

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Correct the documented workflow command.

This section states that .github/workflows/pr.yml runs npm run audit:ci. The workflow runs npm run audit and sets the CI variables explicitly at .github/workflows/pr.yml Line 76 through Line 82. State the actual command, or change the workflow to call npm run audit:ci.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@a11y-audits/tools/README.md` around lines 49 - 53, Update the workflow
command documentation in the README to state that .github/workflows/pr.yml runs
npm run audit, matching the workflow’s explicit CI variable configuration;
alternatively, change the workflow command to npm run audit:ci and keep the
documentation consistent.


That compares the axe results for empty / settings / settings-with-dropdown in both color
schemes against [`baseline.json`](./baseline.json). The baseline is **shrink-only**: a new
rule or a higher node count fails the build; a fix that removes violations should update
`baseline.json` in the same PR so the floor ratchets down.

```bash
# after a fix that clears axe violations:
npm run audit:update-baseline
```

CI does not exercise streaming or populated-conversation states (no live agent). Those
axe results matched the empty-state shell in the original audit, and the settings /
dropdown states carry the distinctive rules (`aria-input-field-name`, the extra
`region` nodes, dark-mode contrast). Re-run the full `npm run audit` locally when a
fix needs the live-agent states.

## What each script does

**`audit.mjs`** — the main sweep. For each of light and dark mode it walks the app through five
Expand Down
Loading