Skip to content

ci: run the visual smoke tests - #213

Merged
shockalotti merged 1 commit into
mainfrom
ci/run-visual-tests
Nov 3, 2026
Merged

shockalotti merged 1 commit into
mainfrom
ci/run-visual-tests

Conversation

@shockalotti

Copy link
Copy Markdown
Contributor

The repo's CLAUDE.md says to run pnpm test:visual after touching ui/**, app.css or the theme, because the suite mounts components in a real Chromium and asserts computed styles. That is the only thing that catches a bits-ui data-state losing its @custom-variant remap: the logic stays green while the component renders unstyled, which neither svelte-check nor a build can see.

Nothing enforced it, so it was a manual step. It was skipped during the recent dependency bumps, including a major bump of vitest-browser-svelte, and only ran because I did it by hand.

Change

- name: Install the browser for visual tests
  working-directory: frontend
  run: pnpm exec playwright install --with-deps chromium

- name: Visual smoke tests (computed styles in a real browser)
  working-directory: frontend
  run: pnpm test:visual

Placed after pnpm build so a broken build reports first, and before the Go steps.

Cost

Chromium download plus system deps, roughly 1 to 2 minutes. The suite itself is under 2 seconds (5 tests).

Verification

This PR is its own test: CI should now show both new steps, and the suite passed locally on current main in 1.29s.

The suite in frontend (vitest browser mode, Chromium via Playwright) asserts
computed styles for the shadcn components, which is what catches a bits-ui
data-state that lost its @custom-variant remap: the logic works, the component
renders unstyled, and neither svelte-check nor a build can see it. CLAUDE.md
already requires running it after touching ui/**, app.css or the theme, but
nothing enforced that, so it was a manual step that could be forgotten. It was
in fact skipped for the recent dependency bumps.

Runs after the frontend build so a broken build reports first, and installs the
Chromium binary the config needs (the pnpm store does not carry it).

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • CI runs the visual smoke suite — adds pnpm exec playwright install --with-deps chromium and pnpm test:visual to the check job, placed after pnpm build and before the Go steps, so the real-browser computed-style suite is enforced on every push to main and every PR.

I confirmed the new steps ran and passed on 6a69e39 (job 125632983912), that test:visual resolves to vitest run against the browser-mode vitest.config.ts (headless Chromium via @vitest/browser-playwright), and that playwright is a devDependency so pnpm exec playwright resolves. The suite is the only src/**/*.test.ts file, so vitest run matches exactly the intended tests.

The only tradeoff worth naming is that the Chromium download (~1–2 min) is not cached across runs; the PR body already prices that in, and caching the browser is optional rather than required for correctness.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash | 𝕏

@shockalotti
shockalotti merged commit 0c789f2 into main Nov 3, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant