Repository navigation
ci: nightly browser suite - #540
Merged
Merged
Conversation
… tests WIP: the suite still has failures on a clean master that are not fixed here. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…roof - opencode-resize: record WebSocket resize frames as well as POST /resize, seed the needsRefresh test with real PTY output, skip the OpenCode close modal test when opencode is not installed - split-pane: send the marker with useMux:false (plain prompts otherwise go through tmux send-keys, which test mode does not have) and retry past Codeman's own post-create clear Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JrzFKEdBLwVfu6ev2ZscJS
Ark0N
pushed a commit
that referenced
this pull request
Oct 9, 2026
Move the nightly cron from 03:17 to 03:23 UTC. GitHub sends scheduled-run
failure notices to whoever last modified the cron line, and after the merge
that is the contributor, so a maintainer commit has to touch it. The
docs below give no clock time, so they cannot drift from the cron.
Drop the "Keep the failure artifacts" step and the blank line before it.
No browser test writes test-results/ or screenshots-echo-diag/ (only the
ignore files name them), and if-no-files-found: ignore made the step upload
nothing without a word. The run log already carries the failure output.
Reword the workflow header. Drop the claim that the skipped suite let two
semantically conflicting PRs merge green: that incident came from
test/mobile/keyboard.test.ts, which this job does not run. Correct the
codex-predictive-echo note: the test uses a fake key in a throwaway
CODEX_HOME and skips itself when codex is missing, so it needs a codex
binary, not an authenticated one.
opencode-resize: record WebSocket resize frames under the socket's own URL
instead of appending '#' + the session id. The URL already carries
/ws/sessions/<id>/terminal, and the suffix let toContain(sessionId) pass for
a resize sent on any session's socket, the bug this test exists to catch.
Reduce the six session-id extractions (opencode-resize and perf-browser) to
data.data?.session?.id. POST /api/sessions always answers in the
{ success, data: { session } } envelope, and the dead fallbacks are what
hid the original breakage.
split-pane-terminal: restore the browser config's 60 s test timeout (the
added 20000 ms override tightened it), and replace the comment that blamed
Codeman's post-create clear. Under vitest the session is an echo PTY, so
that clear comes back as text; the real fix is useMux:false, since a plain
prompt otherwise goes through tmux send-keys, which test mode no-ops.
CLAUDE.md: the CI note now says the gate excludes the Playwright tests in
BROWSER_TEST_GLOBS instead of a stale count of 14, and names
browser-suite.yml; the Testing warning says the browser suite runs nightly.
CONTRIBUTING.md gets the same one-line pointer under Tests.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ark0N
added a commit
that referenced
this pull request
Oct 9, 2026
Move the nightly browser-suite cron from 03:23 to 03:29 UTC, authored as Ark0N. GitHub sends scheduled-run failure notices to whoever last modified the cron line, but its docs do not say whether that means the commit author or the pusher. The previous cron edit (02c65e9) was authored under the maintainer identity, whose noreply@anthropic.com address GitHub resolves to the unrelated login "claude", so under the author reading the nightly's failure notices would never reach the maintainer. With this commit the author, the committer and the pusher are all Ark0N, so every reading lands on the maintainer. Only the minute changes; no doc or test names a clock time. After the first scheduled run on master, confirm with gh api 'repos/Ark0N/Codeman/actions/runs?event=schedule&per_page=1' --jq '.workflow_runs[0].actor.login', which should print Ark0N. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ark0N
pushed a commit
that referenced
this pull request
Oct 9, 2026
Thirteen contributor PRs, each re-checked against its GitHub head, merged with its own merge commit and landing fixes, reviewed, and gated together (524 test files, 10194 tests): #559, #552, #550, #556, #551, #546, #542, #540, #555, #543, #541, #502, #432. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Owner
|
Merged in 1.40.0, thanks @opticon454! A nightly run of the suites the CI gate cannot see was overdue, and fixing the three stale browser tests first means it starts green. Applied on the way in (02c65e9, ccd5258): the cron minute moved, because GitHub sends scheduled-run failure notices to whoever last edited the cron line; the artifact step that never uploaded anything is gone; opencode-resize records resize frames under the socket's own URL, so it can no longer pass on a resize sent to another session; and the split-pane test keeps the browser config's 60 s timeout. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A scheduled (03:17 UTC) and manually runnable GitHub Actions workflow,
Browser suite, that runsnpm run test:browseron a clean ubuntu runner (Node 22, tmux,playwright install --with-deps chromium). It is informational only: it is not a merge gate and never blocks a push or PR. The per-push gate (npm test) is unchanged;ci.ymlonly gains a two-line comment pointing at the new job.Why
config/test-suites.tskeeps the Playwright suite out of the per-push gate, which means a browser-only regression (the Shift+Enter keypress bug was one) can merge green. Running it nightly catches those within a day. It needs no secrets: the tests start their own server and drive a local chromium.Left out on purpose:
test:mobile(per-machine PNG baselines),test:perf(wall-clock), andtest/codex-predictive-echo.test.ts(needs a real, authenticated codex binary).Evidence
I ran it on a fork to see whether it can be green on a clean runner. First run: 119 passed, 5 failed. The failures were stale or environment-dependent tests rather than product bugs, so this PR also fixes them:
test/opencode-resize.test.tsdata.id, butPOST /api/sessionsanswers{ success, data: { session } }.{t:'z'}frames while the socket is up, so the test records those as well asPOST /resize.opencodeis not installed.test/split-pane-terminal.browser.test.ts: plain<text>\rinput goes through tmuxsend-keys, which test mode does not have, so the marker never landed. It is now sent withuseMux:falseand retried past Codeman's own post-createclear.test/perf-browser.test.ts: same session-id envelope fix.Latest fork run (before #534 landed, on 1.34.0): 122 passed, 1 skipped, 1 failed, the one failure being
test/inline-rename.test.ts(rail rename editor clamped), a real product bug that #534 / #526 have since fixed on master. This branch is rebased onto 1.35.0 andinline-rename,opencode-resizeandsplit-pane-terminalpass locally (36 passed, 1 skipped for the OpenCode-not-installed case), so the first nightly run should be green. I have not re-run the whole suite on a runner since the rebase: Actions → Browser suite → Run workflow after merge is the quickest confirmation.Notes
permissions: contents: read.