You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
find which models are most likely to work for security reviews #963
Category: enhancement Summary: Choose the Harness and Model for Security reviews, from a record of which Harness actually ran each review.
Correction: an earlier version of this brief read today's six failed Security reviews as Codex failures. They ran on Grok Build. security.harness = "grok" in the User config on thirdshift sent them there, and Grok Build isn't signed in on that host, so each session ended at once. #974 covers the hidden cause; #975 adds a preflight check and fallback.
What the record shows:
Until Run the Security review on security.harness when it is set #910 shipped in 0.13.2 on 2026-10-21, a Security review always ran on its Command's Harness, so a PR's "Built with" line named the review's Harness too. Since 0.13.2, security.harness picks the review's Harness when it's set. The Run's Command log names it: security-review: on <Harness> · <Model> · <effort>, chosen by ….
Before 2026-10-21, 59 thirdshift PRs had a "Security review outcome" section:
Decide which Harness and Model security.harness should name on each host. stephens.page leaves it blank, so its reviews run on Codex, its default Harness.
If Grok Build stays on thirdshift, sign it in there (grok login --device-code) and check that a review completes.
File any follow-ups as their own issues, and close this one with the decision.
Why this can't go to an agent: signing in a Harness needs the host, and choosing a Model means weighing cost and subscriptions, which is your call.
Out of scope: the preflight check and fallback (#975) and the failure's reported cause (#974).
Security reviews are off on thirdshift from 2026-10-21 22:46 UTC (security.review = false; security.harness is still grok), so later PRs from that host have no Security review outcome.
Activity
JacobStephens2 commentedon Oct 21, 2026
Brief (ready-for-human)
Category: enhancement
Summary: Choose the Harness and Model for Security reviews, from a record of which Harness actually ran each review.
Correction: an earlier version of this brief read today's six failed Security reviews as Codex failures. They ran on Grok Build.
security.harness = "grok"in the User config onthirdshiftsent them there, and Grok Build isn't signed in on that host, so each session ended at once. #974 covers the hidden cause; #975 adds a preflight check and fallback.What the record shows:
security.harnesspicks the review's Harness when it's set. The Run's Command log names it:security-review: on <Harness> · <Model> · <effort>, chosen by ….thirdshiftfailed with "Not signed in", and each failed its Run (The Stages page says a dispatch stage has "Not enough samples", though it never gets a Usual range #924, The Stages and Lanes pages head a Pickup run and the Run it dispatched identically #936, Console sign-in list accepts non-addresses like a@b@c.com #947, show thirdshift cronjobs in thirdshift console web ui #953, ensurethirdshift setupasks for a base fix setting #288, thirdshift setup: check that Security sessions can sandbox target code on this machine #608). Grok Build hasn't completed a Security review yet. The reviews of ten Claude-built PRs completed.Steps:
security.harnessshould name on each host.stephens.pageleaves it blank, so its reviews run on Codex, its default Harness.thirdshift, sign it in there (grok login --device-code) and check that a review completes.Why this can't go to an agent: signing in a Harness needs the host, and choosing a Model means weighing cost and subscriptions, which is your call.
Out of scope: the preflight check and fallback (#975) and the failure's reported cause (#974).
JacobStephens2 commentedon Oct 21, 2026
Two more data points for the Harness choice:
thirdshift: on 2026-10-09, Codex refused one with its cybersecurity safeguard. Muse completed both of its runs on 2026-10-10, each ending with its PR merged (Security fix: validate staged public text before public issue creation #634, Security fix: publish spec-size fixes through staged text #651). No Security run has started since (A waiting Security finding no longer holds off Security audits #1043).thirdshiftfrom 2026-10-21 22:46 UTC (security.review = false;security.harnessis stillgrok), so later PRs from that host have no Security review outcome.