Repository navigation
[Decision] Test Core shard timeout: land 40 as ruled, keep 45, or raise to 50 so the stall guard's cap verdict lands before the job dies #22516
Description
This card carries the value of the Test Core shard timeout-minutes. Its parent, #22075, keeps everything else; its ruled re-size is built as PR #22512 (45 → 40), held in draft until this answer. Filed by domain:devx seat 1 · os-bill · session_01LYXc6ckoWuZyVZpWYizdMh. The platform dates this card.
维护者速读
一小时前你选了 #22075 的 A,其中一项是「按实测把 45 分钟的分片超时下调」。「下调」这两个字是席位写的，席位当时没有核对卡死守卫(stall guard)。dev 实测 396 个分片 job 后发现：超时降到 40,守卫最重要的那条「封顶判定」会有 15% 的情形来不及开口，任务就被超时直接杀掉，只留一个没有说明的超时;保持 45 是 2%;升到 50 是 0%。正常运行的耗时不受这个值影响，它只决定卡死时谁先说话。推荐 C:升到 50。 回复一行即可，例如「C」。
一句话问题。 测试分片卡死时，应该先由守卫报出「卡在哪里」,还是早 5 分钟被超时无声杀掉?
Governing text
- 裁决
6083012192(ci(test-shards): the shard balance is derived on full-run sums while PR and merge_group runs use the affected set — the CLI shard (1/6) measures 34–36 min against 10–20 for the others and sets CI and queue wall time #22075,维护者「A 停在 3 片收口 (Recommended)」):"re-size theTest Coreshardtimeout-minutes… from the measured job-wall distribution after5919483472"。所选选项的说明写的是「按实测分布下调 45 分钟分片超时」。 scripts/measure-stall-guard-headroom.mjs文件头:"The deferred path is the one that matters most and is checked least: … losing that report to an unlabelled job timeout is the exact state the guard exists to abolish.".github/workflows/ci.yml该 job 注释:"the stall guard stays the primary hang detector"(超时只是兜底)。- 是否改协议：不改。三个选项都只改一个数和它的注释。
前提(每条带复核命令)
- 静态预算门禁在 40 下仍然通过(守卫窗口 10 分钟、封顶 20 分钟)。复核:
node scripts/check-stall-guard-budget.mjs --list(在 PR ci(test-core): the Test Core shard wall is 50, so both stall-guard verdicts land before it #22512 heada549ca3a上 exit 0,席位已重跑)。 - 实测值(dev 报告
6085414584,席位未重测):窗口内 396 个Test Core (N/6)job,最慢一个是 27m05s(run 37923333449,3/6);守卫所在步骤结束时最晚已是 job 开始后 26m38s。- 窗口判定(+10 分钟)最晚在 36m38s 落下:40 下来得及,35 下来不及。
- 封顶判定(+20 分钟)最晚在 46m38s 落下,要求超时至少 47 分钟。来不及的比例:45 下 8/396,40 下 60/396,50 下 0/396。
- 复核:
node scripts/measure-stall-guard-headroom.mjs --run 37923333449。
- 正常运行的耗时与这个值无关;它只决定卡死的 job 在第几分钟被杀。复核:读
ci.yml该 job 的timeout-minutes上方注释。
选项
| 选项 | 做什么 | 后果 |
|---|---|---|
| A. 40(按裁决字面，即 PR #22512 现状) | 合并 PR #22512 | 卡死的 job 早 5 分钟结束;封顶判定 15% 的情形失声，变成无说明的超时 |
| B. 保持 45 | PR #22512 改成只更新注释(写入实测窗口和守卫读数),数值不变 | 与今天一样:2% 的情形失声 |
| C. 升到 50 | PR #22512 改成 50,注释写明「由守卫两条判定路径 + 实测墙钟推导」,回退到 30 的条件同步改写 | 守卫两条路径都先于超时开口;卡死的 job 多跑 5 分钟才被兜底杀掉 |
业务含义直译: 超时是火警铃响后的强制断电,守卫是先报「哪里着火」的烟感器。A 是把断电提前 5 分钟，代价是更多时候烟感器还没来得及报;C 是让烟感器总能先报，再断电。
四轴(业务立场)
- ① 项目长远合理性: 两年后的样子：超时只做兜底，数值由守卫的两条判定路径和实测墙钟推导，守卫永远先开口(主流 CI 的超时也是 watchdog 之外的最后一道)。C 是这个终态;A 让兜底抢在守卫前面。
- ② 实际业务拉动: 正常运行不受影响;卡死本身罕见。拉动在于卡死时开发者能否看到「卡在哪」,而不是早 5 分钟拿到一个红叉。
- ③ 防 AI 犯错: 无说明的超时会把流水线 bug 藏起来，下一个 agent 只会重跑;守卫的判定会点名卡住的步骤。C、B 优于 A。
- ④ 创业阶段不扩散: 三个选项都只改一个数，不加门禁、不加零件。
os-decision-facets
① 项目长远合理性:超时是兜底，应由守卫两条判定路径与实测墙钟推导;C 是终态,A 让兜底抢在守卫之前。
② 实际业务拉动:正常运行耗时不变;拉动是卡死时能否看到卡在哪一步。
③ 防 AI 犯错:守卫点名卡住的步骤，无说明超时只会诱导重跑;C、B 优于 A。
④ 创业阶段不扩散:三者都只改一个数，零新增。
Prior rulings read: ruling 6083012192 (#22075, A: the re-size, worded 下调 by the seat's option text); scripts/check-stall-guard-budget.mjs and measure-stall-guard-headroom.mjs headers; ADR none; thread: 0 rulings on the value itself
推荐 C,回退 B。 自检：只看①选 C;②③④ 是否翻转：否。
置信缺口: 窗口只有约 5.6 小时;封顶判定只在缓冲层把活着的测试藏起来时才会用到(守卫自己的说法是这本身是一个流水线 bug),多罕见没有读数。同样的封顶缺口在 dogfood 与 temporal-conformance 两步上也存在(需 33 与 31 分钟，现为 30),本卡不处理。
裁后执行
- A: 席位把 PR ci(test-core): the Test Core shard wall is 50, so both stall-guard verdicts land before it #22512 转 ready 并入队,ci(test-shards): the shard balance is derived on full-run sums while PR and merge_group runs use the affected set — the CLI shard (1/6) measures 34–36 min against 10–20 for the others and sets CI and queue wall time #22075 随之关闭。
- B / C: 席位给原 dev 一轮补丁(同分支、同 PR),改数值和注释;C 同时改写回退到 30 的条件，使之也要求守卫覆盖。复核后入队,ci(test-shards): the shard balance is derived on full-run sums while PR and merge_group runs use the affected set — the CLI shard (1/6) measures 34–36 min against 10–20 for the others and sets CI and queue wall time #22075 随之关闭。
Activity
objectstack-fleet commentedon Oct 10, 2026
Ruling: letter C (raise the
Test Coreshardtimeout-minutesto 50) · maintainer, answering indomain:devxseat 1's session chat · 2026-10-10T00:13Zdomain:devxseat 1 ·os-bill·session_01LYXc6ckoWuZyVZpWYizdMh. Presented as this card's body. The maintainer's answer, verbatim (the option label chosen in the session's question prompt): 「C 升到 50 (Recommended)」. Thread-read: none (no comment on this card before this record). Freshness: the body is unchanged since filing.The ruling
Test Coreshard job'stimeout-minutesbecomes 50. It is derived from the stall guard's two kill paths plus the measured job wall, so both guard verdicts land before the job timeout on every observed run. Reading: the worstp + sis 26m38s (run 37923333449, 3/6). The cap verdict then lands at 46m38s, inside 50 with 3m22s to spare, and the window verdict lands at 36m38s.6083012192's option A on ci(test-shards): the shard balance is derived on full-run sums while PR and merge_group runs use the affected set — the CLI shard (1/6) measures 34–36 min against 10–20 for the others and sets CI and queue wall time #22075, which the seat wrote before the guard was measured. The rest of that ruling stands: 3 slices, Done-when clause 2 withdrawn, close on the timeout re-size.Execution
claude/issue-22075-test-core-timeout-resize): the value 40 → 50, and the rationale comment rewritten to derive the value from both guard paths plus the wall. The revert condition is rewritten so that any lower value must keep both guard paths covered on the measured window, as well as the 80% wall margin. The seat rewrites the PR body after the push.pm:blocked→pm:dispatchedin the same pass. This card closescompletedin this act.dogfood(needs 33, has 30) andtemporal-conformance(needs 31, has 30). These are noted, not filed.