Skip to content

ci(test-shards): the density cap carries the declaration closure's own displacement when a split carries the reader slice - #29838

Merged
objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-29777-partitioner-slice-spread
Nov 3, 2026
Merged

objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-29777-partitioner-slice-spread

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Requested by 田騏 · project thread
Fixes #29777
Clause-②: no
Metadata-protocol: none

维护者速读

  • 改了什么:Test Core 分片器判断「这次要不要把 CLI 切片」时,若切出的方案带着声明构建那一片,密度上限加上闭包计费分摊到其余分片的那一份(783 秒 ÷ 7 ≈ 112 秒);不带那一片时上限不变。
  • 为什么改:每周分片计时刷新被分片器自检拒收。新数据里 CLI 降到 1432 秒、低于平均,原上限等于平均值,而闭包计费必然把其余分片推高约 7%,于是任何切法都过不了,全量运行只能把 CLI 整包串行跑(约 24 分钟,其余分片约 8 分钟)。
  • 风险与回滚:只动一个仓根脚本,不发布任何包;今天的已提交数据切分结果逐字节不变。回滚即 revert 本 PR,影响仅是刷新 PR 再次被拒。
  • 先看:scripts/partition-test-shards.mjs 的 closureAllowance() 与 planShards() 里 WITHIN CAP 那几行;再看自检新增的四个「declaration-closure allowance」用例。
  • 两分钟验证:node scripts/partition-test-shards.mjs --self-test,应见 self-test OK,且 bins 仍为 1600.11/1597.48/1597.14/1599.09/1598.01/1597.78/1599.48/1597.04s。
  • 需要你做的:无,席位落地。

What changes

scripts/partition-test-shards.mjs only, implementing maintainer ruling record 6596963071 (option B, the first of its two slices):

  • closureAllowance(items, shardCount) (new): DECLARATION_CLOSURE_CHARGE_SECONDS / (shardCount - 1) when the items carry the declaration-reader slice's closure charge, 0 otherwise and on a one-shard run. Its docblock carries the measured reason. The charge (783 s of packing weight) moves that much whole-package work onto the other shards. So when the whole-package split is balanced, the density cap (its densest bin) is the mean, and no split carrying the charge can come back within it at any slice count. On the 2026-11-02 refresh the cap read 1455.81 s against 1552.68 / 1552.77 / 1552.63 s for the CLI cut 2 / 3 / 4 ways, so the full list ran the CLI whole.
  • planShards(): the WITHIN CAP check, and the repair placement behind it (placeItems / refineToCap), judge a candidate split against cap + closureAllowance(candidate). The final placement uses the same. Each decision records its allowance.
  • renderSliceDecision(): names the allowance only when it was part of the verdict (a refusal past it, or a slicing only it admitted), so the shard log never claims a cap the split did not meet. Today's lines are unchanged.
  • Self-test: the hard constraint (the committed full list stays within the cap) is scored against the same adjusted cap. The success line names the allowance only when the bins exceed the bare cap. Unchanged, as ruled: the pin "cap = the whole split's densest bin" and the capFixture refusal/admission controls.
  • New pins (4), in the density cap and the per-run slice decision battery, whose floor is raised 14 to 18. They are built on the refresh's package weights frozen in the test (79 packages from PR 28978 head b5b4b790, not read from the dataset file), at 8 shards:
    • the allowance's arithmetic: carried at 8, plain at 8, carried at 1;
    • carried: the CLI x2 is taken, cap 1455.81 s + allowance 111.86 s = 1567.67 s, densest shard 1553.36 s (past the bare cap, within the adjusted one), slices spread, and the log names the allowance;
    • not carried: the same shape with no reader slice is allowed 0 s;
    • still binding: a cap one second short of admitting the carried split refuses it, naming the allowance.

Not changed, as ruled: scripts/test-shard-timings.json, FILE_SHARDED_PACKAGES / PREVIOUS_FILE_SHARDED_PACKAGES, ci.yml's DTS_READER_SLICE and the CLI vitest config's DECLARATION_READER_SLICE. The dataset and map swap (CLI 4 to 2, previous = 4) is the ruling's second slice, a separate lane PR that replaces PR 28978.

Tests

Measured on objectstack-ai/objectstack at head e9569e920e (merge base b0ef25f524).

  • node scripts/partition-test-shards.mjs --self-test, committed dataset: exit 0, and the verdict line is byte-identical to the base's.
  • Byte-identity on today's committed dataset (base vs head). I ran main() for every shard 1/8..8/8 on a full-list document with --exclude @objectstack/dogfood: stdout and stderr are byte-identical on all 8. In the plan JSON (planShards on the full list, as the self-test builds it), items, bins (1600.11/1597.48/1597.14/1599.09/1598.01/1597.78/1599.48/1597.04) and cap (2697.45) are identical. The one difference is the added decision key "allowance": 111.85714285714286.
  • PR 28978's refreshed dataset (b5b4b790), scratch tree, not committed. At head the self-test is red on one check only: pin 3c, "sliced 4 ways ... Lower it to 3". That is the map swap the second slice carries. At base it was red on 7, the first being slice spread. With the second slice's map applied in scratch (CLI 2, previous 4): exit 0, bins 1551.96/1551.44/1553.29/1552.46/1552.43/1553.22/1551.01/1553.36s within the 1455.81s density cap plus its 111.86s declaration-closure allowance, whole legs within 1868 s, CLI x2.
  • Ablation (scratch tree, closureAllowance mutated to return 0, marker grep-confirmed): exit 1. With failures collected, the three cases that depend on the allowance go red and the "not carried" case stays green. Restored by byte comparison; the restored self-test exits 0.
  • Importers' self-tests: nightly-tiers, check-test-suite-ceilings, measure-test-shard-timings, ci/select-shard-timings-run, ci-shard-matrix, check-test-completeness all exit 0.

Acceptance notes

  • The adjusted cap leaves a 14 s margin on the refreshed shape (1553.36 s against 1567.67 s).
  • Pin 3c has no hysteresis: x2 holds on the refreshed dataset by 20 s (716 s against 736.45 s), so a later refresh can flip the derived count again. The ruling places that outside its scope, as its own finding.
  • No changeset: a root script publishes nothing, so skip-changeset.

Verification

Derived gates: 31 of 31 exit 0, reconciled with --ran

node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack at e9569e920e derived 31 commands (17 pnpm, 14 node). All 31 were run with exit codes captured before any pipe, after pnpm install --frozen-lockfile. A first pass without node_modules gave PREREQUISITE NOT MET (exit 3) on 6 commands and 1 environment finding; all reran green after the install. pnpm check:pm-dispatch-gates exit 0 (fast tier, 1945 cases). --ran: "31 derived famil(ies) accounted for, 31 run, 0 NOT-MEASURED (a DERIVED zero, all 31 recorded an exit code and none of them is 3)". The derivation names one path-scheduled CI step (Plan Test Core shards) and the type-check lanes as NOT MEASURED locally; CI runs those.

Session: https://claude.ai/code/session_017ihL2gLghEqeDKTbDTC7Zd

🤖 Generated with Claude Code

…n displacement when the split carries the reader slice

The reader slice's closure charge (783 s of packing weight) moves that much
whole-package work onto the other shards. When the whole-package split is
balanced the density cap is its mean, so no split carrying the charge could
ever come back within it, at any slice count: on the 2026-11-02 refresh the
cap read 1455.81 s and the CLI cut 2/3/4 ways reached 1552.68/1552.77/1552.63 s,
so the full list ran the CLI whole as one 1432 s serial task.

A candidate split that carries the reader slice is now judged, and repaired,
against the cap plus DECLARATION_CLOSURE_CHARGE_SECONDS / (shardCount - 1);
a split without it keeps the bare cap. The self-test's hard constraint is
scored against the same adjusted cap; the cap-definition pin and the
capFixture controls are unchanged. Four new cases pin the allowance on the
refresh's frozen package weights: x2 is admitted within 1567.67 s, a split
with no reader slice is allowed nothing, and a split past the cap plus its
allowance is refused by name.

Today's committed dataset splits byte-identically: every shard's stdout and
stderr, the items, the bins and the cap; the slice decision record gains one
`allowance` key.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ihL2gLghEqeDKTbDTC7Zd
…en the bins exceed the bare cap

The success line said "within the <cap> density cap" from the bare cap alone;
once a dataset slices only through the allowance (bins 1553 s against a
1455.81 s cap on the 2026-11-02 refresh) that sentence would be false. It
now names the allowance exactly when the bare cap did not decide, so today's
line is byte-identical.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ihL2gLghEqeDKTbDTC7Zd
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

domain:devx#2 · marchtian · session_017ihL2gLghEqeDKTbDTC7Zd

席位核对 — ACCEPT · domain:devx#2 · head e9569e920e · 2026-11-03T06:56Z

Checked against GitHub and the PR head, not against the report (os-dev-report 6597424525 on #29777).

  • Shape: a draft to main with Fixes #29777 once, Clause-②: no and Metadata-protocol: none. No assignees, no reviewers, skip-changeset (a root script publishes nothing). +168/−15 in scripts/partition-test-shards.mjs only.
  • Path surface: check-governed-merges --pr 29838 reads NOT governed (183 changed lines).
  • The ruling (6596963071), read in the diff:
    • closureAllowance(items, shardCount) returns DECLARATION_CLOSURE_CHARGE_SECONDS / (shardCount − 1) only when an item carries the closure, and 0 otherwise or on one shard. planShards judges WITHIN-CAP, placement and the final placement against cap + allowance, and each decision records its allowance.
    • The self-test's hard constraint on the committed full list is scored against the same adjusted cap. The pin "cap = the whole split's densest shard" and capFixture are untouched.
    • Not touched, as ruled: the dataset, FILE_SHARDED_PACKAGES, ci.yml's DTS_READER_SLICE, and the CLI vitest DECLARATION_READER_SLICE.
  • Byte-identity today: the seat ran --self-test at this head itself. It exits 0, with bins 1600.11/1597.48/1597.14/1599.09/1598.01/1597.78/1599.48/1597.04 s, cap 2697.45 s and CLI x4, which is today's output. The dev's per-shard stdout/stderr diff, base b0ef25f524 against head, is identical on all 8 shards.
  • Four new pins use b5b4b79's weights frozen inside the test: the allowance's arithmetic; x2 taken against 1567.67 s with the densest shard at 1553.36 s; no allowance without the slice; and the cap still refusing by name one second short. The ablation (allowance forced to 0) turns the three dependent pins red. The battery floor moves 14→18.
  • Accepted deviation: the success line names the allowance only when the bins exceed the bare cap, so today's line is unchanged and the next one does not misstate the bound.
  • Gates (the dev's readings): dispatch-gates --commands 31 derived, 31 run, all exit 0; --ran 0 NOT-MEASURED. The seat lands once every check on this head is green.
  • What Fixes closes: this card is the partitioner refusing the refresh. After this PR the refreshed dataset fails only pin 3c, which is the map change the refresh itself must carry. That is the lane's own refresh PR, in the ci(test-shards): refresh the Test Core dataset and cut the CLI into 4 slices on plain-sum placement, and pin the whole-package leg #26186 shape, replacing chore(ci): refresh the Test Core shard-timings dataset #28978. The seat tracks it on chore(ci): refresh the Test Core shard-timings dataset #28978. Pin 3c's 20 s margin is finding ci(test-shards): the slice-count pin has no hysteresis, so a 20 s drift flips the CLI map and turns the weekly refresh red again #29826.
  • Out of scope, not carded: none beyond ci(test-shards): the slice-count pin has no hysteresis, so a 20 s drift flips the CLI map and turns the weekly refresh red again #29826. Skills-docs: none — CI-internal script.
  • Landing: non-governed. The seat marks the PR ready and enables auto-merge; the queue waits for green.

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review November 3, 2026 06:57
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge November 3, 2026 06:57
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit 3cdb162 Nov 3, 2026
47 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-29777-partitioner-slice-spread branch November 3, 2026 07:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

1 participant