ci(test-leftovers): report what each test job leaves in the tree; retire Rerun Safety

[objectstack-fleet]

Fixes #23880
Clause-②: no

维护者速读

改了什么

• 新增(只报告,永不让任务失败): CI 里每个跑包测试的任务,即 Test Core 的 6 个分片,以及 main 全量跑里的 Dogfood Regression Gate 3 个分片,在构建之后、测试之前给工作树拍一次快照(包括被 .gitignore 忽略的文件);测试结束后列出这次测试新增的每一个路径,按包目录分组,写成任务摘要里的一张表,每个包一条警告注解。脚本在任何情况下都以 0 退出,步骤上也没有 continue-on-error:它不是门禁,不会让任何任务变红。
• 退役: Rerun Safety 夜间双跑(在同一个工作树里把整个测试套件跑两遍)整体删除:工作流文件、判定脚本、package.json 别名、lint 里的自测步骤、红灯卡片的登记(CARD_CADENCES 条目与 scheduled-full-run-card.yml 的监听名单),以及其它文件里对它的引用。

为什么改

裁决 乙(本卡评论 6235054403)。维护者先指出:「这个问题不是应该派卡解决吗？但测试运行时常会往工作目录里写东西：临时数据库、.objectstack/data/*.json、缓存、生成文件。写完不清理，下一次在同一个目录里跑，就会带着上次的残留开始。」席位列出甲、乙、丙三个选项后,维护者回复「同意 乙」。

双跑在它被读过的三十个夜里没有抓到一次残留污染,真实的失败全是负载下的超时;它测的是"后果"(第二遍变红),从不点名文件。而 .objectstack 在 .gitignore 里,git status 根本看不见这一类残留。新报告直接点名"哪个包留下了哪个文件",每个 PR 都跑。

风险与代价(含回滚)

• 看不见的: 写到仓库目录之外的状态,例如 $HOME、系统临时目录(/tmp)、数据库服务器。快照前已经存在、又被测试改写的文件,以及被测试改动的已跟踪文件(这一类 git status 本来就能看到),也不在报告里。
• 失去的: 双跑能看到的"跨包污染的后果",即一个包的测试弄脏了另一个包读到的状态、第二遍才变红。新报告看原因(谁留下了文件),不看后果。
• 报告的盲区: 从 turbo 缓存回放的测试任务并没有真正运行,也就不会留下文件。main 每小时全量跑里多数测试任务是回放,所以 CI 上的报告只覆盖真正执行了的包。下文的本地读数是强制全部执行后得到的。
• 成本: 每个测试分片多两步,本地实测快照约 3 秒、报告约 2 秒。
• 回滚: revert 这一个 PR,夜间双跑、它的判定脚本和红灯卡片全部恢复。

席位意见(domain:devx 2 号席位,2026-10-17T19:24Z 定稿)

• 同意合并。 席位逐项核对过:
  • 报告脚本在任何路径都以 0 退出,包括崩溃和未知参数;两个步骤都没有 continue-on-error。
  • 快照放在构建之后、测试之前,报告带 if: always()。
  • 退役后 grep 只剩 4 处过去式的历史说明,没有任何仍在生效的引用。
  • CI 卡片和 Lint 卡片的标题、标记逐字节不变。
• 开发提出的问题: temporal-conformance 不接报告,席位答 A。它只在全量跑里出现,测的套件 Test Core 已经覆盖。
• 报告的盲区要知道: turbo 从缓存回放的测试任务没有真正执行,所以 main 上的报告只覆盖改动过的包。完整读数以下文的本地强制全量为准。
• 今天唯一一处残留: examples/app-showcase/.objectstack/data/showcase_external.db,来自 dogfood 测试。落地后按裁决交给下一任席位开卡。

你要做的

• 这是 Tier H PR(改动了 .github/workflows/**),需要 os-zhuang 或 hotlong 审批。
• 落地后,席位读 main 上第一次 CI 的报告,按包在各自的 lane 开卡(本地读数见下)。
• 报告是否升级为阻断门禁,等 main 报告干净后由你决定;本 PR 不做这件事。

---

What the report says today: the local full-suite listing

On 015e6eab7 (base cb8756b00; the later merge of origin/main brought 13 commits, none touching a file in this diff). Commands: a full build (pnpm turbo run build --filter=!@objectstack/docs --concurrency=2, 76 of 76 tasks), then report-test-leftovers.mjs snapshot (88275 paths), then every test and test:repo task forced to execute (pnpm turbo run test test:repo --only --force --continue --concurrency=2, under os-verify-lock, 2h48m), then report.

package directory	paths added	path	who wrote it
examples/app-showcase	1	examples/app-showcase/.objectstack/data/showcase_external.db	mtime 16:54:26Z, while @objectstack/dogfood#test was the only task running. The showcase's showcase_external datasource and external-fixture.ts resolve .objectstack/data/showcase_external.db against the process cwd.

• Not counted: 82 turbo task logs and 76 vitest results caches.
• Rewritten instead of added: 0. Every one of the 88275 snapshot paths was stat'ed after the run, and none had an mtime later than the snapshot, so nothing was hiding as a rewrite.
• Measurement limits:
  • @objectstack/organizations#test was killed by the container's memory cgroup at 16:41 (Memory cgroup out of memory: Killed process 26265 (node), exit 137), so its row is partial.
  • @objectstack/cli#test ran to the end with 3 failed files out of 464 (data-commands.absent-database.integration, hook-timeout-override-refusal, boot-preparation-parity), under a load average of about 8 to 10 on 4 cores. This diff touches the CLI by one comment line in another file, and the two unit files pass when re-run alone at 076add1f6 (2 passed, 41 passed); the integration file was not re-run.
  • 80 of 82 tasks passed.
• In CI, the report groups by where a path lands, not by which test wrote it. In a dogfood shard the writer is the dogfood suite even though the row reads examples/app-showcase.

What this PR does

Adds a report-only leftover list to every job in ci.yml that runs package tests, and retires Rerun Safety completely, per ruling 乙 (comment 6235054403). No product code changes, and no test behaviour changes.

scripts/ci/report-test-leftovers.mjs (new)

• snapshot --out FILE lists every untracked path in the tree, ignored paths included, into a file under $RUNNER_TEMP (outside the tree; a path inside the tree is refused, since the report would count it).
• report --before FILE --label TEXT lists the tree again and prints every path the test run ADDED, grouped by the nearest workspace package directory ((root) for anything outside one): a table in $GITHUB_STEP_SUMMARY, the paths in the log (50 per package, then a count), and one ::warning:: annotation per package with its count and up to 5 example paths. GitHub shows 10 warnings per step, so past 10 packages the last annotation is a roll-up naming the rest; the summary always has every row.
• The listing is git ls-files --others -z with no exclude flag. Without --exclude-standard git applies no ignore rules, so ignored files are listed one by one. --exclude-standard is blind to .objectstack/; --ignored --directory collapses an ignored directory to one entry, hiding a new file inside an existing .objectstack/data/; git status --porcelain --ignored --untracked-files=all lists the same paths but also refreshes the index and diffs tracked files. Measured on this worktree: 85066 paths in 0.4 s after install, 88275 after a full build.
• It never fails a job. snapshot and report exit 0 on findings, on bad arguments, on a missing snapshot and on a crash; a failure to measure is one NOT MEASURED warning. The ci.yml steps carry no continue-on-error. The one failure the script cannot catch is its own source not loading; its --self-test loads it on every pull request in Lint & Repo Gates, so that breakage is red there first.
• Not counted, a closed list, each counted in the summary: turbo-state (.turbo/ at the root: turbo's cache and the --summarize run summaries Test Core reads back and uploads); turbo-task-log (PKG/.turbo/turbo-TASK.log); vitest-results-cache (PKG/node_modules/.vite/vitest/, vitest 4.1.11's per-file duration and pass/fail record, read only to order the next run); own-build-output (the declared outputs of a task a test task depends on without ^, derived from turbo.json on every run: today @objectstack/cli#build, @objectstack/metadata#build and @objectstack/spec#gen:migration-registry).

ci.yml

• Test Core (k/6): Snapshot the working tree before this shard's tests sits after both dependency-closure builds and before Run this shard's tests; Report what this shard's tests left in the working tree sits directly after the test step with if: always(), above the attestation pair.
• Dogfood Regression Gate (k/3) (full battery only): the same pair around Boot example apps and exercise real user flows.
• Not wired, by name: dogfood-verify runs the objectstack verify CLI, not a package test suite; build-core, build-docs and console-pin run no package tests; temporal-conformance is a fork, below.

Retirement

• Deleted .github/workflows/rerun-safety-nightly.yml and scripts/rerun-safety-verdict.mjs; removed the check:rerun-safety-verdict script from package.json and its Rerun-safety verdict table self-test step from lint.yml.
• scripts/ci/scheduled-full-run.mjs: removed the Rerun Safety entry of CARD_CADENCES, the cardIdentity('Rerun Safety') setup line and the three self-test cases that pinned it (its title/marker, its nightly body words, and the nightly-vs-hourly non-match). The CI and Lint & Type Check cards are byte-identical (proof below).
• scheduled-full-run-card.yml: Rerun Safety removed from workflow_run.workflows; the comment keeps the rule that every watched name needs a CARD_CADENCES entry.
• scripts/check-stall-guard-budget.mjs: the population paragraph no longer names rerun-safety; the census and self-test find sites from the workflow files, so no self-test case pinned it.
• Re-pointed schedule comments in board-snapshot.yml, docs-deploy.yml, prerelease-pin-watch.yml and test-nightly-tiers.yml; past tense in scripts/docs-audit/README.md and in the comment of packages/cli/test/published-entry-node-env-source-reroute.test.ts (comment only).
• lint.yml gains Test-leftover report self-test in Lint · CI, release & engine gates, beside the other ci-script self-tests, which is what check:self-test-wired requires for a script CI runs.

Retirement grep at 076add1f6, git grep -n -i "rerun.safety\|rerun-safety": 4 hits, 0 live. All four are history sentences in past tense, kept on purpose:

hit	why it stays
packages/cli/test/published-entry-node-env-source-reroute.test.ts:112	where a timeout measurement was taken ("a since-retired nightly that ran the whole suite twice")
scripts/check-stall-guard-budget.mjs:193	the dated census "Measured when this landed"; dropping one term would make its totals false
scripts/ci/report-test-leftovers.mjs:20	the new header naming what it replaces ("retired by the same ruling")
scripts/docs-audit/README.md:902	the 2026-08-18 list of scheduled workflows, now "on that date" and "(retired since)"

#24158 is not addressed by this diff; the seat retires it at landing, as the ruling says.

The premise, measured per job (ruling item 5)

Test Core (k/6) separates test-created paths from build outputs:

• Before the snapshot: Build this shard's dependency closure (--filter=PKG^... per package) and Build the sliced package's dependency closure (--filter=PKG per slice).
• After it, the test step schedules test test:repo with their dependsOn. The ^build closure is already on disk, so a replay rewrites existing paths and adds none. Three tasks without ^ can run after the snapshot: @objectstack/cli#build, @objectstack/metadata#build and @objectstack/spec#gen:migration-registry, whose output exists from pnpm install. Their declared outputs are the own-build-output rule, derived from turbo.json.
• Turbo cache restore happens before the snapshot, and test tasks declare outputs: [], so a replayed test restores only its log.
• --summarize writes .turbo/runs/, which the drift check, the timings capture and the artifact upload read: turbo-state.
• Coverage: no vitest.config.* enables it (git grep -n "enabled: true" -- '*vitest.config.*' gives 0 hits); coverage runs in coverage-nightly.yml.
• vitest writes its results cache under PKG/node_modules/.vite/vitest/: vitest-results-cache.

Dogfood Regression Gate (k/3): Build the dogfood package's dependency closure runs before the snapshot, and @objectstack/dogfood#test depends on ^build alone, so nothing is built after it. There is no --summarize.

Measured: the own builds. On 015e6eab7, after deleting packages/cli/dist and packages/metadata/dist: snapshot, then pnpm turbo run build --filter=@objectstack/cli --filter=@objectstack/metadata --only --force, then report. 688 paths were added, all 688 under own-build-output, 0 counted. The build was forced, so it wrote everything it writes, not only what a replay restores.

Not wired, reported as a fork: temporal-conformance (full battery only). It runs package tests, but it interleaves four builds with four test runs: build driver-sql... then test it; build the non-SQL backends then test them; build metadata-protocol... then test; build runtime... then test. One snapshot after the first build would charge the next three builds' dist/ to the tests. The ruling names the per-PR path and lists Test Core and the dogfood jobs, so this PR leaves it unwired and names it here instead of shipping a partial check. The options are in the report's open question.

Proof

• Self-test: node scripts/ci/report-test-leftovers.mjs --self-test at 076add1f6 prints report-test-leftovers --self-test: OK (47 cases over 6 batteries), exit 0. Batteries: the diff (the four dispatch cases: a new ignored .objectstack/data/x.json under its package, build outputs in both snapshots not reported, a deleted path not reported, root paths as (root)); the fixed exclusions; the turbo.json derivation; rendering; a real throwaway git tree; and every in-job failure path exiting 0.
• Ablation on blob 30c07df010bc with scripts/ablation-replace.mjs: the listing gained --exclude-standard, so it missed ignored paths (anchor 1 to 0, blob 30c07df010bc to 873a382177b1). Result: report-test-leftovers --self-test: 5 FAILED (of 47 cases), including "a NEW ignored file inside an ignored directory that already existed is listed". Restored: blob equals HEAD and git diff HEAD is empty. The direction was red, as expected.
• scheduled-full-run: --self-test prints scheduled-full-run: self-test OK and --check-concurrency passes for both workflows. A one-off byte comparison of the module at cb8756b00 against the new one, for CI and Lint & Type Check (workflowName, cadence, title, marker, labels, three body shapes), gave byte-identical: 16 of 16, and the new module refuses cardIdentity('Rerun Safety') by name.
• check:stall-guard-budget: OK (40 workflow file(s), 75 job(s), 832 step(s), 9 guard-wrapped step(s); ...), and its --self-test passes with 61 assertions (62 before; the real-repository battery lost one job group and sits at its floor of 13).
• YAML: every edited workflow parses with yaml (0 error(s), 0 warning(s)). In ci.yml the pair sits at steps 10 and 12 of 22 in test and 8 and 10 of 16 in dogfood; the report step has if: always() and no continue-on-error, and the last two steps are still the attestation pair.

Gates

• node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack at 076add1f6 (15 paths against merge base d9a749bae) derived 105 commands. Each ran with its exit code captured before any pipe, and all 105 exited 0. They include check:stall-guard-budget, check:self-test-wired, check:self-test-workflow-commands, check:shard-attestation, check:workflow-status-functions, check:workflow-step-name-quoting, check:declared-population-live, check:pm-dispatch-gates (411 s) and the new self-test. --ran with exit codes printed Run reconciliation — 105 derived, 105 run, 0 NOT-MEASURED, 0 UNRUN.
• An earlier pass at 95219d01e had one red. check:declared-population-live read the snapshot schema id test-leftovers-snapshot/1 as a path declaration. 015e6eab7 fixes it with the no-path-population marker (the self-test reads no tracked file) and the id test-leftovers-snapshot.v1.
• Edited gate scripts, their own suites: check:stall-guard-budget runs its --self-test first, and scheduled-full-run.mjs --self-test is in the derived list. No *.test.* file names either script.
• node scripts/pm/check-governed-merges.mjs --branch claude/issue-23880-test-leftover-report: GOVERNED, landing tier H (.github/workflows/** ×8), exit 3. Size: 2333 changed lines (+1008 / -1325) over 15 files.
• Changeset: none. Scripts, workflows and one test comment publish nothing, so the seat applies skip-changeset.
• CI check-runs on this PR were not waited on; CI convergence is the seat's.

NOT MEASURED

• The report's real output in CI. Read it on the first main CI run after landing: Test Core on the hourly full run, and Dogfood on the full battery. This PR's own Test Core shards run the new steps for the affected set, so their summaries are a first reading.
• On the hourly run most test tasks replay from the turbo cache and add nothing, so a CI reading covers executed packages only. The forced local listing above is the complete one.

Acceptance notes

• CADENCE_WORDS in scheduled-full-run.mjs keeps its nightly vocabulary, with no watched workflow mapped to it after this PR. The dispatch scoped the removal to the CARD_CADENCES entry and the self-test cases pinning it. A later nightly card can reuse it, or a later edit can drop it. Carrier: none.
• The scheduled-full-run identity battery keeps its floor of 34 with 36 cases left. The stall-guard real-repository battery is now exactly at its floor.
• test-nightly-tiers.yml and coverage-nightly.yml also run package tests, outside ci.yml. The ruling scopes the report to ci.yml.
• GitHub shows 10 warning annotations per step, so past 10 packages the report uses a roll-up annotation.

---

Generated by Claude Code

[github-actions]

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see

• a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
• a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 8259ea03e2959129c688639f69d0782c3a820f89 → packageMentionDocs.

[objectstack-fleet]

席位核对 · ACCEPT, Tier H · PR #24211 (draft, head 076add1f6, Fixes #23880) · domain:devx seat 2 · session_016PVjjuVPte5XEJvaQx1E4t · 2026-10-17T19:26Z

Checked on GitHub and against the sources, not on the report's say-so. ⛔ The seat does not approve or merge a governed PR. This PR waits for an authorised approval from os-zhuang or hotlong, requested in this act.

速读

• 改了什么:
  • 每个 PR 都跑的 Test Core 6 个分片,以及全量跑的 Dogfood Regression Gate 3 个分片,在测试前后各给工作树拍一次快照,把测试新留下的文件按包列进任务摘要。
  • 只报告,永不失败。
  • 夜间 Rerun Safety 双跑整体退役。
• 为什么: 裁决乙 6235054403,维护者原话「同意 乙」。双跑三十个夜里没抓到过一起残留污染,而且从不点名文件;.objectstack 被 .gitignore 忽略,git status 看不见。
• 看不见的:
  • 写到仓库目录之外的状态,比如 $HOME、/tmp。
  • 跨包污染的后果。
  • 从 turbo 缓存回放、没有真正执行的测试任务。
• 成本: 每个分片多两步,约 5 秒。
• 回滚: revert 这一个 PR。

Read by this seat

• Shape:
  • 15 files, +1008 / −1325. They match the claim's surface 6235066107.
  • check-governed-merges --pr 24211: GOVERNED, 8 of 15 paths on the register, landing tier H.
  • The body opens Fixes #23880, then Clause-②: no.
  • The seat applied skip-changeset and needs-user-decision in this act. No package publishes anything from this diff.
• Never fails a job:
  • main() returns 0 for snapshot, report and an unknown mode.
  • inJob catches every throw and hooks uncaughtException / unhandledRejection to process.exit(0).
  • Neither step uses continue-on-error. The report step carries if: always().
• Placement: the snapshot sits after the closure builds and the report right after the test step, in both Test Core (k/6) and Dogfood Regression Gate (k/3). The attestation pair stays last.
• Retirement: git grep -i "rerun.safety|rerun-safety" on the head leaves 4 past-tense history lines, at published-entry-node-env-source-reroute.test.ts:112, check-stall-guard-budget.mjs:193, report-test-leftovers.mjs:20 and docs-audit/README.md:902. There are 0 live references.
• Proof in the report:
  • the self-test passes 47 cases;
  • the ablation, which drops ignored paths, turns 5 cases red, and the restore was confirmed;
  • the own-build premise holds: 688 added build outputs, 0 counted;
  • the CI and Lint & Type Check cards are byte-identical, 16 of 16;
  • 105 derived gates exit 0, and --ran shows 0 NOT-MEASURED.
• Today's listing: the local forced full suite (82 tasks) found one leftover, examples/app-showcase/.objectstack/data/showcase_external.db from the dogfood suite. It is handed to the next devx seat for its card, per ruling 乙 item 2.
• Open question, answered by the seat: temporal-conformance stays unwired (A). It runs only in the full battery, and its suites already run under Test Core. Option C's widened exclusion would hide the very class this report exists for.
• CI on this head: 27 success, 5 skipped and 9 in progress. Check Changeset was red before the label.

Landing: after an authorised approval and green CI, the seat readies the PR, arms auto-merge, confirms the merge by blob identity, closes #24158 as not planned, and posts the landing record on #23880.

---

Generated by Claude Code