diff --git a/docs/architecture/rfcs/monorepo-distribution-split-v0.md b/docs/architecture/rfcs/monorepo-distribution-split-v0.md index 3e030cbc4f..e7dcd45329 100644 --- a/docs/architecture/rfcs/monorepo-distribution-split-v0.md +++ b/docs/architecture/rfcs/monorepo-distribution-split-v0.md @@ -1450,6 +1450,175 @@ comparison before adopting a performance change. This checkpoint creates no new owner, budget, preset or default. R0's complete census, R1/minimal, D2/D5/D6, I3/D3, hard-lease/context/model and split-install acceptance remain open; independent exact-head review and required validation govern adoption. +### 2026-11-07 — Governed CLI composition cost and archived closeout boundary + +The complete small-state CLI sequence is compared on main +`8e86f173f08bd55eb833d8cdf42360f93bf989fe` and a private experimental overlay of +#8240 at `4df990c9e66465171f980176fba4e3602d41f826` plus #8243 at +`668bed2f03a274f273d39c187121a1a4ede67165`. Only their seven production Python +paths enter the candidate. The independently installed monolithic wheels have +verified source bytes for 1,836 runtime resources each, 1,998 total files and +approximately 8.91 MiB compressed. All environments use CPython 3.13.5, +Node 26.0.0, real Node-owned SQLite 3.53.1 and the same 57 dependency versions +on macOS arm64. This overlay is an author experiment; it changes neither the +installed host nor a shipped preset, authority provider or public runtime. + +Inputs, code/resource/wheel hashes, workload, cache policy, instrumentation, +serial pair order, sampling and interpretation are frozen before sampling. +Each CLI invocation has fresh Python, an empty owned bytecode-cache prefix and +bytecode writes disabled. Each episode starts its own cold Node service at +prequota and reuses it for subsequent phases; compile-cache and temporary roots +are independent. Disk cache and concurrent host load remain uncontrolled. +Pairs alternate baseline/candidate order. The 1,500-second cap includes fixture +setup, execution, teardown and sampling orchestration. Pilot, negative-case, +packaging and instrumented-census costs are recorded separately. + +Each fixture uses ordinary hard-lease claim/admission, a separate Git worktree, +complete contiguous native initial reads, an independently verified JSON sum +artifact (`45`), Todo completion, fresh evidence-linked Vision/writeback, one +spend, terminal result/readback and exact completion/writeback/spend/terminal +retries. Global synchronization and external sinks are explicitly suppressed; +standard fanout is outside this experiment. The small state has one active +Todo; large state adds 100 completed background Todos with substantial text. +Large-state failures remain failures and do not become accepted cost samples. + +All 60 planned observations finish in **1,164.996 seconds**, within the unchanged +cap: 52 small-state episodes complete and eight large-state episodes retain the +same terminal failure. Code, script and freeze inventories remain unchanged. +Small-state groups have 8/6/6/6 serial interleaved pairs; large groups have one +pair each and qualify only the observed compatibility failure. + +| Small-state path | Pairs | Baseline seconds, median / p95 | Candidate seconds, median / p95 | Median paired difference, seconds | Exploratory 95% percentile interval | +| --- | --- | --- | --- | --- | --- | +| Source / Sqlite | 8 | 16.942 / 17.173 | 14.986 / 15.399 | -1.806 | -2.156 to -1.669 | +| Source / File | 6 | 16.751 / 17.036 | 14.725 / 14.917 | -2.086 | -2.159 to -1.875 | +| Wheel / Sqlite | 6 | 17.066 / 22.878 | 15.169 / 19.378 | -1.932 | -5.246 to 2.348 | +| Wheel / File | 6 | 16.689 / 17.454 | 14.801 / 15.195 | -1.917 | -2.491 to -1.553 | + +The p95 uses nearest rank and is the observed maximum at these sample sizes; +it is not a tail bound. The interval uses 10,000 fixed-seed paired resamples. +Differences of arm medians and median paired differences are distinct. Arm +medians improve approximately 11–12%; wheel/SQLite's interval crosses zero and +retains both directions of tail variation. This is scoped evidence for the +small-state path, not a universal latency threshold or long-horizon benefit. + +| Small-state path | Target plus waited descendants CPU seconds, median (base → candidate) | Maximum per-invocation target/waited RSS MiB, median (base → candidate) | Owned broker CPU seconds, median (base → candidate) | Broker end RSS MiB, median (base → candidate) | +| --- | --- | --- | --- | --- | +| Source / Sqlite | 10.536 → 9.190 | 126.60 → 90.02 | 0.767 → 0.775 | 126.59 → 125.28 | +| Source / File | 10.507 → 9.101 | 127.77 → 87.15 | 0.716 → 0.702 | 126.77 → 126.70 | +| Wheel / Sqlite | 10.782 → 9.289 | 134.83 → 91.51 | 0.799 → 0.797 | 125.77 → 126.91 | +| Wheel / File | 10.606 → 9.348 | 132.01 → 88.21 | 0.729 → 0.723 | 127.97 → 127.77 | + +RSS is the maximum of individual CLI invocation peaks, not a concurrent +process-tree total. Broker counters exclude retirement and use Darwin's +calibrated timebase. Default SQLite target/waited CPU falls about 13–14%; broker +CPU/RSS have no consistent improvement. Source/SQLite prequota median falls +3.425 → 1.468 seconds; guard remains 1.845 → 1.863 seconds and becomes the +largest candidate phase. Wheel/SQLite prequota is 3.463 → 1.581 seconds, while +guard is 1.844 → 2.033 seconds. The gain is concentrated in fresh Python +prequota loading, not faster authority execution. + +Setup and retirement medians across groups are 2.15–2.29 and 0.36–0.40 seconds. +Including setup, teardown and driver overhead, source/SQLite median is +20.238 → 18.146 seconds and wheel/SQLite 20.393 → 19.793 seconds. Independently +building/installing the two wheels costs 43.881 seconds. Setup, validation, +recovery and census may overlap and must not be summed. Preparatory authoring, +complete native/thread/process-tree I/O, model tokens, model/human costs and +long-horizon outcomes remain incompletely measured. Kernel-observed broker disk +writes stay at roughly 5.85 MiB per SQLite episode and 220 KiB per File episode; +those counters are not total filesystem traffic. + +Across all 60 observations the artifact bytes, original bindings, complete +settlement obligations, one writeback and one debit agree. Required native +initial bodies are 23,498–24,442 bytes for small states; `bytes / 4` is only a +rough token proxy, not tokenizer output or proof of model retention. The +candidate's extra 85 source / 165 wheel bytes are path/reference presentation; +normalized settlement obligations compare clause for clause without deletion. + +With 101 Todos, ordinary completion automatically archives the selected Todo +at its original early presentation position, together with 90 background +Todos. Validation, writeback and one spend commit, but the later original-Turn +`todo complete --no-follow-up` rejects with `todo_archived`. All four +source/wheel × File/SQLite pairs reproduce this on both arms. No failed episode +is counted as accepted or used to claim a complete large-state speedup. The +existing `todo_terminal_lifecycle.ts` and `todo_archive_selection.ts` owners +must reconcile archival with original-receipt closeout. Related #5677 already +owns freshness/terminal recovery; its published 0/1/4-history matrix does not +qualify this automatic-archive case. Preserve fresh archived-Todo, changed +actor/lease/evidence and duplicate-effect refusals when extending that owner. + +A separate census over all eight source/wheel × File/SQLite variants preserves +invalid Vision/workspace refusals. Four additional failure checks per variant +cover missing Node admission, missing provider, mismatched capture digest and +released-claim completion: no writeback/debit commits. Prequota's soft failure +continues to quota; it does not grant admission. These checks take 69.559 seconds; +the instrumented census takes 90.358 seconds, with overlap between phases. +Every sampled/census broker has observed shutdown and OS exit; pilot's initial +wrong-source retirement/pending wait remains in history and was subsequently +resolved in its owned namespaces. + +Excluding the census's two extra refusals, complete-episode RPCs are 271 → 270 +in every variant; spend alone is 83 → 82. Observed receive traffic is +999,060–1,004,527 → 966,984–972,825 bytes; send traffic is +1,025,077–1,037,023 → 1,026,956–1,040,098 bytes. Extra send bytes reflect the +longer candidate paths. Parent read-opens are 13,532–13,561 → 11,700–11,728; +write-opens 285 → 284. Instrumented prequota module graphs are source +1,404 → 480 and wheel 1,403 → 479, including instrumentation imports; these +are not uninstrumented loading budgets. Cold Node/readiness takes 0.217–0.249 +seconds and version probing 0.026–0.032 seconds in the census. Single census +timings do not establish a repeated latency gain; socket coverage and open +counts do not include all native/broker I/O. + +**Evidence-conditioned next action:** carry the archive case to the existing +terminal/freshness owner and #5677 integration before qualifying the full +large-state lifecycle. Preserve the original operation/read/receipt identity +and every refusal boundary; do not change archive policy merely to pass this +workload. Then characterize the remaining `quota should-run` parser/loading and +shared projection caller before selecting another performance change. The +small-state data support continued exact-head review of #8240/#8243, not merge +or installed-host adoption. Retain I3 for at least one minor release and keep +M1/#8011, full R0/V10/R1, D2/D5/D6/D3, context/model/split-install, actual adoption +and complete-cost acceptance open. No new budget, preset, provider or authority +is introduced. + +For a source replay, fetch the exact PR heads and apply only the production +`loopx/` diffs from their recorded merge bases (`79665b97a0a453380da02e15b1b62eee895147df` +and `97cc1e0c40e766c73193b8077526a7f565dc561b`) to an isolated archive of the +baseline. `uv sync --extra test` supplies the console. Build the two monolithic +wheels independently with `uv build --wheel`; validate the Chat bundle through +its existing owner before building. Reuse the source fixtures in +`tests/control_plane/test_quota_settlement_cli.py`: author the synthetic Goal +and background Todos before `_initialize_fixture_authority`, select File or +SQLite and `hard_lease`, and use ordinary claim for execution. Declare the +artifact write scope in both the Todo and Goal boundary, and use the existing +Git-fixture helper plus a distinct worktree. Scope every command below to that +fixture's registry/runtime and worktree; the guard supplies the full read and +recovery commands. Keep the same Goal, Agent, Todo, Turn and claim operation: + +```text +loopx heartbeat-prompt --thin --codex-app --goal-id --agent-id +loopx heartbeat-prequota -g -a +loopx todo claim --goal-id --todo-id --agent-id --claimed-by --claim-operation-id +loopx quota should-run --goal-id --agent-id --todo-id --turn-instance-id --codex-app --context-delivery on +# Execute first_read_command and every emitted contiguous next_command. +# Produce {"sum_0_through_9":45}; verify it independently before completion. +loopx todo complete --goal-id --agent-id --todo-id --turn-instance-id --claim-operation-id --evidence artifact:verified-sum +loopx refresh-state --goal-id --agent-id --todo-id --turn-instance-id --classification synthetic_cli_cost_artifact --delivery-boundary semantic_closeout --delivery-batch-scale single_surface --delivery-outcome outcome_progress --agent-vision-json --delivery-workspace-path --no-global-sync --suppress-external-sinks +loopx quota spend-slot --goal-id --agent-id --todo-id --turn-instance-id --slots 1 --source heartbeat --execute +loopx todo complete --goal-id --agent-id --todo-id --turn-instance-id --claim-operation-id --evidence artifact:verified-sum --no-follow-up +loopx todo list --goal-id --todo-id --include-archived +``` + +Declare the fixture's observed shell, filesystem-write, network and evidence-poll +capabilities on admission/writeback/spend; use the actual installed console for +generated reader commands. Repeat completion, writeback, spend and terminal with +unchanged arguments. Missing Node uses an isolated PATH, missing provider an +empty disposable runtime, stale source a mismatched capture digest, and stale +claim a normally released lease. These cases must refuse before writeback/debit; +invalid Vision/workspace also remain refusals. The temporary measurement harness +and raw captures stay private; this recipe describes the repeated workload and +its oracles, not a distributed benchmark runner or public reproduction receipt. + ## Appendix B: Decision log | Date | Decision | Owner / approval | Alternatives | Normative sections changed | diff --git a/docs/architecture/rfcs/monorepo-distribution-split-v0.zh-CN.md b/docs/architecture/rfcs/monorepo-distribution-split-v0.zh-CN.md index 9ac069fb79..d1ae12fed5 100644 --- a/docs/architecture/rfcs/monorepo-distribution-split-v0.zh-CN.md +++ b/docs/architecture/rfcs/monorepo-distribution-split-v0.zh-CN.md @@ -1086,6 +1086,143 @@ owner;采纳性能变更前必须另做匹配的真实路径对照。本 check 预算、preset 或默认行为。R0 完整清点、R1/minimal、D2/D5/D6、I3/D3、 hard-lease/context/model 和拆包安装验收仍开放;采纳仍受独立 exact-head 评审及必需验证约束。 +### 2026-11-07 — 完整 governed CLI 组合成本与归档后终结边界 + +对照 main `8e86f173f08bd55eb833d8cdf42360f93bf989fe` 和私有实验组合: +#8240 `4df990c9e66465171f980176fba4e3602d41f826` 加上 +#8243 `668bed2f03a274f273d39c187121a1a4ede67165`。候选只应用两提案的七个 +生产 Python 路径差分。分别构建、独立安装的单体 wheel 各有 1,998 个文件、 +约 8.91 MiB 压缩大小,1,836 个 runtime resource 与对应源码逐字节核对。 +全部环境在 macOS arm64 上使用 CPython 3.13.5、Node 26.0.0、真实 Node 所有的 +SQLite 3.53.1 和相同的 57 个依赖版本。这里只是作者实验组合,没有升级已安装 +host,也没有修改交付 preset、authority provider 或公共 runtime。 + +采样前冻结代码/resource/wheel hash、工作负载、缓存、instrumentation、串行对照 +顺序、采样和解释规则。每次 CLI 都是新 Python、独立空 bytecode cache prefix, +并禁止写入 bytecode。每个 episode 在 prequota 启动独立冷 Node,之后仅在同一轮 +复用;compile cache 和临时根隔离。磁盘缓存及其他 host 负载不受控;每对交替 +base/candidate 顺序。1,500 秒上限包含 fixture 准备、执行、退出与采样编排。 +pilot、负例、打包和独立 instrumentation census 分别记账。 + +流程覆盖普通 hard-lease claim/admission、独立 Git worktree、完整连续 native +initial read、独立验证的 JSON 求和产物 `45`、Todo completion、fresh evidence-linked +Vision/writeback、一次 spend、terminal result/readback,以及 completion/writeback/ +spend/terminal 的原参数重放。明确关闭 global sync 和 external sinks;本实验不 +覆盖 standard fanout。小状态有一个活动 Todo;大状态增加 100 个带较长文本的 +已完成背景 Todo。大状态失败保留为失败,不算成功成本样本。 + +全部 60 个计划观测在 **1,164.996 秒**内完成,未改变上限:52 个小状态 episode +完成,八个大状态 episode 保留相同终结失败。代码、脚本与冻结 inventory 未变。 +小状态四组各有 8/6/6/6 对串行交错样本;大状态各一对,仅验证观察到的兼容性失败。 + +| 小状态路径 | 对数 | Baseline 秒,median / p95 | Candidate 秒,median / p95 | 配对差的 median,秒 | 探索性 95% percentile 区间 | +| --- | --- | --- | --- | --- | --- | +| Source / Sqlite | 8 | 16.942 / 17.173 | 14.986 / 15.399 | -1.806 | -2.156 to -1.669 | +| Source / File | 6 | 16.751 / 17.036 | 14.725 / 14.917 | -2.086 | -2.159 to -1.875 | +| Wheel / Sqlite | 6 | 17.066 / 22.878 | 15.169 / 19.378 | -1.932 | -5.246 to 2.348 | +| Wheel / File | 6 | 16.689 / 17.454 | 14.801 / 15.195 | -1.917 | -2.491 to -1.553 | + +p95 使用 nearest rank;在这个样本量下就是观测最大值,不能作为尾延时上界。 +区间来自固定 seed 的 10,000 次配对重采样。两臂 median 的差与配对差的 median +是不同统计量。两臂 median 改善约 11–12%;wheel/SQLite 区间跨过零,保留了两个 +方向的尾部波动。这是小状态路径的局部证据,不是通用速度门槛或长程收益。 + +| 小状态路径 | Target 加已等待 descendants CPU 秒,median(base → candidate) | 每次调用 target/waited RSS 最大值 MiB,median(base → candidate) | 目标 broker CPU 秒,median(base → candidate) | Broker 结束时 RSS MiB,median(base → candidate) | +| --- | --- | --- | --- | --- | +| Source / Sqlite | 10.536 → 9.190 | 126.60 → 90.02 | 0.767 → 0.775 | 126.59 → 125.28 | +| Source / File | 10.507 → 9.101 | 127.77 → 87.15 | 0.716 → 0.702 | 126.77 → 126.70 | +| Wheel / Sqlite | 10.782 → 9.289 | 134.83 → 91.51 | 0.799 → 0.797 | 125.77 → 126.91 | +| Wheel / File | 10.606 → 9.348 | 132.01 → 88.21 | 0.729 → 0.723 | 127.97 → 127.77 | + +RSS 是各次 CLI 调用峰值的最大值,不是同时存活进程树的总峰值。Broker counter +不含退出阶段,使用校准后的 Darwin timebase。默认 SQLite 的 target/waited CPU +下降约 13–14%;broker CPU/RSS 没有一致改善。Source/SQLite prequota median +从 3.425 降到 1.468 秒;guard 为 1.845 → 1.863 秒,成为候选最重阶段。 +Wheel/SQLite prequota 为 3.463 → 1.581 秒,guard 为 1.844 → 2.033 秒。 +降耗主要集中在新 Python 的 prequota loading,不能归因为 authority 执行加速。 + +各组 setup median 为 2.15–2.29 秒,retirement 为 0.36–0.40 秒。包含 setup、 +退出和 driver 开销后,source/SQLite median 为 20.238 → 18.146 秒,wheel/SQLite +为 20.393 → 19.793 秒。两个 wheel 独立构建/安装用时 43.881 秒。准备、验证、恢复 +与 census 有重叠,不能相加。准备阶段作者成本、完整 native/thread/进程树 I/O、 +model token、model/人工成本及长程结果仍未完整测量。内核观察到的 broker 磁盘写入 +保持约每轮 SQLite 5.85 MiB、File 220 KiB;这不是全部文件系统流量。 + +60 个观测的产物字节、原绑定、完整结算义务、一次 writeback 和一次 debit 一致。 +小状态必读 native initial body 为 23,498–24,442 bytes;`bytes / 4` 仅是粗 token +proxy,不是 tokenizer 结果或 model retention 证明。候选多出的 source 85 / wheel +165 bytes 来自路径/reference 展示;规范化后的结算义务逐条一致,没有删减。 + +101 个 Todo 时,普通 completion 会把原展示位置靠前的选中 Todo 和 90 个背景 +Todo 自动归档。Validation、writeback、一次 spend 已提交,后续原 Turn 的 +`todo complete --no-follow-up` 却因 `todo_archived` 拒绝。四组 source/wheel × +File/SQLite 对照在两臂均复现。失败轮没有计入验收成功,也不用于宣称完整大状态 +加速。既有 `todo_terminal_lifecycle.ts`、`todo_archive_selection.ts` owner 需要协调 +自动归档和原 receipt 的终结。相关 #5677 已负责 freshness/terminal recovery;其 +公开的 0/1/4 history 矩阵没有验证本次自动归档分支。扩展既有 owner 时,保留 fresh +archived-Todo、变更 actor/lease/evidence 和重复 effect 的拒绝。 + +独立 census 在八个 source/wheel × File/SQLite 变体中保留 invalid Vision/workspace +拒绝。每个变体另有四个失败检查:missing Node admission、missing provider、capture +digest 不匹配,以及 released-claim completion;没有提交 writeback/debit。 +Prequota 的 soft failure 继续进入 quota,不授予 admission。负例检查用时 69.559 秒, +instrumented census 用时 90.358 秒,两阶段有重叠。采样/census 的所有 broker 都已 +观察到 shutdown 与 OS exit;pilot 初次 wrong-source retirement/pending wait 的失败 +保留在历史中,随后在其独立 namespace 完成退出回读。 + +剔除 census 额外两个拒绝后,各变体完整 episode 的 RPC 均为 271 → 270,spend +单独为 83 → 82。实际 receive 流量为 999,060–1,004,527 → 966,984–972,825 bytes; +send 为 1,025,077–1,037,023 → 1,026,956–1,040,098 bytes,增加来自候选更长的路径。 +Parent read-open 为 13,532–13,561 → 11,700–11,728,write-open 为 285 → 284。 +带 instrumentation 的 prequota module graph 为 source 1,404 → 480、wheel +1,403 → 479,包含 instrumentation 自身 import,不是未插桩的 loading 预算。 +Census 的冷 Node/readiness 为 0.217–0.249 秒,version probe 为 0.026–0.032 秒。 +单次插桩用时不能证明重复延时收益;socket/open 计数也未覆盖全部 native/broker I/O。 + +**基于证据的下一步:**把归档分支交给既有 terminal/freshness owner 和 #5677 +集成验证,再验证完整大状态生命周期。保留原 operation/read/receipt identity 与 +所有拒绝边界;不为让这个工作负载通过而修改归档策略。随后 characterize 剩余 +`quota should-run` parser/loading 与共享 projection caller,再选择性能改动。 +小状态数据支持继续对 #8240/#8243 的精确 head 做 review,不授予 merge 或已安装 +host 采纳资格。I3 至少保留一个 minor release;M1/#8011、完整 R0/V10/R1、 +D2/D5/D6/D3、context/model/split-install、真实 adoption 和完整成本验收仍开放。 +没有新增 budget、preset、provider 或 authority。 + +Source 重放时,拉取两个精确 PR head,仅将生产 `loopx/` 差分应用到隔离的 +baseline archive;记录的 merge base 分别为 +`79665b97a0a453380da02e15b1b62eee895147df`、 +`97cc1e0c40e766c73193b8077526a7f565dc561b`。使用 `uv sync --extra test` 准备 +console,并用 `uv build --wheel` 分别构建单体 wheel;构建前通过既有 owner 验证 +Chat bundle。复用 `tests/control_plane/test_quota_settlement_cli.py` 的 source fixture: +在 `_initialize_fixture_authority` 前完整编写 synthetic Goal/背景 Todo,选择 File +或 SQLite 与 `hard_lease`,通过普通 claim 准备执行。Todo 和 Goal boundary 同时 +声明产物 write scope,复用 Git fixture helper 并创建独立 worktree。 + +以下重放序列的每条命令都必须指定该 fixture 的 registry/runtime/worktree;guard +提供完整 read/recovery 命令。保持同一 Goal、Agent、Todo、Turn 和 claim operation: + +```text +loopx heartbeat-prompt --thin --codex-app --goal-id --agent-id +loopx heartbeat-prequota -g -a +loopx todo claim --goal-id --todo-id --agent-id --claimed-by --claim-operation-id +loopx quota should-run --goal-id --agent-id --todo-id --turn-instance-id --codex-app --context-delivery on +# Execute first_read_command and every emitted contiguous next_command. +# Produce {"sum_0_through_9":45}; verify it independently before completion. +loopx todo complete --goal-id --agent-id --todo-id --turn-instance-id --claim-operation-id --evidence artifact:verified-sum +loopx refresh-state --goal-id --agent-id --todo-id --turn-instance-id --classification synthetic_cli_cost_artifact --delivery-boundary semantic_closeout --delivery-batch-scale single_surface --delivery-outcome outcome_progress --agent-vision-json --delivery-workspace-path --no-global-sync --suppress-external-sinks +loopx quota spend-slot --goal-id --agent-id --todo-id --turn-instance-id --slots 1 --source heartbeat --execute +loopx todo complete --goal-id --agent-id --todo-id --turn-instance-id --claim-operation-id --evidence artifact:verified-sum --no-follow-up +loopx todo list --goal-id --todo-id --include-archived +``` + +Admission/writeback/spend 声明该 fixture 已观察到的 shell、filesystem-write、network +和 evidence-poll capability;生成的 reader 命令使用实际安装的 console。用不变参数 +重放 completion、writeback、spend、terminal。Missing Node 使用隔离 PATH,missing +provider 使用空的 disposable runtime,stale source 使用不匹配的 capture digest, +stale claim 使用普通 release 后的 lease。它们必须在 writeback/debit 前拒绝;invalid +Vision/workspace 同样保留拒绝。临时测量 harness 和 raw capture 保持私有;这个 recipe +描述重复的 workload/oracle,不是分发的 benchmark runner 或公共复现 receipt。 + ## 附录 B:决策日志 | 日期 | 决策 | 负责人 / 批准 | 备选 | 变更的规范章节 |