Repository navigation
test(control-plane): keep hot-path capture out of inline JSON smokes - #8182
huangruiteng merged 1 commit into
Conversation
017b4ba to
b71ac4e
Compare
Five shipped public smokes read inline quota decision fields, but since the projection contract moved those payloads onto the hot path they can receive a reader reference instead, which turns the nightly full-public sweep red. Declare the transport the way the canary harness and the four existing smokes already do. Signed-off-by: Jiaqiiii523 <293262669+Jiaqiiii523@users.noreply.github.com>
b71ac4e to
fd8287a
Compare
loopx-agent
left a comment
There was a problem hiding this comment.
Reviewer: model_agent | gpt-6.1-sol | OpenAI | runtime_reported | reasoning_effort=max
动机 / Motivation
运行公开 smoke 和最小 CLI Turn 示例的维护者,需要读取决策字段来判断任务是否可执行、缓存是否正确及工作是否完成。
未设置传输环境变量时,原来的五个 smoke 收到 reader 导航对象,因缺少 should_run、agent_lane_next_action 或 agent_todo_summary 而失败;修复后在原有断言处读取 inline JSON,六个相关入口均完成。
独立运行复现了基线五项失败,PR 同头六项通过;第六项共享 fixture 消费者在基线也通过,未将它计作新修复。 这恢复了后续缓存、Todo 和结算检查的信号,避免传输层差异提前中断验证。
范围是四个 smoke/fixture 的传输选择;不修改产品默认 capture、权限、调度、决策语义或性能预算,也不宣称整个 nightly 已恢复。
改动思路 / Approach
区分 smoke 要验证的机器决策数据与模型使用的渐进式展示。既有 canary helper 已在子进程环境选择 LOOPX_CONTEXT_DELIVERY=off;本 PR 在现有 helper 内沿用这个合同,生产默认保持原样。当前 outcome-first/recovery 经验在此适用为内部验证维护,不推导新 UI、权限或完整产品交付要求。
| 路径 | 基线直接执行 | 本 PR |
|---|---|---|
| 五个 inline 消费者 | 导航对象缺少决策字段,提前失败 | 同一 CLI、同一语义断言,显式 inline |
| 第六个 metrics 消费者 | 通过 | 共享 fixture 后仍通过 |
| 普通/原生 capture 测试 | 独立验证传输合同 | 保留并重新执行 |
具体改动 / Changes
4 文件 +21/-2,仅导入、注释和局部环境选择;没有删断言、提高预算或修改生产代码。两个 subprocess helper 复制环境再设置 off;Todo fixture 与 最小 Turn 示例 使用 patch.dict,返回及异常均恢复环境。Todo fixture 的 lifecycle、deferred-capacity、capability-gap-metrics 三个调用者均实跑。
独立基线 d0985c35aa3d05fd5de964e719aaf69f45d72535:cache/perf/minimal 三项缺少 should_run,lifecycle 缺少 agent_lane_next_action,deferred 缺少 agent_todo_summary。同头六项全通过。cache 仍检查失效、损坏、并发写和隐私拒绝;最小 Turn 仍检查实际产物、一次扣费及幂等重放。性能 smoke 的 3.5s 限额保留,本机 status/quota 为0.233s/0.420s,坏状态为0.188s/0.234s;这是 inline 决策路径的证据。
规格先于 diff 读取:docs/development/good-smokes.md,不可变版本 d0985c35aa3d05fd5de964e719aaf69f45d72535。Start With The Contract:保留真实 CLI 与状态边界;Prefer Semantic Pressure:保留拒绝/恢复断言,补充独立环境展开核验;Consolidate Without Losing Coverage:共享现有 Todo helper,未增加重复 smoke;Validate The Change:执行聚焦与风险选择验证。以上四条均在本头实现,未以 PR 自述替代独立 oracle。
对主干的风险 / Risks
最大风险是进程内 off 泄漏到后续调用,或只修 inline 后漏掉默认 reader 回归。16 个 helper 用例覆盖未设置/on/auto/off 环境与正常/异常展开,恢复完整环境;其中强制异常使用合成入口,仅证明 helper 生命周期。另有44项现有测试通过:tests/control_plane/test_quota_shared_json_identity.py 和 tests/test_automation_cadence.py,包含实际 File/SQLite 普通/原生 CLI、分页/select reader、错误摘要拒绝及来源事实保持。
standard premerge 通过5项直接检查与10项选择检查,含语义词汇、Todo、路由、monitor 和四文件公开边界;同头 CQR 有效,changed-file Ruff、DCO trailer、diff whitespace 通过。未改共享语义词汇或持久格式。相邻同作者 PR8183/8184/7398 的当前 key-file 清单无本次文件重叠;无重复 smoke 增生证据,不据此审定那些 PR。
保留现有专用 subprocess helper 的计时、cwd 和错误合同;跨多个 helper 的重写无必要。作者提供的历史 nightly 失败仅作诊断,未查询 CI,未归因剩余失败。撤回该 smoke-only commit 即可回滚,无数据迁移。
我的整体评价 / Verdict
这个内部维护任务已交付可复现的修复,没有发现阻塞项。生产默认 capture 与产品入口保持,长期价值是恢复现有验证信号;不据测试数量推导模型效果、完整成本改善或父级目标完成。
四个改动路径均为 examples 下的 Python smoke/fixture,没有修改任何渲染页面或产品文案,因此视觉审阅不适用。
未执行 Windows、真实 Host 安装、付费模型或完整 nightly;未测默认 capture 路径的性能。
English verdict: APPROVE — HEAD fd8287a. The inline smoke transport repair is verified; production defaults are unchanged. This review does not authorize merge.
Goal And Delivered Outcome
full-public-smokes-shard-2, tested headfaab5f5c29).should_run,agent_todo_summary,agent_lane_next_action) and the projection-cache record that rides alongside them. Since the hot-path projection contract merged in fix(quota): bound mixed diagnostic summaries without losing unfamiliar records #7587, those same CLI calls can answer with a reader reference (context_capture_navigation_v1) instead of the decision body, so the smokes assert against a payload that no longer carries the fields they name.failedin thefull-public-smokes-shard-2artifact for headfaab5f5c29(110 checks, 12 failed). After, each exits 0 when run without any ambient transport setting — see thereal_entrypointandregression_parityrows below.360e6e064; the five in this PR were never part of that report.Author Declaration
Implemented against
loopx/control_plane/testing/canary_harness.py(subprocess env) andexamples/autonomous-replan-obligation-smoke.py(in-processpatch.dict). Repositorymainat62e234c939996ad42f2b1168451cc9a656dce11d.examples/control_plane/status-projection-cache-smoke.py,examples/control_plane/status-quota-perf-budget-smoke.pypython3 examples/run-smokes.py --suite full-public --script examples/control_plane/status-projection-cache-smoke.pyexamples/control_plane/todo_lifecycle_fixtures.py::run_fixture_clipython3 examples/control_plane/todo-lifecycle-cli-smoke.pytests/control_plane/test_quota_shared_json_identity.py,tests/control_plane/codex_checkpoint_fixture.py,tests/test_automation_cadence.pytodo-durability-fixture,todo-readmodel-boundary,todo-first-open-summary) are green on a currentmaincheckout here, two of them being theopen_todo_itemsimport already addressed in fix(status): restore the open_todo_items and todo_item_task_class compat re-exports #8160.Scope And Continuation
Why the smoke is the stale side, not the projection
Following the reader reference instead of opting out was the first option I tried, and it cannot restore what these smokes check: the captured selection set for a
quota should-runhot-path read carries 35 pointers, and none of them is the projection-cache record (written,hit,path) thatassert_cache_writtenandassert_cache_expiredread, nor ashould_runfield at the inline location the smoke names. Opting out of the model-facing transport keeps the declared behaviour under test (cache write/hit/expiry, decision fields, and status/quota latency against a large ignored state tree) and leaves the reference shape covered by the tests listed above.Validation
fd8287af6ab6d7e184143668f5f729d91dade553real_entrypointpassedpython3 examples/run-smokes.py --suite full-public --script <path>on the tested revision: allstatus: passed, returncode 0.regression_paritypassedshould_run,agent_todo_summary,agent_lane_next_action, and onecontext_capture_navigation_v1payload); on the tested revision all five exit 0. The sixth consumer of the shared fixture driver,examples/issue-fix-capability-gap-metrics-smoke.py, also exits 0.staticpassedpython3 -m ruff checkon the four changed files.unitnot_runloopx/code.integrationnot_rund0985c35aa3d05fd5de964e719aaf69f45d72535with no content change —git patch-id --stableis identical before and after (f3ce546bc69ef120722a), and the five smokes plus the sixth consumer of the shared fixture driver were re-run on the rebased revision (all exit 0), against the same revision without the change (exit 1).full-publicsweep turns green overall is not claimable here, because seven other failures in that shard are out of scope.Frontend / Visual Evidence
Type of Change
LoopX Area
Technical Direction
Shared-authority RFC fixture impact
Boundary Checklist
none.Signed-off-bytrailer.