Skip to content

test(control-plane): keep hot-path capture out of inline JSON smokes - #8182

Merged
huangruiteng merged 1 commit into
loopx-project:mainfrom
Jiaqiiii523:fix/status-smoke-inline-quota-json
Nov 7, 2026
Merged

huangruiteng merged 1 commit into
loopx-project:mainfrom
Jiaqiiii523:fix/status-smoke-inline-quota-json

Conversation

@Jiaqiiii523

@Jiaqiiii523 Jiaqiiii523 commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Goal And Delivered Outcome

  • Outcome basis / optional anchor: self-contained maintenance outcome. No issue claims this gap; the public evidence is the nightly sweep run https://github.com/loopx-project/loopx/actions/runs/42132115859 (full-public-smokes-shard-2, tested head faab5f5c29).
  • Goal/source and gap: five shipped public smokes read inline quota decision fields (should_run, agent_todo_summary, agent_lane_next_action) and the projection-cache record that rides alongside them. Since the hot-path projection contract merged in fix(quota): bound mixed diagnostic summaries without losing unfamiliar records #7587, those same CLI calls can answer with a reader reference (context_capture_navigation_v1) instead of the decision body, so the smokes assert against a payload that no longer carries the fields they name.
  • Observable before → after, with the validation row that proves it: before, all five are listed as failed in the full-public-smokes-shard-2 artifact for head faab5f5c29 (110 checks, 12 failed). After, each exits 0 when run without any ambient transport setting — see the real_entrypoint and regression_parity rows below.
  • Issue/task and intended base: no issue is closed here. Related to [Bug]: two shipped public smokes now contradict #7587's projection contract and fail on main (hot-path quota payload) #7653, whose own two smokes were already updated by 360e6e064; the five in this PR were never part of that report.

Author Declaration

  • Written by: model_agent (Qoder Desktop coding agent, Anthropic Claude model family).

Implemented against

  • Specification and revision: the hot-path projection contract as merged in fix(quota): bound mixed diagnostic summaries without losing unfamiliar records #7587 and the transport opt-out already used by shipped consumers — loopx/control_plane/testing/canary_harness.py (subprocess env) and examples/autonomous-replan-obligation-smoke.py (in-process patch.dict). Repository main at 62e234c939996ad42f2b1168451cc9a656dce11d.
Criterion (spec clause) Disposition Symbol / path Test or command
A consumer that reads machine JSON declares its transport instead of inheriting the ambient default implemented examples/control_plane/status-projection-cache-smoke.py, examples/control_plane/status-quota-perf-budget-smoke.py python3 examples/run-smokes.py --suite full-public --script examples/control_plane/status-projection-cache-smoke.py
The same declaration reaches the shared fixture driver, so every smoke that uses it stops inheriting the default implemented examples/control_plane/todo_lifecycle_fixtures.py::run_fixture_cli python3 examples/control_plane/todo-lifecycle-cli-smoke.py
Coverage of the reference shape is left in place rather than moved into the smokes implemented tests/control_plane/test_quota_shared_json_identity.py, tests/control_plane/codex_checkpoint_fixture.py, tests/test_automation_cadence.py existing lanes, unchanged by this diff
  • Self-check before submission: I read the failing assertion in each smoke, the nightly artifact that lists them as failed, and the pointer set the reference actually carries. I deliberately did not touch the seven other failed checks in that shard: four fail on different assertions, and three (todo-durability-fixture, todo-readmodel-boundary, todo-first-open-summary) are green on a current main checkout here, two of them being the open_todo_items import already addressed in fix(status): restore the open_todo_items and todo_item_task_class compat re-exports #8160.

Scope And Continuation

  • Completed scope and remaining work: complete within this scope for the five smokes named above. The remaining shard-2 failures are separate defects with different causes; no follow-up issue is invented here.
  • Slice boundary / successor: N/A — this is a consumer-side declaration, reversible on its own, and it changes no production behaviour.

Why the smoke is the stale side, not the projection

Following the reader reference instead of opting out was the first option I tried, and it cannot restore what these smokes check: the captured selection set for a quota should-run hot-path read carries 35 pointers, and none of them is the projection-cache record (written, hit, path) that assert_cache_written and assert_cache_expired read, nor a should_run field at the inline location the smoke names. Opting out of the model-facing transport keeps the declared behaviour under test (cache write/hit/expiry, decision fields, and status/quota latency against a large ignored state tree) and leaves the reference shape covered by the tests listed above.

Validation

  • Tested revision: fd8287af6ab6d7e184143668f5f729d91dade553
  • Run state: finished
  • Input classes: synthetic
Check kind Result Public-safe evidence / limitation
real_entrypoint passed Each of the five smokes run through python3 examples/run-smokes.py --suite full-public --script <path> on the tested revision: all status: passed, returncode 0.
regression_parity passed Same host, same interpreter, no ambient transport variable: on the base revision all five exit 1 (assertions on should_run, agent_todo_summary, agent_lane_next_action, and one context_capture_navigation_v1 payload); on the tested revision all five exit 0. The sixth consumer of the shared fixture driver, examples/issue-fix-capability-gap-metrics-smoke.py, also exits 0.
static passed python3 -m ruff check on the four changed files.
unit not_run No test module covers these paths; they are shipped smoke scripts executed directly, and this PR changes no loopx/ code.
integration not_run Not applicable: no production module, workflow file, or schema is modified.
  • Rebase note: rebased onto d0985c35aa3d05fd5de964e719aaf69f45d72535 with no content change — git patch-id --stable is identical before and after (f3ce546bc69ef120722a), and the five smokes plus the sixth consumer of the shared fixture driver were re-run on the rebased revision (all exit 0), against the same revision without the change (exit 1).
  • Coverage and gaps: the changed lines are only how a smoke declares its CLI transport, and the checks above execute each changed file end to end. Gap named honestly: this is a test-only change, so it is validated by the smokes themselves rather than by a new assertion — a future regression that removes the declaration would surface as the same nightly failure rather than as a unit test. Whether the nightly full-public sweep turns green overall is not claimable here, because seven other failures in that shard are out of scope.

Frontend / Visual Evidence

  • UI impact: none
  • Before: N/A
  • After: N/A
  • First viewport and preview: N/A
  • States and viewports shown: N/A
  • Source data: none
  • Attention review: N/A
  • Craft review: N/A

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Refactoring (no functional changes)
  • Documentation update
  • Test update

LoopX Area

  • Control plane (goals, todos, quota, scheduler, registry, runtime)
  • Benchmark boundary (adapters, runners, verifiers, scoring, evidence)
  • Capability or extension (providers, adapters, skills)
  • Public docs or presentation surface (README, protocols, dashboard)
  • Build, packaging, installer, or CI
  • Host or runtime integration

Technical Direction

  • Direction / acceptance reference, when applicable: Core control-plane hardening (shipped public smoke reliability).

Shared-authority RFC fixture impact

  • Production-scale fixture schema: N/A
  • Semantic dimensions changed, or reviewed no-impact rationale: N/A
  • Provider conformance arms run: N/A
  • Read-only legacy/file/PostgreSQL three-arm rehearsal: N/A

Boundary Checklist

  • Neither the diff nor this PR body/comments/attachments disclose private state, credentials, raw traces or verifier output, internal links, or local machine paths.
  • I did not duplicate maintainer-owned benchmark work unless a maintainer split out a public issue for it.
  • I kept the change scoped to the linked issue/task.
  • Existing English/Chinese RFC pairs both have a diff, or no paired RFC changed: no RFC file changed.
  • I completed the visual evidence section for UI changes, or marked UI impact none.
  • Every commit includes a DCO Signed-off-by trailer.

Second rebase in this session: moved onto d0985c35aa3d05fd5de964e719aaf69f45d72535 (main advanced twice while these were open). git patch-id --stable unchanged per branch (f3ce546bc69ef120722a, 69bc07982a79a5840364, cf8f91054e388d99a8b7), and the branch's own checks were re-run on that revision: the six smoke consumers exit 0 with the change and exit 1 without it, the guard pair is 26 passed against a 3 failed, 472 passed baseline identical on the untouched base, and the TypeScript lane is 6225 tests / 2 fail against 6224 / 2 fail with the same two failing test names, tsc --noEmit clean.

@Jiaqiiii523
Jiaqiiii523 force-pushed the fix/status-smoke-inline-quota-json branch from 017b4ba to b71ac4e Compare November 6, 2026 14:37
Five shipped public smokes read inline quota decision fields, but since
the projection contract moved those payloads onto the hot path they can
receive a reader reference instead, which turns the nightly full-public
sweep red. Declare the transport the way the canary harness and the four
existing smokes already do.

Signed-off-by: Jiaqiiii523 <293262669+Jiaqiiii523@users.noreply.github.com>
@Jiaqiiii523
Jiaqiiii523 force-pushed the fix/status-smoke-inline-quota-json branch from b71ac4e to fd8287a Compare November 6, 2026 14:56

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent | gpt-6.1-sol | OpenAI | runtime_reported | reasoning_effort=max

动机 / Motivation

运行公开 smoke 和最小 CLI Turn 示例的维护者,需要读取决策字段来判断任务是否可执行、缓存是否正确及工作是否完成。

未设置传输环境变量时,原来的五个 smoke 收到 reader 导航对象,因缺少 should_run、agent_lane_next_action 或 agent_todo_summary 而失败;修复后在原有断言处读取 inline JSON,六个相关入口均完成。

独立运行复现了基线五项失败,PR 同头六项通过;第六项共享 fixture 消费者在基线也通过,未将它计作新修复。 这恢复了后续缓存、Todo 和结算检查的信号,避免传输层差异提前中断验证。

范围是四个 smoke/fixture 的传输选择;不修改产品默认 capture、权限、调度、决策语义或性能预算,也不宣称整个 nightly 已恢复。

改动思路 / Approach

区分 smoke 要验证的机器决策数据与模型使用的渐进式展示。既有 canary helper 已在子进程环境选择 LOOPX_CONTEXT_DELIVERY=off;本 PR 在现有 helper 内沿用这个合同,生产默认保持原样。当前 outcome-first/recovery 经验在此适用为内部验证维护,不推导新 UI、权限或完整产品交付要求。

路径 基线直接执行 本 PR
五个 inline 消费者 导航对象缺少决策字段,提前失败 同一 CLI、同一语义断言,显式 inline
第六个 metrics 消费者 通过 共享 fixture 后仍通过
普通/原生 capture 测试 独立验证传输合同 保留并重新执行

具体改动 / Changes

4 文件 +21/-2,仅导入、注释和局部环境选择;没有删断言、提高预算或修改生产代码。两个 subprocess helper 复制环境再设置 off;Todo fixture 与 最小 Turn 示例 使用 patch.dict,返回及异常均恢复环境。Todo fixture 的 lifecycle、deferred-capacity、capability-gap-metrics 三个调用者均实跑。

独立基线 d0985c35aa3d05fd5de964e719aaf69f45d72535:cache/perf/minimal 三项缺少 should_run,lifecycle 缺少 agent_lane_next_action,deferred 缺少 agent_todo_summary。同头六项全通过。cache 仍检查失效、损坏、并发写和隐私拒绝;最小 Turn 仍检查实际产物、一次扣费及幂等重放。性能 smoke 的 3.5s 限额保留,本机 status/quota 为0.233s/0.420s,坏状态为0.188s/0.234s;这是 inline 决策路径的证据。

规格先于 diff 读取:docs/development/good-smokes.md,不可变版本 d0985c35aa3d05fd5de964e719aaf69f45d72535。Start With The Contract:保留真实 CLI 与状态边界;Prefer Semantic Pressure:保留拒绝/恢复断言,补充独立环境展开核验;Consolidate Without Losing Coverage:共享现有 Todo helper,未增加重复 smoke;Validate The Change:执行聚焦与风险选择验证。以上四条均在本头实现,未以 PR 自述替代独立 oracle。

对主干的风险 / Risks

最大风险是进程内 off 泄漏到后续调用,或只修 inline 后漏掉默认 reader 回归。16 个 helper 用例覆盖未设置/on/auto/off 环境与正常/异常展开,恢复完整环境;其中强制异常使用合成入口,仅证明 helper 生命周期。另有44项现有测试通过:tests/control_plane/test_quota_shared_json_identity.py 和 tests/test_automation_cadence.py,包含实际 File/SQLite 普通/原生 CLI、分页/select reader、错误摘要拒绝及来源事实保持。

standard premerge 通过5项直接检查与10项选择检查,含语义词汇、Todo、路由、monitor 和四文件公开边界;同头 CQR 有效,changed-file Ruff、DCO trailer、diff whitespace 通过。未改共享语义词汇或持久格式。相邻同作者 PR8183/8184/7398 的当前 key-file 清单无本次文件重叠;无重复 smoke 增生证据,不据此审定那些 PR。

保留现有专用 subprocess helper 的计时、cwd 和错误合同;跨多个 helper 的重写无必要。作者提供的历史 nightly 失败仅作诊断,未查询 CI,未归因剩余失败。撤回该 smoke-only commit 即可回滚,无数据迁移。

我的整体评价 / Verdict

这个内部维护任务已交付可复现的修复,没有发现阻塞项。生产默认 capture 与产品入口保持,长期价值是恢复现有验证信号;不据测试数量推导模型效果、完整成本改善或父级目标完成。

四个改动路径均为 examples 下的 Python smoke/fixture,没有修改任何渲染页面或产品文案,因此视觉审阅不适用。

未执行 Windows、真实 Host 安装、付费模型或完整 nightly;未测默认 capture 路径的性能。

English verdict: APPROVE — HEAD fd8287a. The inline smoke transport repair is verified; production defaults are unchanged. This review does not authorize merge.

@huangruiteng
huangruiteng merged commit ebb39f4 into loopx-project:main Nov 7, 2026
23 of 38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants