Skip to content

Remove the PROGRESS.md progress file feature - #409

Merged
ifdotpy merged 1 commit into
mainfrom
remove/progress-md
Oct 22, 2026
Merged

ifdotpy merged 1 commit into
mainfrom
remove/progress-md

Conversation

@ifdotpy

@ifdotpy ifdotpy commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner

Remove the whole PROGRESS.md feature: Studio no longer asks lead agents to write <stateDir>/progress/<agentId>/PROGRESS.md (VM leads: <layrHome>/.studio/progress/<id>/PROGRESS.md), and no longer watches, reads, measures or shows it.

Removed

  • Web renderer: the progress panel above the composer (AgentPanel, ProgressMarkdown, progress cache, layout reports, the "progress hidden" preference scope), and its specs and status notes.
  • client-core: the progress module (progress.read, progress.peek, progress.cache_*).
  • HTTP API: GET /api/panel, POST /api/panel/layout, the schemas ProgressPanel, PanelLayoutBody, PanelLayoutReport, PanelLayoutResult, and the panel member of ResourceRef. Changed in openapi.json, then pnpm run contracts:generate (new api.ts and schema hash). The Python models, the constraint baseline (acceptedOperationRemovals) and the HTTP conformance ledger follow.
  • Rust runtime: progress_panel.rs, progress_watch.rs, the native progress watch in the sync stream, the ProgressWatch port, the progress-layout server subcommand, the progress folder creation in workspace readiness, the progress folder as a sandbox writable root (thread config, turn sandbox, monitor commands), the progress text in developer instructions and turn context, progress_template in preparation-resources.json, and the shared values MAX_PROGRESS_FILE_BYTES, PROGRESS_FILE_LIMIT_LABEL, PROGRESS_LAYOUT_*.
  • Python server: codex_progress.py, codex_progress_watch.py, codex_progress_layout.py, codex_panel.py, the routes, the sync hub watchdog, and the same instruction, sandbox and context parts. VM guest: the agent.progress method and the .studio/progress provisioning in line.bind.
  • Skills and docs: the text that asks leads to update PROGRESS.md, codex-workspace/references/panel.md, and the sections in docs/orchestration.md, docs/sync-protocol.md, docs/product-guide.md, docs/windows-server.md, the runtime READMEs and the operation map in docs/plans/rust-runtime-work-breakdown.md. docs/verification stays as history.
  • Tests that exist only for the feature (progress file, runtime, layout and panel contracts, watcher tests, panel browser specs). Tests that used panel only as a sample resource kind now use queue.

Kept for compatibility

  • The orchestration_context tool schema does not change. Its topic enum still has panel. A catalog change would change the catalog digest of live threads and start a catalog refresh. topic=panel now returns PROGRESS_FILE_RETIRED_NOTICE, and topic=background no longer has the workflow entry.
  • origin/main added PanelGuidance, a pre-SQL read of panel.md for topic=panel. It is removed with the panel guide. The test callback_panel_returns_the_retired_progress_notice checks that topic=panel returns the notice and reads no skill file.
  • PROGRESS_FILE_RETIRED_NOTICE is owned by runtime-schema/src/shared_values.rs. The Python copy has a parity check in shared-values-contract.py.
  • A lead whose known context versions have progressFile (old leads) gets one turn-context block [Studio progress file retired] with the notice, and the manifest records the notice version, so it is sent once. New leads and workers never get it.
  • Stored data stays readable and unused. Nothing is deleted: old <stateDir>/progress/ folders, agent PROGRESS.md files, PROGRESS.layout.json, the client client_progress table, preparedContext.versions.progressFile and codex-progress-hidden:* preferences. The server still accepts that preference key, so old stored preferences pass validation. No migration.
  • The provider launch command of Codex and Claude app-servers does not change.

Resume safety

The receipt check (thread_prepare::reduce_receipt, ThreadRejection::Stale) compares the prepared and current snapshots by receipt identity, agent, epoch, account, connection, cwd, settings, thread and attempt. It does not compare developer instructions or guidance. Both snapshots come from the same build. The role text is frozen in nativeRoleGuidance, so old leads keep their role text. A changed instruction digest of a loaded thread only causes a new thread/resume, as for any settings change.

New test account_worker::tests::closure::a_lead_prepared_with_the_old_progress_file_resumes_its_thread: a lead with an old frozen role that names PROGRESS.md, a preparedContext and a delivered manifest with progressFile, and a progress/<id>/PROGRESS.md file. It checks:

  • The resume has no progress text and no progress writable root.
  • The reply is accepted with no error.
  • The first turn has the notice once. Its sandbox writes only the cwd.
  • The second turn has no notice.
  • The old file content is unchanged.

The turn-preparation corpus has Rust-vs-Python cases for the notice: old version gives the notice, retired version gives none, worker gives none.

Checks

  • just test-fast on 66f87c4 (rebased on origin/main 96c6b55): passed, 825 s.
  • just test-comprehensive on 6a26b98 (on origin/main 69637aa): every step passed except one load timeout.
    • Steps that passed: server 6257 passed, 0 failed, 225 skipped; server bridge 103 passed; renderer build; Rust Clippy.
    • Rust nextest: 3092 passed, 1 failed. The failure was studio-runtime-server::integration wave7_p01_registry::registry_drains_strict_account_pages_after_one_global_wake, a 23 s wait that ran out at load about 140. Alone it passed 2 of 2 (13 to 14 s).
    • The recipe stops at the first failed step, so I ran its remaining steps one by one on the same commit, and all passed: web typecheck:focused, test:api-contract-types, typecheck:client-helper, web vitest unit (1241 passed, 2 skipped), tooling (36 passed), browser smoke (9 passed).
  • Between the two runs, origin/main got b7c5c05 and 96c6b55, which do not touch files of this PR. The rebase had no conflicts, and just test-fast passed again on the new commit.
  • Focused Rust tests on 6a26b98: 138 passed. They include callback_panel_returns_the_retired_progress_notice and a_lead_prepared_with_the_old_progress_file_resumes_its_thread.
  • studio-contracts --check: passed.
  • An earlier just test-fast run on 6a26b98 failed in tooling tests only because the shell had /usr/bin/python3 (3.9) first on PATH. It passed with Homebrew Python first, and origin/main failed the same way with 3.9.

Earlier runs: on 7e4b32043 and 7be4fc8f0 at load near 200, rust-messages-http-contract.py::MessagesHTTPContract.test_managed_send_matches_python_rows_for_every_delivery_and_replay failed once in the comprehensive server step and once alone. At lower load the same suite file passed 4 of 4 on the rebased branch and 1 of 1 each on 7e4b32043, 815919e and 23fc606.

@ifdotpy
ifdotpy force-pushed the remove/progress-md branch 2 times, most recently from 3097560 to 6a26b98 Compare October 22, 2026 14:56
Studio no longer asks lead agents to write a per-agent PROGRESS.md file,
and no longer watches, reads, measures or shows it.

Removed: the progress panel in the web renderer and its client-core
cache, GET /api/panel, POST /api/panel/layout, the "panel" sync
resource, the native progress-file watchers, the progress-layout server
subcommand, the progress folder writable root, the progress text in
developer instructions and turn context, the codex-workspace panel
guide, the skill text that asked leads to update the file, and their
tests and docs, in the Rust and Python servers and the VM guest.

Kept for compatibility: the orchestration_context tool schema is
unchanged, so catalog digests of live threads do not change; topic
panel now returns PROGRESS_FILE_RETIRED_NOTICE. A lead whose known
context has a progressFile version gets the notice once in its turn
context. Old progress folders, agent files, client cache rows and
preferences stay untouched and unused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ifdotpy
ifdotpy force-pushed the remove/progress-md branch from 6a26b98 to 66f87c4 Compare October 22, 2026 16:05
@ifdotpy
ifdotpy merged commit ae891c4 into main Oct 22, 2026
8 checks passed
@ifdotpy
ifdotpy deleted the remove/progress-md branch October 22, 2026 16:05
nowaalex pushed a commit that referenced this pull request Oct 22, 2026
Recapture stream expectations after PR #409 replaces panel resources with queue.

Keep the desktop live-provider projection and both sync stream document
comparisons without interpreter startup. Remove the unused differential print
probe and Python request-corpus generator; update the crate test instructions.
Dropped coverage: live Python differential comparisons and corpus regeneration
from Python (the generated corpus was never committed).

Fixture generator command (run before oracle removal at ae891c4):
python3 - <<'PY'
import json, re, subprocess
from pathlib import Path
root = Path('workspaces/runtime/packages')
interpreter = subprocess.check_output([
    'python3', str(root / '../apps/server/src/codex_python.py')
], text=True).strip()
source = subprocess.check_output([
    'git', 'show',
    'ae891c42:workspaces/runtime/packages/runtime-http/src/routes/sync_stream.rs'
], text=True)
fixtures = root / 'runtime-http/src/routes/fixtures'
fixtures.mkdir(exist_ok=True)
for name, mode in [
    ('stream_documents_match_python_resource_hub', 'documents'),
    ('post_publication_documents_match_python_with_interleaved_consumers', 'stream'),
]:
    body = source.split('async fn ' + name, 1)[1].split('let cases = json!(', 1)[1].split('\n        ]);', 1)[0] + '\n        ]'
    rate = dict(turnId='turn', active=False, estimated=True, rate=1.25, outputTokens=400)
    def rates(match):
        value = int(match[1])
        return json.dumps(dict(rates=dict(agent=dict(turnId='turn', active=True, estimated=False, rate=value, outputTokens=value)), teams={}))
    body = re.sub(r'test_rates\((\d+)\)', rates, body)
    body = body.replace(':rate}', ':' + json.dumps(rate) + '}')
    output = subprocess.check_output([
        interpreter, str(root / 'runtime-services/tests/wave7_p14_oracle.py'), mode
    ], input=json.dumps(json.loads(body)), text=True)
    (fixtures / ('sync_stream_' + mode + '.json')).write_text(json.dumps(json.loads(output), indent=2) + '\n')
PY

Desktop reuses the identical projection.json fixture owned by the
runtime-services provider_versions tests (A4e lane).
nowaalex pushed a commit that referenced this pull request Oct 22, 2026
Freeze reference outputs as JSON and transcript snapshots as text SQL/blob
seeds. Reuse the canonical Rust monitor seed and A4e rules fixture, retaining
every Rust behavioral assertion and both default/full corpus matrices.

Rebase the lane onto origin/main ae891c4 and the A4e dependency commits
48d3c106 and 3306e045 (rebased as e4e6832 and 20ab306). Recapture every
A4g oracle against main's Python implementation after PR #409 removed the
panel resource. P14 now has 278 default cases and 316 full cases.

Generator command, from a separate untouched ae891c4 checkout:
cd /work/ssd2/worktrees/py-a4g-main-oracles
python3 /home/alex/.local/state/codex-agents/issue373/generate-a4g-main409.py
The generator resolves the prepared interpreter, uses isolated provider homes,
saves JSON, and dumps SQLite as text SQL. FTS content is preserved in its
shadow tables. Restore WAL mode and clone frozen blob bytes per test.

Verification in fresh /work/ssd2/worktrees/py-a4g-main409, outside the original
worker checkout, under CODEX_AGENTS_PYTHON=/nonexistent:
focused nextest 67/67 pass in default mode and 67/67 pass with
STUDIO_RUNTIME_CORPUS_FULL=1. Oxfmt, Rustfmt and git diff --check pass.
Broad clippy-runtime/test-fast/test-comprehensive gates belong to the lead.
Dropped coverage: live Python execution/comparison only; no Rust assertions.
The retired panel case was removed by upstream main, not this migration.
nowaalex pushed a commit that referenced this pull request Oct 22, 2026
Replace turn lifecycle, compiler, thread preparation, submission-gate, P08,
and P12 oracle processes with captured JSON data and native fixture readers.
Preserve the complete corpora, transaction/crash tests, typed refusal locations,
and literal/token boundaries. Register the compact-fixture guard in test-fast.
Correct pre-existing full-corpus expectations for the bound connection and
located epoch refusals; scheduler startup preserves malformed agent rows.

Capture base: ae891c4 (#409 included).
Generator commands (run before backend deletion):
  python3 /tmp/generate-a4c.py
  python3 /tmp/generate-a4c-lifecycle.py
  STUDIO_RUNTIME_CORPUS_FULL=1 cargo test -p studio-runtime-services --test integration -- --test-threads=1
The last command recorded thread_prepare/oracle.py and submit_gates/oracle.py
JSONL requests and responses through temporary test-only recorder hooks.
  python3 /tmp/compact-a4c.py <captured JSON paths>
  python3 /tmp/recompact-a4c.py <captured JSON paths>
  python3 /tmp/fix-a4c409-fixtures.py
Generator sources and original JSONL inputs are retained in the session's
issue373/py-a4c-generators evidence directory. The frozen JSON uses repeat,
concatenation, reference and object-extension descriptors; it has no runtime
Python dependency. Existing Python script deletion belongs to the lead.

Dropped coverage: obsolete Python-only guard/invariant self-tests, runtime
cross-language clock-shape/revision audits, and redundant malformed-tool oracle
comparisons. Rust malformed-tool assertions and P08 receipt-signature assertions
remain. All lifecycle transition vectors remain, including malformed-row safety.

Validation: 37 focused integration tests pass with
CODEX_AGENTS_PYTHON=/nonexistent and STUDIO_RUNTIME_CORPUS_FULL=1.
The lifecycle report contains all 183 vectors with no unexpected differences.
Cargo fmt, Oxfmt and Oxlint pass on the refreshed captures.
The lead owns test-fast, clippy-runtime and test-comprehensive on integration.

Current-main corpus coverage: 183 lifecycle, 275 compiler and 300 thread
preparation vectors, plus submission gates and P08/P12 assertions.
nowaalex pushed a commit that referenced this pull request Oct 22, 2026
Freeze the historical oracle outputs and closed SQLite state, retain the
Rust behavioral assertions, and remove unused browser/Claude oracle scripts.
The shared typed fixture loader restores SQLite durability settings. Keep the
existing full-corpus variants and typed title expectations/scenarios.

Fixtures were captured from base 96c6b55 using:
CODEX_AGENTS_PYTHON=/work/ssd2/py-a4d-capture/record-python CARGO_TARGET_DIR=/work/ssd2/targets/PY-A4d CARGO_BUILD_JOBS=4 timeout 1200 cargo nextest run -j 4 -p studio-runtime-services --test integration -E 'test(title_generation_) | test(voice_provider_corpus) | test(radio_policy_corpus) | test(wave7_p19_radio) | test(wave9_login_parity) | test(wave9_safety_buffering_executor) | test(payload_safety_oracle)' --profile ci
python3 /work/ssd2/py-a4d-capture/extract.py
python3 /work/ssd2/py-a4d-capture/freeze-radio.py
Full variants: rerun the original wave7_p19_radio and payload_safety_oracle test binary filters with STUDIO_RUNTIME_CORPUS_FULL=1 through the same recording interpreter, then select the full records into expected-full.json and payload_safety_*-full.json.
The recorder invokes each original oracle and saves stdout, SQLite iterdump,
and payload blob bytes. Generation scripts and raw records are retained at
/work/ssd2/py-a4d-capture for audit; no runtime Python dependency is retained.

Before: CODEX_AGENTS_PYTHON=/nonexistent title_generation_corpus fails because
the guarded Python oracle cannot start. Original capture run: 70 passed.
After: CODEX_AGENTS_PYTHON=/nonexistent selected integration corpus: 70 passed, including the checkout-normalization guard.
Full-corpus plus browser/Claude selection: 46 passed with Python unavailable.
Dropped coverage: obsolete radio oracle subprocess timeout handling; unused
browser/Claude oracle scripts had no registered Rust callers. The Rust radio
boundary, title effects, voice receipts, login replay and safety checks remain.

Validation: cargo fmt -p studio-runtime-services -- --check; selected Oxfmt
checks; repositoryTemporaryStateFindings returns no findings. The integration
lead owns test-fast, clippy-runtime and test-comprehensive on the batch.

Rebased onto ae891c4 (#409).
Regenerated the developer-instruction-dependent safety fixture from the main
oracle using: python3 /work/ssd2/py-a4d-capture/recapture-current-main.py
The frozen database SQL is unchanged; other captures contain no turn/developer
instruction payloads and retain their historical oracle outputs.

Make the frozen safety comparison portable across worktrees. The complete
provider-params assertion failed at stage 5 because the skill-source headers
embedded the original checkout path. A fresh-worktree workspace nextest run
with a real interpreter reproduced it (10 pass, 1 failure). Its left/right
instruction diff changes only Source and Shared tool guidance path roots.
Normalize those exact headers while keeping skill identity, instruction body
and all method-specific fields compared. Decode the volatile envelope through
a typed struct. Add a deterministic fast-tier guard for both normalization and
retained content/parameter differences; leave frozen expectations unchanged.

Corrected fresh-worktree workspace selection: 12 passed with real Python.
Local no-Python selection: 70 passed. Facade tooling tests: 13 passed.
Rebased full/browser selection: 46 passed; source unit selection: 31 passed.
Evidence: /work/ssd2/py-a4d-capture/instruction.diff and fresh-workspace-*.log.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant