Thread view: open threads with a byte-budgeted history window and page older history on scroll #226

Summary

Opening a thread sends its whole history: every message, every tool activity with its payload, every plan and every checkpoint. That happens on first open, on a reload with no cache, and again whenever a resume can't answer unchanged. A long agent thread can be megabytes, and on a remote or slow link the chat stays blank until all of it arrives, even though the timeline first shows only the last few turns.

Open threads with a byte-budgeted window of recent history, and load older history in pages when the user scrolls up, through a new history-page RPC. Measure with #225 first.

Current behavior (verified at a0ea849ef92b)

Server

• subscribeThread (thread_stream resume handling at apps/server/src/production/orchestration_rpc.rs:1422-1441) sends unchanged when the client's resume revision is still current. Otherwise it sends a full snapshot, then field/array patches (thread_patch, :1325-1373; 100 ms coalesce, :1172).
• The snapshot is thread_snapshot_value (:1572-1616) over load_thread_snapshot (apps/server/src/orchestration/engine.rs:8607), which loads all of the thread's rows. thread_detail_rows (orchestration_rpc.rs:1738-1773) keeps every row for the thread, and thread_detail (:1618-1726) serializes them all with no limit:
  • all messages with full text and attachments (:1622-1666);
  • all activities with their full payload (:1667-1686, payload at :1679);
  • all proposed plans with full markdown (:1687-1705);
  • all checkpoints with their full files lists (:1706-1724; capping those is in Thread loading: resume cached threads as unchanged after a cold start, cap checkpoint file lists, and compress HTTP assets #227);
  • reasoning rows (:1589-1614).
• Revisions are keyed per server process (ThreadRevisionKeys, :1185-1190), so a restarted server always sends a full snapshot.

Contract

• OrchestrationThread (packages/contracts/src/orchestration.ts:533-565) holds messages, reasoning, proposedPlans, activities and checkpoints as plain arrays, with no notion of "more history on the server".
• OrchestrationSubscribeThreadInput (:704-720) has threadId, patches and resume. OrchestrationThreadStreamItem (:1667-1691) is snapshot | event | patch | unchanged.

Client

• packages/client-runtime/src/state/threads.ts holds and caches the whole thread (persist → cache.saveThread, :85-99) and applies items with threadReducer.ts.
• The timeline is virtualized with LegendList (apps/web/src/components/chat/MessagesTimeline.tsx:32, list at :678), so it already renders only what's on screen, but it needs the whole thread in memory.
• Features that read the whole loaded thread today: Find in thread (apps/web/src/components/chat/threadFind/, which searches virtualized and folded rows, Find in thread: search a chat's messages and plans, including virtualized and folded rows #175), the "Worked for" / "+N previous tool calls" folding (ChatView.logic.ts), turn diff summaries (apps/web/src/hooks/useTurnDiffSummaries.ts) and the Plan panel.

Not measured. No BiBCode numbers yet for thread-open bytes on a long thread. #225 produces them.

Proposed change

1. Window on open. The first snapshot holds the newest history that fits all of these: about 1 MiB of serialized JSON, 10 turns, and 500 rows (messages + activities + plans + reasoning). Always include the latest turn in full, whatever its size, and anything pending (approvals, queued messages, a streaming message).
2. Long turns are paged inside the turn. When one turn alone is over the budget (a 900-row tool run), send its newest rows plus a count of the older rows, and fetch those in 75-row pages.
3. Window metadata. The snapshot says whether older history exists and gives an opaque cursor for it (for example history: { hasOlder, cursor, omittedTurns, omittedRows }). Patches keep working on the window. Rows the window leaves out are never sent as a removal.
4. History-page RPC, for example orchestration.getThreadHistoryPage { threadId, cursor, limit } → rows older than the cursor plus the next cursor. It needs the same auth scope as orchestration.subscribeThread (apps/server/src/auth/scope.rs:57), registration through the rpc/session.rs wrappers, and check:contracts fixtures. It's a database read, so it doesn't belong in is_long_running, unless the measurement says otherwise.
5. Client timeline. Scrolling near the top of the timeline loads the next older page and keeps the reading position fixed (no jump). A "Load earlier messages" row stays as an explicit fallback. Loaded older pages are kept in the thread state for the session; whether they're cached is an open question.
6. Find in thread (Find in thread: search a chat's messages and plans, including virtualized and folded rows #175) and other whole-thread features must state what they cover: either they ask the server, or they say "searched loaded history" and offer to load the rest. Decide in the design doc.
7. Compatibility. The client asks for a window with an input flag on subscribeThread (as patches does). An older server ignores the flag and sends the full thread, which the client keeps handling. A newer server sends a full thread to a client that doesn't send the flag.

T3 Code reference (their claims, not measured by us)

• pingdotgg/t3code#24589: bounded thread history window. They report p99 thread-open size 1.42 MB → 768 KB, max 5.24 MB → 2.35 MB, and a 900-row turn going from 1.07 MB to 120 KB.
• #26240: large work groups load on expand, with 150 items inline.
• The 1 MiB / 10 turns / 500 rows / 75-row page numbers are T3's. They're a starting point to check against our own measurements, not something we've confirmed fits BiBCode's data.

Acceptance criteria

• First: Performance: measure load bytes and time on a 3G link and gate them with a CI byte budget #225's baseline exists. The PR shows the 3G "cold connect + open the big thread", "thread open" and "reconnect" rows before and after, and lowers the byte budgets for the thread snapshot.
• A design doc docs/plans/YYYY-MM-DD-thread-history-window-design.md, approved before code. It settles the budget numbers, the cursor shape, how patches and revisions behave at the window's edge, the Find in thread behavior, and the caching of older pages.
• Rust tests in apps/server/src/production/orchestration_rpc.rs (or apps/server/tests/):
  • a thread over the budget opens with a window under 1 MiB / 10 turns / 500 rows, and hasOlder is true;
  • a single 900-row turn opens with its newest rows and an older-row count;
  • paging with the cursor returns every older row exactly once, in order, and ends with hasOlder: false;
  • a new message while a window is open arrives as a patch, and nothing outside the window is sent as removed;
  • resume with a current revision still answers unchanged;
  • a subscriber that doesn't send the window flag gets the full thread, byte-identical to today.
• vp run check:contracts: new RPC and stream-shape fixtures in packages/contracts/fixtures/ (including thread-stream/), with rpcRustParity.test.ts and threadStreamParity.test.ts passing.
• Client-runtime tests in packages/client-runtime/src/state/: threads-sync.test.ts covers opening a window, loading two older pages, a patch arriving between pages, and "resumes from the revision it applied and keeps its thread when unchanged" still passing. A test counts requests: scrolling to the top of an N-page thread sends exactly N−1 page requests.
• Web: MessagesTimeline tests (happy-dom, run from apps/web) check that loading an older page keeps the first visible row in place, and that "Load earlier messages" works with no scroll. threadFind tests cover the chosen behavior for unloaded history.
• Desktop e2e: vp run test:ui:desktop, including apps/desktop/e2e/specs/document-navigation.e2e.ts, chat-reasoning-rows.e2e.ts and chat-activity-panel.e2e.ts.
• Manual: on a remote environment at 3G, a long thread paints its last turns quickly, and scrolling up loads older turns without jumps. Existing behavior unchanged on a short thread.
• If the client cache's shape changes, it has a migration (clientStateMigrations.ts) and a test.
• Gates: cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace, vp check (zero warnings), vp run typecheck, vp run check:contracts, Release Smoke.
• UI.md and vercel-react-best-practices reviews are recorded. docs/testing/ runbooks are reviewed and updated or stated "remain accurate".
• Profiler coverage line in the delivery report; the new RPC is registered through rpc/session.rs. Message and activity text never reaches profiler records or logs.

Living docs to update

• docs/architecture/rpc-and-orchestration.md (snapshot section around :2548-2560): the window, the cursor and the page RPC.
• docs/architecture/connection-runtime.md (:43-45, "resuming from the revision it holds"): what a resumed client holds when it has a window.
• docs/user/workspace-ui.md (timeline): older history loads on scroll; what Find in thread covers.
• docs/reference/encyclopedia.md: "history window".

Not in scope

• The sidebar / shell stream (Orchestration: shell stream sends only changed threads instead of the full list every 500 ms #223).
• Checkpoint file lists (capped in Thread loading: resume cached threads as unchanged after a cold start, cap checkpoint file lists, and compress HTTP assets #227).
• Lazy timeline images (Files: preview images, SVG, PDF and audio through the capped asset route; lazy-load timeline images #224).
• Archived threads.

Related

• Performance: measure load bytes and time on a 3G link and gate them with a CI byte budget #225: the measurement this depends on.
• Orchestration: shell stream sends only changed threads instead of the full list every 500 ms #223: the shell-side counterpart (only changed threads, finished-thread cap).
• Thread loading: resume cached threads as unchanged after a cold start, cap checkpoint file lists, and compress HTTP assets #227: smaller load fixes (cached-thread resume, checkpoint cap, HTTP compression).
• Each reconnect sends two subscribeThread requests and can resend a full snapshot twice #197 (closed): reconnect resent full thread snapshots; fixed with unchanged.
• client.interaction: scrolling up a long chat timeline saturates the main thread #103 (closed): scrolling up a long timeline saturated the main thread.
• Find in thread: search a chat's messages and plans, including virtualized and folded rows #175 (closed): Find in thread.
• Cap Claude tool-result bodies on the orchestration event #148 (closed): caps Claude tool-result bodies on the orchestration event, which already bounds one source of large activities.
• macOS x64: PageUp during the sent-row glide sometimes doesn't page up #207, Timeline: a scroll key during the sent-row glide can jump the reading line about 144 px #214: open timeline scroll issues to keep in mind when adding scroll-triggered loading.

Open questions

• Are T3's budget numbers right for BiBCode, or should they come from our measured p50/p99 thread sizes?
• Should loaded older pages be saved in the client thread cache, or only the window?
• Should Find in thread search the server for unloaded history, or search loaded history and say so?
• How should the thread revision treat the window: one revision for the whole thread, or for the window only?