diff --git a/.changeset/remove-repl-tool.md b/.changeset/remove-repl-tool.md new file mode 100644 index 00000000..f3e9186c --- /dev/null +++ b/.changeset/remove-repl-tool.md @@ -0,0 +1,29 @@ +--- +"@automatalabs/mcp-server": major +"@automatalabs/workflows": major +"@automatalabs/acp-agents": patch +"@automatalabs/pi-acp": patch +--- + +Remove the `repl` MCP tool and the `@automatalabs/repl-engine` package behind it. + +The MCP server's model-facing surface is now the `workflow` tool plus the capability-gated `workflow_monitor` view (and the app-only `workflow-events`, `workflow-runs`, `workflow-notifications` tools). The interactive per-project QuickJS REPL, its broker, its snapshot store, and everything in the server that existed only to serve it are gone. `@automatalabs/repl-engine` is deleted from the workspace and will receive no further releases; no other package imported it. + +**`@automatalabs/mcp-server` (breaking)** + +- The `repl` tool is no longer registered; `SERVER_INSTRUCTIONS` describes `workflow` only. +- Removed exports: `replToolInputShape`, `replToolOutputShape`, `ReplToolOptions`, `createReplProjectState`, `ensureReplWorkspace`, `disposeReplProjectState`, `resetReplProjectState`, `renameAsideNeverOverwriting`, `ReplProjectState`, `ReplPresenceLedger`. +- `CreateWorkflowServerOptions` drops `replRunner`, `replPresence`, `replEvalBreakChannel`, `replDrainBoundMs` and `disconnectReplClientOnClose`. `replClientId` had one non-REPL job — scoping `workflow_monitor` notification claims per legacy-era MCP client — and is kept under its honest name, `clientId`. +- `WorkflowServerControl` drops `replBreakUrl()` (previously required), `replDefaultProjectDir()` and `disposeReplEvalBreakChannel()`. The shutdown hook the stdio entry used through the last of those is now the generic optional `dispose()`. +- The in-process stdio entry serves over the SDK's `StdioServerTransport`; the worker-thread relay transport that existed to break a synchronous eval out of band is removed, and the shim no longer intercepts `tools/call` to fire it. +- The daemon drops the REPL client-presence ledger and drain: `CreateDaemonOptions.replRunner` / `replDrainBoundMs` / `sessionTtlMs` / `evalBreakChannel`, `DaemonHandle.activeReplDrainCount()`, `WorkflowProjectRegistry.disposeReplStates()`, `ProjectContext.repl`, `DaemonInfo.replBreakUrl`, the `SessionRegistry` presence hooks (`onConnectionOpened`, `onLastConnectionClosed`, `onSessionDeleted`) and `evictDrainable`'s `keep` veto. Daemon idleness is sessions, runs and in-flight requests. +- Removed environment knobs: `AGENTPRISM_REPL_EVAL_TIMEOUT_MS`, `AGENTPRISM_REPL_DRAIN_BOUND_MS`. +- Existing per-project `repl/` stores on disk are left untouched and are no longer read. + +**`@automatalabs/workflows` (breaking)** + +- The MCP server bundled behind `npx @automatalabs/workflows mcp` no longer serves the `repl` tool, and the package no longer depends on `@automatalabs/repl-engine`. The programmatic SDK is unchanged. + +**`@automatalabs/acp-agents`, `@automatalabs/pi-acp`** + +- Documentation only: comments and README passages that attributed `InteractiveSession.awaitCurrentTurn()`, the `_session/loaded_turn` extension, the turn-text passthroughs, `onHandoff` and `runner.defaultBackendId()` to "the REPL broker" now describe them as the host re-attach surface they are. Those SDK and wire surfaces are unchanged and remain supported. diff --git a/AGENTS.md b/AGENTS.md index 01d726bd..e598bf0f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -9,7 +9,7 @@ Before changing code, read the relevant parts of: - [`CONTRIBUTING.md`](CONTRIBUTING.md) — development, tests, generated artifacts, dependency gates, attribution, PRs, and releases. - [`README.md`](README.md) — product surface and package map. - [`docs/api.md`](docs/api.md) — supported integration APIs. -- [`docs/authoring/`](docs/authoring/) — canonical workflow and REPL authoring documentation shipped through MCP. +- [`docs/authoring/`](docs/authoring/) — canonical workflow authoring documentation shipped through MCP. Then read the code. The source and its tests are the current state; prose describes it and never governs it. When prose and code disagree, fix the prose. @@ -36,20 +36,19 @@ Do not add temporary compatibility layers unless the user explicitly requests on ## Architecture and package boundaries -This is a pnpm monorepo of ten `@automatalabs/*` packages: +This is a pnpm monorepo of nine `@automatalabs/*` packages: - `shared-types`: shared seams and wire/result types. - `workflow-engine`: deterministic workflow execution, journaling, resume, checkpoints, and isolation. - `acp-agents`: ACP client and backend integration for Claude, Codex, OpenCode, pi, and custom agents. - `acp-server`: connection-pinned ACP proxy and backend-discovery server. - `workflows`: the public SDK facade composing the engine and ACP runner. -- `repl-engine`: persistent QuickJS REPL orchestration over the same backend stack. -- `mcp-server`: MCP composition root exposing `workflow`, `repl`, the Apps-capable `workflow_monitor`, and SEP-2640 authoring skills. +- `mcp-server`: MCP composition root exposing `workflow`, the Apps-capable `workflow_monitor`, and SEP-2640 authoring skills. - `pi-acp`: in-process pi ACP server. - `codex-acp`: published fork maintained as a non-squashed upstream subtree. - `agentprism-otel`: optional observability bridge. -Keep `workflow-engine` backend-agnostic and `acp-agents` engine-agnostic; they meet through `shared-types`. The primary runtime direction is `mcp-server → {workflows, repl-engine, shared-types}`, `acp-server → acp-agents`, `workflows → {workflow-engine, acp-agents, shared-types}`, `repl-engine → {workflows, acp-agents, shared-types}`, and `acp-agents → {codex-acp, pi-acp, shared-types}`. +Keep `workflow-engine` backend-agnostic and `acp-agents` engine-agnostic; they meet through `shared-types`. The primary runtime direction is `mcp-server → {workflows, shared-types}`, `acp-server → acp-agents`, `workflows → {workflow-engine, acp-agents, shared-types}`, and `acp-agents → {codex-acp, pi-acp, shared-types}`. For MCP server work, preserve the deliberate SDK boundary: production server code uses the split MCP SDK v2 packages, legacy 2025 and modern `2026-07-28` traffic share one implementation through era-specific transport seams, and no v1 SDK object may be passed into a v2 API. `@modelcontextprotocol/ext-apps` remains browser-build/test-side; production server code must not import its v1 server helpers. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ac47ec84..d1a6e790 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,6 +1,6 @@ # Contributing -This is a **pnpm workspace** (monorepo) of ten packages under the `@automatalabs` scope. The user-facing overview is in [`README.md`](README.md); the integration API is in [`docs/api.md`](docs/api.md). +This is a **pnpm workspace** (monorepo) of nine packages under the `@automatalabs` scope. The user-facing overview is in [`README.md`](README.md); the integration API is in [`docs/api.md`](docs/api.md). ## Prerequisites @@ -30,14 +30,13 @@ pnpm typecheck # pnpm -r exec tsc --noEmit | `packages/workflow-engine` | The deterministic engine (realm, parallel/pipeline, journal/resume, budget, worktree). | | `packages/acp-agents` | ACP client + Claude/Codex/OpenCode/pi/custom backends (the `AgentRunner` implementation, pooling, auth/session lifecycle). | | `packages/acp-server` | Connection-pinned ACP V1 proxy over stdio, Streamable HTTP, or WebSocket, with extension-negotiated backend discovery (bin `agentprism-acp-server`). | -| `packages/mcp-server` | The stdio MCP server / composition root (bin `agentprism-workflow`; the `workflow` and `repl` tools plus the Apps-capable `workflow_monitor` launcher — no auth tools). | +| `packages/mcp-server` | The stdio MCP server / composition root (bin `agentprism-workflow`; the `workflow` tool plus the Apps-capable `workflow_monitor` launcher — no auth tools). | | `packages/workflows` | The importable SDK facade. | | `packages/agentprism-otel` | Optional OpenTelemetry bridge for `WorkflowManager` events. | -| `packages/repl-engine` | The REPL orchestrator engine: persistent JS REPL in a QuickJS-in-WASM VM (workspace lifecycle, eval + job drain, per-VM memory limits, per-eval interrupts). | | `packages/pi-acp` | Standalone in-process ACP server and library adapter for the pi coding agent. | | `packages/codex-acp` | Our codex-acp fork (full upstream history, non-squashed subtree): the ACP server the Codex backend spawns. | -`workflow-engine` and `acp-agents` are **siblings** — neither imports the other; they meet only at the `AgentRunner` seam in `shared-types`. `workflows` is the single facade that composes them; `mcp-server` builds on `workflows`, while `acp-server` builds directly on `acp-agents`. So the primary dependency direction is `mcp-server → workflows → { workflow-engine, acp-agents, shared-types }` and `acp-server → acp-agents`. `agentprism-otel` is an independent leaf with an `@opentelemetry/api` peer dependency; it observes the manager structurally and is not in that runtime chain. `repl-engine` is **not** a leaf: it composes the `quickjs-wasi` shim with `workflows`, `acp-agents`, and `shared-types`, and its `repl` MCP tool is registered in `mcp-server` (which depends on `repl-engine`) — the `repl-orchestrator` roadmap phase is implemented (`docs/roadmap/repl-orchestrator.md`) and the package is published independently. +`workflow-engine` and `acp-agents` are **siblings** — neither imports the other; they meet only at the `AgentRunner` seam in `shared-types`. `workflows` is the single facade that composes them; `mcp-server` builds on `workflows`, while `acp-server` builds directly on `acp-agents`. So the primary dependency direction is `mcp-server → workflows → { workflow-engine, acp-agents, shared-types }` and `acp-server → acp-agents`. `agentprism-otel` is an independent leaf with an `@opentelemetry/api` peer dependency; it observes the manager structurally and is not in that runtime chain. ### Conventions diff --git a/README.md b/README.md index b413ae13..25e872ec 100644 --- a/README.md +++ b/README.md @@ -15,9 +15,9 @@ Run **dynamic, multi-agent workflow scripts** — `agent()`, `parallel()`, `pipe **Your agent authors** a small JavaScript *script* (`export const meta`, then call `agent()` / `parallel()` / `pipeline()`); the engine runs it in a sandboxed realm, fanning each `agent()` call out to an [Agent Client Protocol](https://agentclientprotocol.com) (ACP) backend. It's available two ways: - **As a TypeScript SDK** — `@automatalabs/workflows` — embed the runner in your own program. -- **As a stdio MCP server** — `@automatalabs/mcp-server`, built on the SDK — expose `workflow` and `repl` tools to any MCP host (Claude Code, Zed, …). +- **As a stdio MCP server** — `@automatalabs/mcp-server`, built on the SDK — expose the `workflow` tool to any MCP host (Claude Code, Zed, …). -> All ten `@automatalabs/*` packages are **published on npm** — see [Install](#install). Two are primary workflow entry points: the `@automatalabs/workflows` SDK and the `@automatalabs/mcp-server` stdio server. `@automatalabs/acp-server` is the extension-aware ACP aggregation entry point. +> All nine `@automatalabs/*` packages are **published on npm** — see [Install](#install). Two are primary workflow entry points: the `@automatalabs/workflows` SDK and the `@automatalabs/mcp-server` stdio server. `@automatalabs/acp-server` is the extension-aware ACP aggregation entry point. --- @@ -117,8 +117,6 @@ One process plays **two protocol roles at once**: it's an **MCP server** (or a l The deterministic engine (sandboxed `vm` realm, `parallel`/`pipeline`, journal/resume, worktree isolation) is independent of *how* a single agent runs and of *how* the tool is exposed. -The MCP server also exposes a second, **interactive** route: the `repl` tool. Instead of running a deterministic script to completion, it holds a persistent **QuickJS-in-WASM VM per project** (the [`@automatalabs/repl-engine`](packages/repl-engine) tier), and the client's own agent writes live JavaScript that spawns subagents over the same ACP path — workspace state (bindings, pending calls, checkpoints, logged values) persisting between tool calls and across daemon restarts. Workflows is the batch orchestrator; `repl` is the live steering plane. See [The `repl` tool](packages/mcp-server/README.md#the-repl-tool). - --- ## Requirements @@ -162,7 +160,7 @@ These are the packages you interact with directly. The first two are the primary | Package | What it is | |---|---| | **`@automatalabs/workflows`** | The canonical public **SDK** — a thin facade that runs workflow scripts programmatically over the default ACP backend, and re-exports the supported engine + backend integration surface. Start here. | -| **`@automatalabs/mcp-server`** | The stdio **MCP server** (bin: `agentprism-workflow`) exposing the `workflow` tool (asynchronous run/resume, setup response, bounded status/result, permission response, stop, and an Apps monitor) and the `repl` tool (a persistent JavaScript REPL for live subagent orchestration) — built on `@automatalabs/workflows` and `@automatalabs/repl-engine`. | +| **`@automatalabs/mcp-server`** | The stdio **MCP server** (bin: `agentprism-workflow`) exposing the `workflow` tool (asynchronous run/resume, setup response, bounded status/result, permission response, stop, and an Apps monitor) — built on `@automatalabs/workflows`. | | **`@automatalabs/acp-server`** | The extension-aware **ACP proxy** (bin: `agentprism-acp-server`) over stdio, Streamable HTTP, or WebSocket: probe every configured backend on a discovery connection, then pin each operational connection to Claude, Codex, OpenCode, pi, or a custom ACP server. | | **`@automatalabs/pi-acp`** | The standalone stdio **ACP server** (bin: `pi-acp`) embedding the pi coding agent in-process; exact-pinned and spawned by the first-class `pi` backend. | @@ -172,17 +170,16 @@ One optional integration package attaches to the SDK's manager surface: |---|---| | **`@automatalabs/agentprism-otel`** | OpenTelemetry traces and metrics for a `WorkflowManager`; peer-depends only on `@opentelemetry/api` and no-ops when the host has no OTel SDK. | -The five packages below are **internal building blocks**. Most are composed by the SDK (`@automatalabs/workflows` → `workflow-engine`, `acp-agents`, `shared-types`); the exceptions are `@automatalabs/repl-engine`, which **depends on** the SDK and is composed by the **MCP server** (which registers its `repl` tool), and `@automatalabs/codex-acp`, which is spawned by `acp-agents`. You normally don't depend on any of them directly: `@automatalabs/workflows` is the public entry point for the supported orchestration surface. +The four packages below are **internal building blocks**. Most are composed by the SDK (`@automatalabs/workflows` → `workflow-engine`, `acp-agents`, `shared-types`); the exception is `@automatalabs/codex-acp`, which is spawned by `acp-agents`. You normally don't depend on any of them directly: `@automatalabs/workflows` is the public entry point for the supported orchestration surface. | Package | What it is | |---|---| | **`@automatalabs/acp-agents`** | The ACP client + Claude/Codex/OpenCode/pi/custom backends (the `AgentRunner` implementation, connection pooling, auth/session lifecycle, structured output, permissions, usage) and the `AcpAgent` SDK (one dedicated process per held-open agent, forks, cold reopen). Internal — public entry is `@automatalabs/workflows`. | | **`@automatalabs/workflow-engine`** | The deterministic engine: the script realm, `parallel`/`pipeline`, journal/resume, and worktree isolation. Internal — public entry is `@automatalabs/workflows`. | -| **`@automatalabs/repl-engine`** | The published REPL orchestrator engine: a persistent JavaScript REPL in a capability-free QuickJS-in-WASM VM (workspace lifecycle, eval + job drain, per-VM memory limits, per-eval interrupts, trap-free completion reads, the append-only call store and enveloped snapshots). Its `repl` MCP tool is registered in `mcp-server` (the roadmap's `repl-orchestrator`, phase E — implemented); it depends on `workflows`, `acp-agents` (subagents are ACP sessions), and `shared-types`. | | **`@automatalabs/codex-acp`** | The workspace fork of `agentclientprotocol/codex-acp` (imported with full history) — the ACP server the Codex backend spawns, baking turn-level `outputSchema` forwarding into its shipped dist. Consumed by `@automatalabs/acp-agents` as `workspace:*`; you never depend on it directly. | | **`@automatalabs/shared-types`** | The `AgentRunner` seam + shared types the others compose against. Internal — public entry is `@automatalabs/workflows`. | -Dependency direction: `mcp-server` → `{ workflows, repl-engine, shared-types }`; `acp-server` → `acp-agents`; `workflows` → `{ workflow-engine, acp-agents, shared-types }`; `acp-agents` → `{ codex-acp, pi-acp, shared-types }`; `repl-engine` → `{ workflows, acp-agents, shared-types }`. The SDK (`workflows`) is the single facade that composes the deterministic engine and the ACP backend, which meet only at the `AgentRunner` seam in `shared-types`. The engine never names a backend; the agents never know they're inside a workflow. `acp-agents` spawns the bundled `codex-acp` / `pi-acp` ACP servers as its Codex and pi backends. `repl-engine` composes the QuickJS-in-WASM shim with `workflows` (for the shared per-project key) and `acp-agents` (the REPL's subagents are ACP sessions against the same backends the SDK drives), and ships its `repl` tool in `mcp-server`. +Dependency direction: `mcp-server` → `{ workflows, shared-types }`; `acp-server` → `acp-agents`; `workflows` → `{ workflow-engine, acp-agents, shared-types }`; `acp-agents` → `{ codex-acp, pi-acp, shared-types }`. The SDK (`workflows`) is the single facade that composes the deterministic engine and the ACP backend, which meet only at the `AgentRunner` seam in `shared-types`. The engine never names a backend; the agents never know they're inside a workflow. `acp-agents` spawns the bundled `codex-acp` / `pi-acp` ACP servers as its Codex and pi backends. ### Published ACP registry @@ -523,7 +520,7 @@ both call rows and activity. Its structured payload, including `latestActivity`, UTF-8 bytes and its text at 8,192 bytes. Paused, failed, and aborted outcomes also include a redacted final-20 `logTail` immediately. -The model-facing tools are `workflow`, `repl`, and the capability-gated `workflow_monitor`; `repl` is a persistent QuickJS-in-WASM JavaScript VM (one per project) for live, stateful orchestration. The server also advertises `agentprism-workflow-authoring` through the MCP Skills Extension, and prompt-capable hosts get the compact user-controlled **`author-workflow`** MCP prompt (optional `task` argument). Backend auth belongs to the agents' credential sources (`claude /login`, `codex login`, `opencode auth login`, Pi provider environment keys, or `~/.pi/agent/auth.json`) — configured credentials need no extra step. An `AUTH_REQUIRED` fault pauses the workflow with `reason: "auth_required"` and a non-secret `authContext` naming the backend; configure that credential out-of-band, then call `{ "action":"resume", "runId":"…" }` for the paused source. Programmatic auth/provider management lives in the `@automatalabs/workflows` SDK runner APIs. +The model-facing tools are `workflow` and the capability-gated `workflow_monitor`. The server also advertises `agentprism-workflow-authoring` through the MCP Skills Extension, and prompt-capable hosts get the compact user-controlled **`author-workflow`** MCP prompt (optional `task` argument). Backend auth belongs to the agents' credential sources (`claude /login`, `codex login`, `opencode auth login`, Pi provider environment keys, or `~/.pi/agent/auth.json`) — configured credentials need no extra step. An `AUTH_REQUIRED` fault pauses the workflow with `reason: "auth_required"` and a non-secret `authContext` naming the backend; configure that credential out-of-band, then call `{ "action":"resume", "runId":"…" }` for the paused source. Programmatic auth/provider management lives in the `@automatalabs/workflows` SDK runner APIs. --- diff --git a/docs/api.md b/docs/api.md index b2d45d2c..f4833e47 100644 --- a/docs/api.md +++ b/docs/api.md @@ -1,6 +1,6 @@ # API reference -The integrator-facing surface of the `@automatalabs/*` packages, in one place. This documents the supported integration APIs; package barrels also expose lower-level protocol utilities for advanced hosts, which remain typed but are not all repeated here. Version references are current for `workflows` 0.54.0, `acp-agents` 0.41.3, `workflow-engine` 0.38.0, `shared-types` 0.32.0, `mcp-server` 0.34.0, `repl-engine` 0.4.6, `agentprism-otel` 0.1.2, `pi-acp` 0.6.1, and `codex-acp` 2.1.1. +The integrator-facing surface of the `@automatalabs/*` packages, in one place. This documents the supported integration APIs; package barrels also expose lower-level protocol utilities for advanced hosts, which remain typed but are not all repeated here. Version references are current for `workflows` 0.54.0, `acp-agents` 0.41.3, `workflow-engine` 0.38.0, `shared-types` 0.32.0, `mcp-server` 0.34.0, `agentprism-otel` 0.1.2, `pi-acp` 0.6.1, and `codex-acp` 2.1.1. Packages (all published to npm, Apache-2.0, ESM-only, Node >= 22): @@ -11,11 +11,10 @@ Packages (all published to npm, Apache-2.0, ESM-only, Node >= 22): | `@automatalabs/acp-agents` | The ACP runner: pooled Claude/Codex/OpenCode/pi ACP processes, model routing, structured output, events, interactive sessions, the no-prompt harness config catalog (`probeHarnessConfig`), and the [`AcpAgent` SDK](#acpagent-sdk) (one dedicated process per held-open agent, forks, cold reopen) | You want agent execution without the workflow engine | | `@automatalabs/acp-server` | ACP V1 proxy over stdio, Streamable HTTP, or WebSocket, with negotiated backend discovery and one backend pinned per operational connection | You want one extension-aware ACP endpoint for all configured backends | | `@automatalabs/shared-types` | The seam contracts: `AgentRunner`, `RunOptions`, `WorkflowError` (+ codes), workflow result/meta types | You implement a custom runner or need `instanceof WorkflowError` across packages | -| `@automatalabs/mcp-server` | Stdio MCP server (bin `agentprism-workflow`) exposing the `workflow` tool (asynchronous run/resume, setup response, bounded status/result, live permission response, stop, and an Apps monitor) and the `repl` tool (a persistent per-project JavaScript REPL for live subagent orchestration) | You drive workflows from Claude Code / an MCP client | +| `@automatalabs/mcp-server` | Stdio MCP server (bin `agentprism-workflow`) exposing the `workflow` tool (asynchronous run/resume, setup response, bounded status/result, live permission response, stop, and an Apps monitor) | You drive workflows from Claude Code / an MCP client | | `@automatalabs/agentprism-otel` | Optional OpenTelemetry bridge for `WorkflowManager` traces and metrics | Your host owns an OTel SDK and wants run/agent/tool observability | | `@automatalabs/pi-acp` | Standalone in-process pi coding-agent ACP server (bin `pi-acp`) with a side-effect-free library entry | You use the first-class `pi` backend or embed the ACP server directly | | `@automatalabs/codex-acp` | Fork of `@agentclientprotocol/codex-acp` adding turn-level `outputSchema` forwarding | Installed automatically by `acp-agents`; only pin it directly to override the version | -| `@automatalabs/repl-engine` | The REPL orchestrator engine: persistent JavaScript REPL in a QuickJS-in-WASM VM — workspace lifecycle, eval + job drain, per-VM memory limits, per-eval interrupts | You build a persistent-JS-REPL surface (this package is the engine tier under `mcp-server`'s `repl` tool — see [MCP server](#mcp-server)) | --- @@ -1328,7 +1327,7 @@ const forked = await runner.forkSession({ sessionId, cwd: "/abs/dir" }); `listSessions()` returns the SDK `ListSessionsResponse` (`sessions: SessionInfo[]`, plus `nextCursor?`); `deleteSession()` resolves to `void`. `loadSession()`, `resumeSession()`, and `forkSession()` return live `InteractiveSession`s tracked and released like `openSession()` sessions. Their signature is `(opts: ReattachSessionOptions) => Promise`: they accept the same session-scoped fields as `openSession()` plus the required `sessionId`, and `mcpServers` defaults to `[]` on the wire. For `loadSession()` and `resumeSession()`, that id is the session being reopened. For `forkSession()`, it is the **source** session id; ACP returns a **new** independent session seeded with the source's conversation context, and that new id is exposed as both `forked.sessionId` and `forked.sessionRef.sessionId`. -`loadSession()` registers the caller-supplied id before sending `session/load`, so replayed `session/update` history is accumulated and permissions during replay are routed. After it resolves, replay is visible in `session.text` / `session.history`. `InteractiveSession.awaitCurrentTurn()` resolves with the loaded session's founding turn (the turn that was in flight when the host died) — the REPL broker's re-attach arm — using the **`_session/loaded_turn` vendor extension** (the `_session/steering` precedent; advertised at initialize via `InitializeResponse._meta.loadedTurn.supported === true`, served by the in-repo `@automatalabs/pi-acp` and `@automatalabs/codex-acp`): right after the load response the seam asks `_session/loaded_turn/query` whether the founding turn is still running — `completed` (observably completed while the host was down; the replay's trailing assistant message is its FINAL message, resolved immediately with the real accumulated text, stop reason synthesized `end_turn`), `interrupted` (ended without a terminal message, nothing running — the safe-re-issue class), or `running` (kept attached, waiting for the authoritative `_session/loaded_turn/ended` notification — pushed with the stop reason, or the error, when the turn ends; a quiet gap is only a progress-stream gap, never terminal evidence, and the wait is bounded by `AGENTPRISM_ACP_LOADED_TURN_MAX_WAIT_MS`, default 15 min). A backend WITHOUT the extension (the built-in claude and opencode backends today) is classified by the **observation path** — the post-load continuation watch plus the replay probe under the **connection-death contract** (phase-F review round 2, restricted to the VERIFIED BUILT-INS in round 3 — a custom registry backend's connection-death behavior is not live-verified, so its quiet observation window is not terminal evidence and degrades to the keep-attached still-running wait): the built-in ACP servers terminate in-flight turns when the client connection closes (live-verified — claude-agent-acp/pi-acp exit on connection close and cancel, `opencode acp` exits on stdin EOF, codex-acp ends/kills the codex process) and their persisted transcripts hold only completed messages, so after a daemon crash the founding turn is NEVER still running at the backend and the replay's trailing content is authoritative — an assistant message is the turn's terminal message (completed-while-down, resolved with the real accumulated text), anything else means the turn died mid-way (the safe-re-issue class — nothing running, no duplication possible). The one caveat — content still in flight when the load response resolved — is absorbed by the bounded post-load continuation watch (`AGENTPRISM_ACP_LOADED_TURN_OBSERVE_MS`, default 1 s): any CONTENT update after the load boundary is live continuation, the authoritative still-running signal, and flips the classification to the keep-attached wait. A query FAILURE on an extension backend falls through to the same observation path (a possibly-running call is never released-and-re-issued). A `running` turn past the max-wait bound rejects with the re-armable `LoadedTurnStillRunningError` (the broker re-arms the seam on the still-attached session — a later terminal notification or a cancel still settles the call), and a turn that failed at the backend rejects with `LoadedTurnFailedError` (a definite rejection, never a re-issue). A seam that rejects with the NON-re-armable still-running class (a third-party adapter that can never observe the terminal state) is NOT re-invoked — an immediate recursive re-arm would spin in an unbounded microtask/warning loop (phase-F review round 3): the broker keeps the loaded session attached and waits for the terminal state from the session-level `_session/loaded_turn/ended` surface (when the backend pushes one anyway), the call's cancel (settled as the recoverable `AGENT_CANCELLED`), the session's release (the process died — the safe-re-issue class), or the client-presence drain's forced stop (settled durably at the bound). `resumeSession()` reattaches without replay. `forkSession()` can register only after `session/fork` returns its new id, matching `session/new`; subsequent updates, permissions, and prompts route exclusively under that response id. All three adopt response `configOptions`/`modes`; a routed model id is then sent verbatim, while `mode` is validated and applied strictly from the response mode catalog. The upstream SDK marks `session/fork` **UNSTABLE** / `@experimental`; this wrapper may need to track future protocol changes. +`loadSession()` registers the caller-supplied id before sending `session/load`, so replayed `session/update` history is accumulated and permissions during replay are routed. After it resolves, replay is visible in `session.text` / `session.history`. `InteractiveSession.awaitCurrentTurn()` resolves with the loaded session's founding turn (the turn that was in flight when the host died) — a restarted host's re-attach arm — using the **`_session/loaded_turn` vendor extension** (the `_session/steering` precedent; advertised at initialize via `InitializeResponse._meta.loadedTurn.supported === true`, served by the in-repo `@automatalabs/pi-acp` and `@automatalabs/codex-acp`): right after the load response the seam asks `_session/loaded_turn/query` whether the founding turn is still running — `completed` (observably completed while the host was down; the replay's trailing assistant message is its FINAL message, resolved immediately with the real accumulated text, stop reason synthesized `end_turn`), `interrupted` (ended without a terminal message, nothing running — the safe-re-issue class), or `running` (kept attached, waiting for the authoritative `_session/loaded_turn/ended` notification — pushed with the stop reason, or the error, when the turn ends; a quiet gap is only a progress-stream gap, never terminal evidence, and the wait is bounded by `AGENTPRISM_ACP_LOADED_TURN_MAX_WAIT_MS`, default 15 min). A backend WITHOUT the extension (the built-in claude and opencode backends today) is classified by the **observation path** — the post-load continuation watch plus the replay probe under the **connection-death contract** (phase-F review round 2, restricted to the VERIFIED BUILT-INS in round 3 — a custom registry backend's connection-death behavior is not live-verified, so its quiet observation window is not terminal evidence and degrades to the keep-attached still-running wait): the built-in ACP servers terminate in-flight turns when the client connection closes (live-verified — claude-agent-acp/pi-acp exit on connection close and cancel, `opencode acp` exits on stdin EOF, codex-acp ends/kills the codex process) and their persisted transcripts hold only completed messages, so after a daemon crash the founding turn is NEVER still running at the backend and the replay's trailing content is authoritative — an assistant message is the turn's terminal message (completed-while-down, resolved with the real accumulated text), anything else means the turn died mid-way (the safe-re-issue class — nothing running, no duplication possible). The one caveat — content still in flight when the load response resolved — is absorbed by the bounded post-load continuation watch (`AGENTPRISM_ACP_LOADED_TURN_OBSERVE_MS`, default 1 s): any CONTENT update after the load boundary is live continuation, the authoritative still-running signal, and flips the classification to the keep-attached wait. A query FAILURE on an extension backend falls through to the same observation path (a possibly-running call is never released-and-re-issued). A `running` turn past the max-wait bound rejects with the re-armable `LoadedTurnStillRunningError` (the broker re-arms the seam on the still-attached session — a later terminal notification or a cancel still settles the call), and a turn that failed at the backend rejects with `LoadedTurnFailedError` (a definite rejection, never a re-issue). A seam that rejects with the NON-re-armable still-running class (a third-party adapter that can never observe the terminal state) is NOT re-invoked — an immediate recursive re-arm would spin in an unbounded microtask/warning loop (phase-F review round 3): the broker keeps the loaded session attached and waits for the terminal state from the session-level `_session/loaded_turn/ended` surface (when the backend pushes one anyway), the call's cancel (settled as the recoverable `AGENT_CANCELLED`), the session's release (the process died — the safe-re-issue class), or the client-presence drain's forced stop (settled durably at the bound). `resumeSession()` reattaches without replay. `forkSession()` can register only after `session/fork` returns its new id, matching `session/new`; subsequent updates, permissions, and prompts route exclusively under that response id. All three adopt response `configOptions`/`modes`; a routed model id is then sent verbatim, while `mode` is validated and applied strictly from the response mode catalog. The upstream SDK marks `session/fork` **UNSTABLE** / `@experimental`; this wrapper may need to track future protocol changes. Where does `sessionId` come from? Three sources: `listSessions()`, an `InteractiveSession.sessionRef` you persisted, or — for one-shot workflow agents — `WorkflowRunResult.agentSessions`. Every `agent()` call that opened a live session lands one `AgentSessionRecord` (`AgentSessionRef` + `callIndex`/`label`/`phase`/`keptOpen`) on the run result (even with `journaling: false` — it rides the result, not the journal), in the journal entry (so resume replays it), and on the `agentEnd` event/snapshot. The one-shot-plan round trip: @@ -1677,7 +1676,7 @@ One runtime class (from `@automatalabs/shared-types`, so `instanceof` holds acro ## MCP server -`npx @automatalabs/mcp-server` (bin `agentprism-workflow`) speaks stdio MCP and exposes model-facing tools: deterministic **`workflow`** and persistent interactive **`repl`**, plus **`workflow_monitor`** for Apps-capable clients. By default the stdio process is a thin shim proxying to the shared per-user workflow daemon (Streamable HTTP on loopback, auto-started, spec 2025-11-25 session management and resumability); `--in-process` serves everything in the one stdio process instead, and HTTP-capable hosts can register the daemon URL directly (`agentprism-workflow daemon url`). The tool contract is identical on every path except one knob: the daemon **requires** `projectDir` on workflow config/run and REPL inputs, while an in-process server defaults it to its own project. +`npx @automatalabs/mcp-server` (bin `agentprism-workflow`) speaks stdio MCP and exposes the model-facing **`workflow`** tool, plus **`workflow_monitor`** for Apps-capable clients. By default the stdio process is a thin shim proxying to the shared per-user workflow daemon (Streamable HTTP on loopback, auto-started, spec 2025-11-25 session management and resumability); `--in-process` serves everything in the one stdio process instead, and HTTP-capable hosts can register the daemon URL directly (`agentprism-workflow daemon url`). The tool contract is identical on every path except one knob: the daemon **requires** `projectDir` on workflow config/run inputs, while an in-process server defaults it to its own project. The server declares the SEP-2640 extension `io.modelcontextprotocol/skills` with `{ directoryRead:true }` and publishes `skill://agentprism-workflow-authoring/SKILL.md`. `skills/list({ cursor? })` returns that static entry in one page; each entry contains complete `frontmatter` and a complete `resources` array of `{ uri, digest, size }`. `skills/get({ uri })` returns the same entry for one exact served skill URI. Skill files are read through `resources/read`; directories are listed non-recursively through `resources/directory/read({ uri, cursor? })`. Unknown skill or directory URIs and cursors the server did not issue fail with Invalid Params (`-32602`). Digests are `sha256:` over the exact raw bytes whose length is `size`. @@ -2012,46 +2011,6 @@ that inspects a run attaches to its later updates and nothing earlier is replaye channel handler drop the notification. See the [package README](../packages/mcp-server/README.md#claude-code-channels) for enablement. -### The `repl` tool - -The server also registers the interactive model-facing tool **`repl`** — a persistent QuickJS-in-WASM JavaScript REPL, **one VM per `projectDir`**, for live, stateful subagent orchestration (the interactive complement to `workflow`'s deterministic scripts). Workspace state — bindings, pending subagent calls, raised checkpoints, logged values — persists in the VM across tool calls, MCP-session churn, and daemon restarts. Its full contract, with worked examples per action, is in the [package README](../packages/mcp-server/README.md#the-repl-tool); the surface in brief: - -```ts -type ReplToolInput = - | { action: "eval"; projectDir?: string; code: string; timeoutMs?: number } // timeoutMs default 60_000, hard cap 120_000 - | { action: "interrupt"; projectDir?: string; id?: string }; -``` - -`projectDir` is required on the shared daemon for **both** actions, and defaults to the server's own project on `--in-process`. The input schema is **strict**: a missing required field and every key outside the selected action's exact set are rejected as Invalid Params (`-32602`), never silently discarded. Every result carries `structuredContent` (the exact same shape as the published `outputSchema`) alongside the human text. `eval` holds the call open pumping settlements up to the soft bound: the **finished** shape `{ output, result }` when everything the code waits on settles within the bound, the **still-running** shape `{ output, running: [call ids] }` when the bound elapses (the eval continues server-side; any later eval — including `""`, the documented idempotent poll — drains what settled, and a poll picks a drained timed-out eval's completion repr up as its own `result`), or the **thrown** shape `{ output }` (the §4.6 error rendering, no completion value). `output` is one newline-joined string — console lines, raised checkpoint lines, error renderings, and one-line durability notices — and is forwarded without a byte ceiling: - -```ts -type ReplToolOutput = - | { output: string; result: string } // eval finished (a guest undefined renders "undefined") - | { output: string; running: string[] } // eval still running (the in-flight c1, c2, … ids) - | { output: string } // eval threw / was broken mid-run - | { interrupt: { outcome: "targeted" | "refused-idle" | "cancelled" | "idle" | "failed" | "none"; callId?: string } } - | { error: string }; // isError: true — a missing project context -``` - -With `id`, `interrupt` cancels that subagent call (the guest promise rejects recoverable, `AGENT_CANCELLED` family); without `id`, it breaks the running eval and reports `refused-idle` when nothing is running. Introspection is in-band through guest functions returning ordinary values: `workspace()` (`{ bindings, inFlight, checkpoints, diagnostics }` — `diagnostics` carries the last reconcile summary, a retained drain error, and `childrenClosed`), `agents()` (the live-agent entries), and `reset()` (teardown after the current eval). A stored snapshot that **refuses** (corrupt, format bump, wasm-hash mismatch) auto-resets: the refused file is renamed aside (`.refused-`, never deleted) and the next eval's output leads with a one-line notice; a restore that lost calls or a drain failure that lost state gets the same one-line-notice treatment. Printing follows the repr rules (direct strings whole; depth 2; 20 entries per level; nested strings 200 chars head+tail) with no byte ceiling. Subagent `agent()` calls and `checkpoint()` draw from **one shared per-workspace id sequence** — `c1`, `c2`, … — answered by `checkpoint.answer("c2", value)` in a later eval; raised checkpoints surface as output lines. Subagents are [`acp-agents`](#acpagentrunner-createacprunner) sessions, 6 concurrent per workspace (additional dispatches queue); the workspace snapshots to the per-project store at every state-changing boundary and restores **lazily on first touch** with a three-way call reconcile (settle / re-attach / re-issue). `repl` shares the `workflow` tool's project model and daemon lifetime. - -## `@automatalabs/repl-engine` - -The published engine tier the `repl` tool registers over (imported by `mcp-server` as `workspace:*` in the monorepo and stamped to an exact version at publish time). It is a persistent JavaScript REPL in a capability-free QuickJS-in-WASM VM; the public surface: - -- **`Workspace`** / **`WorkspaceRegistry`** (`WorkspaceOptions`, `WorkspaceRegistryOptions`, `WorkspaceManifest`, `WorkspaceBinding`) — one VM per workspace, owning the lifecycle (`create` → `eval` → `drainJobs` → `dispose`) and the manifest surface. `ReplVm` (`loadShippedWasm`, `ReplVmOptions`, `ReplEvalOptions`, `ReplDrainOptions`, `ReplEvalOutcome`, `DrainJobError`) is the raw quickjs-wasi shim tier. -- **`Broker`** (`DEFAULT_MAX_CONCURRENT_AGENTS`, `DEFAULT_EVAL_TIMEOUT_MS`, `DEFAULT_DISPOSE_BOUND_MS`, `BrokerOptions`, `BrokerRunner`, `ReplEvalResult`, `CheckpointSummary`, `LiveAgentInfo`, `ReconcileReport`, `WorkspaceManifestReport`, …) — drives subagents as ACP sessions, records results by call id, and reconciles on restore. The call store is `InMemoryCallStore` / `JsonlCallStore` (`CallStore`, `CallRecord`, `CallOutcome`, …). -- **Snapshots and durability** — `serializeSnapshot` / `deserializeSnapshot` / `wasmSha256Of`, `SNAPSHOT_FORMAT` / `SNAPSHOT_FORMAT_VERSION`, `SnapshotEnvelopeError` / `SnapshotRestoreError`, and the per-project `ReplWorkspaceStore` (`REPL_STORE_SUBDIR`, `SNAPSHOT_FILENAME`, `CALL_STORE_FILENAME`). -- **The previewer** — `renderPreviewLine` / `renderCollapsed` / `renderGlobalLine` / `manifestBinding` / `formatByteSize` and the CDP preview types (`ObjectPreview`, `PropertyPreview`, …). Guest output is forwarded as rendered; the previewer also supplies bounded internal metadata such as manifest tokens and checkpoint/task previews. -- **The guest bridge and provenance** — `installGuestBridge`, `GUEST_LIBRARY_VERSION`, the `HOST_*` callback names; `provenanceRecord` / `provenanceView` (`eval N` / `worker cN` / `session restore` labels). -- **The out-of-band eval-break channel** — `createEvalBreakChannel` / `EvalBreakChannel` (the worker-thread relay the MCP shim fires to break a synchronous runaway). - -The full engine contract (guest library, host-call surface, FORMAT.md preview rules, reconcile semantics) is documented in the [package README](../packages/repl-engine/README.md). - -### The `repl` adapter exports from `@automatalabs/mcp-server` - -`@automatalabs/mcp-server` re-exports the REPL adapter surface for hosts mounting the tool themselves: `replToolInputShape` / `replToolOutputShape` (the Zod input/output schemas), the `ReplToolOptions` type, `createReplProjectState` / `ensureReplWorkspace` / `disposeReplProjectState` / `resetReplProjectState` and the `ReplProjectState` type (per-project workspace state), and `ReplPresenceLedger` (the client-presence drain). `createWorkflowServer` registers both `workflow` and `repl`; `CreateWorkflowServerOptions` exposes `replRunner` / `replPresence` / `replClientId` / `replEvalBreakChannel` / `replDrainBoundMs`. Breaking a *fully synchronous* runaway requires the relay stdio transport `main()` installs; a vanilla `StdioServerTransport` bounds it only by the per-eval deadline (`AGENTPRISM_REPL_EVAL_TIMEOUT_MS`, default 30 000 ms). - ## Workflow script DSL Scripts run in a deterministic `vm` realm (`Date.now`/`Math.random`/argless `new Date()` throw — the journal/resume identity depends on it; the realm is a determinism boundary, **not** a security boundary). Realm globals: diff --git a/docs/roadmap/repl-eval-redesign.md b/docs/archive/roadmap/repl-eval-redesign.md similarity index 100% rename from docs/roadmap/repl-eval-redesign.md rename to docs/archive/roadmap/repl-eval-redesign.md diff --git a/docs/roadmap/repl-orchestrator.md b/docs/archive/roadmap/repl-orchestrator.md similarity index 100% rename from docs/roadmap/repl-orchestrator.md rename to docs/archive/roadmap/repl-orchestrator.md diff --git a/docs/roadmap/validate-mcp-action.md b/docs/roadmap/validate-mcp-action.md index 300e0c53..bdd1d2fe 100644 --- a/docs/roadmap/validate-mcp-action.md +++ b/docs/roadmap/validate-mcp-action.md @@ -4,8 +4,7 @@ The validator — static parse, mock dry run with scripted mock answers, and the per-harness config-options probe — ships in `@automatalabs/workflows` as a CLI and a programmatic API, but -the MCP `workflow` tool's actions are `run`/`inspect`/`await`/`stop` only (the server also -registers a separate `repl` tool, which is unrelated to script validation). An MCP host that +the MCP `workflow` tool's actions are `run`/`inspect`/`await`/`stop` only. An MCP host that wants to validate a script before spending tokens has to shell out to the CLI or embed the SDK, neither of which fits hosts that only speak MCP. diff --git a/packages/acp-agents/README.md b/packages/acp-agents/README.md index c2ab4cbb..30347209 100644 --- a/packages/acp-agents/README.md +++ b/packages/acp-agents/README.md @@ -165,7 +165,7 @@ bus only; it is not delivered through `session.on()`. the agent to replay the entire persisted conversation before resolving (the runner marks the LOAD BOUNDARY synchronously after the response). `InteractiveSession.awaitCurrentTurn()` resolves with the founding turn (the turn that was in flight when the host died) so a re-attached call's -continuation fires exactly once — the REPL broker's re-attach arm. Completion evidence is the +continuation fires exactly once — the re-attach arm of a host that survives its own restart. Completion evidence is the vendor **`_session/loaded_turn` extension** (the `_session/steering` precedent), an AUTHORITATIVE turn-terminal channel for loaded sessions advertised at initialize (`InitializeResponse._meta.loadedTurn.supported === true`; pi-acp and codex-acp advertise it): diff --git a/packages/acp-agents/src/acp-client.ts b/packages/acp-agents/src/acp-client.ts index 51ec5847..474ce1ed 100644 --- a/packages/acp-agents/src/acp-client.ts +++ b/packages/acp-agents/src/acp-client.ts @@ -3004,7 +3004,7 @@ export class SessionHandle implements StructuredSource { * The trailing kind is PROGRESS evidence, never completion by itself — * the seam classifies completion from the LOAD BOUNDARY plus whether * any content update followed the load (see `loadBoundaryState`). - * Added for the REPL broker's re-attach arm; additive passthrough to + * For a host's re-attach arm; additive passthrough to * `SessionState`. */ loadedTurnState(): { hasUserMessage: boolean; trailingContentKind: 'assistant-message' | 'other' } { return this.state.loadedTurnState(); @@ -3029,21 +3029,21 @@ export class SessionHandle implements StructuredSource { } /** The most recent instant a session/update arrived for this session - * (the re-attach arm's stream-settled clock). Added for the REPL - * broker's re-attach arm; additive passthrough to `SessionState`. */ + * (the re-attach arm's stream-settled clock). For a host's re-attach + * arm; additive passthrough to `SessionState`. */ lastUpdateAtMs(): number { return this.state.lastUpdateAtMs(); } /** Watch the session/update stream (fires after every applied update; - * returns the unsubscribe thunk). Added for the REPL broker's re-attach + * returns the unsubscribe thunk). For a host's re-attach * arm; additive passthrough to `SessionState`. */ subscribeUpdates(listener: () => void): () => void { return this.state.subscribeUpdates(listener); } /** The founding turn's assistant text (the transcript accumulated after - * the last user-message boundary). Added for the REPL broker's re-attach + * the last user-message boundary). For a host's re-attach * arm; additive passthrough to `SessionState`. */ loadedTurnText(): string { return this.state.loadedTurnText(); @@ -3051,7 +3051,7 @@ export class SessionHandle implements StructuredSource { /** The recorded `_session/loaded_turn/ended` terminal state (the * re-attach arm's authoritative completion evidence), or null when a - * running founding turn has not ended yet. Added for the REPL broker's + * running founding turn has not ended yet. For a host's * re-attach arm; additive passthrough to `SessionState`. */ loadedTurnEndedState(): { stopReason?: string; error?: { name: string; message: string } } | null { return this.state.loadedTurnEndedState(); @@ -3059,8 +3059,8 @@ export class SessionHandle implements StructuredSource { /** Watch the loaded-turn-ended channel (fires when the * `_session/loaded_turn/ended` notification arrives — and immediately - * when one already arrived). Returns the unsubscribe thunk. Added for - * the REPL broker's re-attach arm; additive passthrough to + * when one already arrived). Returns the unsubscribe thunk. For + * a host's re-attach arm; additive passthrough to * `SessionState`. */ subscribeLoadedTurnEnded(listener: () => void): () => void { return this.state.subscribeLoadedTurnEnded(listener); diff --git a/packages/acp-agents/src/interactive.ts b/packages/acp-agents/src/interactive.ts index d816452f..4b087dbe 100644 --- a/packages/acp-agents/src/interactive.ts +++ b/packages/acp-agents/src/interactive.ts @@ -201,7 +201,7 @@ export class InteractiveSession { } /** The latest turn's assistant text (turn-segmented, like `run()`'s - * no-schema result path). Added for the REPL broker's result shaping; + * no-schema result path). For a host's own result shaping; * additive passthrough to `SessionHandle`. */ currentTurnText(): string { return this.session.currentTurnText(); @@ -210,7 +210,7 @@ export class InteractiveSession { /** The latest turn's FINAL assistant message (the schema-extraction * source `run()` uses; prose extraction over the whole turn would * resurrect the first-JSON-wins bug for schema-shaped progress - * messages). Added for the REPL broker's structured-output ladder; + * messages). For a host's own structured-output ladder; * additive passthrough to `SessionHandle`. */ finalMessageText(): string { return this.session.finalMessageText(); @@ -231,7 +231,7 @@ export class InteractiveSession { /** Claude's raw `structured_output` for the latest turn, if any (the * native structured channel the runner's ladder tries first). Added - * for the REPL broker's structured-output ladder; additive passthrough + * for a host's own structured-output ladder; additive passthrough * to `SessionHandle`. */ rawStructuredOutput(): unknown { return this.session.rawStructuredOutput(); @@ -252,8 +252,8 @@ export class InteractiveSession { * prompt-in-flight, image validation) AND the underlying ACP session/prompt request has * actually been invoked — the call below runs synchronously through request construction * and the wire send, so by the time the acknowledgment fires the payload is on the wire: - * the point of no return. A host that records a "delivered" marker for the prompt (the - * REPL broker's queued-steer delivery marker) MUST record it here rather than when the + * the point of no return. A host that records a "delivered" marker for the prompt (a + * queued-steer delivery marker, say) MUST record it here rather than when the * returned promise is created: an async pre-handoff rejection (released session, aborted * signal, or prompt-in-flight) never reaches this line, and a marker recorded before it * would make a restore skip a turn that was never delivered. The acknowledgment firing @@ -409,9 +409,8 @@ export class InteractiveSession { /** * The loaded session's founding-turn completion — the re-attach arm's task - * source (phase D of the REPL orchestrator roadmap; the broker drives this - * on a session re-opened with `runner.loadSession()` after a daemon - * restart). Resolves with the turn that was in flight at the backend when + * source (a host drives this on a session re-opened with + * `runner.loadSession()` after its own restart). Resolves with the turn that was in flight at the backend when * the session was loaded, so a re-attached call's continuation fires * exactly once, through the same record → settle → consume pump as a live * call. diff --git a/packages/acp-agents/src/runner.ts b/packages/acp-agents/src/runner.ts index 93541902..97fa54d1 100644 --- a/packages/acp-agents/src/runner.ts +++ b/packages/acp-agents/src/runner.ts @@ -594,9 +594,8 @@ export class AcpAgentRunner implements AgentRunner, AuthCapableRunner, ProviderC /** The configured DEFAULT backend id — the registry's own routing for * an omitted model (`selectBackend({})`: the `AGENTPRISM_DEFAULT_BACKEND` * env-configured backend when registered, the built-in `claude` - * otherwise). The repl-engine's broker serves this to the guest library - * (verify/judgePanel resolve their reviewer/grader spec through it — a - * real registered segment, never the deleted reserved sentinel). */ + * otherwise). A host that resolves a model spec on a caller's behalf reads it + * to name a real registered segment. */ defaultBackendId(): string { return selectBackend({}, this.backends).id; } diff --git a/packages/acp-agents/test/docs-drift.test.ts b/packages/acp-agents/test/docs-drift.test.ts index 51295b58..e955f565 100644 --- a/packages/acp-agents/test/docs-drift.test.ts +++ b/packages/acp-agents/test/docs-drift.test.ts @@ -234,7 +234,7 @@ test("public package inventories cover every workspace package", () => { manifest: JSON.parse(readRepoFile(`packages/${entry.name}/package.json`)) as { name: string }, })); - assert.equal(manifests.length, 10, "update the documented package-count contract when the workspace changes"); + assert.equal(manifests.length, 9, "update the documented package-count contract when the workspace changes"); for (const path of ["README.md", "docs/api.md"]) { const text = readRepoFile(path); for (const { manifest } of manifests) { @@ -255,7 +255,7 @@ test("public package inventories cover every workspace package", () => { for (const { dir } of manifests) { assert.ok(contributing.includes(`packages/${dir}`), `CONTRIBUTING.md must inventory packages/${dir}`); } - assert.match(contributing, /\(monorepo\) of ten packages/); + assert.match(contributing, /\(monorepo\) of nine packages/); }); test("auth, MCP, and authoring docs retain the implemented contracts", () => { diff --git a/packages/acp-agents/test/interactive.test.ts b/packages/acp-agents/test/interactive.test.ts index 487af583..51723c5c 100644 --- a/packages/acp-agents/test/interactive.test.ts +++ b/packages/acp-agents/test/interactive.test.ts @@ -262,7 +262,7 @@ test("InteractiveSession.prompt fires the handoff acknowledgment only after the }); // Preflight rejection: a second prompt while one is in flight. The // acknowledgment must never fire for a turn the backend was never - // handed (the REPL broker records its delivered marker in it — a + // handed (a host records its delivered marker in it — a // false positive would make a restore skip a never-delivered turn). const first = session.prompt("one", { onHandoff: () => order.push("handoff") }); assert.deepEqual( diff --git a/packages/codex-acp/src/AcpExtensions.ts b/packages/codex-acp/src/AcpExtensions.ts index 18889aa9..e5b96192 100644 --- a/packages/codex-acp/src/AcpExtensions.ts +++ b/packages/codex-acp/src/AcpExtensions.ts @@ -147,8 +147,8 @@ export async function steerSession( /** * The `_session/loaded_turn` vendor extension (the steering-extension - * precedent): turn-TERMINAL state for loaded sessions — the REPL broker's - * re-attach arm's authoritative completion evidence. `query` asks whether + * precedent): turn-TERMINAL state for loaded sessions — a re-attaching + * host's authoritative completion evidence. `query` asks whether * the loaded session's founding turn is still running right now * (`"running"`), observably completed while the host was down * (`"completed"` — the replayed thread's last turn completed, so its diff --git a/packages/codex-acp/src/__tests__/CodexACPAgent/loaded-turn.test.ts b/packages/codex-acp/src/__tests__/CodexACPAgent/loaded-turn.test.ts index 9affedd0..d1783257 100644 --- a/packages/codex-acp/src/__tests__/CodexACPAgent/loaded-turn.test.ts +++ b/packages/codex-acp/src/__tests__/CodexACPAgent/loaded-turn.test.ts @@ -10,7 +10,7 @@ import type {Thread} from "../../app-server/v2"; import {LOADED_TURN_ENDED_METHOD, LOADED_TURN_QUERY_METHOD} from "../../AcpExtensions"; /** - * The `_session/loaded_turn` extension (the REPL broker's re-attach arm's + * The `_session/loaded_turn` extension (a re-attaching host's * authoritative completion evidence): the query answers whether the * loaded session's founding turn is still running right now, and the * ended notification is pushed when a turn that a query classified diff --git a/packages/mcp-server/README.md b/packages/mcp-server/README.md index 4fae965e..4bfecd93 100644 --- a/packages/mcp-server/README.md +++ b/packages/mcp-server/README.md @@ -1,8 +1,8 @@ # @automatalabs/mcp-server -An **[MCP](https://modelcontextprotocol.io) server** for asynchronous execution, bounded status observation, and in-place stopping of dynamic multi-agent workflows. Execution lives in a shared per-user **local daemon** (spec-compliant Streamable HTTP on loopback) so runs survive MCP clients killing their server processes; hosts connect through the bundled **stdio shim** (the default bin, zero config change) or directly over HTTP — see [The workflow daemon](#the-workflow-daemon). Its model-facing tools are **`workflow`** for the strict config/run/resume/setup-response/status/result/permissions-response/stop/pause lifecycle and **`repl`** for persistent interactive orchestration. Version-matched guidance for both is published through the SEP-2640 MCP Skills Extension. Apps-capable clients also get the dedicated `workflow_monitor` launcher. App-only `workflow-events`, `workflow-runs`, and `workflow-notifications` tools feed the [MCP Apps run monitor](#run-monitor-mcp-apps) and never enter the model's tool loop. The `workflow` tool discovers its live backend catalog with `action:"config"` and durably accepts each script before slow preparation and validates it before live execution. Scripts may be supplied inline or by absolute server-side path, and every admitted run exposes its script file as an MCP `file://` resource. Agent backends authenticate from their own credential sources (`claude /login`, `codex login`, `opencode auth login`, provider API keys, or pi's `~/.pi/agent/auth.json`), so there is nothing auth-shaped for a host to manage here. A run that genuinely hits expired/missing credentials pauses with `authContext` and resumes with `action:"resume"` after the backend credentials are configured. Auth and provider *management* APIs live in the [`@automatalabs/workflows`](../workflows) SDK for embedding hosts. +An **[MCP](https://modelcontextprotocol.io) server** for asynchronous execution, bounded status observation, and in-place stopping of dynamic multi-agent workflows. Execution lives in a shared per-user **local daemon** (spec-compliant Streamable HTTP on loopback) so runs survive MCP clients killing their server processes; hosts connect through the bundled **stdio shim** (the default bin, zero config change) or directly over HTTP — see [The workflow daemon](#the-workflow-daemon). Its model-facing tool is **`workflow`**, with the strict config/run/resume/setup-response/status/result/permissions-response/stop/pause lifecycle. Version-matched guidance for it is published through the SEP-2640 MCP Skills Extension. Apps-capable clients also get the dedicated `workflow_monitor` launcher. App-only `workflow-events`, `workflow-runs`, and `workflow-notifications` tools feed the [MCP Apps run monitor](#run-monitor-mcp-apps) and never enter the model's tool loop. The `workflow` tool discovers its live backend catalog with `action:"config"` and durably accepts each script before slow preparation and validates it before live execution. Scripts may be supplied inline or by absolute server-side path, and every admitted run exposes its script file as an MCP `file://` resource. Agent backends authenticate from their own credential sources (`claude /login`, `codex login`, `opencode auth login`, provider API keys, or pi's `~/.pi/agent/auth.json`), so there is nothing auth-shaped for a host to manage here. A run that genuinely hits expired/missing credentials pauses with `authContext` and resumes with `action:"resume"` after the backend credentials are configured. Auth and provider *management* APIs live in the [`@automatalabs/workflows`](../workflows) SDK for embedding hosts. -This package is a **thin MCP adapter**. The `workflow` tool's real work — parsing the workflow script, running the deterministic engine, fanning `agent()` calls out to real coding agents over [ACP](https://agentclientprotocol.com), journaling, and resume — lives in **[`@automatalabs/workflows`](../workflows)**; the `repl` tool's real work — the persistent QuickJS-in-WASM VM, the subagent broker, the CDP-style previewer, and the enveloped-snapshot store — lives in **[`@automatalabs/repl-engine`](../repl-engine)**. The MCP server is the *composition root*: it builds the ACP-backed agent runner, injects it into the workflow engine, registers the `workflow` tool over a per-project `WorkflowManager` and the `repl` tool over a per-project QuickJS VM, and serves them over stdin/stdout. +This package is a **thin MCP adapter**. The `workflow` tool's real work — parsing the workflow script, running the deterministic engine, fanning `agent()` calls out to real coding agents over [ACP](https://agentclientprotocol.com), journaling, and resume — lives in **[`@automatalabs/workflows`](../workflows)**. The MCP server is the *composition root*: it builds the ACP-backed agent runner, injects it into the workflow engine, registers the `workflow` tool over a per-project `WorkflowManager`, and serves it over stdin/stdout. > **Embedding in your own program?** Don't reach for this package — use **[`@automatalabs/workflows`](../workflows)** directly (`runDynamicWorkflow(script, …)`). This server exists to put that same engine behind the MCP protocol. See [Programmatic use](#programmatic-use) below. @@ -14,15 +14,13 @@ This package is a **thin MCP adapter**. The `workflow` tool's real work — pars ``` MCP host (Claude Code / Zed / Cursor / …) - │ tools/call → "workflow" | "repl" (JSON-RPC over stdio) + │ tools/call → "workflow" (JSON-RPC over stdio) ▼ ┌──────────────────────────────────────────────────────┐ │ agentprism-workflow (this package) │ -│ • registers the "workflow" and "repl" tools │ +│ • registers the "workflow" tool │ │ • createAcpRunner() → the workflow engine │ │ • workflow → per-project WorkflowManager │ -│ • repl → per-project QuickJS VM + broker; │ -│ each workspace owns its own AcpAgentRunner │ └──────────────────────────────────────────────────────┘ │ session/new, session/prompt … (ACP over stdio) ▼ @@ -31,12 +29,7 @@ This package is a **thin MCP adapter**. The `workflow` tool's real work — pars ``` For `workflow`, run/resume return durable acknowledgements and `status` reads a bounded snapshot. -The events resource and dedicated monitor provide continuous progress. The `repl` tool holds a -persistent QuickJS VM **per `projectDir`** — the - -same per-project context model — whose state persists across tool calls and daemon restarts through -the per-project `repl/` store, and whose subagent `agent()` calls use the same ACP path shown above -(see [The `repl` tool](#the-repl-tool)). `stdout` is reserved for JSON-RPC framing — every diagnostic +The events resource and dedicated monitor provide continuous progress. `stdout` is reserved for JSON-RPC framing — every diagnostic the server emits goes to `stderr`. --- @@ -157,10 +150,10 @@ With `--in-process`, the old lifecycle applies: on stdin EOF, transport close, ` ### The workflow daemon - **Discovery**: the daemon records `{pid, instanceId, port, url, version, envFingerprint, controlUrl, controlProtocol}` (mode 0600) under `~/.agentprism/workflows/daemons/` — a **family pointer** `.json` naming the current daemon for that env, plus one `instances/.json` per live daemon. The user-scoped mode-0600 `run-control-key.json` authenticates cross-family predecessor control. Malformed key storage fails closed. Shims verify liveness via pid + `/healthz` and never dial a port blind. Concurrent shims race a per-family spawn lock, so a cold start produces exactly one daemon. The records are hints, not truth: `daemon status` and `daemon stop --all` reconcile them against the OS process table, so a daemon that lost or never wrote its record is still listed and stoppable (POSIX; on Windows the records are all there is), and a record whose pid the OS has since reused is pruned rather than signalled. Logs land in `~/.agentprism/workflows/logs/daemon.log`. -- **Succession**: a shim that finds an older control-v1 daemon spawns a successor (ephemeral port), which atomically repoints the family pointer. The predecessor becomes a *lame duck*: it admits no new MCP work, migrates drainable sessions immediately, continues its owned executions/REPL drains, accepts signed internal stop/cancel forwarding, and exits when those responsibilities settle. A daemon **equal to or newer** than the shim is adopted (version is a total order, so clients cannot flip discovery backward). Bootstrap exception: when the stale predecessor predates control v1 and reports active runs or requests, the new shim temporarily adopts it until that work drains; sessions alone never defer the upgrade. Supersession is a one-way door: a superseded daemon stays superseded even if its successor later exits and clears the pointer, so a predecessor never returns to service. `daemon status` shows instance/control identity for every current, draining, and other-family daemon, plus any untracked daemon process. +- **Succession**: a shim that finds an older control-v1 daemon spawns a successor (ephemeral port), which atomically repoints the family pointer. The predecessor becomes a *lame duck*: it admits no new MCP work, migrates drainable sessions immediately, continues its owned executions, accepts signed internal stop/cancel forwarding, and exits when those responsibilities settle. A daemon **equal to or newer** than the shim is adopted (version is a total order, so clients cannot flip discovery backward). Bootstrap exception: when the stale predecessor predates control v1 and reports active runs or requests, the new shim temporarily adopts it until that work drains; sessions alone never defer the upgrade. Supersession is a one-way door: a superseded daemon stays superseded even if its successor later exits and clears the pointer, so a predecessor never returns to service. `daemon status` shows instance/control identity for every current, draining, and other-family daemon, plus any untracked daemon process. - **Port**: default `29888` (`AGENTPRISM_DAEMON_PORT` / `--port`). If the port is held — by a foreign process, or by a draining predecessor still finishing its work — the daemon falls back to an ephemeral port — discovery still works, only hardcoded client URLs need the actual port from `daemon status`. -- **Sessions and projects**: sessions are project-agnostic — every `run` call names its project via the **required `projectDir` argument** (absolute path), so one registration serves any number of projects concurrently. `status`/`stop` take only a runId and locate its project store automatically (live contexts first, then the on-disk store manifests). Each project gets its own `WorkflowManager` — same per-project run stores as before — while all projects share one ACP backend pool. Accepted runs are visible from every session, and `MAX_ACTIVE_RUNS` caps runs **per project** rather than per client process. The `repl` tool's workspace is the same shape of per-project context: **one persistent QuickJS VM per `projectDir`**, restored lazily from the per-project `repl/` store on first touch, persisted at every state-changing boundary, and drained when the project's last MCP client disconnects (both tools share one client-presence ledger, so a `workflow`-only client keeps the workspace's children warm too). See [The `repl` tool](#the-repl-tool). -- **Lifetime**: only signals, `daemon stop`, sustained idleness (default: 15 min with zero sessions, running workflow executions, requests, or REPL drains; `AGENTPRISM_DAEMON_IDLE_TTL_MS`, `0` disables), or completed supersession drain end the daemon. Client disconnects never cancel runs. Dead-client sessions are evicted without touching execution; the shim transparently re-initializes on the spec's 404. A predecessor may remain as an execution owner after its MCP sessions migrate, while the successor routes control by run lease. The REPL client-presence drain has its own bound, `AGENTPRISM_REPL_DRAIN_BOUND_MS` (default 2 h). A request in flight when its daemon crashes is answered by the shim with a JSON-RPC error instead of hanging. +- **Sessions and projects**: sessions are project-agnostic — every `run` call names its project via the **required `projectDir` argument** (absolute path), so one registration serves any number of projects concurrently. `status`/`stop` take only a runId and locate its project store automatically (live contexts first, then the on-disk store manifests). Each project gets its own `WorkflowManager` — same per-project run stores as before — while all projects share one ACP backend pool. Accepted runs are visible from every session, and `MAX_ACTIVE_RUNS` caps runs **per project** rather than per client process. +- **Lifetime**: only signals, `daemon stop`, sustained idleness (default: 15 min with zero sessions, running workflow executions, or requests; `AGENTPRISM_DAEMON_IDLE_TTL_MS`, `0` disables), or completed supersession drain end the daemon. Client disconnects never cancel runs. Dead-client sessions are evicted without touching execution; the shim transparently re-initializes on the spec's 404. A predecessor may remain as an execution owner after its MCP sessions migrate, while the successor routes control by run lease. A request in flight when its daemon crashes is answered by the shim with a JSON-RPC error instead of hanging. - **Security**: the daemon binds `127.0.0.1` only, validates the `Host` header, and enforces the spec's `Origin` validation (403 for non-loopback origins; extend with `AGENTPRISM_DAEMON_ALLOWED_ORIGINS`). The MCP endpoint has no authentication: any local process/user on the machine can reach it — the standard localhost-dev-server trade-off. The non-MCP run-control endpoint additionally requires a timestamped HMAC from the user-scoped mode-0600 key; it never accepts unsigned localhost requests. - **Env is captured at daemon start**: the ACP backend registry (`AGENTPRISM_BACKENDS`, `AGENTPRISM_DEFAULT_BACKEND`, …) is resolved once by the daemon. Clients are keyed by their env fingerprint: a shim whose relevant env differs gets its **own daemon family** (one daemon per distinct env, never contending), so changing the env and restarting the host always takes effect; `--in-process` remains the escape hatch for a fully private server. @@ -215,7 +208,7 @@ If the bin isn't on the host's `PATH`, launch it through `npx` instead: `env` here is inherited by the server process **and** by every agent subprocess it spawns (see [Backends & auth](#backends--auth)), so it is where you put `AGENTPRISM_*` settings and any credentials the agent CLIs need. Every MCP client must configure an effective model directly or through a named-agent definition, resolved tier, phase, or `meta.model`. A backend-only route such as `codex` explicitly uses that backend's configured default model. Missing routing fails with live discovery guidance; neither agent-configuration setup nor automatic backend selection fills it. `AGENTPRISM_DEFAULT_BACKEND` does not configure an otherwise model-less MCP call. -After reload, `workflow` and `repl` appear; Apps-capable hosts also discover `workflow_monitor`. +After reload, `workflow` appears; Apps-capable hosts also discover `workflow_monitor`. --- @@ -503,165 +496,6 @@ client `resources` capability to gate these server-offered primitives. --- -## The `repl` tool - -The interactive model-facing tool is **`repl`**: one persistent **QuickJS-in-WASM JavaScript VM per project**, exposed as a live REPL with **one verb — `eval`** (plus the out-of-band `interrupt`). Where `workflow` runs a *deterministic script to completion*, `repl` is the *interactive* orchestration plane: the client's own agent writes JavaScript that spawns subagents, and workspace state (bindings, pending subagent calls, raised checkpoints, logged values) **persists in the VM between tool calls** — a later `eval` sees the same bindings and awaits the same promises; nothing lives in the transcript. Subagents are ACP sessions run through [`acp-agents`](../acp-agents) — the same backends `workflow` drives — **6 concurrent per workspace**, with dispatches above the cap **queued** for the next free slot (never rejected). - -The VM is capability-free: no filesystem, no network, no timers beyond the `sleep(ms)` guest helper. Its entire effect surface is the host bridge — `agent(modelSpec, task, opts?)`, `checkpoint()` / `checkpoint.answer()`, `console`, and the agent-handle methods `steer` / `queue` / `cancel`. Everything else this repo's workflow authors already know — `parallel`, `pipeline`, `verify`, `judgePanel`, `gate`, `retry`, `loopUntilDry` — is pure JavaScript layered on `agent()`, injected as the in-VM guest library. The full guest surface (and the engine internals) live in the engine package, [`@automatalabs/repl-engine`](../repl-engine#the-guest-library-and-the-bridge-phase-b). - -`agent()` returns a persistent promise-handle. Assign the handle before awaiting it: `const a = agent("codex", "inspect the failure"); const first = await a`. `a.steer(text)` targets only the currently running turn. It never starts or queues another turn and resolves `"injected"`, `"idle"`, or `"unsupported"`; transport and protocol failures reject. `const q = a.queue(text)` creates a distinct FIFO turn on the same session. `q.id` is available immediately, `await q` returns that turn's answer, and `q.cancel()` or an out-of-band interrupt of `q.id` cancels that exact turn. Queueing works on every backend that can continue the session; steering requires the ACP server's raw steering advertisement. Do not write `const a = await agent(...)` when you intend to reuse the handle, because that stores only the answer. Steering while idle returns `"idle"` and loses the instruction by design; callers that require later work must use `queue()`. - -```js -// First REPL eval: -const a = agent("codex", "Investigate the parser failure"); - -// A later REPL eval, only while agents() reports a's turn as running: -const steering = await a.steer("Focus on the parser state machine"); - -// After the founding answer settles, create explicit future work: -const first = await a; -const q1 = a.queue("Implement the fix"); -const q2 = a.queue("Run the focused tests"); -console.log(q1.id, q2.id, steering); -const fixed = await q1; -const tested = await q2; -``` - -Every result carries a machine-readable `structuredContent` — the exact same shape as the published `outputSchema` — alongside a human-readable text block. Guest output is **one newline-joined string** with no byte ceiling, so an agent can flood its own context by printing something enormous. This is accepted and documented — the Python REPL posture. - -### Input parameters - -The tool is an **action union** of exactly two actions. The input schema is **strict**: the MCP SDK validates the primitive fields, then the discriminator enforces each action's exact field set, and every key outside that set is rejected as MCP Invalid Params (`-32602`). - -| Param | Type | Actions | Default | Notes | -| --- | --- | --- | --- | --- | -| `action` | `"eval" \| "interrupt"` | all | — | Required. Selects the operation. | -| `projectDir` | absolute path string | all | daemon: **required**; in-process: the server's own project | The workspace key — one VM per `projectDir`, resolved through the same validated, realpathed per-project context as the `workflow` tool. Workspace state survives MCP-session churn and daemon restarts. | -| `code` | string | `eval` | — | The JavaScript to evaluate. Top-level `await` is accepted; top-level `return` is a syntax error; `console` output is captured. An empty string is valid — the documented idempotent poll (see below). | -| `timeoutMs` | integer 0–120,000 | `eval` | `60000` | The soft bound the eval holds the call open for; values above 120 000 ms are rejected. | -| `id` | string | `interrupt` | — | The call id to cancel. Omitted: break the running eval. | - -`projectDir` is required on the shared daemon for **both** actions. On a single-project (`--in-process`) server it defaults to that server's own project. - -### The two actions - -The examples below run against one workspace, `/work/acme`, in sequence — the state each call leaves is what the next one sees. - -**`eval`** runs `code` in the workspace VM, then **holds the call open pumping settlements server-side** up to the soft bound. Exactly one of three shapes returns: - -- **The finished shape** — everything the code waits on settled within the bound: - - ```json - { "output": "researched the auth flow", "result": "three findings…" } - ``` - - `output` is ONE newline-joined string: console lines (one joined line per `console.*` call, args' reprs joined with a space), raised checkpoint lines (`checkpoint c9: `), uncaught-error renderings (§4.6 attribution), and the one-line durability notices (§6). `result` is the completion value's repr, present whenever the code finished — including the literal string `"undefined"` when the value is the guest `undefined` (a `const`/`let`/`class` declaration or a bare `console.log(...)` statement). - -- **The still-running shape** — the bound elapsed first; the eval *continues server-side*: - - ```json - { "output": "…", "running": ["c1"] } - ``` - - `running` lists the in-flight call ids (the stable `c1, c2, …` vocabulary — what `interrupt` targets and `agents()` reports). **Any later eval drains what settled in the meantime**, and `eval` with `""` is the documented idempotent poll: a no-op script that only reports. A poll whose drained timed-out eval **settled** in the meantime reports that eval's completion repr as its own `result` (a poll with nothing new reports its own `"undefined"`). Re-sending the poll never re-executes work. - -- **The thrown-eval shape** — the code threw (or was broken mid-run by `interrupt`): `output` carries the §4.6 error rendering (name + message, the guest stack's top frames with **line numbers in the submitted code**, and — for a subagent-call error — the call id and resolved backend), with **no `result`**: - - ```json - { "output": "TypeError: x is not a function\n at :1:10" } - ``` - -```json -{ "action": "eval", "projectDir": "/work/acme", - "code": "const research = agent('claude/sonnet', 'Summarize the auth flow in src/auth'); 'started'" } -``` -```json -{ "output": "", "result": "started" } -``` - -The `agent(...)` call took id `c1` and keeps running server-side — start-and-don't-await is idiomatic: `await research` in a later eval picks the answer up. - -**`interrupt`** is the one out-of-band verb (the only operation that cannot be expressed as code: a wedged VM cannot run the code that would unwedge it). - -**With `id`** it cancels one subagent call — ACP `session/cancel` downward (a drained handle's session is re-attached lazily first). `interrupt.outcome` is `cancelled` (cancel sent to a running turn), `idle` (the session exists but has no turn to cancel), `failed` (the lazy re-attach could not reach the backend), or `none` (no live session for that id): - -```json -{ "action": "interrupt", "projectDir": "/work/acme", "id": "c2" } -``` -```json -{ "interrupt": { "outcome": "cancelled", "callId": "c2" } } -``` - -**Without `id`** it breaks the **running eval**. `outcome` is `targeted` when a break was armed against an in-flight eval (a suspended continuation, or a fully synchronous runaway the out-of-band relay broke mid-run), or `refused-idle` — the honest refusal — when nothing breakable is running: - -```json -{ "action": "interrupt", "projectDir": "/work/acme" } -``` -```json -{ "interrupt": { "outcome": "refused-idle" } } -``` - -A missing project context (single-project mode with no adopted default) returns the **error variant** — `{ "error": "…" }` flagged `isError: true`. - -### Output - -Every result carries the machine-readable `structuredContent` below — a `oneOf` over the five variants, published as the tool's `outputSchema` — alongside the human text (the same output string, then a `result:` line or a `running:` line, then the interrupt outcome). The shapes are what the tool **emits at runtime**, and `result`/`running` are **mutually exclusive**: an eval result is exactly one of the finished, still-running, or thrown-eval variants. - -```ts -type ReplToolOutput = - | ReplEvalResult | ReplEvalStillRunning | ReplEvalThrown - | ReplInterruptResult | ReplErrorResult; - -interface ReplEvalResult { // the code finished within the soft bound - output: string; // ONE newline-joined string: console lines (one per call), - // checkpoint lines, error renderings, §6 notices - result: string; // the completion value's §4.4 repr (a guest undefined renders "undefined") -} - -interface ReplEvalStillRunning { // the bound elapsed first; the eval continues server-side - output: string; - running: string[]; // the in-flight call ids (c1, c2, … — what interrupt targets) -} - -interface ReplEvalThrown { // the code threw (or was broken mid-run) - output: string; // the §4.6 error rendering — no completion value exists -} - -interface ReplInterruptResult { - interrupt: { - outcome: "targeted" | "refused-idle" | "cancelled" | "idle" | "failed" | "none"; - callId?: string; // present on the id path - }; -} - -interface ReplErrorResult { // isError: true — a missing project context - error: string; -} -``` - -### The guest API, printing, and checkpoints - -`agent(modelSpec, task, opts?)` spawns an ACP subagent on a registry built-in (currently **Claude, Codex, OpenCode, and pi**) or a registered custom agent. The spec is `"backend/model"` — a bare `"backend"` runs its default model — and an unknown backend segment rejects the call **synchronously**, naming the segment and enumerating the known backends (a spec with no known-backend prefix is an error, never a silent route to the default backend). The option keys are `schema` (a structured-output JSON schema, validated per call), `cwd`, `configOptions` (backend-specific knobs, validated at admission — a typo'd key fails in milliseconds naming the valid alternatives), and `mode`. Use `mode` only when the selected `workflow` `action:"config"` entry's `modes.availableModes` explicitly lists its exact id; `modes:null` means omit it, and never invent `"default"`. For example: `agent("pi//", "research X and report the top 3 findings", { cwd: "/repo" })`. An unknown option key rejects synchronously too. Retain the promise-handle before awaiting it. `steer` is transient active-turn control only; `queue` creates a durable, independently awaitable FIFO turn on the same session; `cancel` targets the current public turn, while a queued handle's `cancel` targets that exact queued turn. - -`checkpoint(question)` parks a promise for a human answer **inside the VM**. The raised checkpoint surfaces as an **output line** — `checkpoint c9: ` — and a later eval's `checkpoint.answer("c9", value)` resolves it. No side protocol: the question rides the ordinary output string and the answer rides the ordinary `eval` input. - -Printing follows Python-ish conventions, **with no byte ceilings anywhere** (§4.4): strings passed **directly** to `console.log` — and a string **completion value** — print **whole** (they are the output the orchestrator asked for); objects/arrays render to **depth 2**, deeper levels as `{…}` / `[…]`; collections render their first **20 entries** per level, then `… +N more`; **nested** strings render head-limited at **200 chars**. Everything deeper/longer is reached by evaluating a narrower expression — the values are alive in the VM; slicing is the API. `_` holds the previous eval's completion value, IPython-style — bindings are the memory. - -Introspection is in-band guest data: `workspace()` returns `{ bindings: [{ name, type, sizeBytes, provenance, task, callId?, status? }], inFlight, checkpoints, diagnostics }` (with `diagnostics` carrying the §6 demotions — the last reconcile summary, a retained drain error, `childrenClosed`); `agents()` lists `{ callId, modelSpec, task, state, supportsSteering, queuedTurns }`, including each unsettled queued turn under its own call ID; `reset()` tears the workspace down after the current eval completes. Subagent output passes through **unfiltered** — backend harness noise (e.g. codex's "Warning: Skill descriptions were shortened…") is forwarded verbatim, never curated away; expect it when the backend prints it. - -### The workspace project model and durability - -Workspaces follow the daemon's project model exactly: **one VM per `projectDir`**, addressed by the same required-in-daemon-mode argument the `workflow` tool uses. MCP-session churn — client restarts, transport eviction — never touches the workspace; the daemon's lifetime plus disk snapshots carry it across everything else. - -- **Snapshots are implicit and boundary-durable.** There is no snapshot action. The workspace is written to the daemon's per-project `repl/` store (beside the workflow state, under the same project key) at **every state-changing boundary** — after each eval, and after each settlement drain that changed VM state — as a self-identifying envelope (the `quickjs.wasm` binary's SHA-256 + a format version + gzip compression). Because durability is boundary-based, a daemon kill loses at most the *in-flight* operation that had not yet reached a boundary; every committed boundary — and, through the append-only call store, every recorded subagent result — is durable and reconciled on the next touch. -- **Restore is lazy, on first touch.** There is no daemon-startup restore sweep. The VM is restored the first time a `repl` call addresses the project: host callbacks are re-registered by name, and every outstanding subagent call is reconciled three ways — **settled from the store** if it completed while the daemon was down, **re-attached** to its still-running ACP session (all four built-in backends advertise `loadSession`), or **re-issued** if it was lost. The reconcile summary demotes to `workspace().diagnostics.reconcile`; the next eval's output carries a one-line notice only when calls were **lost** (`failedLost` non-empty) — losses are never silent. -- **A refused snapshot AUTO-RESETS.** A snapshot that cannot be restored with the running engine — corrupt, a format upgrade, or a `quickjs.wasm` hash mismatch after a package bump — no longer poisons every call until a manual reset. The workspace **auto-resets and starts fresh**, and the refused snapshot file is **renamed aside** (`.refused-`, never deleted — auto-reset must not be silent data destruction). The next eval's `output` **leads with a loud one-line notice** naming the file and the reason. The daemon never crash-loops and never silently discards the data. -- **Subagent processes are client-presence keyed.** Child ACP processes stay warm while any MCP client is connected to the project. On last-client disconnect the workspace **drains**: in-flight subagent turns run to completion (each settlement boundary snapshots, so "close the laptop while two researchers run" ends with the findings durable), bounded by the daemon's session-eviction TTL (`AGENTPRISM_SESSION_TTL_MS`, default 2 h) — a turn that overruns the bound is force-settled as the recoverable `AGENT_CANCELLED` — then idle children close (`childrenClosed: true`). Pending queue items remain durable. A client that reconnects **mid-drain aborts it**, keeping the children warm. On the next connect the workspace is live (or restores), and the next eligible queue head re-attaches its recorded subagent session lazily. A drain that fails (a snapshot-flush error) is never silent: the failure is retained under `workspace().diagnostics.drainError`, the next eval's output carries the one-line loss notice (the failed drain **lost state** — the workspace was not persisted), and the next disconnect retries the drain. - -**Interrupting a running eval is not universal.** An eval that **yields** (suspends on a subagent call or checkpoint) is broken by the QuickJS interrupt handler the next time its continuation runs. A **fully synchronous** runaway wedges the daemon's single thread, so the `interrupt` request cannot even be processed mid-run; it is broken **out of band** by a worker-thread relay that the stdio shim (or the `--in-process` relay transport) fires *before* forwarding the call — **a host connected directly over HTTP has no such relay**, and falls back to the per-eval wall-clock deadline. That deadline (`AGENTPRISM_REPL_EVAL_TIMEOUT_MS`, default 30 000 ms) is the last-resort bound in every mode. The no-id `interrupt` therefore honestly reports `refused-idle` for the cases it cannot break (a never-settling local promise, an older restored guest without the continuation-lease seam). - ---- - ## The `author-workflow` prompt The server also exposes one [MCP prompt](https://modelcontextprotocol.io/docs/concepts/prompts): **`author-workflow`**. Prompts are a *user-controlled* primitive, so this adds no additional tool. @@ -712,7 +546,6 @@ All settings are read from the environment of the `agentprism-workflow` process | `AGENTPRISM_PI_ACP_CMD` | bundled `@automatalabs/pi-acp` | Override the command used to launch pi ACP. | | `AGENTPRISM_PI_ACP_ARGS` | — | Whitespace-separated argv passed only when `AGENTPRISM_PI_ACP_CMD` is set. | | `AGENTPRISM_PERSISTENCE_ROOT` | `~/.agentprism/workflows` | Absolute root for persisted run journals and logs used by resume. | -| `AGENTPRISM_REPL_EVAL_TIMEOUT_MS` | `30000` | Per-eval wall-clock deadline (ms) for a `repl` workspace — the last-resort bound on a runaway eval the interrupt handler and out-of-band relay can't otherwise reach. Used only when it parses to an integer ≥ 1; any invalid, zero, or negative value falls back to the 30 000 ms default. There is no upper bound. | --- @@ -733,7 +566,7 @@ const run = await runDynamicWorkflow( console.log(run.status, run.result); ``` -This MCP-server package does export its own building blocks, for hosts that want to mount the same surface on a transport they control rather than the default stdio one. `createWorkflowServer(runner)` registers the `workflow` and `repl` tools, the workflow authoring skill (plus its `skills/list`, `skills/get`, resource-read, and directory-read surface), the capability-gated `workflow_monitor` launcher, app-only `workflow-events`, `workflow-runs`, and `workflow-notifications` tools, and the `author-workflow` prompt. The `repl` workspaces default to a private client-presence ledger and a server-owned eval-break channel. `CreateWorkflowServerOptions` exposes `protocolEra` for SDK serving factories, plus `replRunner`, `replPresence`, `replClientId`, `replEvalBreakChannel`, and `replDrainBoundMs` for host lifecycle integration (the daemon passes shared instances). Workflow setup and continuation use durable run state; the server has no request-state codec or token verifier: +This MCP-server package does export its own building blocks, for hosts that want to mount the same surface on a transport they control rather than the default stdio one. `createWorkflowServer(runner)` registers the `workflow` tool, the workflow authoring skill (plus its `skills/list`, `skills/get`, resource-read, and directory-read surface), the capability-gated `workflow_monitor` launcher, app-only `workflow-events`, `workflow-runs`, and `workflow-notifications` tools, and the `author-workflow` prompt. `CreateWorkflowServerOptions` exposes `protocolEra` for SDK serving factories, plus `clientId` (the legacy-era MCP client identity that scopes `workflow_monitor` notification claims; the daemon passes each session's id). Workflow setup and continuation use durable run state; the server has no request-state codec or token verifier: ```ts import { createWorkflowServer, WorkflowPermissionBroker } from "@automatalabs/mcp-server"; @@ -751,7 +584,6 @@ await serveStdio(({ era }) => createWorkflowServer(runner, { protocolEra: era, p > **Use an SDK serving entry for dual-era hosting.** A hand-constructed server connected directly to `StdioServerTransport` intentionally serves only the legacy era. `serveStdio(factory)` performs the official modern/legacy arbitration while registering each tool once through the factory. The bundled `main()` additionally supplies its internal relay transport, whose worker-thread stdin reader can fire the out-of-band eval-break for a fully synchronous runaway; a vanilla stdio transport remains bounded by the per-eval deadline for that case. -The REPL-specific exports are `replToolInputShape` / `replToolOutputShape` (the tool's Zod input/output schemas), the `ReplToolOptions` type, `createReplProjectState` / `ensureReplWorkspace` / `disposeReplProjectState` / `resetReplProjectState` and the `ReplProjectState` type (per-project workspace state), and `ReplPresenceLedger` (the client-presence drain). Other workflow-side exports include the individual-validator catalog `workflowToolInputShape`, canonical `workflowToolInputBranches` / `workflowToolCanonicalInputSchema`, strict `workflowToolInputSchema` / `parseWorkflowToolInput`, `clampWorkflowInput`, `CreateWorkflowServerOptions`, `WorkflowExecuteToolInput`, `WorkflowResumeToolInput`, `WorkflowSetupResponseToolInput`, `WorkflowStatusToolInput`, `WorkflowPermissionResponseToolInput`, `WorkflowStopToolInput`, `WorkflowPauseToolInput`, diff --git a/packages/mcp-server/package.json b/packages/mcp-server/package.json index df20266e..234a65aa 100644 --- a/packages/mcp-server/package.json +++ b/packages/mcp-server/package.json @@ -49,7 +49,6 @@ "prepublishOnly": "pnpm run build" }, "dependencies": { - "@automatalabs/repl-engine": "workspace:*", "@automatalabs/shared-types": "workspace:*", "@automatalabs/workflows": "workspace:*", "@modelcontextprotocol/client": "^2.0.0", diff --git a/packages/mcp-server/src/daemon/constants.ts b/packages/mcp-server/src/daemon/constants.ts index cddc5dac..0c906f71 100644 --- a/packages/mcp-server/src/daemon/constants.ts +++ b/packages/mcp-server/src/daemon/constants.ts @@ -32,15 +32,6 @@ export const DAEMON_IDLE_TTL_ENV = "AGENTPRISM_DAEMON_IDLE_TTL_MS"; export const SESSION_IDLE_TTL_MS = 5 * 60_000; export const SESSION_IDLE_TTL_ENV = "AGENTPRISM_SESSION_TTL_MS"; -/** - * The REPL client-presence drain bound: after a project's last client disconnects, in-flight - * subagent turns may drain for up to this long before idle children are closed. Its own knob, - * deliberately decoupled from the session-eviction TTL above (the two used to share one - * constant, which forced dead-client eviction to wait hours). - */ -export const REPL_DRAIN_BOUND_MS = 2 * 60 * 60_000; -export const REPL_DRAIN_BOUND_ENV = "AGENTPRISM_REPL_DRAIN_BOUND_MS"; - export const REAPER_INTERVAL_MS = 60_000; export const EVENT_STORE_MAX_EVENTS_PER_STREAM = 1_000; diff --git a/packages/mcp-server/src/daemon/daemon-info.ts b/packages/mcp-server/src/daemon/daemon-info.ts index 467f86fc..23d59c02 100644 --- a/packages/mcp-server/src/daemon/daemon-info.ts +++ b/packages/mcp-server/src/daemon/daemon-info.ts @@ -38,7 +38,7 @@ import { import { dirname, join } from "node:path"; import { workflowHomeDir } from "@automatalabs/workflows"; -import { DAEMON_NAME, DAEMON_IDLE_TTL_ENV, REPL_DRAIN_BOUND_ENV, SESSION_IDLE_TTL_ENV } from "./constants.js"; +import { DAEMON_NAME, DAEMON_IDLE_TTL_ENV, SESSION_IDLE_TTL_ENV } from "./constants.js"; export interface DaemonInfo { name: typeof DAEMON_NAME; @@ -55,13 +55,6 @@ export interface DaemonInfo { controlUrl?: string; /** Internal run-control protocol version. */ controlProtocol?: 1; - /** The REPL eval-break relay's loopback endpoint (see - * `repl-engine`'s `EvalBreakChannel`): the shim fires the interrupt - * tool's no-id break here while the daemon's main thread is blocked - * in a synchronous eval. Absent on older daemons (the shim then - * skips the out-of-band fire and the per-eval deadline remains the - * bound). */ - replBreakUrl?: string; } export interface SpawnLock { @@ -340,7 +333,7 @@ export function isDaemonProcess(pid: number): boolean { * The lifetime knobs are excluded — they do not change what the daemon serves. */ const ENV_FINGERPRINT_PREFIXES = ["AGENTPRISM_", "OPENCODE_ACP_", "PI_ACP_", "CODEX_ACP_"]; -const ENV_FINGERPRINT_EXCLUDED = new Set([DAEMON_IDLE_TTL_ENV, SESSION_IDLE_TTL_ENV, REPL_DRAIN_BOUND_ENV]); +const ENV_FINGERPRINT_EXCLUDED = new Set([DAEMON_IDLE_TTL_ENV, SESSION_IDLE_TTL_ENV]); export function envFingerprint(env: Record = process.env): string { const relevant = Object.entries(env) diff --git a/packages/mcp-server/src/daemon/daemon-lifecycle.ts b/packages/mcp-server/src/daemon/daemon-lifecycle.ts index 88a8273e..87ded87e 100644 --- a/packages/mcp-server/src/daemon/daemon-lifecycle.ts +++ b/packages/mcp-server/src/daemon/daemon-lifecycle.ts @@ -7,11 +7,11 @@ * * Supersession (a newer daemon owns this family's discovery pointer) turns the daemon into a * lame duck, and the reaper then actively drains it instead of waiting for idleness: - * - every session with no request in flight and no REPL workspace mid-turn is closed so its + * - every session with no request in flight is closed so its * client transparently re-initializes on the successor; workflow execution remains on the * predecessor and is reached through the internal run-control plane; * - durable whole-stop intents are scanned on every reaper cadence; - * - the moment nothing is busy — no sessions, runs, requests, or REPL drains — the daemon exits, + * - the moment nothing is busy — no sessions, runs, or requests — the daemon exits, * regardless of the idle TTL (even a disabled one: a superseded daemon with nothing to do is * garbage, not a long-lived service). */ @@ -78,8 +78,8 @@ export function installDaemonLifecycle(options: DaemonLifecycleOptions): DaemonL ); } // MCP sessions are front-door state, not workflow ownership. Migrate every drainable - // session independently; in-flight requests and busy REPL workspaces remain protected by - // SessionRegistry's eviction predicate. + // session independently; in-flight requests remain protected by SessionRegistry's + // eviction predicate. const migrated = daemon.evictDrainableSessions(); if (migrated.length > 0) { log(`[agentprism-daemon] migrated ${migrated.length} idle session(s) to the successor: ${migrated.join(", ")}`); @@ -91,19 +91,10 @@ export function installDaemonLifecycle(options: DaemonLifecycleOptions): DaemonL } } - // Idleness means NO sessions, NO active workflow runs, AND NO active - // REPL client-presence drain (phase-E review rejection round 2: the - // drain used to be invisible to the accounting, so with the final - // session deleted the default 15-minute idle shutdown could fire - // while a last-client-disconnect drain was legitimately running - // toward its full bound — and the shutdown path then replaced the - // drain's bound with the five-second shutdown deadline, so in-flight - // turns were not guaranteed to drain to completion under the - // documented bound). + // Idleness means no sessions, no active workflow runs, and no request in flight. const busy = daemon.sessions.size > 0 || daemon.activeRunCount() > 0 || - daemon.inflightRequestCount() > 0 || - daemon.activeReplDrainCount() > 0; + daemon.inflightRequestCount() > 0; if (busy) { idleSince = undefined; return; @@ -116,7 +107,7 @@ export function installDaemonLifecycle(options: DaemonLifecycleOptions): DaemonL if (options.idleTtlMs <= 0) return; idleSince ??= Date.now(); if (Date.now() - idleSince >= options.idleTtlMs) { - log(`[agentprism-daemon] idle for ${options.idleTtlMs}ms with no sessions, runs, or repl drains; shutting down`); + log(`[agentprism-daemon] idle for ${options.idleTtlMs}ms with no sessions or runs; shutting down`); void lifecycle.shutdown("idle"); } }, options.reaperIntervalMs ?? REAPER_INTERVAL_MS); diff --git a/packages/mcp-server/src/daemon/http-daemon.ts b/packages/mcp-server/src/daemon/http-daemon.ts index 1b24d233..01a642fa 100644 --- a/packages/mcp-server/src/daemon/http-daemon.ts +++ b/packages/mcp-server/src/daemon/http-daemon.ts @@ -26,16 +26,14 @@ import { toWebRequest, } from "@modelcontextprotocol/node"; import type { AgentRunner } from "@automatalabs/shared-types"; -import type { BrokerRunner, EvalBreakChannel } from "@automatalabs/repl-engine"; import { createWorkflowServer, SERVER_VERSION } from "../server.js"; import { workflowRunEventsUri } from "../workflow-resources.js"; import { WorkflowProjectRegistry } from "../project-registry.js"; -import { ReplPresenceLedger } from "../repl-presence.js"; import { WorkflowPermissionBroker } from "../workflow-permissions.js"; import { workflowLifecycle } from "../workflow-lifecycle.js"; import { workflowToolInputBranches } from "../workflow-tool-input.js"; -import { DAEMON_NAME, HEALTHZ_PATH, MCP_ENDPOINT_PATH, REPL_DRAIN_BOUND_MS } from "./constants.js"; +import { DAEMON_NAME, HEALTHZ_PATH, MCP_ENDPOINT_PATH } from "./constants.js"; import { envFingerprint, readDaemonInfo } from "./daemon-info.js"; import { BoundedEventStore } from "./event-store.js"; import { validateRequest } from "./middleware.js"; @@ -61,29 +59,6 @@ export interface CreateDaemonOptions { host?: string; env?: Record; log?: (line: string) => void; - /** - * The REPL workspaces' ACP runner (the broker's structural seam; - * omitted: every workspace's broker owns its own `AcpAgentRunner`). - */ - replRunner?: BrokerRunner; - /** - * The concrete REPL client-presence drain bound: a project whose last - * client disconnected drains its in-flight subagent turns up to this - * bound, then closes idle children. Defaults to `REPL_DRAIN_BOUND_MS` - * (its own knob — decoupled from the session-eviction TTL, which is - * now short enough to collect dead clients promptly). - */ - replDrainBoundMs?: number; - /** @deprecated alias of `replDrainBoundMs` (the two used to share one constant). */ - sessionTtlMs?: number; - /** The REPL eval-break relay (phase-F review round 2; see - * repl-engine's `EvalBreakChannel`): the worker-thread channel whose - * loopback endpoint the shim fires while the daemon's main thread is - * blocked in a synchronous eval. The daemon passes its own channel - * (single-project servers own one by default — round 3: the - * in-process mode's relay transport fires it, see - * `repl-stdio-transport.ts`). */ - evalBreakChannel?: EvalBreakChannel; /** * This daemon's identity for discovery/succession accounting. Defaults to `process.pid`; * injected in tests. It is the pid reported by /healthz and compared against `daemon.json` @@ -115,21 +90,12 @@ export interface DaemonHandle { sessions: SessionRegistry; projects: WorkflowProjectRegistry; activeRunCount(): number; - /** - * The number of REPL workspaces with a client-presence drain scheduled - * or in flight (the daemon idleness accounting seam — phase-E review - * rejection round 2: a drain may legitimately run for the full - * session-eviction TTL after the last session is gone, and the idle - * shutdown must never replace that bound with the shutdown deadline). - */ - activeReplDrainCount(): number; /** Requests (POSTs) being processed right now, across every session. */ inflightRequestCount(): number; /** True when a newer daemon owns this family's discovery pointer (this one is a lame duck). */ isSuperseded(): boolean; /** - * The lame-duck migration: close every session with no request in flight and no REPL - * workspace mid-turn, so its client transparently re-initializes on the successor. Returns + * The lame-duck migration: close every session with no request in flight, so its client transparently re-initializes on the successor. Returns * the closed session ids. */ evictDrainableSessions(): string[]; @@ -375,22 +341,6 @@ export async function createDaemon(options: CreateDaemonOptions): Promise workflowLifecycle(context, options.runner).respond(input), log, }); - // The REPL client-presence ledger: every session touches the projects it addresses; on - // last-connection-closed a project with no clients left is drained (the doc's - // client-presence policy; the bound reuses the session-eviction TTL). - const replDrainBoundMs = options.replDrainBoundMs ?? options.sessionTtlMs ?? REPL_DRAIN_BOUND_MS; - const replPresence = new ReplPresenceLedger(replDrainBoundMs); - // The three presence signals (phase-E review rejection: only the - // disconnect was wired — a transient standalone-GET drop followed by a - // reconnect of the SAME live session used to leave the session's - // projects draining while the client was connected, because the - // reconnect never re-added its presence). A connection OPEN re-adds - // the session's project presence from its retained affinity; the - // last-connection-closed removes presence and schedules the drain; a - // session DELETE drops the retained affinity. - sessions.onConnectionOpened = (sessionId) => replPresence.reconnect(sessionId); - sessions.onLastConnectionClosed = (sessionId) => replPresence.disconnect(sessionId); - sessions.onSessionDeleted = (sessionId) => replPresence.forget(sessionId); let boundPort = options.port; let modernInflight = 0; @@ -403,17 +353,10 @@ export async function createDaemon(options: CreateDaemonOptions): Promise { - const clientId = `modern:${randomUUID()}`; return createWorkflowServer(options.runner, { projects, requireProjectDir: true, - replRunner: options.replRunner, - replPresence, - replClientId: () => clientId, - replDrainBoundMs, - replEvalBreakChannel: options.evalBreakChannel, protocolEra: "modern", - disconnectReplClientOnClose: true, modernNotifier, runControl, permissionBroker, @@ -518,11 +461,7 @@ export async function createDaemon(options: CreateDaemonOptions): Promise transport.sessionId, - replDrainBoundMs, - replEvalBreakChannel: options.evalBreakChannel, + clientId: () => transport.sessionId, runControl, permissionBroker, }); @@ -613,21 +552,14 @@ export async function createDaemon(options: CreateDaemonOptions): Promise projects.activeRunCount(), - activeReplDrainCount: () => replPresence.drainingCount(), inflightRequestCount: () => sessions.inflightCount() + modernInflight, isSuperseded, - evictDrainableSessions: () => sessions.evictDrainable((sessionId) => replPresence.sessionHasBusyWorkspace(sessionId)), + evictDrainableSessions: () => sessions.evictDrainable(), processPendingControlIntents: () => runControl.processPendingIntents(), async close() { const closed = new Promise((resolvePromise) => { httpServer.close(() => resolvePromise()); }); - // Shutdown drains each repl workspace with the shutdown bound - // before the broker teardown (the reviewer-mandated drain-then- - // close posture; the last-client-disconnect path uses the full - // session-eviction TTL instead). - await projects.disposeReplStates(); - replPresence.disconnectAll(); await sessions.closeAll(); detachModernRunEvent(); detachModernRunDeleted(); diff --git a/packages/mcp-server/src/daemon/run-daemon.ts b/packages/mcp-server/src/daemon/run-daemon.ts index f4ae14ba..1d6a4338 100644 --- a/packages/mcp-server/src/daemon/run-daemon.ts +++ b/packages/mcp-server/src/daemon/run-daemon.ts @@ -15,8 +15,6 @@ import { DAEMON_NAME, DAEMON_PORT_ENV, DEFAULT_DAEMON_PORT, - REPL_DRAIN_BOUND_ENV, - REPL_DRAIN_BOUND_MS, SESSION_IDLE_TTL_ENV, SESSION_IDLE_TTL_MS, } from "./constants.js"; @@ -30,7 +28,6 @@ import { } from "./daemon-info.js"; import { installDaemonLifecycle } from "./daemon-lifecycle.js"; import { createDaemon, DaemonPortInUseError } from "./http-daemon.js"; -import { createEvalBreakChannel } from "@automatalabs/repl-engine"; import { WorkflowPermissionBroker } from "../workflow-permissions.js"; export interface RunDaemonOptions { @@ -78,7 +75,6 @@ export async function runDaemon(options: RunDaemonOptions = {}): Promise<"starte let daemon; const sessionTtlMs = envInt(SESSION_IDLE_TTL_ENV, SESSION_IDLE_TTL_MS); - const replDrainBoundMs = envInt(REPL_DRAIN_BOUND_ENV, REPL_DRAIN_BOUND_MS); const describePortHolder = (port: number): string => { const holder = findDaemonInstanceOnPort(port); if (holder === undefined) return `port ${port} is taken by another process`; @@ -87,13 +83,7 @@ export async function runDaemon(options: RunDaemonOptions = {}): Promise<"starte `(v${holder.info.version}, started ${holder.info.startedAt})` ); }; - // The eval-break relay (phase-F review round 2): a worker-thread - // channel whose loopback endpoint stays reachable while the daemon's - // main thread is blocked in a synchronous eval — the `interrupt` - // tool's no-id break. Its address travels in daemon.json so the shim - // can fire it out of band. - const evalBreakChannel = createEvalBreakChannel(); - const daemonOptions = { runner, permissionBroker, log, replDrainBoundMs, evalBreakChannel, ownInstanceId: instanceId }; + const daemonOptions = { runner, permissionBroker, log, ownInstanceId: instanceId }; if (supersede) { // Succession: the stale predecessor may still hold the default port and is left running // to finish its in-flight work, so never contend for it — bind the explicitly requested @@ -144,10 +134,6 @@ export async function runDaemon(options: RunDaemonOptions = {}): Promise<"starte instanceId: daemon.instanceId, controlUrl: daemon.controlUrl, controlProtocol: 1, - ...(await evalBreakChannel - .breakUrl() - .then((url) => ({ replBreakUrl: url })) - .catch(() => ({}))), }); installDaemonLifecycle({ diff --git a/packages/mcp-server/src/daemon/session-registry.ts b/packages/mcp-server/src/daemon/session-registry.ts index eb77a90b..e8e35080 100644 --- a/packages/mcp-server/src/daemon/session-registry.ts +++ b/packages/mcp-server/src/daemon/session-registry.ts @@ -14,22 +14,6 @@ * distinction to migrate sessions that have nothing in flight — closing such a session costs * the client nothing but a transparent re-initialize on the successor — while never cutting a * request that is being processed. - * - * The presence signals are tri-state, and the REPL ledger needs all three (the roadmap - * doc's client-presence policy): - * - * - `onConnectionOpened` — a connection opened on a live session. A TRANSIENT drop of the - * standalone GET stream closes the session's LAST connection (the signal below) but the - * session itself is still alive; when the client reconnects, this signal re-adds its - * presence (the ledger keeps the session's project affinity across the drop, so the - * reconnect restores presence WITHOUT a new tool call — a scheduled drain must not close - * children while that client is connected). - * - `onLastConnectionClosed` — the session's LAST open connection closed (or the session - * was deleted): the client is gone, project presence is removed, and the REPL drain - * policy evaluates. - * - `onSessionDeleted` — the session record is gone outright (DELETE, transport close, - * eviction): the ledger drops the session's retained project affinity (a re-initialized - * client gets a NEW session id and must re-touch projects). */ import type { NodeStreamableHTTPServerTransport } from "@modelcontextprotocol/node"; import type { WorkflowServer } from "../server.js"; @@ -46,41 +30,6 @@ export interface SessionRecord { export class SessionRegistry { private readonly sessions = new Map(); - /** - * Fired when a connection OPENS on a live session — the daemon's - * client-RECONNECT signal. The daemon wires it to the REPL presence - * ledger, which re-adds the session's project presence from its - * retained affinity (see `repl-presence.ts`): a transient GET drop - * followed by a reconnect of the SAME session must not leave the - * session's projects draining while the client is connected - * (phase-E review rejection: only disconnects were wired, so a - * reconnect did not restore presence until the client's next tool - * call — the already-scheduled drain could close children while that - * client was connected). - */ - onConnectionOpened: ((sessionId: string) => void) | undefined; - /** - * Fired when a session's LAST open connection closed (or the session - * was deleted outright) — the daemon's client-presence signal. The - * daemon wires it to the REPL presence ledger, which drains projects - * whose client set became empty (the roadmap doc's last-client- - * disconnect drain; phase-D review round 2: the registry used to - * maintain connection counts without ever signaling project REPL - * lifecycle). - */ - onLastConnectionClosed: ((sessionId: string) => void) | undefined; - /** - * Fired when the session record is deleted (DELETE, transport close, - * eviction) — the daemon's session-GONE signal. The daemon wires it - * to the REPL presence ledger, which drops the session's retained - * project affinity (the session can never reconnect; a re-initialized - * client carries a new session id). Fired AFTER - * `onLastConnectionClosed` (the disconnect's drain evaluation needs - * the affinity to remove the session's presence from its projects - * first). - */ - onSessionDeleted: ((sessionId: string) => void) | undefined; - add(record: Omit & { inflightRequests?: number }): void { this.sessions.set(record.sessionId, { ...record, inflightRequests: record.inflightRequests ?? 0 }); } @@ -91,11 +40,6 @@ export class SessionRegistry { delete(sessionId: string): void { this.sessions.delete(sessionId); - // Disconnect FIRST (the ledger's presence removal walks the - // session's retained project affinity), then drop the affinity - // itself. - this.onLastConnectionClosed?.(sessionId); - this.onSessionDeleted?.(sessionId); } touch(sessionId: string, now = Date.now()): void { @@ -108,10 +52,6 @@ export class SessionRegistry { if (record === undefined) return; record.openConnections++; record.lastActivityAt = Date.now(); - // A connection opened on a live session: the client (re)connected - // — restore its project presence (the ledger re-adds it from the - // retained affinity). - this.onConnectionOpened?.(sessionId); } connectionClosed(sessionId: string): void { @@ -119,12 +59,6 @@ export class SessionRegistry { if (record === undefined) return; record.openConnections = Math.max(0, record.openConnections - 1); record.lastActivityAt = Date.now(); - if (record.openConnections === 0) { - // The last connection closed: the client is gone (a live client - // always holds an open connection — the standalone GET stream or - // an in-flight POST). Signal the lifecycle hook. - this.onLastConnectionClosed?.(sessionId); - } } /** A request (POST) started being processed on the session. */ @@ -169,13 +103,11 @@ export class SessionRegistry { /** * Close every session with NO request in flight — the lame-duck migration: the client's * next frame (or its standalone GET stream ending) makes it re-initialize on the successor. - * `keep` can veto a session (the daemon keeps sessions whose REPL workspace is mid-turn). */ - evictDrainable(keep: (sessionId: string) => boolean = () => false): string[] { + evictDrainable(): string[] { const evicted: string[] = []; for (const record of this.sessions.values()) { if (record.inflightRequests > 0) continue; - if (keep(record.sessionId)) continue; evicted.push(record.sessionId); void record.transport.close().catch(() => undefined); } diff --git a/packages/mcp-server/src/index.ts b/packages/mcp-server/src/index.ts index c0324c64..3f3faaca 100644 --- a/packages/mcp-server/src/index.ts +++ b/packages/mcp-server/src/index.ts @@ -8,14 +8,10 @@ import { realpathSync } from "node:fs"; import { pathToFileURL } from "node:url"; import { createAcpRunner } from "@automatalabs/workflows"; -import { createEvalBreakChannel } from "@automatalabs/repl-engine"; -import { serveStdio } from "@modelcontextprotocol/server/stdio"; +import { serveStdio, StdioServerTransport } from "@modelcontextprotocol/server/stdio"; -import { REPL_DRAIN_BOUND_MS } from "./daemon/constants.js"; import { installMcpServerLifecycle } from "./lifecycle.js"; import { WorkflowProjectRegistry } from "./project-registry.js"; -import { ReplPresenceLedger } from "./repl-presence.js"; -import { ReplRelayStdioTransport } from "./repl-stdio-transport.js"; import { createWorkflowServer, type WorkflowServer } from "./server.js"; import { workflowRunEventsUri } from "./workflow-resources.js"; import { WorkflowPermissionBroker } from "./workflow-permissions.js"; @@ -117,17 +113,6 @@ export type { GeneratedAuthoringSkill, GeneratedAuthoringSkillResource, } from "./generated/authoring-skills-content.js"; -export { replToolInputShape, replToolOutputShape } from "./repl-tool.js"; -export type { ReplToolOptions } from "./repl-tool.js"; -export { - createReplProjectState, - ensureReplWorkspace, - disposeReplProjectState, - resetReplProjectState, - renameAsideNeverOverwriting, -} from "./repl-project.js"; -export type { ReplProjectState } from "./repl-project.js"; -export { ReplPresenceLedger } from "./repl-presence.js"; export { RUN_MONITOR_RESOURCE_URI, WORKFLOW_EVENTS_TOOL_NAME, @@ -170,11 +155,6 @@ export type { * AgentRunner, inject it into the workflow-engine via the server shell, and serve on * stdin/stdout. Backend auth stays with the agents' own CLI credential stores; a run that * hits AUTH_REQUIRED pauses and continues under the same run ID after an out-of-band CLI login. - * The stdio transport is the RELAY transport (phase-F review round 3): its stdin reader - * lives on a worker thread that fires the server's out-of-band eval-break relay for - * `repl` interrupt calls, so the documented no-id interrupt works for a synchronously - * running eval in this mode too (the daemon mode's shim does the same from a separate - * process). */ export async function main(): Promise { const permissionBroker = new WorkflowPermissionBroker(); @@ -185,17 +165,12 @@ export async function main(): Promise { permissionBroker.attach(runner); const projects = new WorkflowProjectRegistry(runner); const defaultContext = projects.getOrCreate(process.cwd()); - const replPresence = new ReplPresenceLedger(REPL_DRAIN_BOUND_MS); - const evalBreakChannel = createEvalBreakChannel(); let activeServer: WorkflowServer | undefined; let activeEra: "legacy" | "modern" | undefined; - // The relay transport still owns the worker-thread eval-break fast path. serveStdio owns - // protocol-era arbitration and pins one factory instance to this long-lived connection. - const transport = new ReplRelayStdioTransport( - () => evalBreakChannel.breakUrl(), - () => defaultContext.projectDir, - ); + // serveStdio owns protocol-era arbitration and pins one factory instance to this long-lived + // connection; the transport is built here so the lifecycle below can watch it close. + const transport = new StdioServerTransport(); const detachModernEvents = projects.onRunEventPersisted((record) => { if (activeEra !== "modern") return; void activeServer?.server.sendResourceUpdated({ uri: workflowRunEventsUri(record.runId) }).catch(() => undefined); @@ -207,12 +182,7 @@ export async function main(): Promise { manager: defaultContext.manager, activeRuns: defaultContext.activeRuns, projects, - replPresence, - replClientId: () => "stdio-client", - replDrainBoundMs: REPL_DRAIN_BOUND_MS, - replEvalBreakChannel: evalBreakChannel, protocolEra: era, - disconnectReplClientOnClose: true, permissionBroker, }); activeServer = server; @@ -229,14 +199,9 @@ export async function main(): Promise { transport, server: { stopAcceptingWork: () => activeServer?.stopAcceptingWork(), - replBreakUrl: () => evalBreakChannel.breakUrl(), - replDefaultProjectDir: () => defaultContext.projectDir, - async disposeReplEvalBreakChannel() { + async dispose() { detachModernEvents(); permissionBroker.dispose(); - await projects.disposeReplStates(); - replPresence.disconnectAll(); - await evalBreakChannel.dispose(); }, }, }); diff --git a/packages/mcp-server/src/lifecycle.ts b/packages/mcp-server/src/lifecycle.ts index 1159b756..9f3d1016 100644 --- a/packages/mcp-server/src/lifecycle.ts +++ b/packages/mcp-server/src/lifecycle.ts @@ -9,31 +9,10 @@ export type McpServerShutdownReason = "stdin-close" | "stdin-end" | "transport-c /** A small server-owned admission gate, kept separate from the MCP transport lifecycle. */ export interface WorkflowServerControl { stopAcceptingWork(): void; - /** The REPL eval-break relay address (the out-of-band interrupt's - * fire side — phase-F review round 3: the in-process/library server - * owns an eval-break channel by default, so the documented no-id - * interrupt for a SYNCHRONOUSLY running eval is deliverable in every - * supported mode; a host whose main thread is blocked in a sync eval - * POSTs `{ key: projectDir }` here from another thread, exactly like - * the daemon mode's shim does). Resolves when the relay worker is - * listening. */ - replBreakUrl(): Promise; - /** The single-project server's own project key — the context the - * `repl` tool resolves when its projectDir argument is omitted - * (phase-F review round 4: the relay stdio transport fires the - * out-of-band eval-break with this key for an omitted-projectDir - * interrupt, so the documented optional-projectDir interrupt works - * for a synchronously running eval). Undefined in daemon mode - * (projectDir is required there). OPTIONAL for minimal third-party - * implementations. */ - replDefaultProjectDir?(): string | undefined; - /** Dispose the SERVER-OWNED eval-break channel (a caller-provided - * channel stays the caller's to dispose — the daemon owns its own). - * Idempotent; the channel's worker is unref'd, so a process can exit - * without this call. OPTIONAL for minimal third-party server - * implementations that don't own a channel (the lifecycle calls it - * defensively). */ - disposeReplEvalBreakChannel?(): Promise; + /** Release state the serving entry shares across server instances (event subscriptions, the + * permission broker). Runs once at shutdown, after the runner is disposed. OPTIONAL: a server + * that owns nothing of the kind omits it. */ + dispose?(): Promise; } interface DisposableRunner { @@ -158,12 +137,8 @@ export function installMcpServerLifecycle(options: McpServerLifecycleOptions): M shutdownPromise = disposeRunnerWithDeadline(options.runner, deadlineMs).then(() => { removeListeners(); - // The server-owned eval-break channel (the in-process mode's - // default channel) dies with the server; a caller-provided - // channel (the daemon's) is the caller's to dispose. Fire-and- - // forget: the exit below is authoritative, and the channel's - // worker is unref'd. - void options.server.disposeReplEvalBreakChannel?.().catch(() => undefined); + // Fire-and-forget: the exit below is authoritative. + void options.server.dispose?.().catch(() => undefined); processHandle.exit(exitCodeFor(reason)); }); return shutdownPromise; diff --git a/packages/mcp-server/src/project-registry.ts b/packages/mcp-server/src/project-registry.ts index f5b47dc4..29395b8b 100644 --- a/packages/mcp-server/src/project-registry.ts +++ b/packages/mcp-server/src/project-registry.ts @@ -20,9 +20,6 @@ import { workflowHomeDir, type WorkflowRunResult, } from "@automatalabs/workflows"; -import type { ReplProjectState } from "./repl-project.js"; -import { disposeReplProjectState } from "./repl-project.js"; -import { SHUTDOWN_DEADLINE_MS } from "./lifecycle.js"; import { deleteRunControlSidecars } from "./daemon/run-control-store.js"; import { WorkflowNotificationClaims } from "./workflow-notifications.js"; @@ -83,10 +80,6 @@ export interface ProjectContext { projectDir: string; manager: WorkflowManager; activeRuns: ActiveRunRegistry; - /** The REPL workspace's daemon state (phase D): created on first touch - * of the `repl` tool, null until then — a pure-workflow project never - * opens a repl store. See `src/repl-project.ts`. */ - repl?: ReplProjectState; } /** The routing surface WorkflowScriptResources needs — a registry, or a single pinned store. */ @@ -247,47 +240,6 @@ export class WorkflowProjectRegistry implements RunStoreRouter { return total; } - /** Dispose every context's REPL workspace: each one DRAINS with the - * shutdown bound first (in-flight subagent turns settle into the VM - * and snapshot; the reviewer-mandated drain-then-close posture — the - * old path cancelled busy sessions on disposal) — then the broker - * teardown (releasing every held ACP session) and the store close. - * Called by the daemon at shutdown; the workflow managers' own - * lifecycle is untouched. - * - * ONE deadline spans the drain AND the teardown (phase-D review - * round 7: the disposal used to run unbounded — a drain that failed - * or consumed the whole bound then entered a teardown that awaited - * hung cancel/release forever, so daemon shutdown could hang on the - * exact hung backend the drain had already caught). A drain that - * fails or times out leaves the teardown only the remaining bound; - * an expired deadline skips straight to the disposal's bookkeeping - * clear. `boundMs` defaults to the daemon's shutdown deadline (the - * engine's own dispose default mirrors it). */ - async disposeReplStates(boundMs: number = SHUTDOWN_DEADLINE_MS): Promise { - const deadline = Date.now() + Math.max(0, boundMs); - for (const context of this.contexts.values()) { - const state = context.repl; - if (state === undefined) continue; - const broker = state.broker; - if (broker !== null) { - await broker - .drainForDisconnect(Math.max(0, deadline - Date.now())) - .catch(() => undefined); - } - // The teardown's own failures are contained here too: its op-end - // flush retries a boundary the drain's failed flush retained, and - // a second failure (the disk is still broken) must not abort the - // shutdown — the VM release and the store close already ran in - // `disposeReplProjectState`'s FINALLY path (phase-D review round - // 8: a disposal rejection used to skip them entirely), the - // persistence failure was already loud (the drain's failure), and - // the process is exiting anyway. The state on disk keeps the last - // good snapshot. - await disposeReplProjectState(state, Math.max(0, deadline - Date.now())).catch(() => undefined); - } - } - snapshot(): Array<{ projectDir: string; activeRuns: number }> { return [...this.contexts.values()].map((context) => ({ projectDir: context.projectDir, diff --git a/packages/mcp-server/src/repl-presence.ts b/packages/mcp-server/src/repl-presence.ts deleted file mode 100644 index 9560cd88..00000000 --- a/packages/mcp-server/src/repl-presence.ts +++ /dev/null @@ -1,204 +0,0 @@ -/** - * The REPL workspaces' client-presence ledger — the daemon-side half of - * the roadmap doc's client-presence drain policy: "Child ACP processes - * stay warm while any MCP client is connected to the project (the - * daemon's session registry already measures liveness by connection - * presence). On last-client disconnect, in-flight subagent turns drain - * to completion — their results settle into the VM and each settlement - * boundary snapshots — and then idle children close. On the next client - * connect, the workspace is live (or restores from snapshot) and an - * eligible queue head re-attaches its recorded subagent session lazily. - * - * One ledger per daemon. `touch(state, clientId)` marks an MCP session - * as present on a project's workspace (every `repl` tool call from that - * session touches); `disconnect(clientId)` runs when the session's last - * connection closed (the daemon's session registry signals it — see - * `SessionRegistry.onLastConnectionClosed`) or the session was deleted. - * A project whose client set becomes EMPTY has its workspace drained: - * the scheduled `drainReplProject` runs the broker's - * `drainForDisconnect` — in-flight turns drain to completion (each - * settlement boundary snapshots), then every idle child closes - * (`keepSession` keeps the backend sessions re-openable). The concrete - * drain bound is the daemon's session-eviction TTL (the spec-owed - * decision: the bound REUSES the daemon's existing TTL knob rather than - * inventing a new one — in-flight turns already run under the runner's - * own runaway protections; the TTL is the outer ceiling). The workspace - * and broker stay alive; the next eligible queued turn lazily - * re-attaches its recorded backend session (the broker's - * capability-gated lazy re-attach), and a client reconnecting MID-drain - * ABORTS the drain — the broker consults the project's client set every - * iteration and before every destructive phase, so the children stay - * warm while any client is connected (phase-D review round 6: the drain - * used to run to its release phase and close every child regardless of - * presence). - * - * **Disconnect retains the session's project AFFINITY** (which projects - * the session touched), so a reconnect of the SAME live session — a - * transient standalone-GET drop, then the client reconnects without a - * new MCP session — restores its presence from `reconnect(clientId)` - * WITHOUT requiring a new tool call (phase-E review rejection: the - * session registry wired only disconnects, so a transient drop left the - * session's presence gone — the already-scheduled drain could close - * children while that client was connected). Only a session DELETION - * (`forget`) drops the affinity: a deleted session can never reconnect, - * and a re-initialized client carries a new session id. - * - * A drain that FAILS — a snapshot-flush failure mid-drain, for example - * — is never discarded silently (phase-D review round 6): the failure - * is recorded on the project state (`drainError`), surfaced loudly in - * every repl tool result, and the drain latch stays clear so the next - * disconnect retries the drain (the store's writer retains the failed - * boundary's dirty flag for that retry). - * - * Drains are single-flight per project (a second disconnect while a - * drain runs is a no-op), and the ledger keeps the project's client set - * accurate throughout — `touch` during a drain leaves the set non-empty - * for the next disconnect to re-evaluate. - */ - -import type { ReplProjectState } from "./repl-project.js"; -import { drainReplProject, disconnectReplProject, touchReplProject } from "./repl-project.js"; - -/** One MCP session's presence on the projects' repl workspaces. */ -export class ReplPresenceLedger { - /** sessionId → the repl states that session has touched (RETAINED - * across disconnects — the session's project affinity; dropped only - * by `forget` when the session is deleted, see the module docs). */ - private readonly bySession = new Map>(); - /** repl state → the sessions currently present on it. */ - private readonly byProject = new Map>(); - /** repl states with a drain scheduled or running (single-flight). */ - private readonly draining = new Set(); - - constructor(private readonly boundMs: number) {} - - /** The concrete drain bound (the daemon's session-eviction TTL). */ - drainBoundMs(): number { - return this.boundMs; - } - - /** - * Mark an MCP session as present on a project's repl workspace (every - * `repl` tool call from that session touches). Idempotent per - * (session, project). - */ - touch(state: ReplProjectState, clientId: string): void { - let sessions = this.byProject.get(state); - if (sessions === undefined) { - sessions = new Set(); - this.byProject.set(state, sessions); - } - sessions.add(clientId); - touchReplProject(state, clientId); - let projects = this.bySession.get(clientId); - if (projects === undefined) { - projects = new Set(); - this.bySession.set(clientId, projects); - } - projects.add(state); - } - - /** - * Run when an MCP session's last connection closed (or the session was - * deleted): remove its presence from every project it touched; a - * project whose client set became EMPTY is drained (single-flight). - * The session's project AFFINITY is retained (see the module docs) so - * a reconnect of the same live session can restore its presence; the - * drain decision reads the ledger's own per-project set (the - * authoritative presence — the same set `touch`/`reconnect` maintain), - * never a snapshot of the projects' `clients` sets. - */ - disconnect(clientId: string): void { - const projects = this.bySession.get(clientId); - if (projects === undefined) return; - for (const state of projects) { - const sessions = this.byProject.get(state); - let last = false; - if (sessions !== undefined) { - sessions.delete(clientId); - if (sessions.size === 0) { - this.byProject.delete(state); - last = true; - } - } - disconnectReplProject(state, clientId); - if (last) this.scheduleDrain(state); - } - } - - /** - * Run when a connection OPENS on a live session (the daemon's session - * registry signals it — a reconnect of the SAME session after a - * transient drop): restore the session's presence on every project it - * retains affinity with (see the module docs). A project whose drain - * was already scheduled or is mid-flight sees the re-added client and - * skips/aborts it — children stay warm while any client is connected. - */ - reconnect(clientId: string): void { - const projects = this.bySession.get(clientId); - if (projects === undefined) return; - for (const state of projects) { - this.touch(state, clientId); - } - } - - /** - * Run when a session record is deleted (DELETE, transport close, - * eviction): drop the session's retained project affinity. The - * session can never reconnect; a re-initialized client carries a new - * session id and re-touches projects through its tool calls. (The - * registry fires `disconnect` BEFORE this — the presence removal and - * drain evaluation walk the affinity.) - */ - forget(clientId: string): void { - this.bySession.delete(clientId); - } - - /** Every project currently drained or draining (the status seam). */ - drainedProjects(): ReplProjectState[] { - return [...this.draining]; - } - - /** Test seam: how many projects are mid-drain right now. */ - drainingCount(): number { - return this.draining.size; - } - - /** - * True when any workspace the session has affinity with has a subagent turn running. The - * lame-duck migration keeps such sessions: closing one would drain the workspace here while - * the migrated client re-opens it on the successor — a workspace split across two daemons. - */ - sessionHasBusyWorkspace(clientId: string): boolean { - const projects = this.bySession.get(clientId); - if (projects === undefined) return false; - for (const state of projects) { - if (state.broker !== null && state.broker.busySessionCount() > 0) return true; - } - return false; - } - - /** Drop every session's presence (daemon shutdown): the scheduled - * drains see the projects' client sets emptied and run to completion - * on the already-disposed brokers — a no-op there, cleared here. */ - disconnectAll(): void { - for (const clientId of [...this.bySession.keys()]) this.disconnect(clientId); - } - - private scheduleDrain(state: ReplProjectState): void { - if (this.draining.has(state)) return; - this.draining.add(state); - void drainReplProject(state, this.boundMs) - .catch(() => { - // The drain runs detached — there is no caller to propagate to. - // The failure is NOT silent: `drainReplProject` recorded it on - // the project state (`drainError`), every repl tool result - // surfaces it loudly, and the drain latch stayed clear so the - // next disconnect retries (phase-D review round 6: the failure - // used to vanish here). - }) - .finally(() => { - this.draining.delete(state); - }); - } -} diff --git a/packages/mcp-server/src/repl-project.ts b/packages/mcp-server/src/repl-project.ts deleted file mode 100644 index 97da8e71..00000000 --- a/packages/mcp-server/src/repl-project.ts +++ /dev/null @@ -1,603 +0,0 @@ -/** - * The REPL workspace's daemon wiring — phase D of the REPL orchestrator - * roadmap (docs/roadmap/repl-orchestrator.md): the per-project context - * opens the daemon's `repl/` store, attaches the broker's state-changing - * boundary sink, and on FIRST TOUCH either restores the stored workspace - * (VM from the enveloped snapshot, then the three-way reconcile) or - * creates a fresh one. This is the production wiring the phase D review - * demanded: `ReplWorkspaceStore` used to be exported/tested only, with no - * daemon project context opening it — the workspace did not survive - * daemon restarts. - * - * ## First-touch semantics (spec-owed decision) - * - * `ensureReplWorkspace` runs once per project context, on the first `repl` - * tool call that addresses the project: - * - * - **No stored snapshot** → a fresh workspace is created, the broker is - * attached (call store + snapshot sink + the interrupt signal), and - * every state-changing boundary persists from then on. - * - **A stored snapshot that loads** → the VM is restored with the same - * wasm binary the envelope's hash was compared against, the broker is - * attached, and `reconcile()` runs the three-way arm (completed-while- - * down → settle from the call store; still resumable at the backend → - * re-attach via `loadSession`; lost → re-issue). The reconcile report - * and the source (`restored`) are recorded on the state; the report - * demotes to `workspace().diagnostics` (§6.2), with a one-line notice - * in the next eval's output only when calls were LOST (`failedLost`). - * - **A stored snapshot that REFUSES** (corrupt/truncated, a format - * version bump, a wasm-hash mismatch naming both hashes, or a - * payload that passes every at-rest check but cannot be RESTORED — a - * corrupted in-range VM header, `SnapshotRestoreError`) → §6.1 - * AUTO-RESET: the refused file is renamed aside (`.refused-`, - * NEVER deleted — auto-reset must not be silent data destruction; - * the destination is COLLISION-SAFE — a same-millisecond second - * refusal bumps a counter suffix instead of overwriting an earlier - * aside), the CALL LEDGER is cleared with it (a fresh VM restarts - * ids at `c1`, and the store's first-wins replay must never hand a - * new call an old record's completion), a fresh workspace starts, - * and the next eval's output leads with a loud one-line notice - * naming the file and the reason. The daemon never crash-loops and - * never silently discards the data; a version bump therefore routes - * old snapshots through this path on first touch, exactly as the - * redesign intends. - * - * First touches are SINGLE-FLIGHT: concurrent first-touch calls share - * one in-flight promise (phase-D review round 2: an asynchronous null - * check followed by create/restore used to race — two concurrent first - * touches could create two VMs and brokers for one project, attach both - * to the same call store and snapshot path, and overwrite the shared - * state, violating the one-VM-per-project and single-writer persistence - * model). The state's `generation` counter makes `dispose`/`reset` - * during a first touch abort the touch's materialization (the created - * workspace is torn down without being registered). - * - * ## The eval-break interrupt and the eval deadline (spec-owed mechanism) - * - * The `interrupt` tool's eval-break path (no call id) targets the - * RUNNING eval through the broker (`Broker.armEvalBreak` — phase-E - * review rejection round 1: the signal used to live here as a - * project-wide boolean that an idle workspace's next eval — or an - * unrelated drain — could consume; the broker now tracks the in-flight - * eval's completion, refuses to arm when nothing is running, and the - * armed signal is consumed by the first subsequent execution of that - * eval — a settlement drain resuming its continuation, or a direct - * eval's own drain when a synchronous host-callback settlement like - * `checkpoint.answer` resumes it — phase-E review rejection round 2), - * breaking it MID-RUN through the quickjs interrupt handler. The daemon - * is single-threaded, so a request cannot be PROCESSED while a - * fully synchronous (never-yielding) eval executes — phase-F review - * round 2: the OUT-OF-BAND eval-break channel closes that gap (a - * worker-thread relay the MCP shim fires before forwarding; the - * running eval's quickjs interrupt handler consumes the shared-memory - * break flag mid-execution — see `repl-engine`'s `eval-break-channel.ts`), - * and the per-eval wall-clock deadline (the harness's eval guard) - * remains the last-resort bound: every eval and settlement drain runs - * under `BrokerOptions.evalTimeoutMs` enforced by the quickjs interrupt - * handler, so a runaway eval can never hang the workspace forever. An - * eval that YIELDS (suspends on a subagent call or a checkpoint) is - * interruptible at its next execution; the wait tool's pumps release - * the broker chain between iterations, so an interrupt lands promptly - * mid-wait. The call-cancel path - * (`interrupt { id }`) is immediate: it drives ACP `session/cancel` - * downward (lazily re-attaching a drained handle's recorded session - * first). - * - * ## Client presence and the drain (spec-owed decision) - * - * The doc's client-presence policy is wired here: `touchReplProject` - * marks an MCP session present (every `repl` tool call touches); - * `disconnectReplProject` runs when the session's last connection closed - * (the daemon's session registry signals it via the `ReplPresenceLedger` - * — see `src/repl-presence.ts`). A project with NO clients is drained: - * in-flight subagent turns drain to completion (their results settle - * into the VM and each settlement boundary snapshots — "close the laptop - * while two researchers run" ends with the findings durable), then idle - * children close. The concrete drain bound is the daemon's - * `REPL_DRAIN_BOUND_MS` (`AGENTPRISM_REPL_DRAIN_BOUND_MS`; it used to - * reuse the session-eviction TTL, which has since been decoupled so dead - * clients are collected promptly — the runner's own runaway protections - * already bound individual turns). - * The workspace and broker stay alive; the next eligible queued turn - * lazily re-attaches its recorded backend session (the broker's - * capability-gated lazy re-attach). - * - * ## Ownership - * - * The state owns its store and (through the broker) its ACP runner; the - * workspace is the caller's per-context lifetime. - * `disposeReplProjectState` drains with the shutdown bound, tears the - * broker down (releasing every held ACP session and its processes) and - * closes the store; `resetReplProjectState` additionally deletes the - * whole `repl/` directory (the `reset` tool's engine-side). - */ - -import { - Broker, - ReplWorkspaceStore, - SnapshotEnvelopeError, - Workspace, - type BrokerRunner, - type EvalBreakChannel, - type ReconcileReport, - type ReplStoreOptions, - type WasmModule, -} from "@automatalabs/repl-engine"; - -import { existsSync, renameSync } from "node:fs"; - -import { SHUTDOWN_DEADLINE_MS } from "./lifecycle.js"; - -export interface ReplProjectState { - readonly projectDir: string; - /** The daemon's per-project repl store (snapshot + call store). */ - readonly store: ReplWorkspaceStore; - /** The live VM workspace; null until the first touch. */ - workspace: Workspace | null; - /** The attached broker; null until the first touch. */ - broker: Broker | null; - /** Where the workspace came from on first touch. */ - source: "restored" | "fresh" | null; - /** The last restore's three-way reconcile report (restored only). */ - reconcileReport: ReconcileReport | null; - /** The MCP sessions currently present on this workspace (the - * client-presence ledger's per-project set). */ - readonly clients: Set; - /** The in-flight first-touch promise (single-flight; null when idle). */ - firstTouch: Promise | null; - /** Bumped by dispose/reset: an in-flight first touch whose generation - * changed aborts its materialization. */ - generation: number; - /** True once the client-presence drain ran (children closed; the - * workspace stays live and re-attaches lazily). The latch resets on - * every client touch, and the drain's skip guard double-checks the - * broker's authoritative warmth — a second disconnect after a - * re-attach must drain again (phase-D review). */ - drained: boolean; - /** The last client-presence drain's failure (a snapshot-flush failure - * mid-drain, for example), recorded LOUDLY by the presence ledger and - * surfaced by the §6.2 [C]14 one-line notice in the next eval's output - * (the failure lost state — the workspace was not persisted — so it - * is never silent). The drain latch stays clear on failure, so the - * next disconnect retries. */ - drainError: { name: string; message: string } | null; - /** §6.1: a refused stored snapshot was AUTO-RESET at first touch — the - * file was renamed aside (`.refused-`, never deleted) and a fresh - * workspace started. The next eval's output leads with the loud - * one-line notice naming the file and the refusal reason; consumed on - * first render. */ - autoResetNotice: { file: string; reason: string } | null; - /** §6.2 [C]14: pending one-line LOSS notices — a restore that lost - * calls (`failedLost` non-empty) and a client-presence drain failure - * that lost state. Each surfaces ONCE, in the next eval's output - * (losses are never silent); consumed on render. */ - lossNotices: string[]; - /** The §3.1 empty-eval poll seam: the continuation tokens of evals - * THIS tool returned as still-running (their held calls ended with - * the bound elapsed, so no wait will ever read their token-keyed - * settlement). The engine's `claimSweptEvalSettlement` reads an - * entry only when its token is in this set — a concurrent client's - * still-pumping wait can never lose its attribution — and the - * claimed token leaves the set (its settlement was reported). */ - timedOutEvalTokens: Set; -} - -/** Open (and create, on first touch) the project's repl state. */ -export function createReplProjectState( - projectDir: string, - options: ReplStoreOptions = {}, -): ReplProjectState { - return { - projectDir, - store: ReplWorkspaceStore.open(projectDir, options), - workspace: null, - broker: null, - source: null, - reconcileReport: null, - clients: new Set(), - firstTouch: null, - generation: 0, - drained: false, - drainError: null, - autoResetNotice: null, - lossNotices: [], - timedOutEvalTokens: new Set(), - }; -} - -/** The per-eval wall-clock deadline in ms (see module docs; the harness's - * eval guard). Overridable for tests; the daemon wires the env knob. */ -export const DEFAULT_REPL_EVAL_TIMEOUT_MS = 30_000; - -/** - * The first touch: restore the stored workspace (reconcile included) or - * create a fresh one. A refused snapshot is CONTAINED on the state (see - * module docs) — the daemon never crash-loops and never silently - * discards data. Genuine host failures (wasm load, VM instantiation) - * still propagate. SINGLE-FLIGHT: concurrent first-touch calls share one - * in-flight promise (phase-D review round 2 — see module docs), and - * `dispose`/`reset` during the touch aborts its materialization via the - * generation counter. `runner` is optional (default: the broker owns a - * fresh AcpAgentRunner); tests inject a fake and own its lifetime. - */ -export async function ensureReplWorkspace( - state: ReplProjectState, - wasm: WasmModule, - runner?: BrokerRunner, - evalTimeoutMs: number = DEFAULT_REPL_EVAL_TIMEOUT_MS, - evalBreakChannel?: EvalBreakChannel, -): Promise { - // The in-flight first-touch promise is awaited BEFORE the workspace - // fast path (phase-D review rejection: the fast path used to check - // `state.workspace` first, while `doFirstTouch` publishes the - // workspace/broker before awaiting the restore's reconcile — a - // concurrent request could bypass an in-progress restore - // reconciliation and observe or use partially restored state). While - // `firstTouch` is non-null the touch (reconcile included) is not - // complete, so every concurrent toucher shares it. - const flight = state.firstTouch; - if (flight !== null) return flight; - if (state.workspace !== null) return; - const promise = doFirstTouch(state, wasm, runner, evalTimeoutMs, evalBreakChannel); - state.firstTouch = promise; - try { - await promise; - } finally { - if (state.firstTouch === promise) state.firstTouch = null; - } -} - -/** The single-flight first touch body (see `ensureReplWorkspace`). */ -async function doFirstTouch( - state: ReplProjectState, - wasm: WasmModule, - runner: BrokerRunner | undefined, - evalTimeoutMs: number, - evalBreakChannel: EvalBreakChannel | undefined, -): Promise { - const generation = state.generation; - const attach = async (workspace: Workspace): Promise => { - if (state.generation !== generation) { - // dispose/reset won the race: never materialize the workspace. - workspace.dispose(); - throw new Error("repl workspace touch aborted by reset/dispose"); - } - const broker = await Broker.attach(workspace, { - runner, - store: state.store.callStore(), - snapshotSink: state.store.snapshotWriter(workspace, wasm), - // The eval-break signal no longer lives here — the broker owns it - // (see `Broker.armEvalBreak`; phase-E review rejection: the - // project-wide boolean used to be consumable by an unrelated eval - // or drain). The per-eval wall-clock deadline still bounds every - // eval and drain, and the OUT-OF-BAND eval-break channel (phase-F - // review round 2) makes the interrupt tool's no-id path - // deliverable to a synchronously running eval. - evalTimeoutMs, - evalBreakChannel, - }); - if (state.generation !== generation) { - await broker.dispose(); - workspace.dispose(); - throw new Error("repl workspace touch aborted by reset/dispose"); - } - state.workspace = workspace; - state.broker = broker; - }; - if (state.store.hasSnapshot()) { - try { - const restored = state.store.loadSnapshot(wasm); - const workspace = await Workspace.restore(state.projectDir, restored.snapshot, { wasm }); - await attach(workspace); - // The restore's source/report are published only AFTER the - // reconciliation completes, and its completion is - // GENERATION-CHECKED (phase-D review rejection: the old code wrote - // `source` before awaiting the reconcile and wrote the report - // after it with no generation check, so a reset/dispose during a - // parked restore-time loadSession left a stale "restored" report - // on the torn-down state — the broker's own disposal fences - // released the late-loaded sessions, and the report of a reconcile - // that outlived the state it describes must not be published). - const broker = state.broker!; - const report = await broker.reconcile(); - if (state.generation !== generation) { - // reset/dispose won while the reconciliation was in flight: the - // report belongs to a torn-down state — it is dropped, and the - // touch aborts loudly exactly like the attach race. - throw new Error("repl workspace touch aborted by reset/dispose"); - } - state.source = "restored"; - state.reconcileReport = report; - // §6.2 [C]14: a restore that LOST calls (failedLost non-empty) is - // never silent — a one-line notice leads the next eval's output - // (the full report lives in workspace().diagnostics.reconcile). - if (report.failedLost.length > 0) { - state.lossNotices.push( - `restore lost ${report.failedLost.length} call(s) (${report.failedLost.join(", ")}) — ` + - `their outcomes were unknowable and they were settled failed/re-issued; the full reconcile ` + - `report lives in workspace().diagnostics.reconcile`, - ); - } - return; - } catch (error) { - if (error instanceof SnapshotEnvelopeError) { - // §6.1 — a refused snapshot AUTO-RESETS: the file is renamed - // aside (`.refused-`, never deleted — auto-reset must not - // be silent data destruction) and a FRESH workspace starts. The - // envelope family covers the whole load path — the decode-time - // refusals (hash/version/gzip/shape) AND the restore-time - // corruption (`SnapshotRestoreError` — a payload that passed - // every at-rest check but failed to materialize). The next - // eval's output leads with the loud one-line notice naming the - // file and the reason (consumed once by the tool). - // - // A restore that got as far as attaching a workspace/broker - // before refusing (a restore-time corruption surfacing at the - // reconcile arm) is torn down before the fresh workspace starts - // — never two live workspaces for one project. - const attachedBroker = state.broker; - const attachedWorkspace = state.workspace; - state.broker = null; - state.workspace = null; - if (attachedBroker !== null) { - try { - await attachedBroker.dispose(SHUTDOWN_DEADLINE_MS); - } catch { - // Best-effort: the fresh workspace must still start. - } - } - attachedWorkspace?.dispose(); - const aside = renameAsideNeverOverwriting(state.store.snapshotPath, Date.now()); - try { - renameSync(state.store.snapshotPath, aside); - } catch (renameError) { - // The rename is the data-safety guarantee — if it fails the - // fresh workspace's first snapshot write would silently - // replace the refused file. Fail loudly instead. - throw new Error( - `the stored snapshot refused (${error.message}) and could not be renamed aside: ` + - `${renameError instanceof Error ? renameError.message : String(renameError)}`, - ); - } - // §6.1 auto-reset is a FULL reset: the CALL LEDGER is cleared - // with the snapshot (review finding — the old code renamed only - // `snapshot.bin` and left `calls.jsonl` intact, so the fresh - // VM's ids restarting at c1 hit the store's first-wins replay - // and a new c1 inherited an old c1's record AND completion). - // `store.reset()` closes the call store and wipes the `repl/` - // directory ENTRY-WISE, preserving the renamed-aside - // `.refused-*` file (§6.1 [C]13 — never deleted), and the next - // `callStore()` reopens an empty ledger for the fresh ids. - state.store.reset(); - state.autoResetNotice = { file: aside, reason: error.message }; - // Fall through: the fresh workspace starts below. - } else { - throw error; - } - } - } - const workspace = await Workspace.create(state.projectDir, { wasm }); - await attach(workspace); - state.source = "fresh"; -} - -/** - * §6.1 [C]13: the refused snapshot's rename-aside destination — - * collision-safe, never an overwrite. POSIX `renameSync` SILENTLY - * REPLACES an existing destination, so a plain - * `.refused-` name could delete an earlier - * refused snapshot when two auto-resets land in the same millisecond - * (a second refusal on a fresh snapshot, or a test driving two - * refusals) — refused snapshots are never deleted. The daemon is - * single-threaded, so the existence check and the rename below cannot - * race; a collision bumps a counter suffix instead of replacing. - * Exported for the collision regression test. - */ -export function renameAsideNeverOverwriting(snapshotPath: string, atMs: number): string { - let attempt = 0; - for (;;) { - const suffix = attempt === 0 ? `${atMs}` : `${atMs}-${attempt}`; - const candidate = `${snapshotPath}.refused-${suffix}`; - if (!existsSync(candidate)) return candidate; - attempt += 1; - } -} - -/** Mark an MCP session present on this project's workspace (the - * client-presence ledger's touch side; every `repl` tool call touches). - * The workspace stays warm while any session is present, and the drain - * latch resets: a present client makes the workspace warmable again, so - * the NEXT disconnect must drain whatever the workspace warmed (phase-D - * review: drain → reconnect → a queue head re-attaches children → a second - * disconnect used to skip the drain and leave the reattached children - * running). */ -export function touchReplProject(state: ReplProjectState, clientId: string): void { - state.clients.add(clientId); - state.drained = false; -} - -/** Remove an MCP session's presence (the ledger's disconnect side); a - * project with no clients left is drained by the ledger (see - * `ReplPresenceLedger.disconnect`). */ -export function disconnectReplProject(state: ReplProjectState, clientId: string): void { - state.clients.delete(clientId); -} - -/** - * The client-presence drain (the ledger's scheduled half): in-flight - * subagent turns DRAIN TO COMPLETION (each settlement boundary snapshots - * — the findings land durable in the workspace), bounded by `boundMs` - * (the daemon's session-eviction TTL — the spec-owed concrete bound), - * then every idle child closes. The workspace and broker stay alive; the - * next eligible queue head lazily re-attaches its recorded backend session. - * A client that reconnected before the drain started - * skips it (presence is re-checked); one that reconnects MID-DRAIN - * ABORTS it — the broker's `drainForDisconnect` consults this state's - * client set every iteration and before every destructive phase, so the - * children stay warm while any client is connected (phase-D review - * round 6: the drain used to run to its release phase and close every - * child regardless of presence). - * - * A failing drain — a snapshot-flush failure mid-drain, for example — - * is recorded on the state (`drainError`, surfaced loudly in every repl - * tool result) and the drain latch stays clear, so the next disconnect - * retries the drain (phase-D review round 6: the failure used to be - * discarded silently, and a failed snapshot write left the boundary - * clean — the dirty boundary is retained for retry by the store's - * writer). - * - * The latch is not a permanent skip: `touchReplProject` clears it on - * every connect, and a stale latch (the broker reports warm children — - * a lazy re-attach after the latch was set) never skips the drain - * (phase-D review: drain → reconnect → queue dispatch → second disconnect - * left the reattached child running). - */ -export async function drainReplProject(state: ReplProjectState, boundMs: number): Promise { - if (state.broker === null || state.clients.size > 0) return; - if (state.drained && state.broker.isDrained) return; - try { - // The mid-drain presence probe: the drain aborts the moment a - // client is connected again (children stay warm). - const drained = await state.broker.drainForDisconnect(boundMs, () => state.clients.size > 0); - if (state.broker !== null) { - state.drained = drained; - if (drained) state.drainError = null; - } - } catch (error) { - // Loud + retained: the ledger records the failure on the state and - // every repl tool result surfaces it; the drain latch stays clear - // so the next disconnect retries. The rethrow reaches the ledger's - // catch — the drain runs detached, so the state record IS the - // loudness. - state.drainError = { - name: error instanceof Error ? error.name : 'Error', - message: error instanceof Error ? error.message : String(error), - }; - // §6.2: the retained drain error lives under - // workspace().diagnostics.drainError (the demoted diagnostics - // home) — the broker's own drain paths retain their internal - // failures there, and the tool layer pushes ITS observation of a - // rethrown failure into the same record. - state.broker?.retainDrainError(state.drainError.name, state.drainError.message); - // §6.2 [C]14: the failed drain LOST STATE (the workspace was not - // persisted — the store's dirty boundary is retained for retry) and - // losses are never silent: a one-line notice leads the next eval's - // output (consumed once; the record itself stays until the next - // drain succeeds or reset clears it). - state.lossNotices.push( - `warn: the last client-presence drain failed (${state.drainError.name}: ${state.drainError.message}) — ` + - `the workspace state was not persisted; the next disconnect retries the drain`, - ); - throw error; - } -} - -/** Detach a stale in-flight first-touch flight (reset/dispose during a - * parked restore-time reconcile): the flight is dropped from the state - * so a fresh touch starts a NEW first touch instead of awaiting the - * never-resolving promise forever, and its eventual rejection — the - * generation check aborting the stale touch when the parked - * loadSession finally lands — is marked handled (the original toucher - * still observes it; a detached promise must never become an unhandled - * rejection). Phase-D review rejection: reset/dispose used to leave - * `state.firstTouch` parked — the generation check only ran after - * `broker.reconcile()` resolved — so every subsequent touch returned - * the stale promise and hung forever. */ -function detachFirstTouch(state: ReplProjectState): void { - const flight = state.firstTouch; - state.firstTouch = null; - if (flight !== null) { - void flight.catch(() => undefined); - } -} - -/** Teardown the live workspace and broker (releasing every held ACP - * session) and close the store. The `repl/` directory is kept (a later - * touch restores from it). The broker's disposal drains what it can - * with the shutdown bound before cancelling (the daemon's shutdown - * path; the last-client-disconnect path uses the full drain bound). - * `boundMs` defaults to the daemon's shutdown deadline; the shutdown - * path passes the REMAINING time after its drain (phase-D review round - * 7: the teardown used to run unbounded — a failed or deadline-expired - * drain was followed by a disposal that awaited hung cancel/release - * forever, so daemon shutdown could hang on the exact hung backend the - * drain had already caught). */ -export async function disposeReplProjectState( - state: ReplProjectState, - boundMs: number = SHUTDOWN_DEADLINE_MS, -): Promise { - const { broker, workspace } = state; - state.broker = null; - state.workspace = null; - state.generation++; - detachFirstTouch(state); - try { - if (broker !== null) await broker.dispose(boundMs); - } finally { - // The VM release and the store close run in the FINALLY path - // (phase-D review round 8: a disposal rejection — its op-end flush - // retrying the retained dirty boundary from a failed drain and - // failing again, or an owned-runner teardown failure — used to skip - // both, leaving the actual VM and the call store open while the - // state already claimed to be torn down; the registry swallows the - // disposal's rejection at shutdown, so the cleanup must never - // depend on the disposal resolving). - workspace?.dispose(); - state.store.close(); - } -} - -/** The `reset()` guest function's host-side effect: teardown the - * workspace and delete the `repl/` store's contents. The broker - * teardown is bounded like the shutdown path's (a hung backend must - * not hang it either). Renamed-aside refused snapshots - * (`snapshot.bin.refused-*`) survive the wipe (§6.1 [C]13 — never - * deleted; the store's `reset()` preserves them). The stale - * first-touch flight is detached - * like the shutdown path's (see `disposeReplProjectState`): a reset - * during a parked restore-time reconcile must not leave every - * subsequent touch awaiting the never-resolving promise forever. - * - * The client PRESENCE is deliberately NOT reset: `state.clients` (and - * the presence ledger's maps — they are always in sync) track - * CONNECTION liveness, not workspace state. The workspace is dropped, - * but the clients that are connected to the project stay connected — - * clearing the set here would desync it from the ledger, and a later - * disconnect of the reset-issuing client would drain work started - * after the reset while another project client is still connected - * (phase-E review rejection: reset used to clear `state.clients`, so - * the drain decision — which the ledger derives from its own maps — - * could fire against a project that still had a connected client). - * The `drained` latch resets (the next disconnect must re-evaluate - * whatever the fresh workspace warmed) and `drainError` clears (the - * dropped state's stale failure is gone with it). */ -export async function resetReplProjectState( - state: ReplProjectState, - boundMs: number = SHUTDOWN_DEADLINE_MS, -): Promise { - const { broker, workspace } = state; - state.broker = null; - state.workspace = null; - state.generation++; - detachFirstTouch(state); - try { - if (broker !== null) await broker.dispose(boundMs); - } finally { - // The VM release and the store reset run in the FINALLY path — a - // disposal rejection must never leave the VM or the `repl/` store - // behind while the state claims to be reset (see - // `disposeReplProjectState`). - workspace?.dispose(); - state.store.reset(); - } - state.source = null; - state.reconcileReport = null; - state.autoResetNotice = null; - state.lossNotices = []; - // Presence survives the reset: the connected clients remain present - // (see the module docs above) — the next touch re-establishes the - // workspace under the SAME presence, and the drain policy keeps - // working against the ledger's authoritative per-project set. - state.drained = false; - state.drainError = null; -} diff --git a/packages/mcp-server/src/repl-stdio-relay-worker.ts b/packages/mcp-server/src/repl-stdio-relay-worker.ts deleted file mode 100644 index 8ac81543..00000000 --- a/packages/mcp-server/src/repl-stdio-relay-worker.ts +++ /dev/null @@ -1,225 +0,0 @@ -/** - * The `ReplRelayStdioTransport`'s worker thread (see - * `repl-stdio-transport.ts`): owns the STDIN READ of the single-project - * in-process MCP server, so a `repl` interrupt can reach the - * out-of-band eval-break relay while the main thread is blocked in a - * synchronous eval (the daemon mode's shim does the same from a - * separate process; here the reader thread plays the shim's fire side). - * - * ## Reading stdin from a worker thread - * - * `process.stdin` in a worker thread is NOT wired to the real fd (it - * reports EOF immediately), and `fs.read` on the raw fd 0 returns - * EAGAIN whenever the pipe is momentarily empty — libuv's child stdio - * pipes are non-blocking — which a naive read stream treats as fatal. - * The pump below therefore reads fd 0 directly and treats EAGAIN as - * "no data right now": it yields for a few milliseconds and retries, - * so the worker's event loop stays free for the relay's fire-and-forget - * fetch between lines. A blocking fd (a shell pipe, a terminal) simply - * blocks in the read until data or EOF arrives. - * - * ## Wire contract - * - * Every newline-delimited JSON-RPC frame is forwarded VERBATIM to the - * main thread. A `tools/call` frame for the `repl` tool with - * `action: "interrupt"` and NO call id additionally fires the relay - * first — `POST /break` with the REALPATH'd `key` (exactly the daemon's - * canonical project key, phase-F review round 3: the raw caller- - * supplied path used to be posted verbatim, so a symlinked or - * non-normalized projectDir got a relay 404). An unresolvable path is - * skipped — the server's own validation refuses the call. An interrupt - * that OMITS projectDir fires the relay with the SINGLE-PROJECT - * SERVER'S OWN PROJECT KEY (phase-F review round 4: the repl tool - * resolves the omitted projectDir to the registry's adopted default - * context, so the relay must too — the old code skipped the relay - * entirely, and the runaway eval ran to the per-eval deadline before - * the interrupt could be processed). EOF on stdin posts the EOF - * marker; the transport closes. - * - * ## Decoding discipline - * - * The raw byte stream is decoded through a STREAMING UTF-8 decoder - * (`StringDecoder`), never per-chunk `Buffer.toString`: a multibyte - * character split across two reads must survive intact — the frame - * forwarding is byte-identical, and a per-chunk decode replaced the - * split character with U+FFFD, corrupting the JSON-RPC payload - * (phase-F review round 4: a multibyte payload's decoded length - * changed on the wire). - */ - -import { readSync } from "node:fs"; -import { realpathSync } from "node:fs"; -import { isAbsolute } from "node:path"; -import { StringDecoder } from "node:string_decoder"; -import { parentPort, workerData } from "node:worker_threads"; - -interface WorkerData { - breakUrl?: string; - /** The single-project server's own project key — the context the - * repl tool resolves when projectDir is omitted (`stores()[0]`). - * Undefined in daemon mode (projectDir is required there) and when - * the transport has no default project. */ - defaultProjectKey?: string; -} - -const EOF_MARKER = "\u0000__repl_stdio_eof__\u0000"; -// `workerData` is null outside a worker thread (the unit tests import -// this module in the main thread) — the defaults keep the relay inert. -const { breakUrl, defaultProjectKey } = (workerData ?? {}) as WorkerData; -const READ_CHUNK = 64 * 1024; -/** The EAGAIN retry yield: the pipe is momentarily empty — check again - * shortly (the worker's only jobs are this pump and the relay's - * fire-and-forget fetch, so a few milliseconds of slack is nothing). */ -const EAGAIN_RETRY_MS = 5; - -/** The newline-delimited frame splitter: decodes the raw byte stream - * through a STREAMING UTF-8 decoder (a multibyte character split - * across reads must not be corrupted — phase-F review round 4: the - * per-chunk `Buffer.toString("utf8")` replaced the split character - * with U+FFFD, so the claimed byte-identical forwarding was false for - * multibyte payloads) and emits complete lines. Exported for the unit - * tests; the module is import-safe outside a worker because the pump - * only runs when `parentPort` is present. */ -export class RelayFrameSplitter { - private readonly decoder = new StringDecoder("utf8"); - private pending = ""; - - constructor(private readonly onLine: (line: string) => void) {} - - /** Decode one read chunk and emit every complete line it contains. */ - push(chunk: Buffer): void { - this.pending += this.decoder.write(chunk); - for (;;) { - const newline = this.pending.indexOf("\n"); - if (newline < 0) break; - this.onLine(this.pending.slice(0, newline)); - this.pending = this.pending.slice(newline + 1); - } - } - - /** EOF: flush the decoder's held partial character (a TRUNCATED - * UTF-8 sequence at the stream's end decodes to the replacement - * char) and emit a final unterminated frame, if any. */ - end(): void { - this.pending += this.decoder.end(); - if (this.pending.length > 0) { - const line = this.pending; - this.pending = ""; - this.onLine(line); - } - } -} - -/** Resolve the relay key for one interrupt call: the REALPATH'd - * projectDir (exactly the daemon's canonical project key — a symlink - * or non-normalized path must reach the workspace's registered key), - * or — when projectDir is OMITTED — the single-project server's own - * project key, VERBATIM (the repl tool resolves the omitted projectDir - * to the registry's adopted default context, and the broker registers - * that context's projectDir as-is; phase-F review round 4). `undefined` - * when the call cannot be keyed at all (non-absolute or unresolvable - * path, or no default key available) — the server's own validation - * refuses the call, or the transport has no default project. Exported - * for the unit tests. */ -export function relayBreakKey( - projectDir: unknown, - defaultProjectKey: string | undefined, -): string | undefined { - if (typeof projectDir === "string") { - if (!isAbsolute(projectDir)) return undefined; - try { - return realpathSync(projectDir); - } catch { - return undefined; // invalid/unresolvable — the server's own validation refuses the call - } - } - // projectDir omitted — single-project mode (round 4: the relay used - // to skip the omitted-projectDir interrupt entirely, so the runaway - // eval ran to the per-eval deadline and the interrupt then reported - // refused-idle). - return defaultProjectKey; -} - -/** Fire the out-of-band break (best-effort, fire-and-forget: the - * server's own interrupt processing clears the flag when it lands; a - * dead relay degrades to the per-eval deadline bound). */ -function fireOutOfBandBreak(projectDir: unknown): void { - if (breakUrl === undefined) return; - const key = relayBreakKey(projectDir, defaultProjectKey); - if (key === undefined) return; - void fetch(breakUrl, { - method: "POST", - headers: { "content-type": "application/json" }, - body: JSON.stringify({ key }), - signal: AbortSignal.timeout(1000), - }).catch(() => { - // Best-effort: a dead relay must never break the forwarding path. - }); -} - -/** Handle one complete line: detect the repl interrupt and fire the - * relay, then forward the RAW frame verbatim. */ -function handleLine(line: string): void { - if (parentPort !== null) { - // Parse only to detect the interrupt; the raw frame is forwarded - // either way. - try { - const message = JSON.parse(line) as { - method?: unknown; - params?: { name?: unknown; arguments?: Record }; - }; - if ( - message.method === "tools/call" && - message.params?.name === "repl" && - message.params.arguments?.action === "interrupt" && - message.params.arguments.id === undefined - ) { - fireOutOfBandBreak(message.params.arguments.projectDir); - } - } catch { - // Not a JSON-RPC frame — forward verbatim below. - } - parentPort.postMessage(line); - } -} - -/** The stdin pump: read fd 0 directly (see the module docs for the - * EAGAIN discipline), split the newline-delimited frame stream through - * the STREAMING UTF-8 decoder, and handle each line. EOF (a - * zero-length read) posts the EOF marker and stops. A fatal read error - * is reported to the parent and stops the pump — the transport then - * closes like any broken pipe. */ -function pump(): void { - const buffer = Buffer.alloc(READ_CHUNK); - const readOnce = (): number => readSync(0, buffer, 0, READ_CHUNK, null); - const splitter = new RelayFrameSplitter(handleLine); - const step = (): void => { - let n: number; - try { - n = readOnce(); - } catch (error) { - if ((error as NodeJS.ErrnoException).code === "EAGAIN") { - setTimeout(step, EAGAIN_RETRY_MS); - return; - } - parentPort?.postMessage(`\u0000__repl_stdio_error__\u0000${String(error)}`); - return; - } - if (n === 0) { - // EOF: flush a final unterminated frame (a truncated multibyte - // sequence at the stream's end decodes to the replacement char), - // then close the transport. - splitter.end(); - parentPort?.postMessage(EOF_MARKER); - return; - } - splitter.push(buffer.subarray(0, n)); - step(); - }; - step(); -} - -// The pump owns fd 0 and only runs inside the worker thread — importing -// this module in the main thread (the unit tests) must not touch stdin -// (`parentPort` is null there). -if (parentPort !== null) pump(); diff --git a/packages/mcp-server/src/repl-stdio-transport.ts b/packages/mcp-server/src/repl-stdio-transport.ts deleted file mode 100644 index a8643934..00000000 --- a/packages/mcp-server/src/repl-stdio-transport.ts +++ /dev/null @@ -1,178 +0,0 @@ -/** - * The single-project (in-process) stdio transport with the REPL - * eval-break relay — phase-F review round 3: the public in-process/ - * library server must implement the documented no-id interrupt behavior - * for a SYNCHRONOUSLY running eval, not only the daemon mode. - * - * ## Why a transport at all - * - * The pre-daemon `--in-process` mode (and the library `main()` path) - * serves MCP directly over stdio from the server's own process. A - * `while (true) {}` eval blocks that process's main thread, so the - * client's interrupt request sitting in the stdin pipe cannot be - * PROCESSED — the documented "break a runaway eval" would be - * unimplemented in this supported mode. The daemon mode closes the same - * gap with a separate shim PROCESS that fires the eval-break relay - * (a worker thread) before forwarding. In-process there is no shim - * process, so this transport moves the STDIN READER into a worker - * thread: the reader stays live while the main thread is wedged in the - * VM, recognizes `repl` interrupt calls (no call id), fires the - * server's out-of-band eval-break relay (the same `POST /break` - * contract the shim uses), and forwards every raw frame to the main - * thread for normal processing once the eval ends or breaks. The - * per-eval wall-clock deadline remains the last-resort bound, exactly - * like the daemon mode's. - * - * ## Frame discipline - * - * The reader worker parses each newline-delimited JSON-RPC frame only - * to detect the interrupt; every frame — detected or not — is forwarded - * verbatim to the main thread (the server's `onmessage`), so the - * protocol layer sees a byte-identical stream to `StdioServerTransport`'s - * and the single-project mode keeps its exact framing semantics. The - * worker is the ONLY stdin reader (this transport replaces the SDK's - * stdio transport), so there is no reader conflict. The byte stream is - * decoded through a STREAMING UTF-8 decoder in the worker — a multibyte - * character split across reads survives intact (phase-F review round 4: - * per-chunk decoding corrupted split characters). - * - * ## The omitted-projectDir interrupt (phase-F review round 4) - * - * The repl tool documents projectDir as OPTIONAL in single-project - * mode (it resolves the server's own adopted project). The reader - * worker therefore receives the transport's DEFAULT PROJECT KEY and - * fires the relay with it when the interrupt call omits projectDir — - * the old reader skipped the relay for an omitted projectDir, so the - * documented interrupt silently degraded to the per-eval deadline in - * the one configuration where the tool's own docs promise the default - * project works. - * - * ## Send completion (phase-F review round 4) - * - * `send` resolves only when the frame left the user-space buffer: a - * `write()` that reports backpressure is followed by a wait on the - * `drain` event — exactly the `StdioServerTransport` semantics this - * transport replaces. The old fire-and-forget write allowed unbounded - * buffering against a slow client and violated the transport contract - * for every in-process MCP exchange. - */ - -import { existsSync } from "node:fs"; -import { fileURLToPath } from "node:url"; -import { Worker } from "node:worker_threads"; -import type { Transport, JSONRPCMessage } from "@modelcontextprotocol/server"; - -/** The worker's EOF marker (the stdin pipe closed — the main thread - * closes the transport, mirroring `StdioServerTransport`'s close-on- - * stdin-end semantics). */ -const EOF_MARKER = "\u0000__repl_stdio_eof__\u0000"; - -/** The relay worker entry: the compiled `repl-stdio-relay-worker.js` in - * dist, or the TypeScript source when running from src (tsx dev/tests — - * the worker inherits the parent's loader, so the .ts runs directly). */ -function relayWorkerEntryUrl(): URL { - const tsEntry = new URL("./repl-stdio-relay-worker.ts", import.meta.url); - if (existsSync(fileURLToPath(tsEntry))) return tsEntry; - return new URL("./repl-stdio-relay-worker.js", import.meta.url); -} - -/** The stdout seam (injectable for tests): the minimal surface the - * transport needs — `write` (backpressure-reporting), `once`/`off` - * for the `drain` and `error` events. `process.stdout` satisfies it - * structurally. */ -export interface ReplRelayStdioSink { - write(chunk: string): boolean; - once(event: "drain" | "error", listener: () => void): unknown; - off(event: "drain" | "error", listener: () => void): unknown; -} - -/** - * The stdio transport whose stdin reader lives on a worker thread and - * fires the REPL eval-break relay for `repl` interrupt calls without a - * call id (see the module docs). `breakUrlSource` supplies the relay - * address (the server's owned eval-break channel); it is resolved at - * `start()` — the channel's worker boots in milliseconds, and a relay - * that never becomes available degrades to the per-eval deadline bound - * (the transport still forwards every frame). - * `defaultProjectKeySource` supplies the single-project server's own - * project key — the relay's key for an interrupt that omits projectDir - * (the tool resolves the omitted projectDir to exactly that context). - */ -export class ReplRelayStdioTransport implements Transport { - onclose?: () => void; - onerror?: (error: Error) => void; - onmessage?: (message: JSONRPCMessage) => void; - - private readonly breakUrlSource: () => Promise; - private readonly defaultProjectKeySource: () => string | undefined; - private readonly stdout: ReplRelayStdioSink; - private worker: Worker | undefined; - private started = false; - private closed = false; - - constructor( - breakUrlSource: () => Promise, - defaultProjectKeySource: () => string | undefined = () => undefined, - stdout: ReplRelayStdioSink = process.stdout, - ) { - this.breakUrlSource = breakUrlSource; - this.defaultProjectKeySource = defaultProjectKeySource; - this.stdout = stdout; - } - - async start(): Promise { - if (this.started) return; - this.started = true; - const [breakUrl, defaultProjectKey] = await Promise.all([ - this.breakUrlSource().catch(() => undefined), - Promise.resolve(this.defaultProjectKeySource()), - ]); - if (this.closed) return; - this.worker = new Worker(relayWorkerEntryUrl(), { - workerData: { breakUrl, defaultProjectKey }, - }); - this.worker.unref(); - this.worker.on("message", (payload: string) => { - if (payload === EOF_MARKER) { - void this.close(); - return; - } - try { - this.onmessage?.(JSON.parse(payload) as JSONRPCMessage); - } catch (error) { - this.onerror?.(error instanceof Error ? error : new Error(String(error))); - } - }); - this.worker.on("error", (error) => this.onerror?.(error)); - this.worker.on("exit", () => { - if (!this.closed) void this.close(); - }); - } - - async send(message: JSONRPCMessage): Promise { - // stdout stays on the main thread (a blocked main thread is not - // sending anything anyway — the eval's result is only produced - // after the execution ends). SEND COMPLETION MEANS FLUSHED: - // when write() reports backpressure, the promise waits for the - // drain event — exactly the StdioServerTransport semantics this - // transport replaces (phase-F review round 4: the write used to - // resolve immediately, allowing unbounded buffering against a slow - // client and violating send-completion for all in-process MCP - // traffic). - if (this.stdout.write(`${JSON.stringify(message)}\n`)) return; - await new Promise((resolve) => { - this.stdout.once("drain", () => resolve()); - }); - } - - async close(): Promise { - if (this.closed) return; - this.closed = true; - const worker = this.worker; - this.worker = undefined; - if (worker !== undefined) { - await worker.terminate().catch(() => undefined); - } - this.onclose?.(); - } -} diff --git a/packages/mcp-server/src/repl-tool.ts b/packages/mcp-server/src/repl-tool.ts deleted file mode 100644 index 366952f1..00000000 --- a/packages/mcp-server/src/repl-tool.ts +++ /dev/null @@ -1,730 +0,0 @@ -/** - * The `repl` MCP tool — the eval-plane redesign's surface (the roadmap - * bible, docs/roadmap/repl-eval-redesign.md §3): TWO actions on the - * same enum-shaped tool the repo's `workflow` tool uses. - * - * - `eval { projectDir, code, timeoutMs? }` — the ONE verb. Runs `code` - * in the workspace VM (top-level `await` allowed; top-level `return` - * a syntax error; empty string valid), then HOLDS THE CALL OPEN - * pumping settlements server-side (the fusion of v1's eval with v1's - * wait pump) up to the soft bound — default 60 000 ms, per-call - * `timeoutMs` override, hard cap 120 000 ms. Everything the code is - * waiting on settles within the bound → the FINISHED shape - * `{ output, result? }` (`result` is the completion value's §4.4 - * repr). The bound elapses first → the honest STILL-RUNNING shape - * `{ output, running: [call ids] }`; the eval continues server-side - * and ANY later eval — including `""`, the documented idempotent - * poll — drains and reports what settled. `output` is ONE - * newline-joined string: console lines (one per call), raised - * checkpoint lines (`checkpoint c9: `), and uncaught-error - * renderings (§4.6). No `pending`/`completed`/`checkpoints`/ - * `outputTruncated`/`truncated`/`referenced` fields exist on the - * wire — the budget/cap apparatus is deleted (§7); an agent CAN - * flood its own context, the Python posture. - * - `interrupt { projectDir, id? }` — the one out-of-band verb: with - * `id`, cancel that subagent call (the guest promise rejects - * recoverable, `AGENT_CANCELLED` family); without `id`, break the - * running eval — every running eval is broken mid-run (the - * out-of-band eval-break channel and the quickjs interrupt handler - * stand as built) or, when it is suspended on nothing resumable or - * its continuation cannot be keyed, TERMINATED (released) outright; - * `refused-idle` is honest only when NOTHING is running. - * - * Durability is kept and hidden (§6): bindings AND in-flight subagent - * turns survive daemon restarts exactly as before, but the ceremony - * left the surface — a refused stored snapshot AUTO-RESETS (the file - * is renamed aside `.refused-`, never deleted, and the next eval's - * output leads with a loud one-line notice naming the file and the - * reason); reconcile summaries and retained drain errors live under - * `workspace().diagnostics`, except a restore that lost calls or a - * drain failure that lost state, which still get a one-line notice in - * the next eval's output (losses are never silent). - */ -import { ProtocolError, ProtocolErrorCode } from "@modelcontextprotocol/server"; -import type { McpServer } from "@modelcontextprotocol/server"; -import type { Broker, BrokerRunner, EvalBreakChannel, WasmModule } from "@automatalabs/repl-engine"; -import { isAbsolute } from "node:path"; -import { z } from "zod"; - -import { resolveProjectDir, type WorkflowProjectRegistry, type ProjectContext } from "./project-registry.js"; -import { createReplProjectState, ensureReplWorkspace, type ReplProjectState } from "./repl-project.js"; -import type { ReplPresenceLedger } from "./repl-presence.js"; - -/** The soft-bound eval's default hold (§3.1 [D]: default 60 000 ms). */ -export const DEFAULT_REPL_EVAL_BOUND_MS = 60_000; -/** The soft-bound eval's hard cap (§3.1 [D]: 120 000 ms — the same - * numbers v1's `wait` used). */ -export const MAX_REPL_EVAL_BOUND_MS = 120_000; -/** The fused eval's re-poll interval when the suspended eval awaits - * nothing pumpable by call ids (a checkpoint, a sleep) — a short hold - * so host-timer settlements still resolve within the call; the - * absolute bound caps the hold. */ -const REPL_EVAL_POLL_GAP_MS = 100; - -export const replToolInputShape = { - action: z - .enum(["eval", "interrupt"]) - .describe( - "Operation. eval runs code in the workspace's persistent VM and holds the call open pumping " + - "settlements up to the soft bound; interrupt cancels one subagent call (by id) or breaks the " + - "running eval (no id; honestly refused when nothing is running).", - ), - projectDir: z - .string() - .min(1) - .refine((value) => isAbsolute(value), "projectDir must be an absolute path") - .optional() - .describe( - "Absolute project directory the workspace lives in: one VM per projectDir, addressed exactly like the " + - "workflow tool's projectDir (the same validated, realpathed per-project context; the workspace state " + - "survives MCP-session churn and daemon restarts through the per-project repl store). Required on the " + - "shared workflow daemon; optional (defaults to this server's own project) in single-project mode.", - ), - code: z - .string() - .optional() - .describe( - "The JavaScript to eval (top-level await accepted; `return` is a syntax error; console output is captured). " + - "An empty string is valid — the documented idempotent poll: a no-op script that drains and reports " + - "whatever settled since the last eval.", - ), - timeoutMs: z - .number() - .int() - .min(0) - .max(MAX_REPL_EVAL_BOUND_MS) - .optional() - .describe( - `Bounded server-side hold for this eval (default ${DEFAULT_REPL_EVAL_BOUND_MS} ms, hard cap ${MAX_REPL_EVAL_BOUND_MS} ms): ` + - "the call is held open pumping settlements up to the bound. Everything the code waits on settles " + - "within the bound → the finished shape { output, result? }; the bound elapses first → the " + - "still-running shape { output, running } with the eval continuing server-side (any later eval drains).", - ), - id: z - .string() - .optional() - .describe( - "The call id to cancel (interrupt action). Omitted: break the running eval (honestly refused when no eval is in flight).", - ), -}; - -export interface ReplToolOptions { - projects: WorkflowProjectRegistry; - /** The engine's compiled quickjs.wasm (a shared promise — the - * envelope's identity check compares its hash at restore). */ - wasm: Promise; - /** The workspaces' ACP runner (optional: each workspace's broker owns - * its own when omitted — tests inject a fake). */ - runner?: BrokerRunner; - /** The per-eval wall-clock deadline in ms (the harness's eval guard; - * a currently-running runaway eval is always breakable through the - * quickjs interrupt handler — see `src/repl-project.ts`). Read from - * `AGENTPRISM_REPL_EVAL_TIMEOUT_MS`, default - * `DEFAULT_REPL_EVAL_TIMEOUT_MS`. */ - evalTimeoutMs: number; - /** Mirrors the workflow tool's daemon-mode projectDir requirement. */ - requireProjectDir: boolean; - /** The client-presence ledger (the doc's last-client-disconnect drain; - * see `repl-presence.ts`). */ - presence: ReplPresenceLedger; - /** This server's client id (the MCP session id in daemon mode), used - * to touch presence. */ - clientId: () => string | undefined; - /** The OUT-OF-BAND eval-break channel (phase-F review round 2; see - * repl-engine's `EvalBreakChannel`): the worker-thread relay the MCP - * shim fires while the daemon's main thread is blocked in a - * synchronous eval, so the interrupt tool's no-id path breaks the - * eval mid-run instead of waiting for the per-eval deadline. The - * server always wires one — the daemon passes its own, and - * single-project servers own one by default (round 3) whose relay - * the stdio transport's worker-reader fires (see - * `repl-stdio-transport.ts`). */ - evalBreakChannel?: EvalBreakChannel; - /** When true, the server is shutting down and rejects new calls. */ - acceptingWork: () => boolean; -} - -/** One parsed `repl` tool input — the action discriminator's output - * (the workflow tool's pattern): the MCP SDK validates the primitive - * fields, then the discriminator enforces each action's EXACT field - * set — `eval` without `code`, or `interrupt` with `code`/`timeoutMs`, - * are both rejected at the boundary. */ -export type ParsedReplToolInput = - | { action: "eval"; projectDir?: string; code: string; timeoutMs: number } - | { action: "interrupt"; projectDir?: string; id?: string }; - -/** Which fields belong to which action (the discriminator's exact-shape - * vocabulary — every other key is rejected at the boundary). */ -const replInputFields = ["action", "projectDir", "code", "timeoutMs", "id"] as const; -type ReplInputField = (typeof replInputFields)[number]; - -const REPL_ACTION_FIELDS: Record> = { - eval: new Set(["action", "projectDir", "code", "timeoutMs"]), - interrupt: new Set(["action", "projectDir", "id"]), -}; - -function invalidReplInput(message: string): never { - throw new ProtocolError(ProtocolErrorCode.InvalidParams, `Invalid repl tool input: ${message}`); -} - -/** Apply the action discriminator after the MCP SDK has validated the - * primitive fields: every action's EXACT field set is enforced here. - * EVERY key outside the action's set is rejected — deleted surface - * like the v1 `refs` parameter (and the wait/status/reset fields) - * fails at the boundary instead of being silently discarded, and - * missing required fields are rejected too. `requireProjectDir` - * mirrors the workflow tool's daemon-mode rule: projectDir is - * required for both actions there. */ -export function parseReplToolInput( - raw: Record, - options: { requireProjectDir: boolean }, -): ParsedReplToolInput { - const action = replToolInputShape.action.parse(raw.action); - const allowed = REPL_ACTION_FIELDS[action]; - for (const field of Object.keys(raw)) { - if (field === "action") continue; - if (!allowed.has(field as ReplInputField)) { - invalidReplInput(`action "${action}" cannot include ${field}`); - } - } - const projectDir = raw.projectDir === undefined ? undefined : replToolInputShape.projectDir.parse(raw.projectDir); - if (projectDir === undefined && options.requireProjectDir) { - invalidReplInput("projectDir is required on the shared workflow daemon"); - } - switch (action) { - case "eval": { - const code = replToolInputShape.code.parse(raw.code); - if (code === undefined) { - invalidReplInput("eval requires a code string"); - } - // An EMPTY script is valid JavaScript AND the documented poll - // idiom — only the ABSENT field is rejected, at the exact-shape - // boundary. - const timeoutMs = replToolInputShape.timeoutMs.parse(raw.timeoutMs ?? DEFAULT_REPL_EVAL_BOUND_MS) - ?? DEFAULT_REPL_EVAL_BOUND_MS; - return { action, projectDir, code, timeoutMs }; - } - case "interrupt": { - const id = raw.id === undefined ? undefined : replToolInputShape.id.parse(raw.id); - return { action, projectDir, id }; - } - } -} - -/** The project context a repl call addresses; undefined = no project. */ -function resolveContext( - options: ReplToolOptions, - projectDir: string | undefined, -): ProjectContext | undefined { - if (projectDir === undefined) { - // Single-project mode: the registry's adopted default context. - return options.projects.stores()[0]; - } - const resolution = resolveProjectDir(projectDir); - if (!resolution.ok) { - throw new ProtocolError(ProtocolErrorCode.InvalidParams, `Invalid repl tool input: ${resolution.message}`); - } - return options.projects.getOrCreate(resolution.projectDir); -} - -/** The pending one-line notices (§6.1 auto-reset, §6.2 [C]14 loss - * notices): each is rendered ONCE, leading the next eval's `output`, - * and consumed on render. */ -function takeNotices(state: ReplProjectState): string[] { - const notices: string[] = []; - if (state.autoResetNotice !== null) { - const { file, reason } = state.autoResetNotice; - notices.push( - `REPL workspace auto-reset: the stored snapshot refused (${reason}) — the file was renamed aside to ` + - `${file} (never deleted) and a fresh workspace started`, - ); - state.autoResetNotice = null; - } - notices.push(...state.lossNotices.splice(0)); - return notices; -} - -/** §4.5 reset(): the reset-owning eval disposes its own broker in the - * operation's post-hook (AFTER the eval result was rendered). Clear the - * project state's live references and drop the whole repl/ store (the - * deleted v1 `reset` action's engine-side) so the NEXT touch creates a - * FRESH workspace, never a restore of the torn-down one. Returns true - * when a reset tore the workspace down. - * - * The pending §6.1/§6.2 notices are deliberately NOT cleared here: - * they belong to the STORE's refusal history (the renamed-aside - * `.refused-*` file survives `store.reset()` — §6.1 [C]13, never - * deleted) and are consumed exactly once when an eval's output - * renders them (the review finding: the sync ran before the render, - * so a first-eval `reset()` after an auto-reset erased the leading - * refusal notice). */ -function syncReplStateAfterOp(state: ReplProjectState): boolean { - if (state.broker !== null && state.broker.isDisposed) { - state.broker = null; - state.workspace = null; - state.store.reset(); - state.source = null; - state.reconcileReport = null; - state.drained = false; - state.drainError = null; - state.timedOutEvalTokens.clear(); - return true; - } - return false; -} - -/** The interrupt action's structured outcome. */ -const interruptOutcomeShape = z.object({ - outcome: z.enum(["targeted", "refused-idle", "cancelled", "idle", "failed", "none"]), - callId: z.string().optional(), -}); - -/** The machine-readable output of the `repl` tool (published as the - * tool's `outputSchema`, mirrored by every result's - * `structuredContent`): the bible's §3.1 eval shape — ONE - * newline-joined string of the printed stream (console lines, - * checkpoint lines, error renderings, and the §6 notices), the - * completion value's repr when the code finished, the in-flight call - * ids when the bound elapsed. `result` and `running` are MUTUALLY - * EXCLUSIVE — an eval result is exactly one of the finished shape - * `{ output, result }`, the still-running shape `{ output, running }`, - * or the bare `{ output }` of an eval whose code threw (the §4.6 - * rendering, no completion value). The interrupt variant carries the - * honest outcome; the error variant a structured error string. - * NOTHING else — the v1 - * pending/completed/checkpoints/outputTruncated/truncated/referenced - * fields are deleted with the cap apparatus (§7). */ -export const replToolOutputShape = z - .object({ - output: z.string().optional(), - result: z.string().optional(), - running: z.array(z.string()).optional(), - interrupt: interruptOutcomeShape.optional(), - error: z.string().optional(), - }) - .superRefine((value, context) => { - const keys = new Set(Object.keys(value)); - const has = (field: string) => keys.has(field); - const only = (...fields: string[]) => [...keys].every((key) => fields.includes(key)); - let valid: boolean; - if (has("error")) { - valid = only("error"); - } else if (has("interrupt")) { - valid = only("interrupt"); - } else { - // The eval variant: the output string is required, `result` and - // `running` are mutually exclusive (the finished shape vs the - // bound-elapsed shape), and nothing else rides along. - valid = has("output") && only("output", "result", "running") && !(has("result") && has("running")); - } - if (!valid) { - context.addIssue({ code: "custom", message: "output does not match a repl result variant" }); - } - }) - .meta({ - oneOf: [ - { - title: "eval", - required: ["output", "result"], - properties: { output: { type: "string" }, result: { type: "string" } }, - not: { anyOf: [{ required: ["running"] }, { required: ["interrupt"] }, { required: ["error"] }] }, - }, - { - title: "eval-still-running", - required: ["output", "running"], - properties: { output: { type: "string" }, running: { type: "array", items: { type: "string" } } }, - not: { anyOf: [{ required: ["result"] }, { required: ["interrupt"] }, { required: ["error"] }] }, - }, - { - title: "eval-error", - required: ["output"], - properties: { output: { type: "string" } }, - not: { anyOf: [{ required: ["result"] }, { required: ["running"] }, { required: ["interrupt"] }, { required: ["error"] }] }, - }, - { - title: "interrupt", - required: ["interrupt"], - properties: { interrupt: interruptOutcomeShape }, - not: { anyOf: [{ required: ["output"] }, { required: ["result"] }, { required: ["running"] }, { required: ["error"] }] }, - }, - { - title: "error", - required: ["error"], - properties: { error: { type: "string" } }, - // The runtime validator accepts ONLY the bare `error` key — the - // published branch must mirror it exactly, so `error`+`result` - // and `error`+`running` objects are advertised-invalid too - // (§3.1 [C]1: the published schema mirrors the runtime shape). - not: { - anyOf: [ - { required: ["output"] }, - { required: ["interrupt"] }, - { required: ["result"] }, - { required: ["running"] }, - ], - }, - }, - ], - }); - -/** Assemble the eval result: the §3.1 wire shape - * `{ output, result?, running? }` as `structuredContent` (mirroring - * the published output schema exactly), plus the bounded human text — - * the output string, then the `result:` line, then the `running:` - * line. Notices (§6.1/§6.2) lead the output. */ -function evalResult( - outputLines: string[], - result: string | undefined, - running: string[] | undefined, - notices: string[], -): { structuredContent: Record; content: { type: "text"; text: string }[] } { - const output = [...notices, ...outputLines].join("\n"); - const structured: Record = { output }; - if (result !== undefined) structured.result = result; - if (running !== undefined) structured.running = running; - const textLines = [...notices, ...outputLines]; - if (result !== undefined) textLines.push(`result: ${result}`); - if (running !== undefined) textLines.push(`running: ${running.join(", ")}`); - return { - structuredContent: structured, - content: [{ type: "text", text: textLines.join("\n") }], - }; -} - -/** Register the `repl` tool on the server. */ -export function registerReplTool(mcp: McpServer, options: ReplToolOptions): void { - const { projects, wasm, requireProjectDir } = options; - mcp.registerTool( - "repl", - { - description: - "A persistent QuickJS-in-WASM JavaScript VM you drive interactively to orchestrate subagents — one VM per " + - "projectDir, addressed by the same project model as the workflow tool. Two actions: eval runs code and " + - "holds the call open pumping settlements; interrupt cancels one subagent call (by id) or breaks the " + - "running eval (no id). Named bindings, pending subagent calls, raised checkpoints, and `_` (the previous " + - "eval's completion value) PERSIST in the VM between calls — a later eval sees the same variables and awaits " + - "the same promises. Console logging produces output text only and creates no persistent value; nothing lives " + - "in the transcript. " + - // The guest API. - "Inside code (JavaScript; top-level await is allowed, top-level return is a syntax error; console output " + - "is captured) the host bridge provides agent(modelSpec, task, opts?) → Promise: spawn an ACP subagent on " + - "a registry built-in (currently Claude, Codex, OpenCode, and pi) or a registered custom agent. The spec " + - "is \"backend/model\" (a bare \"backend\" runs its default model); an unknown backend rejects the call " + - "immediately, naming the known backends. The opts keys are schema (a structured-output JSON schema, " + - "validated per call), cwd, configOptions (backend-specific knobs, validated at admission), and mode. " + - "Before setting mode, use workflow action:\"config\" for that exact modelSpec and read the harness-owned mode descriptions. Trusted autonomous implementation/review uses advertised Claude bypassPermissions or Codex agent; Claude auto uses a model classifier and may request permission. Pin only an exact advertised id. " + - "Unknown option keys reject synchronously. agent() returns a persistent promise-handle. Assign the handle " + - "before awaiting it: `const a = agent(\"codex\", \"inspect the failure\"); const first = await a`. " + - "a.steer(text) targets only the currently running turn. It never starts or queues another turn and resolves " + - "\"injected\", \"idle\", or \"unsupported\"; transport and protocol failures reject. Steering while " + - "idle returns \"idle\" and loses the instruction by design. `const q = a.queue(text)` creates a distinct " + - "FIFO turn on the same session. q.id is available immediately, await q returns that turn's answer, and " + - "q.cancel() or an out-of-band interrupt of q.id cancels that exact turn. Queueing works on every backend " + - "that can continue the session; steering requires the ACP server's raw steering advertisement. Do not write " + - "`const a = await agent(...)` when you intend to reuse the handle, because that stores only the answer. " + - "Persistent-workspace example — first eval: `const a = agent(\"codex\", \"Investigate the parser " + - "failure\")`; a later eval, only while agents() reports a's turn as running: `const steering = await " + - "a.steer(\"Focus on the parser state machine\")`; after the founding answer settles: `const first = await " + - "a; const q1 = a.queue(\"Implement the fix\"); const q2 = a.queue(\"Run the focused tests\"); " + - "console.log(q1.id, q2.id, steering); const fixed = await q1; const tested = await q2`. " + - "checkpoint(question) parks a promise for a human answer, resolved by checkpoint.answer(id, value) in a " + - "later eval. parallel, pipeline, verify, judgePanel, gate, retry, " + - "loopUntilDry, and sleep(ms) round out the guest library. Introspection is in-band: workspace() returns " + - "{ bindings, inFlight, checkpoints, diagnostics }; agents() lists live agents with their call ids and " + - "states; reset() tears the workspace down. `_` holds the previous eval's completion value. No fs, no " + - "net, no timers beyond sleep. Subagents (6 concurrent per workspace) take stable ids c1, c2, … used by " + - "interrupt and reported by agents(). " + - // The soft-bound eval loop. - "eval { code } runs the code, then HOLDS THE CALL OPEN pumping settlements up to a soft bound (default " + - "60 000 ms; per-call timeoutMs override; hard cap 120 000 ms). If everything the code waits on settles " + - "within the bound the result is the finished shape { output, result? } — output is ONE newline-joined " + - "string (console lines, checkpoint lines like \"checkpoint c9: \", error renderings), result " + - "the completion value's repr. If the bound elapses first the result is the still-running shape " + - "{ output, running: [call ids] } and the eval continues server-side — any later eval drains what " + - "settled, and eval with \"\" (the empty script) is the documented idempotent poll: it re-executes " + - "nothing, only reports. " + - // Durability (kept, hidden) + §5 hygiene. - "State survives MCP-session churn and daemon restarts: every eval and every settlement drain that " + - "changed state persists the workspace to the daemon's per-project repl store, and the first touch of a " + - "stored workspace restores it and reconciles every outstanding call. A stored snapshot that refuses " + - "(corrupt, a format upgrade, a wasm-binary mismatch) AUTO-RESETS — the file is renamed aside, never " + - "deleted, and the next eval's output leads with a notice naming the file and reason. Reconcile reports " + - "and drain errors live in workspace().diagnostics. On last-client disconnect the workspace drains " + - "in-flight subagent turns to completion and closes idle children; the next eligible queued turn re-attaches " + - "its founding session lazily. Subagent " + - "output passes through UNFILTERED — backend harness noise (e.g. codex's \"Warning: Skill descriptions " + - "were shortened…\") is forwarded verbatim, never curated away. Every result carries the machine-readable " + - "shape (see the output schema) as structuredContent alongside the human text.", - // STRICT at the wire too: the MCP SDK strips unknown keys from a - // non-strict object schema before the handler runs, so a deleted - // surface like `refs` would be silently discarded instead of - // rejected. The strict schema makes the wire fail on EVERY key - // outside the two actions' exact sets (§3.3 [C]4 / §7). - inputSchema: z.object(replToolInputShape).strict(), - outputSchema: replToolOutputShape, - }, - async (rawArgs) => { - if (!options.acceptingWork()) { - throw new ProtocolError( - ProtocolErrorCode.InternalError, - "Workflow server is shutting down and is no longer accepting tool calls.", - ); - } - // The action discriminator (the workflow tool's pattern): the MCP - // SDK validates the primitive fields, then the discriminator - // enforces each action's EXACT field set — `eval` without code, - // `interrupt` with code/timeoutMs, and projectDir-missing on the - // daemon are all rejected HERE, never deferred to late handler - // checks. - const input = parseReplToolInput(rawArgs as Record, { requireProjectDir }); - const { action, projectDir } = input; - const context = resolveContext(options, projectDir); - if (context === undefined) { - return { - structuredContent: { - error: `No project context is available for projectDir "${projectDir}".`, - }, - content: [{ type: "text", text: `No project context is available for projectDir "${projectDir}".` }], - isError: true, - }; - } - - // All actions touch the workspace: the session is marked present - // on the project (the client-presence ledger — its - // last-connection-closed signal drives the doc's drain). - context.repl ??= createReplProjectState(context.projectDir); - const state = context.repl; - options.presence.touch(state, options.clientId() ?? "unknown"); - - // The first touch (restore + reconcile, or the §6.1 auto-reset of - // a refused snapshot) happens on BOTH actions — an interrupt on a - // restored workspace must be able to target the restored suspended - // eval. The auto-reset/loss notices stay pending: only the next - // EVAL's output carries them. - await ensureReplWorkspace(state, await wasm, options.runner, options.evalTimeoutMs, options.evalBreakChannel); - const broker = state.broker!; - - if (action === "interrupt") { - return handleInterrupt(options, context.projectDir, broker, input); - } - - // ── eval: the soft-bound fused pump ──────────────────────────── - const bound = Math.min(input.timeoutMs, MAX_REPL_EVAL_BOUND_MS); - const deadline = Date.now() + bound; - let evalOutcome: Awaited>; - try { - evalOutcome = await broker.eval(input.code); - } catch (error) { - // A reset-owning eval that completed during this eval's pump - // tears the workspace down BEFORE the submitted code runs (the - // engine's documented order) — clear the state so the NEXT - // touch re-creates, and surface the honest failure. The pending - // notices stay on the state: no output was rendered to consume - // them, so the NEXT eval still leads with them. - syncReplStateAfterOp(state); - throw error; - } - // A reset-owning eval disposes the broker and the post-op sync - // tears the store down; the eval result may still need to render - // below (the renamed `.refused-*` file survives the store reset — - // §6.1 [C]13). - syncReplStateAfterOp(state); - const outputLines = [...evalOutcome.output]; - let finalResult: string | undefined; - let finalRunning: string[] | undefined; - if (evalOutcome.kind !== "pending") { - // Finished in-eval: a value (its repr in `result`) or an error - // (the §4.6 rendering in `output`). Nothing the code waits on - // remains — the finished shape ships immediately. - finalResult = evalOutcome.result; - if (input.code === "") { - // The empty eval IS the documented idempotent poll (§3.1 - // [C]3): its own completion is the guest `undefined` (repr - // "undefined") — and a previous eval that timed out may have - // settled in the meantime, its completion value swept under - // that eval's token. Claim the oldest such settlement: the - // poll reports the drained late value instead of its own - // empty-script undefined. A claimed `error` settlement's - // rendering already drained into `output` — the poll keeps - // its own undefined result then. - const swept = broker.claimSweptEvalSettlement(state.timedOutEvalTokens); - if (swept !== undefined) { - state.timedOutEvalTokens.delete(swept.token); - if (swept.kind === "value" && swept.result !== undefined) { - finalResult = swept.result; - } - } - } - } else { - // Suspended: hold the call open pumping settlements up to the - // bound. Each wait is passed the ids KNOWN to be pending at ITS - // entry (the eval's own suspension surface first, then each - // wait's last pending read), so a continuation that dispatches - // more calls is chased within the same call; a suspended eval - // awaiting nothing pumpable by call ids (a checkpoint, a sleep, - // a local promise) is re-polled on a short gap so host-timer - // settlements still resolve in-call. The wait's token-keyed seam - // reports exactly THIS eval's completion. Passing the KNOWN ids - // (never the ids-omitted form) also keeps the still-running shape - // honest under chain contention: a concurrent serialized - // operation that holds the broker through the whole remaining - // bound makes the broker's pending surface UNREADABLE (it would - // read empty) — the known in-flight ids stay reported, never - // replaced by an empty guess (§3.1 [D]3/[C]1). - let lastRunning = evalOutcome.pending; - let finished = false; - for (;;) { - const remaining = deadline - Date.now(); - if (remaining <= 0) break; - let waitResult: Awaited>["result"]; - let drained: boolean; - try { - const waited = await broker.waitForCalls(lastRunning, remaining, evalOutcome.evalToken); - waitResult = waited.result; - drained = waited.drained; - } catch (error) { - // A concurrent reset tore the broker down mid-hold — clear the - // state so the next touch re-creates, and surface the honest - // failure. The pending §6.1/§6.2 notices are NOT taken yet - // (they are consumed only when a result that carries them is - // actually returned — review finding): they stay on the state - // so the next successful eval still leads with them. - syncReplStateAfterOp(state); - throw error; - } - outputLines.push(...waitResult.output); - lastRunning = waitResult.pending; - if (syncReplStateAfterOp(state) && waitResult.kind === "pending") { - // A reset() tore the workspace down mid-hold (this eval's own - // reset completed — or a concurrent client's): the workspace - // is gone; report the honest still-running shape and let the - // next touch re-create. - break; - } - if (waitResult.kind !== "pending") { - // The eval's continuation completed during the pumps — the - // finished shape with its completion repr (or the late error - // rendering already in the output lines). - finalResult = waitResult.result; - finished = true; - break; - } - if (Date.now() >= deadline) break; - if (waitResult.pending.length === 0 || !drained) { - await new Promise((resolve) => setTimeout(resolve, Math.min(REPL_EVAL_POLL_GAP_MS, deadline - Date.now()))); - if (Date.now() >= deadline) break; - } - // `drained` with pending ids left: the continuation dispatched - // more calls — chase them within the remaining bound. - } - if (!finished) { - // The bound elapsed first: the honest still-running shape. The - // eval continues server-side; any later eval (including `""`) - // drains what settled. Its continuation token joins the poll - // seam: when the eval settles later, a subsequent empty eval - // claims the swept settlement under this token and reports the - // late completion value as ITS `result` (§3.1 [C]3). - if (evalOutcome.evalToken !== undefined) { - state.timedOutEvalTokens.add(evalOutcome.evalToken); - } - finalRunning = lastRunning; - } - } - // §6.1/§6.2: the pending notices are consumed ONLY here, at the - // single point a result that carries them is built — a - // `waitForCalls` failure (or any other throwing path) above - // renders NO eval result, so the notices stay on the state and - // the NEXT successful eval still leads with them (review - // finding: taking them before the held settlement pump lost them - // on the pump's throwing path). - const notices = takeNotices(state); - return evalResult(outputLines, finalResult, finalRunning, notices); - }, - ); -} - -/** The interrupt action (§3.2, unchanged from v1): with `id`, cancel - * that subagent call; without `id`, BREAK THE RUNNING EVAL through the - * broker's eval-break arm (suspended evals) and the OUT-OF-BAND - * eval-break channel (a synchronous runaway blocking the daemon's - * event loop — the MCP shim fires the worker-thread relay before - * forwarding, and the running eval's quickjs interrupt handler - * consumes the break flag mid-execution). Honest `refused-idle` when - * nothing is running. */ -async function handleInterrupt( - options: ReplToolOptions, - projectDir: string, - broker: Broker, - input: Extract, -): Promise<{ structuredContent: Record; content: { type: "text"; text: string }[] }> { - if (input.id === undefined) { - const targeted = await broker.armEvalBreak(); - options.evalBreakChannel?.clearBreak(projectDir); - // The honest out-of-band outcome: the running eval broke via the - // relay while the daemon was blocked (the break's delivery record — - // the `armEvalBreak` refusal above is expected for a SYNC eval, - // which is never continuation-tracked). The record is consumed on - // read, so a later interrupt never inherits an earlier break's - // delivery. - if (!targeted && broker.consumeOutOfBandBreakReport() !== null) { - return { - structuredContent: { - interrupt: { outcome: "targeted" }, - }, - content: [ - { - type: "text", - text: - `workspace ${projectDir}: the running eval was broken OUT OF BAND — the relay delivered ` + - `the break while the daemon's main thread was blocked in the eval, and the quickjs interrupt ` + - `handler broke it mid-run`, - }, - ], - }; - } - if (!targeted) { - return { - structuredContent: { - interrupt: { outcome: "refused-idle" }, - }, - content: [ - { - type: "text", - text: - `workspace ${projectDir}: no running eval to interrupt — no eval is in flight; nothing was armed`, - }, - ], - }; - } - return { - structuredContent: { - interrupt: { outcome: "targeted" }, - }, - content: [ - { - type: "text", - text: - `workspace ${projectDir}: interrupting the running eval — the eval-break signal is set and the eval's next ` + - `execution (a settlement drain resuming its continuation, or a direct eval's drain) is broken mid-run by the ` + - `quickjs interrupt handler; an eval suspended on nothing resumable (a never-settling local promise) is ` + - `terminated outright — its tracked continuation is released immediately`, - }, - ], - }; - } - const outcome = await broker.cancelCall(input.id); - const text = - outcome === "cancelled" - ? `interrupt ${input.id}: ACP session/cancel sent` - : outcome === "idle" - ? `interrupt ${input.id}: the session was idle — nothing to cancel` - : outcome === "failed" - ? `interrupt ${input.id}: could not reach the backend session (lazy re-attach failed)` - : `interrupt ${input.id}: no live session to cancel`; - return { - structuredContent: { - interrupt: { outcome, callId: input.id }, - }, - content: [{ type: "text", text }], - }; -} diff --git a/packages/mcp-server/src/server.ts b/packages/mcp-server/src/server.ts index 49f09bb9..d6e53049 100644 --- a/packages/mcp-server/src/server.ts +++ b/packages/mcp-server/src/server.ts @@ -12,8 +12,8 @@ import type { // packages/mcp-server/src/server.ts // -// The MCP shell: constructs an McpServer, registers the `workflow` and `repl` model-facing -// tools, serves their version-matched Agent Skills, and adds the user-controlled +// The MCP shell: constructs an McpServer, registers the `workflow` model-facing +// tool, serves its version-matched Agent Skill, and adds the user-controlled // `author-workflow` prompt. This is the composition root where all three packages meet — the injected acp-agents // AgentRunner is wired into a workflow-engine WorkflowManager (DI) and every tool call runs // through WorkflowManager.runSync. @@ -44,13 +44,6 @@ import { } from "./workflow-lifecycle.js"; import { createProgressReporter } from "./progress.js"; import { CLAUDE_CHANNEL_CAPABILITY, ClaudeChannelNotifier } from "./channel-notifier.js"; -import { - createEvalBreakChannel, - loadShippedWasm, - type BrokerRunner, - type EvalBreakChannel, -} from "@automatalabs/repl-engine"; - import { clampWorkflowInput, parseWorkflowToolInput, @@ -81,11 +74,7 @@ import type { } from "./workflow-tool-output.js"; import { registerAuthoringPrompt } from "./authoring-prompt.js"; import { registerAuthoringSkills, SKILLS_EXTENSION_ID } from "./authoring-skills.js"; -import { registerReplTool } from "./repl-tool.js"; -import { ReplPresenceLedger } from "./repl-presence.js"; import { CapabilityAwareToolCatalog } from "./tool-catalog.js"; -import { createReplProjectState, DEFAULT_REPL_EVAL_TIMEOUT_MS } from "./repl-project.js"; -import { REPL_DRAIN_BOUND_MS } from "./daemon/constants.js"; import type { WorkflowRunControlRouter } from "./daemon/run-control.js"; import { configSummary, @@ -124,12 +113,12 @@ export const SERVER_VERSION: string = : (require("../package.json") as { version: string }).version; // Server-wide guidance returned in the MCP initialize response (ServerOptions.instructions), -// surfaced by hosts to orient the calling agent to the two model-facing tools and the +// surfaced by hosts to orient the calling agent to the model-facing `workflow` tool and the // version-matched workflow Agent Skill. Kept short and behavioral — exhaustive guidance is loaded through // the host's skill activation path only when needed. export const SERVER_INSTRUCTIONS = [ - "This server exposes workflow and repl orchestration tools, plus workflow_monitor for Apps-capable hosts. They " + - "spawn subagents over the same ACP backends — the registry built-ins Claude, Codex, OpenCode, and " + + "This server exposes the workflow orchestration tool, plus workflow_monitor for Apps-capable hosts. It " + + "spawns subagents over the same ACP backends — the registry built-ins Claude, Codex, OpenCode, and " + "pi, plus any registered custom agents — and key durable state by an absolute projectDir " + "(required on the shared daemon; defaulted by a single-project server). Backend credentials come " + "from each agent's own login, so there is nothing auth-shaped to configure here.", @@ -141,12 +130,6 @@ export const SERVER_INSTRUCTIONS = [ "a durable runId for bounded status, permissions-response, result, pause, and stop calls; resume continues " + "the exact run (paused or stopped) from its durable admission and journal. action:\"config\" discovers the live backend " + "and model option catalog. Every agent call must resolve an explicit model route (backend-only routes are valid). Accepted runs prepare durably; custom backends require approval. Checkpoints always require an explicit answer.", - "• repl — INTERACTIVE STATEFUL orchestration. A persistent per-project JavaScript VM driven with " + - "action:\"eval\". Named bindings, pending subagent handles, queued turns, checkpoints, and `_` " + - "persist between calls and survive daemon restarts. Use it when the next orchestration step depends " + - "on inspecting intermediate results.", - "Rule of thumb: use workflow when you can script the whole plan ahead of time; use repl when you " + - "want a live session that evolves call by call.", "Claude Code channels: when this server is loaded as a channel, the updates the run monitor would " + "show arrive as \" run_id=\"…\" " + "kind=\"terminal|paused|checkpoint|permission|setup\" status=\"…\"> events for every workflow run " + @@ -795,7 +778,7 @@ function formatStatusSummary(result: WorkflowStatusToolResult): string { } /** - * Build the MCP server with the `workflow` and `repl` model-facing tools, their Agent Skills, + * Build the MCP server with the `workflow` model-facing tool, its Agent Skill, * plus the user-controlled `author-workflow` prompt. Prompts are a separate MCP primitive and never enter the model's tool-selection * loop). Backend auth is the agents' own concern (their CLI credential stores); a run that * genuinely hits AUTH_REQUIRED pauses with authContext and resumes after an out-of-band CLI @@ -804,17 +787,6 @@ function formatStatusSummary(result: WorkflowStatusToolResult): string { * through manager.runSync or startInBackground. The returned McpServer is not yet connected — the caller attaches a * transport (see index.ts). */ -/** The per-eval wall-clock deadline (see `repl-project.ts`); the - * `AGENTPRISM_REPL_EVAL_TIMEOUT_MS` env knob, clamped to >= 1 ms. */ -function replEvalTimeoutMs(): number { - const env = process.env.AGENTPRISM_REPL_EVAL_TIMEOUT_MS; - if (env !== undefined) { - const parsed = Number.parseInt(env, 10); - if (Number.isFinite(parsed) && parsed >= 1) return parsed; - } - return DEFAULT_REPL_EVAL_TIMEOUT_MS; -} - export interface CreateWorkflowServerOptions { /** Pin a pre-built manager as this server's own project (composition/back-compat seam). */ manager?: WorkflowManager; @@ -831,42 +803,14 @@ export interface CreateWorkflowServerOptions { */ requireProjectDir?: boolean; /** - * The REPL workspaces' ACP runner (the broker's structural seam). Omitted: every - * workspace's broker owns its own `AcpAgentRunner` (disposed with the workspace). Tests - * inject a fake and own its lifetime. - */ - replRunner?: BrokerRunner; - /** - * The REPL client-presence ledger (daemon mode: one ledger per daemon, shared by every - * session; single-project mode: a private ledger). Drives the doc's last-client- - * disconnect drain. Omitted: a private ledger is created (the single-project mode's - * own client presence). + * This server's MCP client identity on a legacy-era connection (daemon mode: the per-session + * transport's id). Scopes `workflow_monitor` notification claims so one client's claim never + * answers for another's. Omitted: a single-client server uses one fixed scope. Modern-era + * requests carry their own scope and ignore it. */ - replPresence?: ReplPresenceLedger; - /** - * This server's MCP session id (daemon mode: the per-session transport's id, resolved - * per call; single-project mode: a fixed client id). The `repl` tool touches presence - * under it. - */ - replClientId?: () => string | undefined; - /** The REPL eval-break relay (phase-F review round 2; daemon mode — - * the shim fires it while the daemon's main thread is blocked in a - * synchronous eval). OMITTED in single-project mode: the server owns - * a channel of its own by default (round 3 — the documented no-id - * interrupt must work in every supported mode; the stdio transport's - * worker-reader fires it, and `replBreakUrl()` exposes the relay to - * library hosts). */ - replEvalBreakChannel?: EvalBreakChannel; - /** - * The concrete client-presence drain bound — the daemon reuses its session-eviction - * TTL (the spec-owed decision; see `repl-presence.ts`). Defaults to - * `SESSION_IDLE_TTL_MS`. - */ - replDrainBoundMs?: number; + clientId?: () => string | undefined; /** Protocol era selected by an SDK serving entry. Hand-connected servers remain legacy. */ protocolEra?: "legacy" | "modern"; - /** Modern request instances have request-scoped presence and disconnect when the instance closes. */ - disconnectReplClientOnClose?: boolean; /** Daemon-scoped publisher for modern subscriptions/listen change delivery. */ modernNotifier?: ServerNotifier; /** Daemon-only location-transparent run-control router. */ @@ -891,38 +835,10 @@ export function createWorkflowServer( ); const toolCatalog = new CapabilityAwareToolCatalog(mcp, options.protocolEra ?? "legacy"); let acceptingWork = true; - // The REPL eval-break channel (phase-F review round 3): the in-process/ - // library server OWNS one by default — the documented no-id interrupt - // for a synchronously running eval is deliverable in every supported - // mode, not only daemon mode (the daemon passes its own channel and - // owns its lifetime; `disposeReplEvalBreakChannel` disposes only a - // server-owned channel). The relay address is exposed as - // `replBreakUrl()` on the server control — the stdio transport's - // worker-reader fires it (see `repl-stdio-transport.ts`), and a - // library host can fire it from another thread. - const ownsReplEvalBreakChannel = options.replEvalBreakChannel === undefined; - const replEvalBreakChannel = options.replEvalBreakChannel ?? createEvalBreakChannel(); const server = Object.assign(mcp, { stopAcceptingWork() { acceptingWork = false; }, - replBreakUrl() { - return replEvalBreakChannel.breakUrl(); - }, - replDefaultProjectDir() { - // The single-project server's own project: the FIRST registry - // context — exactly what the repl tool's projectDir-omitted - // resolution returns (`resolveContext`: `stores()[0]`). The - // relay transport fires its out-of-band break under this key - // when the client omits projectDir (phase-F review round 4: the - // omitted-projectDir interrupt used to skip the relay entirely - // and run to the per-eval deadline). Undefined in daemon mode - // (projectDir is required there) and when no context exists yet. - return projects.stores()[0]?.projectDir; - }, - async disposeReplEvalBreakChannel() { - if (ownsReplEvalBreakChannel) await replEvalBreakChannel.dispose(); - }, }); // registerCapabilities is illegal after a transport attaches. Merge the complete resources @@ -956,17 +872,6 @@ export function createWorkflowServer( registerAuthoringSkills(mcp, { registerResourceReader: (uri, read) => scriptResources.registerExternalResourceReader(uri, read), }); - // The REPL client-presence ledger (see `repl-presence.ts`): one per - // server, shared by the repl tool AND the workflow tool — a session - // that addresses a project through WORKFLOW calls is present on that - // project exactly like one that touched the repl workspace (phase-E - // review rejection round 2: the workflow handler resolved the same - // project context without registering presence, so a workflow-only - // client's presence was invisible to the last-client-disconnect drain - // and a repl client's disconnect could drain children while the - // workflow client was still connected). - const replPresence = options.replPresence ?? new ReplPresenceLedger(options.replDrainBoundMs ?? REPL_DRAIN_BOUND_MS); - /** Route a parsed input to its project context; undefined = runId found in no known store. */ const resolveContext = (input: ReturnType): ProjectContext | undefined => { if ( @@ -993,24 +898,6 @@ export function createWorkflowServer( registerAuthoringPrompt(mcp); const probeRunner = workflowProbeRunner(runner); - // The REPL tool (roadmap doc's Surface section; phase D wiring): one - // persistent VM per project context, restored from the daemon's - // per-project repl store on first touch and reconciled; the snapshot - // sink attached by `ensureReplWorkspace` persists every state-changing - // boundary. The wasm is the engine's shipped binary (its hash is the - // snapshot envelope's identity — a version bump refuses loudly). - registerReplTool(mcp, { - projects, - wasm: loadShippedWasm(), - requireProjectDir, - runner: options.replRunner, - evalTimeoutMs: replEvalTimeoutMs(), - presence: replPresence, - clientId: options.replClientId ?? (() => "single-project"), - evalBreakChannel: replEvalBreakChannel, - acceptingWork: () => acceptingWork, - }); - const workflowToolOutputSchema = workflowToolOutputShape; const workflowToolConfig = { title: "Run and manage deterministic agent workflows", @@ -1125,19 +1012,6 @@ export function createWorkflowServer( isError: true, }; } - // Project-presence registration for the REPL's client-presence - // drain (phase-E review rejection round 2): the workflow tool - // resolves the SAME per-project context the repl tool addresses, - // and a session that calls it is connected to the project for the - // doc's "any MCP client connected to the project" warmth rule. - // The repl STATE is created if missing — a pure-workflow project - // keeps a stateless context (no VM: the workspace is materialized - // only on the first repl tool touch); the state is what the - // presence ledger keys presence by, so a workflow-only client B - // staying connected keeps the workspace warm when repl-client A - // disconnects. - if (context.repl === undefined) context.repl = createReplProjectState(context.projectDir); - replPresence.touch(context.repl, options.replClientId?.() ?? "unknown"); const manager = context.manager; const activeRuns = context.activeRuns; if ("runId" in parsedInput) workflowLifecycle(context, runner).recover(parsedInput.runId); @@ -1762,7 +1636,7 @@ export function createWorkflowServer( if (!projects.storeFor(request.runId)) throw new ProtocolError(ProtocolErrorCode.InvalidParams, `No workflow run found for ${request.runId}`); const scope = options.protocolEra === "modern" ? `modern:${request.scopeId ?? request.viewId}` - : `legacy:${options.replClientId?.() ?? "single-project"}`; + : `legacy:${options.clientId?.() ?? "single-project"}`; return projects.notificationClaims.handle(scope, request); }, readEventsPage: (request) => scriptResources.readEventsPage(request), @@ -1805,21 +1679,6 @@ export function createWorkflowServer( } }; - if (options.disconnectReplClientOnClose) { - const previousOnClose = mcp.server.onclose; - mcp.server.onclose = () => { - try { - previousOnClose?.(); - } finally { - const clientId = options.replClientId?.(); - if (clientId !== undefined) { - replPresence.disconnect(clientId); - replPresence.forget(clientId); - } - } - }; - } - if (channel) { const previousOnClose = mcp.server.onclose; mcp.server.onclose = () => { diff --git a/packages/mcp-server/src/shim/shim.ts b/packages/mcp-server/src/shim/shim.ts index 9f056899..d6720ced 100644 --- a/packages/mcp-server/src/shim/shim.ts +++ b/packages/mcp-server/src/shim/shim.ts @@ -24,8 +24,6 @@ * without inventing initialize or session state. */ -import { realpathSync } from "node:fs"; -import { isAbsolute } from "node:path"; import { SdkHttpError, StreamableHTTPClientTransport, @@ -77,48 +75,9 @@ function isRecoverableError(error: unknown): boolean { export async function runShim(options: RunShimOptions): Promise { const log = (line: string) => console.error(line); const info = await ensureDaemonRunning({ bundlePath: options.bundlePath, port: options.port, log }); - // The REPL eval-break relay (phase-F review round 2): the worker- - // thread channel whose loopback endpoint stays reachable while the - // daemon's main thread is blocked in a synchronous eval. The shim - // fires the `repl` interrupt tool's no-id break here BEFORE forwarding - // the request to the daemon — the out-of-band delivery that makes the - // documented quickjs-interrupt behavior real for a never-yielding - // eval (the daemon processes the forwarded request only after the - // eval ends or breaks; the relay's flag is what breaks it mid-run). - let replBreakUrl: string | undefined = info.replBreakUrl; let exiting = false; let compatibilityDrainTimer: NodeJS.Timeout | undefined; - /** Fire the out-of-band break for a `repl` interrupt without an id - * (best-effort, fire-and-forget: the daemon's own processing clears - * the flag when it lands; a missing/stale relay URL or a dead relay - * degrades to the per-eval deadline bound). The key is REALPATH'd - * exactly like the daemon's own project validation (phase-F review - * round 3: the raw caller-supplied path used to be posted verbatim, - * while the tool realpaths it and the channel registers the - * canonical path — a valid absolute symlink or a path with - * redundant components therefore got a relay 404 and could not - * interrupt the running eval). An unresolvable path is skipped: the - * tool call itself is refused as an invalid projectDir. */ - function fireOutOfBandBreak(projectDir: unknown): void { - if (typeof projectDir !== "string" || replBreakUrl === undefined) return; - let key: string; - try { - if (!isAbsolute(projectDir)) return; - key = realpathSync(projectDir); - } catch { - return; // invalid/unresolvable — the daemon's own validation refuses the call - } - void fetch(replBreakUrl, { - method: "POST", - headers: { "content-type": "application/json" }, - body: JSON.stringify({ key }), - signal: AbortSignal.timeout(1000), - }).catch(() => { - // Best-effort: a dead relay must never break the forwarding path. - }); - } - const stdio = new StdioServerTransport(); let cachedInitialize: JSONRPCMessage | undefined; @@ -323,7 +282,6 @@ export async function runShim(options: RunShimOptions): Promise { retireTransport(http, reason); // Re-ensure: the daemon may have restarted (new port) or been superseded by a newer one. const fresh = await ensureDaemonRunning({ bundlePath: options.bundlePath, port: options.port, log }); - replBreakUrl = fresh.replBreakUrl; armCompatibilityDrain(fresh); http = makeHttpTransport(fresh.url); await http.start(); @@ -414,22 +372,6 @@ export async function runShim(options: RunShimOptions): Promise { if (message.method === "resources/subscribe") subscribedUris.add(uri); else subscribedUris.delete(uri); } - } else if (message.method === "tools/call") { - // The out-of-band repl eval-break (phase-F review round 2): a - // `repl` interrupt WITHOUT a call id fires the relay first — the - // daemon may be blocked in a synchronous eval, and the relay's - // worker thread is the only path that can reach it mid-run. The - // forwarded request is still sent (when the daemon is - // responsive, its own arm/refuse/clear handling owns the - // break; the flag is consumed on first observation or cleared - // by the daemon — never a stale break). - const params = message.params as { name?: unknown; arguments?: Record } | undefined; - if (params?.name === "repl") { - const args = params.arguments ?? {}; - if (args.action === "interrupt" && args.id === undefined) { - fireOutOfBandBreak(args.projectDir); - } - } } } else if (isJSONRPCNotification(message) && message.method === "notifications/cancelled") { const requestId = (message.params as { requestId?: unknown } | undefined)?.requestId; diff --git a/packages/mcp-server/src/wasm-ambient.d.ts b/packages/mcp-server/src/wasm-ambient.d.ts deleted file mode 100644 index ca21743b..00000000 --- a/packages/mcp-server/src/wasm-ambient.d.ts +++ /dev/null @@ -1,19 +0,0 @@ -/** - * Minimal ambient types for the WebAssembly surface this package's type - * graph touches. The repo's base tsconfig lib is `ES2022 + - * ESNext.Disposable` (no DOM), so the global `WebAssembly` namespace is - * not declared. This package typechecks `@automatalabs/repl-engine`'s - * source (its `types` field points at `src/index.ts`, the repo's - * workspace convention), whose `loadShippedWasm` calls - * `WebAssembly.compile`; the engine declares its own ambient - * (`packages/repl-engine/src/wasm-ambient.d.ts`) for ITS compilation, - * which is not visible to this package's program — hence this mirror. - * At runtime the `WebAssembly` global is provided by Node. - */ - -type BufferSource = ArrayBufferView | ArrayBuffer; - -declare namespace WebAssembly { - interface Module {} - function compile(bytes: BufferSource): Promise; -} diff --git a/packages/mcp-server/src/workflow-permissions.ts b/packages/mcp-server/src/workflow-permissions.ts index aa63992e..5c73dee9 100644 --- a/packages/mcp-server/src/workflow-permissions.ts +++ b/packages/mcp-server/src/workflow-permissions.ts @@ -285,8 +285,8 @@ export class WorkflowPermissionBroker { private detachEvents: (() => void) | undefined; readonly resolver: PermissionResolver = (request, context) => { - // The daemon's runner is shared with the REPL. Engine workflow calls always stamp both an - // engine runId and callIndex; other callers retain the SDK's autonomous auto-response path. + // Engine workflow calls always stamp both an engine runId and callIndex; any other caller of + // the shared runner retains the SDK's autonomous auto-response path. if ( context.backendId === "pi" || context.runId === undefined || diff --git a/packages/mcp-server/test/app-ui.test.ts b/packages/mcp-server/test/app-ui.test.ts index 09a41bb4..b9a60ffe 100644 --- a/packages/mcp-server/test/app-ui.test.ts +++ b/packages/mcp-server/test/app-ui.test.ts @@ -39,7 +39,7 @@ test("only workflow_monitor carries the panel resource; app-only support tools c const { tools } = await client.listTools(); assert.deepEqual( tools.map((tool) => tool.name).sort(), - ["repl", "workflow", WORKFLOW_MONITOR_TOOL_NAME, WORKFLOW_EVENTS_TOOL_NAME, WORKFLOW_RUNS_TOOL_NAME, WORKFLOW_NOTIFICATIONS_TOOL_NAME].sort(), + ["workflow", WORKFLOW_MONITOR_TOOL_NAME, WORKFLOW_EVENTS_TOOL_NAME, WORKFLOW_RUNS_TOOL_NAME, WORKFLOW_NOTIFICATIONS_TOOL_NAME].sort(), ); const workflow = tools.find((tool) => tool.name === "workflow"); @@ -95,7 +95,7 @@ test("only the exact well-formed extensions capability receives the MCP Apps sur malformedString, ]) { const tools = (await session.client.listTools()).tools; - assert.deepEqual(tools.map((tool) => tool.name).sort(), ["repl", "workflow"]); + assert.deepEqual(tools.map((tool) => tool.name).sort(), ["workflow"]); const workflow = tools.find((tool) => tool.name === "workflow"); assert.ok(workflow); assert.equal(workflow._meta, undefined, "text workflow has no UI metadata"); diff --git a/packages/mcp-server/test/authoring-prompt.test.ts b/packages/mcp-server/test/authoring-prompt.test.ts index a815f9fd..db1c52dd 100644 --- a/packages/mcp-server/test/authoring-prompt.test.ts +++ b/packages/mcp-server/test/authoring-prompt.test.ts @@ -70,7 +70,7 @@ test("prompt registration leaves a non-Apps client's core tool inventory unchang const { tools } = await client.listTools(); assert.deepEqual( tools.map((tool) => tool.name).sort(), - ["repl", "workflow"], + ["workflow"], ); } finally { await dispose(); diff --git a/packages/mcp-server/test/daemon/daemon-info.test.ts b/packages/mcp-server/test/daemon/daemon-info.test.ts index 1d788df3..9efa8847 100644 --- a/packages/mcp-server/test/daemon/daemon-info.test.ts +++ b/packages/mcp-server/test/daemon/daemon-info.test.ts @@ -103,7 +103,7 @@ test("envFingerprint tracks runner-relevant vars and ignores unrelated or TTL-on assert.notEqual(base, envFingerprint({ AGENTPRISM_BACKENDS: '{"a":2}' })); assert.notEqual(base, envFingerprint({})); assert.equal( - envFingerprint({ AGENTPRISM_DAEMON_IDLE_TTL_MS: "1", AGENTPRISM_SESSION_TTL_MS: "2", AGENTPRISM_REPL_DRAIN_BOUND_MS: "3" }), + envFingerprint({ AGENTPRISM_DAEMON_IDLE_TTL_MS: "1", AGENTPRISM_SESSION_TTL_MS: "2" }), envFingerprint({}), ); }); diff --git a/packages/mcp-server/test/daemon/daemon-lifecycle.test.ts b/packages/mcp-server/test/daemon/daemon-lifecycle.test.ts index 37c8745e..a78e3a95 100644 --- a/packages/mcp-server/test/daemon/daemon-lifecycle.test.ts +++ b/packages/mcp-server/test/daemon/daemon-lifecycle.test.ts @@ -1,12 +1,7 @@ /** - * The daemon process-lifetime accounting (phase-E review rejection round - * 2): idleness means no sessions, no active workflow runs, AND no active - * REPL client-presence drain. A last-client-disconnect drain may - * legitimately run for the full session-eviction TTL after the final - * session is gone; the default idle shutdown must never replace that - * drain's bound with the five-second shutdown deadline — so the reaper's - * busy check counts `activeReplDrainCount()` exactly like sessions and - * runs, and the idle clock only starts once every drain completed. + * The daemon process-lifetime accounting: idleness means no sessions, no active workflow runs, + * and no request in flight; the idle clock only starts once all three are quiet. A superseded + * daemon migrates its drainable sessions and exits as soon as nothing is busy. */ import assert from "node:assert/strict"; @@ -26,7 +21,6 @@ function sleep(ms: number): Promise { function fakeHandle(state: { sessions: number; activeRuns: number; - activeDrains: number; superseded?: boolean; /** Sessions closed by the lame-duck migration; `evictDrainable` empties `sessions`. */ migrated?: number; @@ -38,9 +32,8 @@ function fakeHandle(state: { instanceId: "test-instance", controlUrl: "http://127.0.0.1:0/_agentprism/control/v1/run", sessions: { get size() { return state.sessions; }, evictIdle: () => [], inflightCount: () => 0 } as never, - projects: { disposeReplStates: async () => undefined } as never, + projects: {} as never, activeRunCount: () => state.activeRuns, - activeReplDrainCount: () => state.activeDrains, inflightRequestCount: () => 0, isSuperseded: () => state.superseded ?? false, evictDrainableSessions: () => { @@ -78,33 +71,8 @@ function fakeProcess(): { return { handle, exits }; } -test("idle shutdown never fires while a REPL client-presence drain is in flight — the drain is counted like sessions and runs; shutdown fires only after the drain completed", async () => { - const state = { sessions: 0, activeRuns: 0, activeDrains: 1 }; - const process = fakeProcess(); - // The drain runs with no sessions and no workflow runs: the OLD - // accounting (sessions + runs only) would have idled the daemon out — - // and the shutdown path would have replaced the drain's - // session-eviction-TTL bound with the five-second shutdown deadline. - installDaemonLifecycle({ - daemon: fakeHandle(state), - runner: { dispose: async () => undefined } as unknown as AgentRunner, - ownPid: 424242, - idleTtlMs: 80, - sessionTtlMs: 1_000, - reaperIntervalMs: 20, - process: process.handle, - log: () => undefined, - }); - await sleep(160); - assert.deepEqual(process.exits, [], "idle shutdown must not fire while the drain is in flight"); - // The drain completes: the idle clock starts, and the daemon idles out. - state.activeDrains = 0; - await sleep(220); - assert.deepEqual(process.exits, [0], "idle shutdown fires after the drain completed"); -}); - -test("sessions and runs still count as busy alongside drains (the pre-existing accounting is unchanged); a drain that starts later restarts the idle clock", async () => { - const state = { sessions: 1, activeRuns: 0, activeDrains: 0 }; +test("sessions and runs count as busy; a run that starts as the last session closes restarts the idle clock", async () => { + const state = { sessions: 1, activeRuns: 0 }; const process = fakeProcess(); installDaemonLifecycle({ daemon: fakeHandle(state), @@ -118,20 +86,19 @@ test("sessions and runs still count as busy alongside drains (the pre-existing a }); await sleep(160); assert.deepEqual(process.exits, [], "a live session holds the daemon open"); - // The last session closes and a drain starts: the idle clock must not - // accumulate across the transition (the drain counts as busy from the - // moment it is scheduled). + // The last session closes while a run is active: the idle clock must not accumulate across + // the transition. state.sessions = 0; - state.activeDrains = 1; + state.activeRuns = 1; await sleep(160); - assert.deepEqual(process.exits, [], "the drain restarted the idle clock"); - state.activeDrains = 0; + assert.deepEqual(process.exits, [], "the active run restarted the idle clock"); + state.activeRuns = 0; await sleep(220); - assert.deepEqual(process.exits, [0], "idle shutdown fires once the drain completed"); + assert.deepEqual(process.exits, [0], "idle shutdown fires once the run finished"); }); test("a superseded daemon does not wait for the idle TTL: it migrates idle sessions and exits as soon as nothing is busy, even with idle shutdown disabled", async () => { - const state = { sessions: 3, activeRuns: 0, activeDrains: 0, superseded: true, migrated: 0 }; + const state = { sessions: 3, activeRuns: 0, superseded: true, migrated: 0 }; const { handle, exits } = fakeProcess(); const logs: string[] = []; const lifecycle = installDaemonLifecycle({ @@ -157,7 +124,7 @@ test("a superseded daemon does not wait for the idle TTL: it migrates idle sessi }); test("a superseded daemon migrates drainable MCP sessions while retaining execution ownership, then exits once runs finish", async () => { - const state = { sessions: 2, activeRuns: 1, activeDrains: 0, superseded: true, migrated: 0 }; + const state = { sessions: 2, activeRuns: 1, superseded: true, migrated: 0 }; const { handle, exits } = fakeProcess(); const lifecycle = installDaemonLifecycle({ daemon: fakeHandle(state), diff --git a/packages/mcp-server/test/daemon/http-daemon.test.ts b/packages/mcp-server/test/daemon/http-daemon.test.ts index df290392..6862e06b 100644 --- a/packages/mcp-server/test/daemon/http-daemon.test.ts +++ b/packages/mcp-server/test/daemon/http-daemon.test.ts @@ -276,94 +276,3 @@ test("middleware rejects bad Origin over real HTTP; unknown paths 404", async () await daemon.close(); } }); - -test("the session registry signals last-connection-closed and the repl presence drain closes idle children", async () => { - // Phase-D review round 2: the daemon's session registry measures - // liveness by connection presence and SIGNALS project repl lifecycle — - // a client whose last connection closed leaves its projects' workspaces - // drained (in-flight turns complete — each settlement boundary - // snapshots — then idle children close; the next connect re-attaches - // lazily). The drain itself is pinned in repl-review2.test.ts; this - // test pins the daemon wiring: registry signal → presence ledger → - // drain over a real HTTP session. - type Turn = { resolve: (turn: import("@automatalabs/repl-engine").BrokerTurn) => void }; - let pendingTurn: Turn | undefined; - const releases: number[] = []; - const fakeSession = { - sessionId: "fake-s1", - backendId: "pi", - initializeMeta: { steering: { supported: true } }, - async prompt(): Promise { - return new Promise((resolve) => { - pendingTurn = { resolve }; - }); - }, - async steer(): Promise { - return { outcome: "injected" }; - }, - async cancel(): Promise {}, - async release(): Promise { - releases.push(1); - }, - currentTurnText(): string { - return "done text"; - }, - finalMessageText(): string { - return "done text"; - }, - rawStructuredOutput(): unknown { - return undefined; - }, - } as import("@automatalabs/repl-engine").BrokerSession; - const runner = { - sessions: 0, - listBackends(): string[] { - return ["pi"]; - }, - defaultBackendId(): string { - return "pi"; - }, - async openSession(): Promise { - this.sessions++; - return fakeSession; - }, - async loadSession(): Promise { - throw new Error("no load"); - }, - async dispose(): Promise {}, - } as unknown as import("@automatalabs/repl-engine").BrokerRunner; - - const daemon = await createDaemon({ - runner: okRunner(), - port: 0, - env: {}, - log: () => undefined, - replRunner: runner, - sessionTtlMs: 60_000, - } as never); - const project = makeProjectDir("repl-drain"); - const a = await connectHttp(daemon.url); - try { - const started = await a.client.callTool({ - name: "repl", - arguments: { action: "eval", projectDir: project, code: 'const p = agent("pi/x", "task"); "started"' }, - }); - assert.ok(!(started as { isError?: boolean }).isError, textOf(started)); - await new Promise((resolve) => setTimeout(resolve, 50)); - assert.equal(runner.sessions, 1, "the session opened"); - // The client disconnects: the session's connections close, the - // registry signals, and the project's workspace starts draining — - // the in-flight turn is WAITED OUT, not cancelled. - await a.dispose(); - await new Promise((resolve) => setTimeout(resolve, 50)); - assert.equal(releases.length, 0, "the drain waits for the in-flight turn (never cancels it)"); - assert.ok(pendingTurn !== undefined, "the founding turn is still in flight"); - pendingTurn!.resolve({ stopReason: "end_turn", text: "done text" }); - for (let attempt = 0; attempt < 100 && releases.length === 0; attempt++) { - await new Promise((resolve) => setTimeout(resolve, 20)); - } - assert.equal(releases.length, 1, "the idle child closed after the drain completed the turn"); - } finally { - await daemon.close(); - } -}); diff --git a/packages/mcp-server/test/daemon/repl-break.e2e.test.ts b/packages/mcp-server/test/daemon/repl-break.e2e.test.ts deleted file mode 100644 index 0c4a83e1..00000000 --- a/packages/mcp-server/test/daemon/repl-break.e2e.test.ts +++ /dev/null @@ -1,248 +0,0 @@ -// End-to-end over the BUILT dist: the out-of-band eval-break (phase-F -// review round 2) — the `repl` interrupt tool's no-id path delivered to -// a SYNCHRONOUSLY running eval. A never-yielding eval (`while (true) {}`) -// blocks the daemon's single thread, so the interrupt request itself -// cannot be processed; the daemon's eval-break relay (a worker thread, -// advertised in daemon.json) is the one path that can reach the running -// eval: the quickjs interrupt handler consumes the relay's shared-memory -// flag mid-execution and breaks the eval. Two delivery paths are pinned: -import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; -import { Client, StreamableHTTPClientTransport } from "@modelcontextprotocol/client"; -import type { CallToolResult } from "@modelcontextprotocol/client"; - -// the shim fires the relay automatically when it forwards a repl -// interrupt (the stdio path), and the relay endpoint works standalone -// (the direct-HTTP path — a host that fires it itself). -// -// Requires `pnpm build` first (spawns dist/entry.js --daemon-run). -import assert from "node:assert/strict"; -import { spawn, spawnSync, type ChildProcess } from "node:child_process"; -import { mkdtempSync, mkdirSync, readFileSync, rmSync } from "node:fs"; -import { tmpdir } from "node:os"; -import { join, resolve } from "node:path"; -import { fileURLToPath } from "node:url"; -import test from "node:test"; - -import { envFingerprint } from "../../src/daemon/daemon-info.js"; -const distEntry = resolve(fileURLToPath(import.meta.url), "../../../dist/entry.js"); -const e2eHome = mkdtempSync(join(tmpdir(), "agentprism-repl-break-e2e-")); -const childEnv: Record = { - ...(process.env as Record), - HOME: e2eHome, - AGENTPRISM_DAEMON_PORT: "0", - // The deadline is the failure bound: a broken relay path must fail - // the fast asserts below, not hang the suite for the default 30 s. - AGENTPRISM_REPL_EVAL_TIMEOUT_MS: "20000", - AGENTPRISM_SESSION_TTL_MS: "60000", -}; - -interface E2eDaemonInfo { - pid: number; - port: number; - url: string; - replBreakUrl?: string; -} - -function readInfo(): E2eDaemonInfo | undefined { - try { - return JSON.parse(readFileSync(join(e2eHome, ".agentprism", "workflows", "daemons", `${envFingerprint(childEnv)}.json`), "utf-8")) as E2eDaemonInfo; - } catch { - return undefined; - } -} - -async function waitFor(predicate: () => boolean, what: string, timeoutMs = 15_000): Promise { - const start = Date.now(); - while (!predicate()) { - if (Date.now() - start > timeoutMs) throw new Error(`Timed out waiting for ${what}`); - await new Promise((resolvePromise) => setTimeout(resolvePromise, 100)); - } -} - -let daemon: ChildProcess | undefined; -let daemonInfo: E2eDaemonInfo | undefined; - -async function startDaemon(): Promise { - assert.equal(readInfo(), undefined, "expected a cold start"); - daemon = spawn(process.execPath, [distEntry, "--daemon-run"], { - env: childEnv, - stdio: ["ignore", "ignore", "pipe"], - detached: process.platform !== "win32", - }); - daemon.unref(); - daemon.stderr?.on("data", () => undefined); - await waitFor(() => readInfo() !== undefined && readInfo()!.replBreakUrl !== undefined, "daemon.json with replBreakUrl"); - daemonInfo = readInfo()!; - return daemonInfo; -} - -function replEvalCode(client: Client, projectDir: string, code: string): Promise { - return client.callTool( - { name: "repl", arguments: { action: "eval", projectDir, code } }, - { timeout: 60_000 }, - ); -} - -function replInterrupt(client: Client, projectDir: string): Promise { - return client.callTool( - { name: "repl", arguments: { action: "interrupt", projectDir } }, - { timeout: 60_000 }, - ); -} - -function textOf(result: CallToolResult): string { - return (result.content ?? []) - .filter((block) => block.type === "text") - .map((block) => (block as { text?: string }).text ?? "") - .join("\n"); -} - -async function connectDirect(): Promise { - const client = new Client({ name: "repl-break-e2e", version: "0.0.0" }, { capabilities: {} }); - const transport = new StreamableHTTPClientTransport(new URL(daemonInfo!.url)); - await client.connect(transport); - return client; -} - -async function connectShim(): Promise { - const client = new Client({ name: "repl-break-e2e", version: "0.0.0" }, { capabilities: {} }); - const transport = new StdioClientTransport({ - command: process.execPath, - args: [distEntry], - env: childEnv, - stderr: "ignore", - }); - await client.connect(transport); - return client; -} - -test("the daemon advertises the eval-break relay and the direct relay path breaks a synchronous while(true) eval out of band", async () => { - const info = await startDaemon(); - const projectDir = join(e2eHome, "direct-project"); - mkdirSync(projectDir, { recursive: true }); - const client = await connectDirect(); - try { - // Warm the workspace (the relay's slot registers at first touch). - const warm = await replEvalCode(client, projectDir, "6 * 7"); - assert.ok(textOf(warm).includes("result: 42"), textOf(warm)); - // The synchronous runaway: the eval request cannot be processed by - // the daemon (its main thread is wedged in the VM). - const startedAt = Date.now(); - const runaway = replEvalCode(client, projectDir, "while (true) {}"); - // Give the daemon a moment to enter the eval, then fire the relay — - // the out-of-band delivery: the worker thread arms the shared flag, - // and the running eval's quickjs interrupt handler consumes it - // mid-execution (the arm-after-start rule: the eval was already - // running when the break arrived). - await new Promise((resolvePromise) => setTimeout(resolvePromise, 400)); - const breakResponse = await fetch(`${info.replBreakUrl}`, { - method: "POST", - headers: { "content-type": "application/json" }, - body: JSON.stringify({ key: projectDir }), - }); - assert.equal(breakResponse.status, 204, "the relay armed the break"); - const result = await runaway; - const elapsed = Date.now() - startedAt; - assert.ok(elapsed < 8000, `the eval broke out of band, not at the 20 s deadline: ${elapsed} ms`); - const text = textOf(result); - assert.ok(text.includes("interrupted") || text.includes("error"), `the eval reports the break: ${text}`); - // The workspace stays usable. - const after = await replEvalCode(client, projectDir, "40 + 2"); - assert.ok(textOf(after).includes("result: 42"), textOf(after)); - await client.close().catch(() => undefined); - } finally { - await client.close().catch(() => undefined); - } -}); - -test("the shim fires the relay automatically: a repl interrupt through stdio breaks the synchronous eval and reports the out-of-band outcome", async () => { - const projectDir = join(e2eHome, "shim-project"); - mkdirSync(projectDir, { recursive: true }); - const client = await connectShim(); - try { - const warm = await replEvalCode(client, projectDir, "6 * 7"); - assert.ok(textOf(warm).includes("result: 42"), textOf(warm)); - const startedAt = Date.now(); - const runaway = replEvalCode(client, projectDir, "while (true) {}"); - await new Promise((resolvePromise) => setTimeout(resolvePromise, 400)); - // The interrupt tool call: the shim fires the relay BEFORE - // forwarding — while the daemon is blocked — so the eval breaks - // mid-run, and the daemon's own processing (once unblocked) reports - // the honest out-of-band outcome. - const interrupt = await replInterrupt(client, projectDir); - const text = textOf(interrupt); - assert.ok( - text.includes("out of band") || text.includes("out-of-band") || text.includes("broken OUT OF BAND"), - `the interrupt reports the out-of-band break: ${text}`, - ); - const structured = (interrupt.structuredContent ?? {}) as { interrupt?: { outcome?: string } }; - assert.equal(structured.interrupt?.outcome, "targeted", JSON.stringify(structured)); - const elapsed = Date.now() - startedAt; - const result = await runaway; - const evalText = textOf(result); - assert.ok(evalText.includes("interrupted") || evalText.includes("error"), `the eval reports the break: ${evalText}`); - // The workspace stays usable, and a later no-id interrupt with - // nothing running REFUSES (no stale break ever reaches a later - // eval). - const after = await replEvalCode(client, projectDir, "40 + 2"); - assert.ok(textOf(after).includes("result: 42"), textOf(after)); - const idle = await replInterrupt(client, projectDir); - const idleStructured = (idle.structuredContent ?? {}) as { interrupt?: { outcome?: string } }; - assert.equal(idleStructured.interrupt?.outcome, "refused-idle", JSON.stringify(idleStructured)); - await client.close().catch(() => undefined); - } finally { - await client.close().catch(() => undefined); - } -}); - -test("a stale relay break never breaks a later eval (the arm-after-start rule end to end)", async () => { - const projectDir = join(e2eHome, "stale-project"); - mkdirSync(projectDir, { recursive: true }); - const client = await connectShim(); - try { - const warm = await replEvalCode(client, projectDir, "6 * 7"); - assert.ok(textOf(warm).includes("result: 42"), textOf(warm)); - // Arm the relay while the workspace is IDLE (nothing running), then - // run a fresh eval: the stale flag must be consumed-and-dropped by - // the first execution — the fresh eval runs to completion. - const breakResponse = await fetch(`${daemonInfo!.replBreakUrl}`, { - method: "POST", - headers: { "content-type": "application/json" }, - body: JSON.stringify({ key: projectDir }), - }); - assert.equal(breakResponse.status, 204); - const result = await replEvalCode(client, projectDir, "41 + 1"); - assert.ok(textOf(result).includes("result: 42"), `the stale break did not touch the fresh eval: ${textOf(result)}`); - await client.close().catch(() => undefined); - } finally { - await client.close().catch(() => undefined); - } -}); - -process.on("exit", () => { - if (daemon !== undefined && daemon.pid !== undefined) { - try { - process.kill(daemon.pid, "SIGKILL"); - } catch { - /* best-effort */ - } - } - spawnSync(process.execPath, [distEntry, "daemon", "stop"], { env: childEnv }); - rmSync(e2eHome, { recursive: true, force: true }); -}); - -test.after(() => { - // The test runner exits only when the loop drains: tear down the - // daemon's stdio handles (an unref'd child still holds its pipes) and - // any shim-spawned daemon before the runner finishes. - daemon?.stderr?.destroy(); - daemon?.stdout?.destroy(); - if (daemon !== undefined && daemon.pid !== undefined) { - try { - process.kill(daemon.pid, "SIGKILL"); - } catch { - /* best-effort */ - } - } - spawnSync(process.execPath, [distEntry, "daemon", "stop"], { env: childEnv }); -}); diff --git a/packages/mcp-server/test/daemon/session-registry.test.ts b/packages/mcp-server/test/daemon/session-registry.test.ts index 5f38e8eb..43a11b18 100644 --- a/packages/mcp-server/test/daemon/session-registry.test.ts +++ b/packages/mcp-server/test/daemon/session-registry.test.ts @@ -1,6 +1,5 @@ // SessionRegistry: connections vs requests are tracked separately, and the lame-duck -// migration (`evictDrainable`) never cuts a session with a request in flight or one the -// daemon vetoes (a REPL workspace mid-turn). +// migration (`evictDrainable`) never cuts a session with a request in flight. import assert from "node:assert/strict"; import { test } from "node:test"; @@ -18,29 +17,27 @@ function fakeRecord(registry: SessionRegistry, sessionId: string): { closed: boo return state; } -test("requests and connections are counted independently; evictDrainable skips in-flight and vetoed sessions", () => { +test("requests and connections are counted independently; evictDrainable skips sessions with a request in flight", () => { const registry = new SessionRegistry(); const idle = fakeRecord(registry, "idle"); // GET stream only const busy = fakeRecord(registry, "busy"); // a POST being processed - const vetoed = fakeRecord(registry, "vetoed"); // REPL workspace mid-turn registry.connectionOpened("idle"); registry.connectionOpened("busy"); registry.requestStarted("busy"); assert.equal(registry.inflightCount(), 1); assert.equal(registry.get("busy")?.openConnections, 1); - const migrated = registry.evictDrainable((sessionId) => sessionId === "vetoed"); + const migrated = registry.evictDrainable(); assert.deepEqual(migrated, ["idle"]); assert.equal(idle.closed, true); assert.equal(busy.closed, false, "a session with a request in flight is never cut"); - assert.equal(vetoed.closed, false, "a vetoed session is kept"); - assert.equal(registry.size, 2); + assert.equal(registry.size, 1); // The request finishes → the session becomes drainable. registry.requestFinished("busy"); registry.connectionClosed("busy"); assert.equal(registry.inflightCount(), 0); - assert.deepEqual(registry.evictDrainable(), ["busy", "vetoed"]); + assert.deepEqual(registry.evictDrainable(), ["busy"]); assert.equal(registry.size, 0); }); diff --git a/packages/mcp-server/test/daemon/shim.e2e.test.ts b/packages/mcp-server/test/daemon/shim.e2e.test.ts index a954ead8..d61f440e 100644 --- a/packages/mcp-server/test/daemon/shim.e2e.test.ts +++ b/packages/mcp-server/test/daemon/shim.e2e.test.ts @@ -44,6 +44,28 @@ const NO_AGENT_SCRIPT = [ 'export const meta = { name: "no-agent", description: "no subagents" };', "return 42;", ].join("\n"); +/** + * A request the daemon ACCEPTS and cannot finish before it dies: a live config probe spawns real + * backend adapters (seconds of work, none of it blocking the daemon's event loop, so the response + * stream is open), and the daemon is frozen mid-probe so the answer can never be written. A + * request that never reached the daemon is a different case — the legacy path replays it. + */ +function startUnanswerableRequest(client: Client, daemonPid: number): ReturnType { + const inflight = client.callTool( + { name: "workflow", arguments: { action: "config", projectDir: e2eHome, harnesses: ["claude", "codex"] } }, + { timeout: 50_000 }, + ); + // Observe the failure from the start: the rejection may land before the caller awaits it. + inflight.catch(() => undefined); + setTimeout(() => { + try { + process.kill(daemonPid, "SIGSTOP"); + } catch { + /* already gone */ + } + }, 300); + return inflight; +} interface E2eDaemonInfo { pid: number; @@ -401,13 +423,8 @@ test("a request in flight when the daemon dies is answered with an error (never const before = readInfo(); assert.ok(before); - // A request that will never complete on this daemon: a synchronous never-yielding eval - // blocks the daemon's main thread (the repl-break e2e's fixture). It is in flight when the - // daemon is killed outright. - const inflight = session.client.callTool( - { name: "repl", arguments: { action: "eval", projectDir: e2eHome, code: "while (true) {}" } }, - { timeout: 50_000 }, - ); + // It is in flight when the daemon is killed outright. + const inflight = startUnanswerableRequest(session.client, before.pid); await new Promise((resolvePromise) => setTimeout(resolvePromise, 500)); process.kill(before.pid, "SIGKILL"); await waitFor(() => !pidAlive(before.pid), "old daemon to die"); @@ -433,10 +450,7 @@ test("a modern in-flight request is failed as ambiguous and never replayed after const before = readInfo(); assert.ok(before); - const inflight = session.client.callTool( - { name: "repl", arguments: { action: "eval", projectDir: e2eHome, code: "while (true) {}" } }, - { timeout: 50_000 }, - ); + const inflight = startUnanswerableRequest(session.client, before.pid); const inflightFailure = inflight.then( () => undefined, (error: unknown) => error, @@ -455,7 +469,7 @@ test("a modern in-flight request is failed as ambiguous and never replayed after assert.equal( (await second)?.status, "completed", - "the ambiguous eval was not replayed onto the successor and did not block it", + "the ambiguous request was failed, and the successor serves the same client", ); } finally { await session.close(); diff --git a/packages/mcp-server/test/dual-era.e2e.test.ts b/packages/mcp-server/test/dual-era.e2e.test.ts index 416a39c9..4d38e3af 100644 --- a/packages/mcp-server/test/dual-era.e2e.test.ts +++ b/packages/mcp-server/test/dual-era.e2e.test.ts @@ -148,7 +148,7 @@ async function exerciseEra( } const listed = await connected.client.listTools(); const tools = listed.tools.map((tool) => tool.name).sort(); - assert.deepEqual(tools, ["repl", "workflow", "workflow-events", "workflow-notifications", "workflow-runs", "workflow_monitor"]); + assert.deepEqual(tools, ["workflow", "workflow-events", "workflow-notifications", "workflow-runs", "workflow_monitor"]); const workflow = listed.tools.find((tool) => tool.name === "workflow"); assert.ok(workflow); const panel = await connected.client.readResource({ @@ -635,7 +635,7 @@ test("legacy and modern requests both keep the Apps surface capability-gated", a const connected = await connectHttp(daemon.url, { protocolMode, uiCapability }); try { const listed = await connected.client.listTools(); - assert.deepEqual(listed.tools.map((tool) => tool.name).sort(), ["repl", "workflow"]); + assert.deepEqual(listed.tools.map((tool) => tool.name).sort(), ["workflow"]); assert.equal(listed.tools.find((tool) => tool.name === "workflow")?._meta, undefined); const directAppCall = await connected.client.callTool({ name: "workflow-events", diff --git a/packages/mcp-server/test/in-process-dual-era.e2e.test.ts b/packages/mcp-server/test/in-process-dual-era.e2e.test.ts index dad08c75..fbb51c88 100644 --- a/packages/mcp-server/test/in-process-dual-era.e2e.test.ts +++ b/packages/mcp-server/test/in-process-dual-era.e2e.test.ts @@ -37,7 +37,7 @@ test("--in-process uses serveStdio to serve modern and capability-project the Ap assert.equal(capable.client.getProtocolEra(), "modern"); const tools = await capable.client.listTools(); assert.deepEqual(tools.tools.map((tool) => tool.name).sort(), [ - "repl", "workflow", "workflow-events", "workflow-notifications", "workflow-runs", "workflow_monitor", + "workflow", "workflow-events", "workflow-notifications", "workflow-runs", "workflow_monitor", ]); const status = structured(await runAndObserve(capable.client, { script: SCRIPT }))!; assert.equal(status.status, "completed"); @@ -55,7 +55,7 @@ test("--in-process uses serveStdio to serve modern and capability-project the Ap try { assert.equal(incapable.client.getProtocolEra(), "modern"); const tools = await incapable.client.listTools(); - assert.deepEqual(tools.tools.map((tool) => tool.name).sort(), ["repl", "workflow"]); + assert.deepEqual(tools.tools.map((tool) => tool.name).sort(), ["workflow"]); assert.equal(tools.tools.find((tool) => tool.name === "workflow")?._meta, undefined); } finally { await incapable.close(); diff --git a/packages/mcp-server/test/initialize-race.test.ts b/packages/mcp-server/test/initialize-race.test.ts index 84523af5..2438aec5 100644 --- a/packages/mcp-server/test/initialize-race.test.ts +++ b/packages/mcp-server/test/initialize-race.test.ts @@ -84,7 +84,7 @@ test("the negotiated MCP Apps surface still waits for client capabilities, then -1, // hold indefinitely until we release it ); const before = await capable.client.listTools(); - assert.deepEqual(before.tools.map((tool) => tool.name).sort(), ["repl", "workflow"]); + assert.deepEqual(before.tools.map((tool) => tool.name).sort(), ["workflow"]); assert.equal( before.tools.some((tool) => tool.name === WORKFLOW_EVENTS_TOOL_NAME), false, @@ -113,7 +113,7 @@ test("the negotiated MCP Apps surface still waits for client capabilities, then const plain = await connectWithDelayedInitialized({}, 0); await new Promise((resolve) => setTimeout(resolve, 150)); const plainTools = await plain.client.listTools(); - assert.deepEqual(plainTools.tools.map((tool) => tool.name).sort(), ["repl", "workflow"]); + assert.deepEqual(plainTools.tools.map((tool) => tool.name).sort(), ["workflow"]); assert.equal( plainTools.tools.some((tool) => tool.name === WORKFLOW_EVENTS_TOOL_NAME), false, diff --git a/packages/mcp-server/test/legacy-v1-client.e2e.test.ts b/packages/mcp-server/test/legacy-v1-client.e2e.test.ts index 5e63661c..5d6962ff 100644 --- a/packages/mcp-server/test/legacy-v1-client.e2e.test.ts +++ b/packages/mcp-server/test/legacy-v1-client.e2e.test.ts @@ -27,7 +27,7 @@ test("released SDK v1 client retains the sessionful legacy end-to-end path", asy assert.deepEqual(client.getServerCapabilities()?.extensions?.[SKILLS_EXTENSION_ID], { directoryRead: true }); const tools = await client.listTools(); assert.deepEqual(tools.tools.map((tool) => tool.name).sort(), [ - "repl", "workflow", "workflow-events", "workflow-notifications", "workflow-runs", "workflow_monitor", + "workflow", "workflow-events", "workflow-notifications", "workflow-runs", "workflow_monitor", ]); const accepted = await client.callTool({ name: "workflow", diff --git a/packages/mcp-server/test/live-backend.e2e.test.ts b/packages/mcp-server/test/live-backend.e2e.test.ts index f3188027..86c6e676 100644 --- a/packages/mcp-server/test/live-backend.e2e.test.ts +++ b/packages/mcp-server/test/live-backend.e2e.test.ts @@ -636,88 +636,25 @@ test("live workflow config discovery: no-prompt catalogs create no run and inval const thinking = (exactPiRow.options as Array>).find((option) => option.id === "thinkingLevel"); assert.equal(typeof thinking?.currentValue, "string", JSON.stringify(exactPiRow)); - const replFailure = await client.callTool({ - name: "repl", + // An unadvertised mode is refused at preparation — no run is created and the agent is never + // prompted — and the refusal names the mode, never the (valid) config option beside it. + const invalidMode = await client.callTool({ + name: "workflow", arguments: { - action: "eval", + action: "run", projectDir, - timeoutMs: 120_000, - code: `await agent(${JSON.stringify(`pi/${PI_E2E_MODEL}`)}, "must never prompt", { mode: "default", configOptions: { thinkingLevel: ${JSON.stringify(thinking?.currentValue)} } }).catch(e => e.name + ": " + e.message)`, + script: [ + "export const meta = { name: 'invalid-mode', description: 'an unadvertised session mode is refused before execution' }", + `return await agent("must never prompt", { model: ${JSON.stringify(`pi/${PI_E2E_MODEL}`)}, mode: "default", configOptions: { thinkingLevel: ${JSON.stringify(thinking?.currentValue)} } })`, + ].join("\n"), }, }, { timeout: 240_000, maxTotalTimeout: 240_000 }); - assert.notEqual(replFailure.isError, true, JSON.stringify(replFailure)); - const replResult = (replFailure.structuredContent as Record).result; - assert.equal(typeof replResult, "string", JSON.stringify(replFailure.structuredContent)); - assert.match(replResult as string, /cannot apply session mode "default" \(advertised modes: none\)/); - assert.doesNotMatch(replResult as string, /ConfigOptionsError|offending key|thinkingLevel/); - } finally { - await client.close().catch(() => undefined); - await transport.close().catch(() => undefined); - } -}); - -test("live REPL queue smoke: Claude, OpenCode, Pi, and (when opted in) Codex continue one session through broker-owned FIFO prompts", { - skip: SKIP, - timeout: 600_000, -}, async () => { - assert.ok(existsSync(SERVER_ENTRY), `built server entry missing — run \`pnpm build\` first: ${SERVER_ENTRY}`); - const projectDir = fileURLToPath(new URL("../../..", import.meta.url)); - const env: NodeJS.ProcessEnv = { ...process.env }; - const transport = new StdioClientTransport({ - command: process.execPath, - args: [SERVER_ENTRY], - env: env as Record, - stderr: "pipe", - cwd: projectDir, - }); - const client = new Client({ name: "live-repl-queue", version: "0.0.0" }, { capabilities: {} }); - const specs: Record = { - claude: CLAUDE_E2E_MODEL, - ...(CODEX_LIVE ? { codex: "codex" } : {}), - opencode: OPENCODE_E2E_MODEL, - pi: `pi/${PI_E2E_MODEL}`, - }; - const names = Object.keys(specs); - const suffix = `${Date.now().toString(36)}_${process.pid}`; - const handleName = (name: string): string => `live_${name}_${suffix}`; - const queueName = (name: string): string => `queued_${name}_${suffix}`; - const lastLine = (value: unknown): string => - typeof value === "string" ? (value.trim().split("\n").at(-1)?.trim() ?? "") : ""; - try { - await client.connect(transport); - const foundingSource = names.map((name) => - `const ${handleName(name)} = agent(${JSON.stringify(specs[name])}, ${JSON.stringify(`Reply with exactly FOUNDING_${name.toUpperCase()} and no other text. Do not call tools.`)});`, - ).join("\n") + - `\nJSON.stringify(await Promise.all([${names.map(handleName).join(", ")}]))`; - const founding = await client.callTool({ - name: "repl", - arguments: { action: "eval", projectDir, code: foundingSource, timeoutMs: 120_000 }, - }, { timeout: 240_000, maxTotalTimeout: 240_000 }); - assert.notEqual(founding.isError, true, JSON.stringify(founding)); - const foundingResult = (founding.structuredContent as Record | undefined)?.result; - assert.equal(typeof foundingResult, "string", JSON.stringify(founding.structuredContent)); - const foundingValues = JSON.parse(foundingResult as string) as unknown[]; - assert.deepEqual( - foundingValues.map(lastLine), - names.map((name) => `FOUNDING_${name.toUpperCase()}`), - ); - - const queueSource = names.map((name) => - `const ${queueName(name)} = ${handleName(name)}.queue(${JSON.stringify(`Reply with exactly QUEUE_${name.toUpperCase()} and no other text. Do not call tools.`)});`, - ).join("\n") + - `\nJSON.stringify(await Promise.all([${names.map(queueName).join(", ")}]))`; - const queued = await client.callTool({ - name: "repl", - arguments: { action: "eval", projectDir, code: queueSource, timeoutMs: 120_000 }, - }, { timeout: 240_000, maxTotalTimeout: 240_000 }); - assert.notEqual(queued.isError, true, JSON.stringify(queued)); - const queuedResult = (queued.structuredContent as Record | undefined)?.result; - assert.equal(typeof queuedResult, "string", JSON.stringify(queued.structuredContent)); - const queuedValues = JSON.parse(queuedResult as string) as unknown[]; - assert.deepEqual( - queuedValues.map(lastLine), - names.map((name) => `QUEUE_${name.toUpperCase()}`), - ); + const invalidModeText = JSON.stringify(invalidMode); + assert.equal(invalidMode.isError, true, invalidModeText); + assert.equal(asObject(invalidMode.structuredContent)?.runId, undefined, "a refused preparation creates no workflow run"); + assert.match(invalidModeText, /Workflow run was not started/); + assert.match(invalidModeText, /mode authored value \\"default\\" is not advertised by pi model/); + assert.doesNotMatch(invalidModeText, /ConfigOptionsError|offending key/); } finally { await client.close().catch(() => undefined); await transport.close().catch(() => undefined); diff --git a/packages/mcp-server/test/repl-daemon.test.ts b/packages/mcp-server/test/repl-daemon.test.ts deleted file mode 100644 index f2bcbfd3..00000000 --- a/packages/mcp-server/test/repl-daemon.test.ts +++ /dev/null @@ -1,1112 +0,0 @@ -/** - * Phase E of the REPL-orchestrator roadmap, adapted to the eval-plane - * redesign surface: the `repl` tool's DAEMON-BOUNDARY suite. The - * phase-D suites (repl-tool.test.ts, repl-review2.test.ts) drive - * `createWorkflowServer` over in-memory transports; this suite pins the - * phase-E deliverables against a REAL daemon instance — `createDaemon` - * on an ephemeral loopback port, driven by real SDK Clients over - * StreamableHTTPClientTransport (the `_http-harness` pattern): - * - * - the tool schema: `repl` registers alongside `workflow` with exactly - * the redesign's two-action enum (`eval` / `interrupt`) and field set - * — snapshotting is implicit, there is no user-facing snapshot action, - * - action behaviors on the daemon: projectDir is required in daemon - * mode, and eval / interrupt round-trip over HTTP (eval: the - * soft-bound fused pump — the finished shape when the awaited call - * settles within the bound, the honest still-running shape with the - * running ids when it does not), - * - project keying: two projectDirs are two ISOLATED workspaces on one - * daemon (separate VMs, separate per-project repl stores, a reset of - * one never touches the other), - * - MCP-session churn never touches the workspace: bindings survive a - * client disconnect and a fresh client's reconnect, - * - a TRANSIENT connection drop of the same live session restores its - * project presence on reconnect (the registry's connection-open - * signal re-adds it from the ledger's retained affinity), so the - * scheduled drain aborts and children stay warm, - * - reset() (the §4.5 guest function) does NOT clear client presence - * (connection liveness, not workspace state): with a second client - * connected, the resetting client's disconnect never drains the - * post-reset workspace, - * - the lifecycle drain driven by the daemon's session registry: the - * last-client disconnect drains the in-flight subagent turn to - * completion (mock runner), closes the idle child, and the next - * explicit queued turn lazily re-attaches the recorded backend session, - * - interrupt without an id breaks a RUNNING eval: an eval held open by - * the fused pump is in flight while the interrupt lands (the pump - * releases the broker chain between iterations) and the armed signal - * breaks the resumed continuation MID-RUN via the quickjs interrupt - * handler, - * - the machine-readable output: the tool publishes an outputSchema and - * every result carries the redesign's shapes as structuredContent — - * eval `{ output, result?, running? }` (ONE newline-joined string, - * nothing else), the interrupt outcome, and the error variant. - */ - -import assert from "node:assert/strict"; -import { existsSync } from "node:fs"; -import { join } from "node:path"; -import { test } from "node:test"; -import type { Client } from "@modelcontextprotocol/client"; -import type { - BrokerLoadSessionOptions, - BrokerOpenSessionOptions, - BrokerPromptOptions, - BrokerRunner, - BrokerSession, - BrokerTurn, -} from "@automatalabs/repl-engine"; -import { workflowProjectPaths } from "@automatalabs/workflows"; -import { z } from "zod"; - -import { replToolInputShape, replToolOutputShape } from "../src/index.js"; -import { createDaemon, type DaemonHandle } from "../src/daemon/http-daemon.js"; -import { connectHttp, makeProjectDir } from "./_http-harness.js"; -import { okRunner, textOf, waitForRun } from "./_harness.js"; - -/** The fake held-open ACP session (the broker's structural seam; the - * same shape as repl-tool.test.ts's fake, kept local so this suite - * runs standalone). */ -class FakeSession implements BrokerSession { - readonly sessionId: string; - initializeMeta: Readonly> | undefined; - readonly prompts: Array<{ content: string; resolve: (turn: BrokerTurn) => void; reject: (error: unknown) => void }> = []; - readonly steers: Array<{ content: string; resolve: (outcome: unknown) => void; reject: (error: unknown) => void }> = []; - releases = 0; - cancelCalls = 0; - stopReason = "end_turn"; - readonly completedTexts: string[] = []; - /** The re-attach seam's scripted loaded-turn outcome (null parks it). */ - loadedTurnTextValue: string | null = null; - - constructor(readonly openedWith: BrokerOpenSessionOptions | BrokerLoadSessionOptions) { - this.sessionId = `fake-session-${FakeSession.nextId++}`; - this.initializeMeta = { steering: { supported: true } }; - } - - static nextId = 0; - - prompt(content: string, opts: BrokerPromptOptions = {}): Promise { - return new Promise((resolve, reject) => { - this.prompts.push({ content, resolve, reject }); - opts.onHandoff?.(); - }); - } - - steer(content: string): Promise { - return new Promise((resolve, reject) => { - this.steers.push({ content, resolve, reject }); - }); - } - - awaitCurrentTurn(): Promise { - if (this.loadedTurnTextValue !== null) { - return Promise.resolve({ stopReason: this.stopReason, text: this.loadedTurnTextValue }); - } - return new Promise(() => {}); - } - - cancel(): Promise { - this.cancelCalls++; - for (const pending of this.prompts.splice(0)) { - pending.resolve({ stopReason: "cancelled", text: "" }); - } - return Promise.resolve(); - } - - release(): Promise { - this.releases++; - return Promise.resolve(); - } - - currentTurnText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ""; - } - - finalMessageText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ""; - } - - rawStructuredOutput(): unknown { - return undefined; - } - - completeTurn(text: string): void { - const pending = this.prompts.shift(); - assert.ok(pending, "a prompt turn must be in flight"); - this.completedTexts.push(text); - pending.resolve({ stopReason: this.stopReason, text }); - } -} - -/** The fake runner with the loadSession seam (see repl-tool.test.ts). */ -class FakeRunner implements BrokerRunner { - readonly sessions: FakeSession[] = []; - readonly openedWith: BrokerOpenSessionOptions[] = []; - readonly loadedWith: BrokerLoadSessionOptions[] = []; - - listBackends(): string[] { - return ["pi"]; - } - - defaultBackendId(): string { - return "pi"; - } - - async openSession(opts: BrokerOpenSessionOptions): Promise { - const session = new FakeSession(opts); - this.sessions.push(session); - this.openedWith.push(opts); - return session; - } - - async loadSession(opts: BrokerLoadSessionOptions): Promise { - const session = new FakeSession(opts); - session.loadedTurnTextValue = null; - this.sessions.push(session); - this.loadedWith.push(opts); - return session; - } - - async dispose(): Promise {} - - last(): FakeSession { - assert.ok(this.sessions.length > 0, "a session must exist"); - return this.sessions[this.sessions.length - 1]; - } -} - -/** A real daemon on an ephemeral loopback port with an injected repl - * runner (the suite's mock seam; the drain bound reuses the daemon's - * session-eviction TTL knob — here a short test value). */ -async function startReplDaemon(replRunner: BrokerRunner): Promise { - return createDaemon({ - runner: okRunner(), - port: 0, - env: {}, - log: () => undefined, - replRunner, - sessionTtlMs: 60_000, - }); -} - -/** The runner whose openSession is PARKED until released manually — the - * delayed-open regression seam (phase-E review rejection round 7: - * `interrupt { id }` must cancel a call whose `openSession()` is still - * pending, and the eventual late child must be closed without ever - * prompting). */ -class DelayedOpenRunner extends FakeRunner { - private gate: Promise = Promise.resolve(); - private releaseGate: () => void = () => {}; - - /** Park every openSession until `releaseOpens()`. */ - parkOpens(): void { - this.gate = new Promise((resolve) => { - this.releaseGate = resolve; - }); - } - - releaseOpens(): void { - this.releaseGate(); - } - - async openSession(opts: BrokerOpenSessionOptions): Promise { - await this.gate; - return super.openSession(opts); - } -} - -/** Call the repl tool over HTTP (typed over the raw input). */ -function repl( - session: { client: Client }, - input: { action: string; projectDir?: string; code?: string; timeoutMs?: number; id?: string }, -): ReturnType { - return session.client.callTool({ name: "repl", arguments: input as Record }); -} - -function structuredOf(res: Awaited>): Record { - return (res as { structuredContent?: Record }).structuredContent ?? {}; -} - -/** Evaluate an expression that returns JSON (the §4.5 sliceable- - * introspection idiom). */ -async function evalJson(session: { client: Client }, projectDir: string, expression: string): Promise { - const r = await repl(session, { action: "eval", projectDir, code: `JSON.stringify(${expression})` }); - assert.ok(!isErrorResult(r), textOf(r)); - const sc = structuredOf(r); - assert.ok(typeof sc.result === "string", `the eval resolved with a value: ${JSON.stringify(sc)}`); - return JSON.parse(sc.result as string); -} - -function isErrorResult(res: Awaited>): boolean { - return (res as { isError?: boolean }).isError === true; -} - -async function tick(): Promise { - await new Promise((resolve) => setTimeout(resolve, 0)); -} - -test("the repl tool registers alongside workflow with the redesign's two-action schema; snapshotting is implicit (no snapshot action)", async () => { - const Schema = z.object(replToolInputShape); - // The field set is exactly the redesign's surface. - assert.deepEqual(Object.keys(replToolInputShape).sort(), ["action", "code", "id", "projectDir", "timeoutMs"]); - // No user-facing snapshot action: snapshotting is implicit. - assert.ok(!("snapshot" in replToolInputShape), "snapshot must not be a tool action"); - // The action enum is exactly eval / interrupt. - assert.doesNotThrow(() => Schema.parse({ action: "eval" })); - assert.doesNotThrow(() => Schema.parse({ action: "interrupt" })); - assert.throws(() => Schema.parse({ action: "snapshot" }), /Invalid option/); - assert.throws(() => Schema.parse({ action: "wait" }), /Invalid option/); - assert.throws(() => Schema.parse({ action: "status" }), /Invalid option/); - assert.throws(() => Schema.parse({ action: "reset" }), /Invalid option/); - assert.throws(() => Schema.parse({ action: "nope" }), /Invalid option/); - // projectDir must be an absolute path. - assert.throws(() => Schema.parse({ action: "eval", projectDir: "relative/path", code: "1" }), /absolute path/); - assert.doesNotThrow(() => Schema.parse({ action: "eval", projectDir: "/abs/path", code: "1" })); - // timeoutMs: an integer in [0, 120_000] (the soft-bound eval's cap). - assert.throws(() => Schema.parse({ action: "eval", timeoutMs: -1 })); - assert.throws(() => Schema.parse({ action: "eval", timeoutMs: 120_001 })); - assert.throws(() => Schema.parse({ action: "eval", timeoutMs: 1.5 })); - assert.doesNotThrow(() => Schema.parse({ action: "eval", timeoutMs: 0 })); - assert.doesNotThrow(() => Schema.parse({ action: "eval", timeoutMs: 120_000 })); - - // The OUTPUT schema: the published machine-readable shape parses the - // redesign's eval variant and refuses a malformed one. - const OutputSchema = replToolOutputShape; - assert.doesNotThrow(() => OutputSchema.parse({ output: "", result: "42" })); - assert.doesNotThrow(() => OutputSchema.parse({ output: "", running: ["c1"] })); - assert.doesNotThrow(() => OutputSchema.parse({ interrupt: { outcome: "refused-idle" } })); - assert.doesNotThrow(() => OutputSchema.parse({ error: "nope" })); - assert.throws( - () => OutputSchema.parse({ output: "", interrupt: { outcome: "refused-idle" } }), - /output does not match a repl result variant/, - ); - assert.throws( - () => OutputSchema.parse({ error: "x", interrupt: { outcome: "refused-idle" } }), - /output does not match a repl result variant/, - ); - - // The WIRE schema of the real daemon advertises exactly that shape, - // alongside the workflow tool. - const daemon = await startReplDaemon(new FakeRunner()); - try { - const session = await connectHttp(daemon.url, { listTools: true }); - try { - const tools = await session.client.listTools(); - assert.deepEqual( - tools.tools.map((t) => t.name).sort(), - ["repl", "workflow"], - "plain clients receive the repl and workflow tools", - ); - const wire = tools.tools.find((t) => t.name === "repl")!; - const schema = wire.inputSchema as { properties: Record; required?: string[] }; - assert.deepEqual( - Object.keys(schema.properties).sort(), - ["action", "code", "id", "projectDir", "timeoutMs"], - ); - const action = schema.properties.action as { enum?: string[] }; - assert.deepEqual(action.enum, ["eval", "interrupt"]); - assert.deepEqual(schema.required, ["action"], "action is the only required field"); - // The OUTPUT schema is advertised on the wire too: the redesign's - // eval shape plus the interrupt outcome and the error variant. - const wireOutput = wire.outputSchema as { properties?: Record; oneOf?: Array<{ title?: string; required?: string[] }> }; - assert.ok(wireOutput, "the output schema is published on the wire"); - for (const field of ["output", "result", "running", "interrupt", "error"]) { - assert.ok(field in (wireOutput.properties ?? {}), `output schema field ${field}`); - } - for (const dead of ["pending", "completed", "checkpoints", "outputTruncated", "truncated", "referenced", "drained", "timedOut", "workspaces", "dropped", "action"]) { - assert.ok(!(dead in (wireOutput.properties ?? {})), `deleted output field ${dead}`); - } - assert.equal(wireOutput.oneOf?.length, 5, "the five output variants are published (finished / still-running / thrown eval, interrupt, error)"); - const evalFinished = wireOutput.oneOf?.find((b) => b.title === "eval"); - assert.deepEqual( - evalFinished?.required?.sort(), - ["output", "result"], - "the finished eval branch requires exactly output + result", - ); - const evalRunning = wireOutput.oneOf?.find((b) => b.title === "eval-still-running"); - assert.deepEqual( - evalRunning?.required?.sort(), - ["output", "running"], - "the still-running eval branch requires exactly output + running", - ); - const evalError = wireOutput.oneOf?.find((b) => b.title === "eval-error"); - assert.deepEqual(evalError?.required, ["output"], "the thrown-eval branch requires the output string alone"); - } finally { - await session.dispose(); - } - } finally { - await daemon.close(); - } -}); - -test("daemon mode: projectDir is required; eval/interrupt round-trip over the real HTTP daemon (soft-bound shapes included); reset() is the guest function", async () => { - const runner = new FakeRunner(); - const daemon = await startReplDaemon(runner); - try { - const session = await connectHttp(daemon.url); - try { - const PROJECT = makeProjectDir("repl-actions"); - // Daemon mode REQUIRES projectDir for both actions. - const noDir = await repl(session, { action: "eval", code: "1 + 1" }); - assert.ok(isErrorResult(noDir), textOf(noDir)); - assert.ok(textOf(noDir).includes("projectDir is required on the shared workflow daemon"), textOf(noDir)); - // An unknown action is refused by the schema at the wire boundary. - const badAction = await repl(session, { action: "snapshot", projectDir: PROJECT, code: "1" }); - assert.ok(isErrorResult(badAction), textOf(badAction)); - assert.ok(textOf(badAction).includes("Input validation error"), textOf(badAction)); - // An empty script is VALID JavaScript and the documented poll idiom. - const emptyCode = await repl(session, { action: "eval", projectDir: PROJECT, code: "" }); - assert.ok(!isErrorResult(emptyCode), textOf(emptyCode)); - assert.ok(textOf(emptyCode).includes("result: undefined"), textOf(emptyCode)); - const structuredEmpty = structuredOf(emptyCode); - assert.equal(structuredEmpty.output, ""); - assert.equal(structuredEmpty.result, "undefined"); - // eval round trip. - const evaled = await repl(session, { action: "eval", projectDir: PROJECT, code: "var answer = 40 + 2; answer" }); - assert.ok(!isErrorResult(evaled), textOf(evaled)); - assert.ok(textOf(evaled).includes("result: 42"), textOf(evaled)); - // The workspace manifest through workspace(): metadata (name/type - // token), never content. EVERY binding carries its byte size. - const ws = (await evalJson(session, PROJECT, "workspace()")) as { - bindings: Array<{ name: string; type: string; sizeBytes: number; token?: string }>; - }; - assert.ok(ws.bindings.some((b) => b.name === "answer" && b.type === "number"), JSON.stringify(ws.bindings)); - assert.ok(!JSON.stringify(ws).includes("40 + 2"), `content leaked: ${JSON.stringify(ws)}`); - // A pending subagent call; the soft-bound eval reports the honest - // still-running shape when the bound elapses (the call continues - // server-side). - const started = await repl(session, { action: "eval", projectDir: PROJECT, code: 'const p = agent("pi/x", "task"); "started"' }); - assert.ok(!isErrorResult(started), textOf(started)); - assert.equal(structuredOf(started).result, "started", "start-and-don't-await finishes immediately"); - await tick(); - const timedOut = await repl(session, { - action: "eval", - projectDir: PROJECT, - code: "await p", - timeoutMs: 100, - }); - assert.ok(!isErrorResult(timedOut), textOf(timedOut)); - const scTimedOut = structuredOf(timedOut); - assert.deepEqual(Object.keys(scTimedOut).sort(), ["output", "running"], "the still-running shape"); - assert.deepEqual(scTimedOut.running, ["c1"], "the in-flight call ids"); - assert.ok(textOf(timedOut).includes("running: c1"), textOf(timedOut)); - // The fused pump absorbs a MID-HOLD settlement: the eval call is - // held open while the HTTP request is open, and the backend's - // settlement resolves it in the SAME call (the finished shape). - await repl(session, { action: "eval", projectDir: PROJECT, code: 'const q = agent("pi/x", "task2"); "started"' }); - await tick(); - const waiting = repl(session, { action: "eval", projectDir: PROJECT, code: "await q", timeoutMs: 5000 }); - await tick(); - runner.last().completeTurn("waited result"); - const waited = await waiting; - assert.ok(!isErrorResult(waited), textOf(waited)); - const scWaited = structuredOf(waited); - assert.deepEqual(Object.keys(scWaited).sort(), ["output", "result"], "the finished shape"); - assert.equal(scWaited.result, "waited result"); - // The earlier still-running eval's promise is still live: `await p` - // resolves once ITS turn completes. - runner.sessions[0].completeTurn("p result"); - const picked = await repl(session, { action: "eval", projectDir: PROJECT, code: "await p" }); - assert.equal(structuredOf(picked).result, "p result"); - // interrupt with an id cancels the subagent call (ACP - // session/cancel downward). - await repl(session, { action: "eval", projectDir: PROJECT, code: 'const r = agent("pi/x", "task3"); "started"' }); - await tick(); - const interrupted = await repl(session, { action: "interrupt", projectDir: PROJECT, id: "c3" }); - assert.ok(!isErrorResult(interrupted), textOf(interrupted)); - assert.ok(textOf(interrupted).includes("session/cancel sent"), textOf(interrupted)); - assert.deepEqual(structuredOf(interrupted), { interrupt: { outcome: "cancelled", callId: "c3" } }); - // interrupt without an id BREAKS THE RUNNING EVAL: a runaway loop - // that keeps EXECUTING across drains (each iteration does real - // work, fires the next subagent call, and suspends) is held open - // by the fused pump; the interrupt lands while the eval call is IN - // FLIGHT; the pump's next iteration breaks the loop MID-RUN via - // the quickjs interrupt handler. - const inFlight = repl(session, { - action: "eval", - projectDir: PROJECT, - code: 'const s = agent("pi/x", "task4"); await s; for (;;) { let x = 0; for (let i = 0; i < 200000; i++) x += i; await agent("pi/x", "again"); }', - timeoutMs: 30_000, - }); - await tick(); - await tick(); - const armed = await repl(session, { action: "interrupt", projectDir: PROJECT }); - assert.ok(!isErrorResult(armed), textOf(armed)); - assert.ok(textOf(armed).includes("interrupting the running eval"), textOf(armed)); - assert.deepEqual(structuredOf(armed), { interrupt: { outcome: "targeted" } }); - // The first settlement: the pump resumes the loop's next iteration - // — and the armed signal breaks it MID-RUN. The broken eval can - // never settle: the held eval returns the finished-with-error - // shape promptly, and the interrupted drain is retained in - // workspace().diagnostics (§6.2). - runner.last().completeTurn("resumed"); - const broken = await inFlight; - assert.ok(!isErrorResult(broken), textOf(broken)); - const scBroken = structuredOf(broken); - assert.ok(!("running" in scBroken) && !("result" in scBroken), `the broken eval returned promptly: ${JSON.stringify(scBroken)}`); - const diag = (await evalJson(session, PROJECT, "workspace().diagnostics")) as { drainError: { message: string } | null }; - assert.ok( - diag.drainError !== null && (diag.drainError.message.includes("interrupted") || diag.drainError.message.includes("Job execution error")), - `the interrupted drain is retained in diagnostics: ${JSON.stringify(diag.drainError)}`, - ); - // The signal was consumed by the running eval's execution: the - // next eval is NOT broken, and the VM stays usable. - const afterInterrupt = await repl(session, { action: "eval", projectDir: PROJECT, code: "6 * 7" }); - assert.ok(textOf(afterInterrupt).includes("result: 42"), textOf(afterInterrupt)); - // reset() — the §4.5 guest function: the teardown runs after the - // eval completes; the next eval starts a fresh workspace. - const reset = await repl(session, { action: "eval", projectDir: PROJECT, code: "reset()" }); - assert.ok(!isErrorResult(reset), textOf(reset)); - const gone = await repl(session, { action: "eval", projectDir: PROJECT, code: "typeof answer" }); - assert.ok(!isErrorResult(gone), textOf(gone)); - assert.ok(textOf(gone).includes("undefined"), textOf(gone)); - // A GLOBAL LEXICAL binding (top-level let/const/class — the - // canonical `const research = agent(...)` state) is listed by - // workspace() with its full provenance surface. - const lexed = await repl(session, { action: "eval", projectDir: PROJECT, code: 'const research = agent("pi/x", "task"); "started"' }); - assert.ok(!isErrorResult(lexed), textOf(lexed)); - const wsLex = (await evalJson(session, PROJECT, "workspace()")) as { - bindings: Array<{ name: string; type: string; status?: string; callId?: string; provenance: string | null; task: string | null }>; - }; - const binding = wsLex.bindings.find((b) => b.name === "research"); - assert.ok(binding, JSON.stringify(wsLex.bindings)); - assert.equal(binding.type, "agent handle"); - assert.equal(binding.callId, "c1"); - assert.equal(binding.status, "pending"); - assert.equal(binding.provenance, "eval 2", "the provenance pass counts the fresh workspace's evals"); - assert.equal(binding.task, "task"); - } finally { - await session.dispose(); - } - } finally { - await daemon.close(); - } -}); - -test("workspaces are keyed by projectDir: two projects on one daemon are fully isolated", async () => { - const daemon = await startReplDaemon(new FakeRunner()); - try { - const session = await connectHttp(daemon.url); - try { - const projectA = makeProjectDir("repl-keying-a"); - const projectB = makeProjectDir("repl-keying-b"); - const a = await repl(session, { action: "eval", projectDir: projectA, code: 'const secretA = "alpha"; "A"' }); - assert.ok(!isErrorResult(a), textOf(a)); - // B does not see A's bindings... - const probeB = await repl(session, { action: "eval", projectDir: projectB, code: "typeof secretA" }); - assert.ok(!isErrorResult(probeB), textOf(probeB)); - assert.ok(textOf(probeB).includes("undefined"), textOf(probeB)); - const b = await repl(session, { action: "eval", projectDir: projectB, code: 'const secretB = "beta"; "B"' }); - assert.ok(!isErrorResult(b), textOf(b)); - // ...and A does not see B's. - const probeA = await repl(session, { action: "eval", projectDir: projectA, code: "typeof secretB" }); - assert.ok(!isErrorResult(probeA), textOf(probeA)); - assert.ok(textOf(probeA).includes("undefined"), textOf(probeA)); - // Each project persisted its OWN repl store (one enveloped snapshot - // per project, under the daemon's per-project layout). - const pathsA = workflowProjectPaths(projectA); - const pathsB = workflowProjectPaths(projectB); - const storeA = join(pathsA.rootDir, "repl"); - const storeB = join(pathsB.rootDir, "repl"); - assert.notEqual(storeA, storeB); - assert.ok(existsSync(join(storeA, "snapshot.bin")), "A's snapshot exists"); - assert.ok(existsSync(join(storeB, "snapshot.bin")), "B's snapshot exists"); - // resetting A never touches B: B's binding and store survive, and - // A's stored state is gone — the next touch starts a FRESH - // workspace, never a restore, and the old binding is gone. - await repl(session, { action: "eval", projectDir: projectA, code: "reset()" }); - const bAlive = await repl(session, { action: "eval", projectDir: projectB, code: "secretB" }); - assert.ok(!isErrorResult(bAlive), textOf(bAlive)); - assert.ok(textOf(bAlive).includes("beta"), textOf(bAlive)); - const aGone = await repl(session, { action: "eval", projectDir: projectA, code: "typeof secretA" }); - assert.ok(!isErrorResult(aGone), textOf(aGone)); - assert.ok(textOf(aGone).includes("undefined"), textOf(aGone)); - } finally { - await session.dispose(); - } - } finally { - await daemon.close(); - } -}); - -test("MCP-session churn never touches the workspace: bindings survive a client disconnect and a fresh client's reconnect", async () => { - const daemon = await startReplDaemon(new FakeRunner()); - try { - const PROJECT = makeProjectDir("repl-churn"); - const session1 = await connectHttp(daemon.url); - try { - const evaled = await repl(session1, { action: "eval", projectDir: PROJECT, code: "var x = 40 + 2; x" }); - assert.ok(!isErrorResult(evaled), textOf(evaled)); - assert.ok(textOf(evaled).includes("result: 42"), textOf(evaled)); - } finally { - // The client's last connection closes: the session registry signals - // the presence ledger, which drains the project (no children were - // ever opened — a quick no-op drain). The workspace itself is - // NEVER dropped by session churn. - await session1.dispose(); - } - const session2 = await connectHttp(daemon.url); - try { - // A brand-new MCP session (new session id, new transport) sees the - // same live VM: the binding survived the disconnect. - const continued = await repl(session2, { action: "eval", projectDir: PROJECT, code: "x * 2" }); - assert.ok(!isErrorResult(continued), textOf(continued)); - assert.ok(textOf(continued).includes("result: 84"), textOf(continued)); - const ws = (await evalJson(session2, PROJECT, "workspace()")) as { - bindings: Array<{ name: string; type: string }>; - }; - assert.ok(ws.bindings.some((b) => b.name === "x" && b.type === "number"), JSON.stringify(ws.bindings)); - } finally { - await session2.dispose(); - } - } finally { - await daemon.close(); - } -}); - -test("the session registry drives the client-presence drain on the real daemon: last-client disconnect drains the in-flight turn to completion, closes the idle child, and the next queued turn lazily re-attaches", async () => { - const runner = new FakeRunner(); - const daemon = await startReplDaemon(runner); - try { - const PROJECT = makeProjectDir("repl-drain"); - const session1 = await connectHttp(daemon.url); - try { - const started = await repl(session1, { action: "eval", projectDir: PROJECT, code: 'const p = agent("pi/x", "task"); "started"' }); - assert.ok(!isErrorResult(started), textOf(started)); - await tick(); - } finally { - // The client's last connection closes: the daemon's session - // registry fires onLastConnectionClosed, the presence ledger - // removes the session, and the project's workspace DRAINS — the - // in-flight subagent turn runs to completion (never a cancel). - await session1.dispose(); - } - const session = runner.last(); - // The drain waits for the in-flight turn; its completion settles - // into the VM (each settlement boundary snapshots), then the idle - // child closes. - session.completeTurn("drained result"); - for (let attempt = 0; attempt < 200 && session.releases === 0; attempt++) { - await new Promise((resolve) => setTimeout(resolve, 10)); - } - assert.equal(session.releases, 1, "the idle child closed after the drain"); - // The next client connect: the workspace is still live (the drain - // never drops it), children closed — and queue() on the settled - // handle lazily re-attaches the recorded backend session via the - // capability matrix (loadSession with the SAME session id). - const session2 = await connectHttp(daemon.url); - try { - const ws = (await evalJson(session2, PROJECT, "workspace()")) as { - diagnostics: { childrenClosed: boolean }; - }; - assert.equal(ws.diagnostics.childrenClosed, true, "children closed after the drain"); - const probe = await repl(session2, { action: "eval", projectDir: PROJECT, code: 'p.queue("more"); "fired"' }); - assert.ok(!isErrorResult(probe), textOf(probe)); - await tick(); - assert.equal(runner.loadedWith.length, 1, "the recorded session was loaded lazily on the next connect"); - assert.equal(runner.loadedWith[0].sessionId, session.sessionId, "the SAME backend session"); - // The queued turn starts on the re-attached - // session; completing it lets the next disconnect's drain (and the - // daemon's bounded teardown) finish instead of waiting out their - // bounds on a parked turn. - for (let attempt = 0; attempt < 100 && runner.last().prompts.length === 0; attempt++) { - await new Promise((resolve) => setTimeout(resolve, 10)); - } - runner.last().completeTurn("followed up"); - await tick(); - } finally { - await session2.dispose(); - } - } finally { - await daemon.close(); - } -}); - -test("a transient connection drop of the SAME live session restores its project presence on reconnect — the scheduled drain aborts and children stay warm", async () => { - const runner = new FakeRunner(); - const daemon = await startReplDaemon(runner); - try { - const PROJECT = makeProjectDir("repl-reconnect"); - const session = await connectHttp(daemon.url); - try { - const started = await repl(session, { action: "eval", projectDir: PROJECT, code: 'const p = agent("pi/x", "task"); "started"' }); - assert.ok(!isErrorResult(started), textOf(started)); - await tick(); - const record = daemon.sessions.values()[0]; - assert.ok(record, "the session is registered"); - // The transient drop: the session's LAST connection closes (a - // standalone-GET blip — the session itself stays alive). The - // registry signals the presence ledger, which removes the session's - // presence and schedules the project drain. - daemon.sessions.connectionClosed(record.sessionId); - await new Promise((resolve) => setTimeout(resolve, 20)); - const child = runner.last(); - assert.equal(child.releases, 0, "the drain is waiting on the in-flight turn"); - // The SAME live session reconnects (no new MCP session, no tool - // call): the registry's connection-open signal re-adds the - // session's project presence from its retained affinity, and the - // scheduled drain aborts — the child stays warm. - daemon.sessions.connectionOpened(record.sessionId); - await new Promise((resolve) => setTimeout(resolve, 20)); - assert.equal(child.releases, 0, "the reconnect aborted the drain — the child stays warm"); - assert.equal(child.cancelCalls, 0, "nothing was cancelled"); - // The turn completes normally and settles into the live workspace; - // the workspace stays warm (children not closed). - child.completeTurn("warm after reconnect"); - const got = await repl(session, { action: "eval", projectDir: PROJECT, code: "await p" }); - assert.ok(!isErrorResult(got), textOf(got)); - assert.equal(structuredOf(got).result, "warm after reconnect"); - const ws = (await evalJson(session, PROJECT, "workspace()")) as { - diagnostics: { childrenClosed: boolean }; - }; - assert.equal(ws.diagnostics.childrenClosed, false, "the workspace is warm"); - } finally { - await session.dispose(); - } - } finally { - await daemon.close(); - } -}); - -test("reset() does not clear client presence: with a second client still connected, the resetting client's disconnect does NOT drain the post-reset workspace", async () => { - const runner = new FakeRunner(); - const daemon = await startReplDaemon(runner); - try { - const PROJECT = makeProjectDir("repl-reset-presence"); - const sessionA = await connectHttp(daemon.url); - const sessionB = await connectHttp(daemon.url); - try { - // Both clients are present on the project. - const start = await repl(sessionA, { action: "eval", projectDir: PROJECT, code: 'const p = agent("pi/x", "task"); "started"' }); - assert.ok(!isErrorResult(start), textOf(start)); - await repl(sessionB, { action: "eval", projectDir: PROJECT, code: '"b present"' }); - await tick(); - // A resets: the workspace is dropped, but the CONNECTIONS stay — - // presence is connection liveness, not workspace state. - const reset = await repl(sessionA, { action: "eval", projectDir: PROJECT, code: "reset()" }); - assert.ok(!isErrorResult(reset), textOf(reset)); - // A starts NEW work on the fresh workspace. - const restarted = await repl(sessionA, { action: "eval", projectDir: PROJECT, code: 'const q = agent("pi/x", "task2"); "started2"' }); - assert.ok(!isErrorResult(restarted), textOf(restarted)); - await tick(); - const child = runner.last(); - // A's connection drops while B is still connected: NO drain may - // fire — the post-reset child stays warm (the drain decision sees - // B's presence). - await sessionA.dispose(); - await new Promise((resolve) => setTimeout(resolve, 50)); - assert.equal(child.releases, 0, "no drain while B is connected"); - assert.equal(child.cancelCalls, 0, "nothing was cancelled"); - // The in-flight turn completes and settles into the live - // workspace; B can see the result. - child.completeTurn("post-reset result"); - const got = await repl(sessionB, { action: "eval", projectDir: PROJECT, code: "await q" }); - assert.ok(!isErrorResult(got), textOf(got)); - assert.equal(structuredOf(got).result, "post-reset result"); - // B's disconnect is the LAST client: NOW the drain runs and closes - // the idle child. - await sessionB.dispose(); - for (let attempt = 0; attempt < 200 && child.releases === 0; attempt++) { - await new Promise((resolve) => setTimeout(resolve, 10)); - } - assert.equal(child.releases, 1, "the last-client disconnect drains and closes the idle child"); - } finally { - await sessionA.dispose().catch(() => undefined); - await sessionB.dispose().catch(() => undefined); - } - } finally { - await daemon.close(); - } -}); - -test("workflow calls register project presence: a workflow-only client B keeps the workspace warm when repl-client A disconnects (phase-E review rejection round 2)", async () => { - const runner = new FakeRunner(); - const daemon = await startReplDaemon(runner); - try { - const PROJECT = makeProjectDir("repl-workflow-presence"); - const sessionA = await connectHttp(daemon.url); - const sessionB = await connectHttp(daemon.url); - try { - // A touches the repl workspace (a child opens). - const start = await repl(sessionA, { action: "eval", projectDir: PROJECT, code: 'const p = agent("pi/x", "task"); "started"' }); - assert.ok(!isErrorResult(start), textOf(start)); - await tick(); - const child = runner.last(); - // B addresses the SAME project through the WORKFLOW tool (a - // trivial script — no agents, nothing repl-related). The workflow - // handler resolves the same per-project context and registers the - // session's presence on it. - const ran = await sessionB.client.callTool({ - name: "workflow", - arguments: { - action: "run", - projectDir: PROJECT, - script: 'export const meta = { name: "empty", description: "empty script" };', - }, - }); - assert.ok(!(ran as { isError?: boolean }).isError, textOf(ran)); - assert.equal(structuredOf(ran).accepted, true); - const completed = await waitForRun(sessionB.client, String(structuredOf(ran).runId)); - assert.equal(structuredOf(completed).status, "completed"); - // A's connection drops while B is still connected to the project - // through workflow calls: NO drain may fire — the post-workflow - // child stays warm. - await sessionA.dispose(); - await new Promise((resolve) => setTimeout(resolve, 50)); - assert.equal(child.releases, 0, "no drain while B (a workflow-only client) is connected"); - assert.equal(child.cancelCalls, 0, "nothing was cancelled"); - // The in-flight turn completes and settles into the live workspace. - child.completeTurn("wf-presence result"); - const got = await repl(sessionB, { action: "eval", projectDir: PROJECT, code: "await p" }); - assert.ok(!isErrorResult(got), textOf(got)); - assert.equal(structuredOf(got).result, "wf-presence result"); - // B's disconnect is the LAST client: NOW the drain runs and closes - // the idle child. - await sessionB.dispose(); - for (let attempt = 0; attempt < 200 && child.releases === 0; attempt++) { - await new Promise((resolve) => setTimeout(resolve, 10)); - } - assert.equal(child.releases, 1, "the last-client disconnect drains and closes the idle child"); - } finally { - await sessionA.dispose().catch(() => undefined); - await sessionB.dispose().catch(() => undefined); - } - } finally { - await daemon.close(); - } -}); - -test("every repl action returns the redesign's machine-readable shape as structuredContent — eval { output, result?, running? }, the interrupt outcome, and the error variant (guest output and orchestration metadata were separate fields in v1; the redesign's output is ONE string)", async () => { - const runner = new FakeRunner(); - const daemon = await startReplDaemon(runner); - try { - const session = await connectHttp(daemon.url); - try { - const PROJECT = makeProjectDir("repl-structured"); - // eval: the redesign's shape — ONE newline-joined output string - // (console lines), the result repr, and nothing else. - const evaled = await repl(session, { - action: "eval", - projectDir: PROJECT, - code: 'const research = agent("pi/x", "investigate"); globalThis.answer = 42; console.log("hello"); answer', - }); - assert.ok(!isErrorResult(evaled), textOf(evaled)); - const sc = structuredOf(evaled); - assert.deepEqual(Object.keys(sc).sort(), ["output", "result"]); - assert.equal(sc.output, "hello"); - assert.equal(sc.result, "42"); - assert.ok(textOf(evaled).includes("result: 42"), "the human text stays alongside"); - - // The still-running shape with the mid-hold settlement absorbed. - await tick(); - const waiting = repl(session, { action: "eval", projectDir: PROJECT, code: "await research", timeoutMs: 5000 }); - await tick(); - runner.last().completeTurn("waited result"); - const waited = await waiting; - assert.ok(!isErrorResult(waited), textOf(waited)); - const scWaited = structuredOf(waited); - assert.deepEqual(Object.keys(scWaited).sort(), ["output", "result"]); - assert.equal(scWaited.result, "waited result"); - - // A timed-out hold reports the still-running shape with the ids. - await repl(session, { action: "eval", projectDir: PROJECT, code: 'const q2 = agent("pi/x", "task2"); "started2"' }); - const timedOut = await repl(session, { action: "eval", projectDir: PROJECT, code: "await q2", timeoutMs: 100 }); - const scTimedOut = structuredOf(timedOut); - assert.deepEqual(scTimedOut, { output: "", running: ["c2"] }); - assert.ok(textOf(timedOut).includes("running: c2"), textOf(timedOut)); - - // interrupt with an id: the honest outcome + the call id. - const interrupted = await repl(session, { action: "interrupt", projectDir: PROJECT, id: "c2" }); - assert.ok(!isErrorResult(interrupted), textOf(interrupted)); - const scInterrupt = structuredOf(interrupted); - assert.deepEqual(scInterrupt, { interrupt: { outcome: "cancelled", callId: "c2" } }); - assert.ok(textOf(interrupted).includes("session/cancel sent"), textOf(interrupted)); - // interrupt without an id on an idle workspace: the honest refusal - // (a fresh project — nothing ever ran). - const idleProject = makeProjectDir("repl-structured-idle"); - const refused = await repl(session, { action: "interrupt", projectDir: idleProject }); - const scRefused = structuredOf(refused); - assert.deepEqual(scRefused, { interrupt: { outcome: "refused-idle" } }); - assert.ok(textOf(refused).includes("no running eval to interrupt"), textOf(refused)); - // interrupt without an id on a RUNNING eval: outcome "targeted". - const running = repl(session, { - action: "eval", - projectDir: PROJECT, - code: 'const q3 = agent("pi/x", "task3"); await q3; while (true) {}', - timeoutMs: 30_000, - }); - await tick(); - const targeted = await repl(session, { action: "interrupt", projectDir: PROJECT }); - const scTargeted = structuredOf(targeted); - assert.deepEqual(scTargeted, { interrupt: { outcome: "targeted" } }); - assert.ok(textOf(targeted).includes("interrupting the running eval"), textOf(targeted)); - // The targeted runaway is broken at its next execution — the held - // eval returns promptly (the finished-with-error shape). - await tick(); - runner.last().completeTurn("resumed"); - const broken = await running; - assert.ok(!isErrorResult(broken), textOf(broken)); - assert.ok(!("running" in structuredOf(broken)), "the broken eval returned"); - // reset() via the guest function. - const reset = await repl(session, { action: "eval", projectDir: PROJECT, code: "reset()" }); - assert.ok(!isErrorResult(reset), textOf(reset)); - } finally { - await session.dispose(); - } - } finally { - await daemon.close(); - } -}); - -// ── Round 4: exact action shapes, manifest fields, bounded surface ──── - -test("review round 4: the input is action-discriminated — every action's EXACT field set is enforced at the boundary (eval without code, interrupt with code/timeoutMs, eval with id: all rejected with 'cannot include'/'requires'; irrelevant known fields are never silently accepted)", async () => { - const runner = new FakeRunner(); - const daemon = await startReplDaemon(runner); - try { - const session = await connectHttp(daemon.url); - try { - const PROJECT = makeProjectDir("repl-shapes"); - // Missing required fields: eval WITHOUT the code field is still - // rejected (the absent field fails the exact-shape boundary — only - // the present-but-empty string is valid). - const noCode = await repl(session, { action: "eval", projectDir: PROJECT }); - assert.ok(isErrorResult(noCode), textOf(noCode)); - assert.ok(textOf(noCode).includes("eval requires a code string"), textOf(noCode)); - // Extraneous known fields per action. - const interruptWithTimeout = await repl(session, { action: "interrupt", projectDir: PROJECT, timeoutMs: 100 }); - assert.ok(isErrorResult(interruptWithTimeout), textOf(interruptWithTimeout)); - assert.ok(textOf(interruptWithTimeout).includes('cannot include timeoutMs'), textOf(interruptWithTimeout)); - const interruptWithCode = await repl(session, { action: "interrupt", projectDir: PROJECT, code: "1 + 1" }); - assert.ok(isErrorResult(interruptWithCode), textOf(interruptWithCode)); - assert.ok(textOf(interruptWithCode).includes('cannot include code'), textOf(interruptWithCode)); - const evalWithId = await repl(session, { action: "eval", projectDir: PROJECT, code: "1 + 1", id: "c1" }); - assert.ok(isErrorResult(evalWithId), textOf(evalWithId)); - assert.ok(textOf(evalWithId).includes('cannot include id'), textOf(evalWithId)); - // The workspace was never created by the rejected calls: a well- - // formed eval works. - const ok = await repl(session, { action: "eval", projectDir: PROJECT, code: "6 * 7" }); - assert.ok(!isErrorResult(ok), textOf(ok)); - assert.ok(textOf(ok).includes("result: 42"), textOf(ok)); - } finally { - await session.dispose(); - } - } finally { - await daemon.close(); - } -}); - -test("review round 4: workspace() carries the machine-readable type and live-handle status fields — `agent handle` type, the call id, and pending→settled status transitions as their own fields", async () => { - const runner = new FakeRunner(); - const daemon = await startReplDaemon(runner); - try { - const session = await connectHttp(daemon.url); - try { - const PROJECT = makeProjectDir("repl-manifest-fields"); - const evaled = await repl(session, { - action: "eval", - projectDir: PROJECT, - code: 'const research = agent("pi/x", "investigate"); globalThis.answer = 42; console.log("hello"); answer', - }); - assert.ok(!isErrorResult(evaled), textOf(evaled)); - await tick(); - const ws = (await evalJson(session, PROJECT, "workspace()")) as { - bindings: Array<{ name: string; type: string; callId?: string; status?: string }>; - }; - const handle = ws.bindings.find((b) => b.name === "research"); - assert.ok(handle, "the agent handle binding"); - assert.equal(handle.type, "agent handle", "the machine-readable type"); - assert.equal(handle.callId, "c1", "the call id is its own field"); - assert.equal(handle.status, "pending", "the live-handle status is its own field"); - const plain = ws.bindings.find((b) => b.name === "answer"); - assert.ok(plain, "the plain binding"); - assert.equal(plain.type, "number", "the plain binding's machine-readable type"); - assert.equal(plain.callId, undefined); - assert.equal(plain.status, undefined); - // The handle settles: the status transitions to `settled` (the - // call store is the authority). - runner.last().completeTurn("done"); - await tick(); - const picked = await repl(session, { action: "eval", projectDir: PROJECT, code: "await research" }); - assert.equal(structuredOf(picked).result, "done"); - const wsAfter = (await evalJson(session, PROJECT, "workspace()")) as { - bindings: Array<{ name: string; status?: string }>; - }; - const handleAfter = wsAfter.bindings.find((b) => b.name === "research"); - assert.equal(handleAfter?.status, "settled", "the handle status transitioned"); - } finally { - await session.dispose(); - } - } finally { - await daemon.close(); - } -}); - -test("review round 8 (flipped by the redesign): agents() ships the modelSpec VERBATIM — the old 200-char status cap is deleted with the structured-status surface (§7 kept the 200-char bound only for manifest tokens and task previews)", async () => { - const runner = new FakeRunner(); - const daemon = await startReplDaemon(runner); - try { - const session = await connectHttp(daemon.url); - try { - const PROJECT = makeProjectDir("repl-modelspec-verbatim"); - const hugeSpec = "pi/" + "X".repeat(500); - const evaled = await repl(session, { action: "eval", projectDir: PROJECT, code: `const big = agent(${JSON.stringify(hugeSpec)}, "task"); "started"` }); - assert.ok(!isErrorResult(evaled), textOf(evaled)); - await tick(); - const agents = (await evalJson(session, PROJECT, "agents()")) as Array<{ callId: string; modelSpec: string }>; - assert.equal(agents.length, 1); - assert.equal(agents[0].callId, "c1"); - assert.equal(agents[0].modelSpec, hugeSpec, "the full model spec, verbatim"); - // The workspace manifest's task preview keeps its 200-char metadata - // bound (a retained §7 preview — metadata formatting, not a cap). - const ws = (await evalJson(session, PROJECT, "workspace()")) as { - bindings: Array<{ name: string; task: string | null }>; - }; - assert.equal(ws.bindings.find((b) => b.name === "big")?.task, "task"); - } finally { - await session.dispose(); - } - } finally { - await daemon.close(); - } -}); - -test("review round 8: interrupt { id } cancels a call whose openSession is still pending — the call settles durably as the recoverable AGENT_CANCELLED, and the LATE child is closed without ever prompting", async () => { - const runner = new DelayedOpenRunner(); - runner.parkOpens(); - const PROJECT = makeProjectDir("repl-interrupt-opening"); - const daemon = await startReplDaemon(runner); - try { - const session = await connectHttp(daemon.url); - try { - const evaled = await repl(session, { action: "eval", projectDir: PROJECT, code: `const p = agent("pi/x", "task"); "started"` }); - assert.ok(!isErrorResult(evaled), textOf(evaled)); - await tick(); - // The interrupt lands while openSession is STILL parked: the - // old decision returned 'none' (no live session, no lazy - // re-attach record) and the eventual open went on to prompt a - // supposedly-interrupted call. - const interrupted = await repl(session, { action: "interrupt", projectDir: PROJECT, id: "c1" }); - assert.ok(!isErrorResult(interrupted), textOf(interrupted)); - const si = structuredOf(interrupted); - assert.equal((si.interrupt as { outcome: string }).outcome, "cancelled", `honest outcome: ${JSON.stringify(si.interrupt)}`); - // The guest promise settled NOW with the recoverable error — not - // when the open eventually lands. - const read = await repl(session, { action: "eval", projectDir: PROJECT, code: `await p.catch((e) => "ERR:" + e.message)` }); - assert.ok(!isErrorResult(read), textOf(read)); - const sc1 = structuredOf(read); - assert.ok( - String(sc1.result).includes("turn c1 was cancelled"), - `guest-visible settlement: ${sc1.result}`, - ); - } finally { - await session.dispose(); - } - } finally { - await daemon.close(); - } - // DURABILITY: a daemon restart over the same store restores the - // snapshot (the rejected promise is part of it) and the recorded - // completion — the cancellation is durable, never re-issued, never - // re-opened (a fresh daemon must not open a session for a settled - // call). - const daemon2 = await startReplDaemon(runner); - try { - const session = await connectHttp(daemon2.url); - try { - const read = await repl(session, { action: "eval", projectDir: PROJECT, code: `await p.catch((e) => "ERR:" + e.message)` }); - assert.ok(!isErrorResult(read), textOf(read)); - const sc = structuredOf(read); - assert.ok( - String(sc.result).includes("turn c1 was cancelled"), - `the restart settles the durable cancellation: ${sc.result}`, - ); - } finally { - await session.dispose(); - } - } finally { - await daemon2.close(); - } - // The LATE open lands after everything: the child is closed - // immediately — it never prompts (a supposedly-interrupted call must - // not run a turn), and nothing re-opened across the restart. - runner.releaseOpens(); - await tick(); - assert.equal(runner.sessions.length, 1, "exactly one session ever opened"); - assert.equal(runner.sessions[0].prompts.length, 0, "the stopped call never ran a turn"); - assert.equal(runner.sessions[0].releases, 1, "the late child was closed without prompting"); -}); - -test("review round 9: interrupt { id } on a still-OPENING call is IMMEDIATELY durable — the daemon can be killed right after the interrupt (NO eval in between) and the restart restores the SETTLED workspace: the reconcile's store arm has nothing to settle and the continuation binding carries the settlement provenance", async () => { - const runner = new DelayedOpenRunner(); - runner.parkOpens(); - const PROJECT = makeProjectDir("repl-interrupt-opening-immediate"); - const daemon = await startReplDaemon(runner); - try { - const session = await connectHttp(daemon.url); - try { - // The settlement drain's continuation creates a binding whose - // provenance must travel INSIDE the interrupt's own snapshot. - const evaled = await repl(session, { action: "eval", projectDir: PROJECT, code: `const p = agent("pi/x", "task"); p.catch(() => { globalThis.wasCancelled = true; }); "started"` }); - assert.ok(!isErrorResult(evaled), textOf(evaled)); - await tick(); - const interrupted = await repl(session, { action: "interrupt", projectDir: PROJECT, id: "c1" }); - assert.ok(!isErrorResult(interrupted), textOf(interrupted)); - const si = structuredOf(interrupted); - assert.equal((si.interrupt as { outcome: string }).outcome, "cancelled", `honest outcome: ${JSON.stringify(si.interrupt)}`); - // NO further repl calls — the daemon dies immediately. The - // interrupt's own settlement boundary must already have persisted - // the settled workspace (the op-end flush writes before the tool - // call resolves). - } finally { - await session.dispose(); - } - } finally { - await daemon.close(); - } - // The restart over the same home: the FIRST read is an eval, and it - // must already see the settlement — the restored registry is settled - // (c1 not pending) and the continuation binding carries the - // settlement's provenance FROM THE SNAPSHOT. - const daemon2 = await startReplDaemon(runner); - try { - const session = await connectHttp(daemon2.url); - try { - const ws = (await evalJson(session, PROJECT, "workspace()")) as { - inFlight: string[]; - bindings: Array<{ name: string; provenance: string | null }>; - diagnostics: { reconcile: { settledFromStore: string[] } | null }; - }; - assert.ok(!ws.inFlight.includes("c1"), `the restored registry is settled: ${JSON.stringify(ws.inFlight)}`); - // The discriminator: the interrupt's OWN snapshot carried the - // settlement, so the restart's reconcile has NOTHING for the - // store arm. - assert.ok(ws.diagnostics.reconcile !== null, "the restored workspace carries its reconcile summary"); - assert.deepEqual(ws.diagnostics.reconcile!.settledFromStore, [], "the store arm had nothing to settle — the snapshot already carried the settlement"); - const wasCancelled = ws.bindings.find((b) => b.name === "wasCancelled"); - assert.ok(wasCancelled !== undefined, `the continuation binding survived the restart: ${JSON.stringify(ws.bindings)}`); - assert.equal(wasCancelled.provenance, "worker c1", "the settlement provenance traveled inside the interrupt's snapshot"); - // The guest promise rejects with the durable cancellation. - const read = await repl(session, { action: "eval", projectDir: PROJECT, code: `await p.catch((e) => "ERR:" + e.message)` }); - assert.ok(!isErrorResult(read), textOf(read)); - const sc1 = structuredOf(read); - assert.ok( - String(sc1.result).includes("turn c1 was cancelled"), - `the restart settles the durable cancellation: ${sc1.result}`, - ); - } finally { - await session.dispose(); - } - } finally { - await daemon2.close(); - } - // The LATE open lands after everything: the child is closed - // immediately — it never prompts — and nothing re-opened across the - // immediate restart. - runner.releaseOpens(); - await tick(); - assert.equal(runner.sessions.length, 1, "exactly one session ever opened"); - assert.equal(runner.sessions[0].prompts.length, 0, "the stopped call never ran a turn"); - assert.equal(runner.sessions[0].releases, 1, "the late child was closed without prompting"); -}); diff --git a/packages/mcp-server/test/repl-inprocess-break.e2e.test.ts b/packages/mcp-server/test/repl-inprocess-break.e2e.test.ts deleted file mode 100644 index eef458e4..00000000 --- a/packages/mcp-server/test/repl-inprocess-break.e2e.test.ts +++ /dev/null @@ -1,214 +0,0 @@ -// End-to-end over the BUILT dist: the OUT-OF-BAND eval-break in the -// SINGLE-PROJECT (in-process) stdio mode (phase-F review round 3 — the -// reviewer: the public in-process/library server omitted the channel, -// so its event loop could not process a no-id interrupt during a -// synchronous `while(true)`; the documented interrupt behavior must be -// implemented in every supported mode, and it is not a v1 exclusion). -// -import { StdioClientTransport } from "@modelcontextprotocol/client/stdio"; -import { Client } from "@modelcontextprotocol/client"; -import type { CallToolResult } from "@modelcontextprotocol/client"; - -// The in-process server has no shim process, so its stdio transport's -// stdin reader lives on a WORKER THREAD (`repl-stdio-transport.ts`): -// the reader stays live while the main thread is wedged in the VM, -// recognizes `repl` interrupt calls, and fires the server's own -// eval-break relay (a second worker thread) — the running eval's -// quickjs interrupt handler consumes the shared-memory flag mid-run. -// The relay key is realpath'd exactly like the daemon's project -// validation (a symlinked projectDir must still interrupt). -// -// Requires `pnpm build` first (spawns dist/entry.js --in-process). -import assert from "node:assert/strict"; -import { spawn, type ChildProcess } from "node:child_process"; -import { mkdtempSync, mkdirSync, rmSync, symlinkSync } from "node:fs"; -import { tmpdir } from "node:os"; -import { join, resolve } from "node:path"; -import { fileURLToPath } from "node:url"; -import test from "node:test"; -const distEntry = resolve(fileURLToPath(import.meta.url), "../../dist/entry.js"); - -const TEST_TMP = mkdtempSync(join(tmpdir(), "agentprism-repl-inprocess-")); -let server: ChildProcess | undefined; - -function startServer(home: string): Promise { - const transport = new StdioClientTransport({ - command: process.execPath, - args: [distEntry, "--in-process"], - env: { - ...(process.env as Record), - HOME: home, - // The failure bound: a broken relay path must fail the fast - // asserts below, not hang the suite for the default 30 s. - AGENTPRISM_REPL_EVAL_TIMEOUT_MS: "20000", - }, - stderr: "pipe", - }); - const client = new Client({ name: "repl-inprocess-e2e", version: "0.0.0" }, { capabilities: {} }); - server = (transport as unknown as { _process: ChildProcess })._process; - return client.connect(transport).then(() => client); -} - -function replEvalCode(client: Client, projectDir: string, code: string): Promise { - return client.callTool( - { name: "repl", arguments: { action: "eval", projectDir, code } }, - { timeout: 60_000 }, - ); -} - -function replEvalNoDir(client: Client, code: string): Promise { - return client.callTool({ name: "repl", arguments: { action: "eval", code } }, { - timeout: 60_000, - }); -} - -function replInterruptNoDir(client: Client): Promise { - return client.callTool({ name: "repl", arguments: { action: "interrupt" } }, { - timeout: 60_000, - }); -} - -function replInterrupt(client: Client, projectDir: string): Promise { - return client.callTool( - { name: "repl", arguments: { action: "interrupt", projectDir } }, - { timeout: 60_000 }, - ); -} - -function textOf(result: CallToolResult): string { - return (result.content ?? []) - .filter((block) => block.type === "text") - .map((block) => (block as { text?: string }).text ?? "") - .join("\n"); -} - -test("the in-process stdio server's worker-reader fires the relay: a no-id interrupt breaks a synchronous while(true) eval out of band and the workspace stays usable", async () => { - const home = mkdtempSync(join(TEST_TMP, "home-")); - const projectDir = join(home, "project"); - mkdirSync(projectDir, { recursive: true }); - const client = await startServer(home); - try { - const warm = await replEvalCode(client, projectDir, "6 * 7"); - assert.ok(textOf(warm).includes("result: 42"), textOf(warm)); - const startedAt = Date.now(); - const runaway = replEvalCode(client, projectDir, "while (true) {}"); - // Give the server a moment to enter the eval, then send the - // interrupt: the transport's worker-reader — the only stdin reader, - // live while the main thread is wedged in the VM — fires the - // server's eval-break relay before forwarding the frame, and the - // running eval's quickjs interrupt handler breaks mid-run. - await new Promise((resolvePromise) => setTimeout(resolvePromise, 400)); - const interrupt = await replInterrupt(client, projectDir); - const interruptText = textOf(interrupt); - assert.ok( - interruptText.includes("out of band") || - interruptText.includes("out-of-band") || - interruptText.includes("broken OUT OF BAND"), - `the interrupt reports the out-of-band break: ${interruptText}`, - ); - const structured = (interrupt.structuredContent ?? {}) as { interrupt?: { outcome?: string } }; - assert.equal(structured.interrupt?.outcome, "targeted", JSON.stringify(structured)); - const result = await runaway; - const elapsed = Date.now() - startedAt; - assert.ok(elapsed < 8000, `the eval broke out of band, not at the 20 s deadline: ${elapsed} ms`); - const text = textOf(result); - assert.ok(text.includes("interrupted") || text.includes("error"), `the eval reports the break: ${text}`); - // The workspace stays usable, and a later no-id interrupt with - // nothing running REFUSES (no stale break ever reaches a later - // eval). - const after = await replEvalCode(client, projectDir, "40 + 2"); - assert.ok(textOf(after).includes("result: 42"), textOf(after)); - const idle = await replInterrupt(client, projectDir); - const idleStructured = (idle.structuredContent ?? {}) as { interrupt?: { outcome?: string } }; - assert.equal(idleStructured.interrupt?.outcome, "refused-idle", JSON.stringify(idleStructured)); - await client.close().catch(() => undefined); - } finally { - await client.close().catch(() => undefined); - } -}); - -test("the OMITTED-projectDir interrupt fires the relay with the server's own project key: the documented optional projectDir works for a synchronous runaway (phase-F review round 4: the relay used to skip the omitted-projectDir interrupt, so the eval ran to the per-eval deadline and the interrupt reported refused-idle)", async () => { - const home = mkdtempSync(join(TEST_TMP, "home-nodir-")); - const client = await startServer(home); - try { - // No projectDir anywhere: the repl tool resolves the server's own - // adopted project (its cwd), and the reader worker's relay fires - // under the SAME key (the transport's default project key). - const warm = await replEvalNoDir(client, "6 * 7"); - assert.ok(textOf(warm).includes("result: 42"), textOf(warm)); - const startedAt = Date.now(); - const runaway = replEvalNoDir(client, "while (true) {}"); - await new Promise((resolvePromise) => setTimeout(resolvePromise, 400)); - const interrupt = await replInterruptNoDir(client); - const interruptText = textOf(interrupt); - assert.ok( - interruptText.includes("out of band") || - interruptText.includes("out-of-band") || - interruptText.includes("broken OUT OF BAND"), - `the interrupt reports the out-of-band break: ${interruptText}`, - ); - const structured = (interrupt.structuredContent ?? {}) as { interrupt?: { outcome?: string } }; - assert.equal(structured.interrupt?.outcome, "targeted", JSON.stringify(structured)); - const result = await runaway; - const elapsed = Date.now() - startedAt; - assert.ok(elapsed < 8000, `the eval broke out of band, not at the 20 s deadline: ${elapsed} ms`); - const text = textOf(result); - assert.ok(text.includes("interrupted") || text.includes("error"), `the eval reports the break: ${text}`); - // The default workspace stays usable, and an idle no-dir interrupt - // refuses (no stale break ever reaches a later eval). - const after = await replEvalNoDir(client, "40 + 2"); - assert.ok(textOf(after).includes("result: 42"), textOf(after)); - const idle = await replInterruptNoDir(client); - const idleStructured = (idle.structuredContent ?? {}) as { interrupt?: { outcome?: string } }; - assert.equal(idleStructured.interrupt?.outcome, "refused-idle", JSON.stringify(idleStructured)); - await client.close().catch(() => undefined); - } finally { - await client.close().catch(() => undefined); - } -}); - -test("the relay key is the CANONICAL projectDir: an interrupt through a symlink breaks the running eval (phase-F review round 3: the raw path used to get a relay 404)", async () => { const home = mkdtempSync(join(TEST_TMP, "home-sym-")); - const realDir = join(home, "real-project"); - const symDir = join(home, "linked-project"); - mkdirSync(realDir, { recursive: true }); - symlinkSync(realDir, symDir, "dir"); - const client = await startServer(home); - try { - const warm = await replEvalCode(client, symDir, "6 * 7"); - assert.ok(textOf(warm).includes("result: 42"), textOf(warm)); - const startedAt = Date.now(); - const runaway = replEvalCode(client, symDir, "while (true) {}"); - await new Promise((resolvePromise) => setTimeout(resolvePromise, 400)); - // The interrupt is addressed through the SYMLINK; the reader - // worker realpaths it before arming the relay, exactly like the - // daemon's project validation — the eval must break out of band. - const interrupt = await replInterrupt(client, symDir); - const structured = (interrupt.structuredContent ?? {}) as { interrupt?: { outcome?: string } }; - assert.equal(structured.interrupt?.outcome, "targeted", JSON.stringify(structured)); - const result = await runaway; - const elapsed = Date.now() - startedAt; - assert.ok(elapsed < 8000, `the symlink-addressed eval broke out of band: ${elapsed} ms`); - assert.ok( - textOf(result).includes("interrupted") || textOf(result).includes("error"), - `the eval reports the break: ${textOf(result)}`, - ); - await client.close().catch(() => undefined); - } finally { - await client.close().catch(() => undefined); - } -}); - -process.on("exit", () => { - if (server !== undefined && server.pid !== undefined) { - try { - process.kill(server.pid, "SIGKILL"); - } catch { - /* best-effort */ - } - } - try { - rmSync(TEST_TMP, { recursive: true, force: true }); - } catch { - /* best-effort */ - } -}); diff --git a/packages/mcp-server/test/repl-review2.test.ts b/packages/mcp-server/test/repl-review2.test.ts deleted file mode 100644 index 9a313a3b..00000000 --- a/packages/mcp-server/test/repl-review2.test.ts +++ /dev/null @@ -1,901 +0,0 @@ -/** - * Phase-D review round 2 at the MCP-tool boundary, adapted to the - * eval-plane redesign surface: the fused eval's pump (a continuation's - * console output drains in the SAME held call), the §4.5 workspace() - * introspection (bindings with structure-only tokens, provenance, and - * live-handle status), the single-flight first touch (concurrent - * first-touch calls create exactly one VM and broker), the - * client-presence drain (last-client disconnect drains in-flight turns - * and closes idle children; the next explicit queued turn re-attaches - * lazily), and the per-eval deadline (a currently-running runaway eval - * is always breakable). - */ - -import assert from "node:assert/strict"; -import { existsSync, mkdirSync, mkdtempSync, rmSync } from "node:fs"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; -import { test } from "node:test"; -import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; -import type { - BrokerLoadSessionOptions, - BrokerOpenSessionOptions, - BrokerPromptOptions, - BrokerRunner, - BrokerSession, - BrokerTurn, -} from "@automatalabs/repl-engine"; - -import { createWorkflowServer, resetReplProjectState } from "../src/index.js"; -import { WorkflowProjectRegistry } from "../src/project-registry.js"; -import { ReplPresenceLedger } from "../src/repl-presence.js"; -import { okRunner, textOf, type Connected } from "./_harness.js"; -import { workflowProjectPaths } from "@automatalabs/workflows"; - -/** The fake held-open ACP session (see repl-tool.test.ts). */ -class FakeSession implements BrokerSession { - readonly sessionId: string; - initializeMeta: Readonly> | undefined; - readonly prompts: Array<{ content: string; resolve: (turn: BrokerTurn) => void; reject: (error: unknown) => void }> = []; - readonly steers: Array<{ content: string; resolve: (outcome: unknown) => void; reject: (error: unknown) => void }> = []; - releases = 0; - cancelCalls = 0; - stopReason = "end_turn"; - readonly completedTexts: string[] = []; - /** The re-attach seam's scripted loaded-turn outcome (null parks it). */ - loadedTurnTextValue: string | null = null; - /** A hung cancel (the bounded-teardown regression: the shutdown - * disposal must not await a hung cancel past its bound). */ - hangCancel = false; - /** A hung release (same regression for the release phase). */ - hangRelease = false; - - constructor(readonly openedWith: BrokerOpenSessionOptions | BrokerLoadSessionOptions) { - this.sessionId = `fake-session-${FakeSession.nextId++}`; - this.initializeMeta = { steering: { supported: true } }; - } - - static nextId = 0; - - prompt(content: string, opts: BrokerPromptOptions = {}): Promise { - return new Promise((resolve, reject) => { - this.prompts.push({ content, resolve, reject }); - opts.onHandoff?.(); - }); - } - - steer(content: string): Promise { - return new Promise((resolve, reject) => { - this.steers.push({ content, resolve, reject }); - }); - } - - awaitCurrentTurn(): Promise { - if (this.loadedTurnTextValue !== null) { - return Promise.resolve({ stopReason: this.stopReason, text: this.loadedTurnTextValue }); - } - return new Promise(() => {}); - } - - cancel(): Promise { - this.cancelCalls++; - if (this.hangCancel) return new Promise(() => {}); - for (const pending of this.prompts.splice(0)) { - pending.resolve({ stopReason: "cancelled", text: "" }); - } - return Promise.resolve(); - } - - release(): Promise { - this.releases++; - if (this.hangRelease) return new Promise(() => {}); - return Promise.resolve(); - } - - currentTurnText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ""; - } - - finalMessageText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ""; - } - - rawStructuredOutput(): unknown { - return undefined; - } - - completeTurn(text: string): void { - const pending = this.prompts.shift(); - assert.ok(pending, "a prompt turn must be in flight"); - this.completedTexts.push(text); - pending.resolve({ stopReason: this.stopReason, text }); - } -} - -/** The fake runner with the loadSession seam (see repl-tool.test.ts). */ -class FakeRunner implements BrokerRunner { - readonly sessions: FakeSession[] = []; - readonly openedWith: BrokerOpenSessionOptions[] = []; - readonly loadedWith: BrokerLoadSessionOptions[] = []; - loadedTurnText: string | null = null; - - listBackends(): string[] { - return ["pi"]; - } - - defaultBackendId(): string { - return "pi"; - } - - async openSession(opts: BrokerOpenSessionOptions): Promise { - const session = new FakeSession(opts); - this.sessions.push(session); - this.openedWith.push(opts); - return session; - } - - async loadSession(opts: BrokerLoadSessionOptions): Promise { - const session = new FakeSession(opts); - session.loadedTurnTextValue = this.loadedTurnText; - this.sessions.push(session); - this.loadedWith.push(opts); - return session; - } - - async dispose(): Promise {} - - last(): FakeSession { - assert.ok(this.sessions.length > 0, "a session must exist"); - return this.sessions[this.sessions.length - 1]; - } -} - -/** A fresh project directory per test (the repl store persists across - * tests — each test gets its own project so the store starts clean). */ -function freshProject(): string { - return mkdtempSync(join(tmpdir(), "repl-review2-tool-")); -} - -/** Connect a workflow server with an injected repl runner + presence. */ -async function connectWithRepl( - replRunner: BrokerRunner, - options: { - presence?: ReplPresenceLedger; - clientId?: () => string | undefined; - projects?: WorkflowProjectRegistry; - } = {}, -): Promise { - const server = createWorkflowServer(okRunner(), { - replRunner, - replPresence: options.presence, - replClientId: options.clientId ?? (() => "test-client"), - projects: options.projects, - }); - const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair(); - const client = new Client({ name: "mcp-repl-review2", version: "0.0.0" }, { capabilities: {} }); - await Promise.all([server.connect(serverTransport), client.connect(clientTransport)]); - return { - client, - server, - runner: replRunner as FakeRunner, - async dispose() { - await client.close(); - await server.close(); - }, - }; -} - -async function repl( - connected: Connected, - input: { action: string; projectDir?: string; code?: string; timeoutMs?: number; id?: string }, -) { - return connected.client.callTool({ name: "repl", arguments: input as Record }); -} - -function structuredOf(res: Awaited>): Record { - return (res as { structuredContent?: Record }).structuredContent ?? {}; -} - -/** Evaluate an expression that returns JSON (the §4.5 sliceable- - * introspection idiom). */ -async function evalJson(connected: Connected, projectDir: string, expression: string): Promise { - const r = await repl(connected, { action: "eval", projectDir, code: `JSON.stringify(${expression})` }); - assert.ok(!isErrorResult(r), textOf(r)); - const sc = structuredOf(r); - assert.ok(typeof sc.result === "string", `the eval resolved with a value: ${JSON.stringify(sc)}`); - return JSON.parse(sc.result as string); -} - -function isErrorResult(res: Awaited>): boolean { - return (res as { isError?: boolean }).isError === true; -} - -async function tick(): Promise { - await new Promise((resolve) => setTimeout(resolve, 0)); -} - -test("review2: the fused eval pumps a settled call's continuation INTO THE SAME held call — console output drained by the pumps renders immediately (the v1 wait's same-shape guarantee, fused into eval)", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const connected = await connectWithRepl(runner); - try { - // The eval awaits the subagent; the continuation logs guest-visible - // output when it settles. The tool holds the call open pumping — the - // turn completes mid-hold and the SAME call renders the output. - const held = repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'const p = agent("pi/x", "task").then((v) => { console.log("continuation ran:", v); return v; }); await p', - timeoutMs: 5000, - }); - for (let attempt = 0; attempt < 100 && runner.sessions.length === 0; attempt++) { - await new Promise((resolve) => setTimeout(resolve, 10)); - } - assert.equal(runner.sessions.length, 1, "the founding session opened"); - runner.last().completeTurn("waited result"); - const r = await held; - assert.ok(!isErrorResult(r), textOf(r)); - const sc = structuredOf(r); - assert.equal(sc.output, "continuation ran: waited result", `output rendered: ${JSON.stringify(sc)}`); - assert.equal(sc.result, "waited result"); - } finally { - await connected.dispose(); - } -}); - -test("review2: workspace() renders the workspace manifest as plain data — bindings with structure-only types, provenance, and live-handle status", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const connected = await connectWithRepl(runner); - try { - const r = await repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'globalThis.findings = { zekret: "MARKER".repeat(10), n: 1 }; globalThis.n = 3; globalThis.research = agent("pi/x", "investigate"); "done"', - }); - assert.ok(!isErrorResult(r), textOf(r)); - await tick(); - const ws = (await evalJson(connected, PROJECT, "workspace()")) as { - bindings: Array<{ - name: string; - type: string; - sizeBytes: number; - provenance: string | null; - task: string | null; - callId?: string; - status?: string; - }>; - inFlight: string[]; - }; - const findings = ws.bindings.find((b) => b.name === "findings"); - assert.ok(findings, JSON.stringify(ws.bindings)); - assert.equal(findings.type, "object", "the structure-only type"); - assert.ok(!JSON.stringify(findings).includes("MARKER"), "no value content in the manifest"); - assert.ok(!JSON.stringify(findings).includes("zekret"), "no nested names leak"); - // EVERY binding carries its byte size — primitives included. - const n = ws.bindings.find((b) => b.name === "n"); - assert.equal(n?.type, "number"); - assert.ok((n?.sizeBytes ?? 0) > 0, `primitive size: ${JSON.stringify(n)}`); - // The doc's full provenance surface: which eval produced the value - // and the live-handle status + call id. - const research = ws.bindings.find((b) => b.name === "research"); - assert.equal(research?.type, "agent handle"); - assert.equal(research?.callId, "c1"); - assert.equal(research?.status, "pending"); - assert.equal(research?.provenance, "eval 1"); - assert.equal(research?.task, "investigate", "the founding task text"); - assert.deepEqual(ws.inFlight, ["c1"]); - // agents(): the live agent with its state and task. - const agents = (await evalJson(connected, PROJECT, "agents()")) as Array<{ callId: string; task: string; state: string }>; - assert.equal(agents.length, 1); - assert.equal(agents[0].callId, "c1"); - assert.equal(agents[0].task, "investigate"); - assert.equal(agents[0].state, "running"); - // The handle settles: the live-handle status follows (the honest - // settled — the call store is the authority). - runner.last().completeTurn("DUG-UP"); - await tick(); - await repl(connected, { action: "eval", projectDir: PROJECT, code: "await research" }); - const wsAfter = (await evalJson(connected, PROJECT, "workspace()")) as { - bindings: Array<{ name: string; status?: string }>; - }; - assert.equal(wsAfter.bindings.find((b) => b.name === "research")?.status, "settled", `settled status: ${JSON.stringify(wsAfter.bindings)}`); - assert.ok(!JSON.stringify(wsAfter).includes("DUG-UP"), "worker content never enters the manifest"); - } finally { - await connected.dispose(); - } -}); - -test("review2: concurrent first touches create exactly ONE VM and broker for a project (the single-flight lock)", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const presence = new ReplPresenceLedger(60_000); - const connected = await connectWithRepl(runner, { presence }); - try { - // Park the backend open so the first touch is slow: every concurrent - // first-touch eval must share the single in-flight firstTouch promise. - let releaseOpen!: () => void; - const parkedOpen = new Promise((resolve) => { - releaseOpen = resolve; - }); - const originalOpen = runner.openSession.bind(runner); - runner.openSession = async (opts) => { - await parkedOpen; - return originalOpen(opts); - }; - const results = await Promise.all( - [1, 2, 3].map((i) => - repl(connected, { - action: "eval", - projectDir: PROJECT, - code: `const p${i} = agent("pi/x", "t${i}"); globalThis.n = ${i}`, - }), - ), - ); - for (const result of results) assert.ok(!isErrorResult(result), textOf(result)); - releaseOpen(); - await tick(); - // All three dispatches ran through the ONE broker (three sessions), - // and the ONE provenance registry saw exactly three eval passes — if - // a second VM had been created and abandoned, its passes would have - // landed on a different registry and the eval sequence would be short. - assert.equal(runner.openedWith.length, 3, "all dispatches were served"); - const ws = (await evalJson(connected, PROJECT, "workspace()")) as { - bindings: Array<{ provenance: string | null }>; - }; - assert.ok( - ws.bindings.some((b) => b.provenance === "eval 3"), - `the three passes landed on one registry: ${JSON.stringify(ws.bindings)}`, - ); - } finally { - await connected.dispose(); - } -}); - -test("review2: last-client disconnect drains in-flight turns to completion and closes idle children; the next queued turn lazily re-attaches", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const presence = new ReplPresenceLedger(60_000); - const connected = await connectWithRepl(runner, { presence, clientId: () => "client-A" }); - try { - const r = await repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'const p = agent("pi/x", "task"); "started"', - }); - assert.ok(!isErrorResult(r), textOf(r)); - await tick(); - // The client's last connection closes: the project's workspace drains - // (the in-flight turn completes and settles; then the child closes). - presence.disconnect("client-A"); - await new Promise((resolve) => setTimeout(resolve, 20)); - const session = runner.last(); - session.completeTurn("drained result"); - await new Promise((resolve) => setTimeout(resolve, 20)); - // The drain settles the result into the VM (the continuation sees it) - // and closes the child. - for (let attempt = 0; attempt < 100 && session.releases === 0; attempt++) { - await new Promise((resolve) => setTimeout(resolve, 10)); - } - assert.equal(session.releases, 1, "the idle child closed after the drain"); - const ws = (await evalJson(connected, PROJECT, "workspace()")) as { - diagnostics: { childrenClosed: boolean }; - }; - assert.equal(ws.diagnostics.childrenClosed, true, "children closed after the drain"); - // The next connect (a new client) evaluates: the continuation already - // fired; an explicit queued turn lazily re-attaches the recorded session. - const probe = await repl(connected, { action: "eval", projectDir: PROJECT, code: 'p.queue("again"); "fired"' }); - assert.ok(!isErrorResult(probe), textOf(probe)); - await tick(); - assert.equal(runner.loadedWith.length, 1, "the recorded session was loaded lazily on the next connect"); - assert.equal(runner.loadedWith[0].sessionId, session.sessionId, "the SAME backend session"); - // SECOND DISCONNECT (phase-D review regression): the re-attached child - // is warm again. The project-level drain latch reset when the client - // reconnected (touch), so this disconnect must drain the re-attached - // child too — the latch used to skip every later drain permanently, - // leaving the reattached child running. - const reattached = runner.last(); - presence.disconnect("client-A"); - // The queued turn started on the reattached session: the - // drain waits for it to complete (drain-to-completion is the policy), - // then closes the reattached child. - await new Promise((resolve) => setTimeout(resolve, 20)); - reattached.completeTurn("second-drain result"); - for (let attempt = 0; attempt < 100 && reattached.releases === 0; attempt++) { - await new Promise((resolve) => setTimeout(resolve, 10)); - } - assert.equal(reattached.releases, 1, "the re-attached child closed after the SECOND disconnect"); - const ws2 = (await evalJson(connected, PROJECT, "workspace()")) as { - diagnostics: { childrenClosed: boolean }; - }; - assert.equal(ws2.diagnostics.childrenClosed, true, "children closed after the second drain"); - } finally { - await connected.dispose(); - } -}); - -test("review2: the per-eval deadline bounds a CURRENTLY running runaway eval through the daemon wiring (the eval-timeout env knob)", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const prev = process.env.AGENTPRISM_REPL_EVAL_TIMEOUT_MS; - process.env.AGENTPRISM_REPL_EVAL_TIMEOUT_MS = "200"; - try { - const server = createWorkflowServer(okRunner(), { replRunner: runner }); - const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair(); - const client = new Client({ name: "mcp-repl-review2", version: "0.0.0" }, { capabilities: {} }); - await Promise.all([server.connect(serverTransport), client.connect(clientTransport)]); - try { - const runaway = await repl( - { client, server, async dispose() {} }, - { action: "eval", projectDir: PROJECT, code: "while (true) {}" }, - ); - assert.ok(!isErrorResult(runaway), textOf(runaway)); - assert.ok(textOf(runaway).includes("interrupted"), `the deadline broke the runaway eval: ${textOf(runaway)}`); - const after = await repl( - { client, server, async dispose() {} }, - { action: "eval", projectDir: PROJECT, code: "6 * 7" }, - ); - assert.ok(textOf(after).includes("result: 42"), `the VM stayed usable: ${textOf(after)}`); - } finally { - await client.close(); - await server.close(); - } - } finally { - if (prev === undefined) delete process.env.AGENTPRISM_REPL_EVAL_TIMEOUT_MS; - else process.env.AGENTPRISM_REPL_EVAL_TIMEOUT_MS = prev; - } -}); - -test("review6: a client reconnecting mid-drain ABORTS the drain through the daemon wiring — the child stays warm while any client is connected, nothing is cancelled or released", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const presence = new ReplPresenceLedger(60_000); - const connected = await connectWithRepl(runner, { presence, clientId: () => "client-A" }); - try { - const r = await repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'const p = agent("pi/x", "task"); "started"', - }); - assert.ok(!isErrorResult(r), textOf(r)); - await tick(); - // The last client disconnects: the drain starts (the in-flight turn - // is still running). - presence.disconnect("client-A"); - await new Promise((resolve) => setTimeout(resolve, 20)); - const session = runner.last(); - assert.equal(session.releases, 0, "the drain is waiting for the in-flight turn — the child is still warm"); - // A client RECONNECTS mid-drain (the next tool call touches - // presence): the drain must ABORT — the release phase must never - // close children while any client is connected (phase-D review - // round 6: the drain used to run to completion regardless of - // presence). - const probe = await repl(connected, { action: "eval", projectDir: PROJECT, code: '"back"' }); - assert.ok(!isErrorResult(probe), textOf(probe)); - assert.equal(session.releases, 0, "the drain aborted — the child was NOT released"); - assert.equal(session.cancelCalls, 0, "nothing was cancelled"); - // The still-running turn completes normally after the abort and - // settles into the live workspace. - session.completeTurn("warm result"); - for (let attempt = 0; attempt < 100; attempt++) { - const got = await repl(connected, { action: "eval", projectDir: PROJECT, code: "await p" }); - if (structuredOf(got).result === "warm result") break; - if (attempt === 99) assert.fail(`the turn never settled: ${textOf(got)}`); - await new Promise((resolve) => setTimeout(resolve, 10)); - } - const ws = (await evalJson(connected, PROJECT, "workspace()")) as { - diagnostics: { childrenClosed: boolean }; - }; - assert.equal(ws.diagnostics.childrenClosed, false, "the workspace is warm (the drain aborted)"); - } finally { - await connected.dispose(); - } -}); - -test("review6: a failed client-presence drain gets the §6.2 [C]14 one-line notice in the next eval's output and is retained until the next drain succeeds — the snapshot-flush failure never vanishes", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const presence = new ReplPresenceLedger(60_000); - const connected = await connectWithRepl(runner, { presence, clientId: () => "client-A" }); - try { - const r = await repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'const p = agent("pi/x", "task"); "started"', - }); - assert.ok(!isErrorResult(r), textOf(r)); - await tick(); - // Sabotage the snapshot write: a DIRECTORY at the tmp path makes the - // drain's atomic write fail (EISDIR). - const paths = workflowProjectPaths(PROJECT); - const tmpPath = join(paths.rootDir, "repl", "snapshot.bin.tmp"); - mkdirSync(tmpPath); - // The last client disconnects: the drain runs, the in-flight turn - // completes, and the drain's settlement flush FAILS. The failure - // must not vanish (phase-D review round 6: the ledger used to - // swallow it) — it is recorded on the state and the NEXT eval's - // output carries the one-line notice (the failure lost state). - presence.disconnect("client-A"); - await new Promise((resolve) => setTimeout(resolve, 20)); - const session = runner.last(); - session.completeTurn("drained but not persisted"); - // Wait for the drain op to finish (single-flight ends in the - // finally), then remove the obstruction: the drain failed, the - // boundary stayed dirty, and the NEXT flush retries the SAME state. - for (let attempt = 0; attempt < 100 && presence.drainingCount() > 0; attempt++) { - await new Promise((resolve) => setTimeout(resolve, 10)); - } - assert.equal(presence.drainingCount(), 0, "the drain op finished (and failed)"); - assert.equal(session.releases, 1, "the release phase ran before the flush failure"); - rmSync(tmpPath, { recursive: true, force: true }); - // The next eval succeeds (its end-of-op flush retries the retained - // boundary) and its output leads with the drain-failure notice; the - // call's settlement survived the failed flush in the live VM. - const probe = await repl(connected, { action: "eval", projectDir: PROJECT, code: "await p" }); - assert.ok(!isErrorResult(probe), textOf(probe)); - const sc = structuredOf(probe); - assert.equal(sc.result, "drained but not persisted", `the drained settlement survived the failed flush in the live VM: ${JSON.stringify(sc)}`); - assert.ok( - String(sc.output).includes("client-presence drain failed"), - `the drain failure is surfaced as the one-line notice: ${String(sc.output)}`, - ); - assert.ok(String(sc.output).includes("not persisted"), `the notice names the lost state: ${String(sc.output)}`); - // §6.2: the retained drain error lives under - // workspace().diagnostics.drainError — the demoted diagnostics home - // (the broker's own internal drain failures are retained there, and - // the tool layer pushes ITS observation of the rethrown flush - // failure into the same record). - const diag = (await evalJson(connected, PROJECT, "workspace().diagnostics")) as { - drainError: { message: string } | null; - }; - assert.ok( - diag.drainError !== null && diag.drainError.message.includes("EISDIR"), - `the failed drain is retained in workspace().diagnostics: ${JSON.stringify(diag.drainError)}`, - ); - // The notice is consumed exactly once: a further eval is clean. - const clean = await repl(connected, { action: "eval", projectDir: PROJECT, code: '"clean"' }); - assert.equal(structuredOf(clean).output, "", "the notice was rendered once"); - // The next disconnect retries the drain: the broker's latch says the - // release already completed, so the retry finishes the bookkeeping - // and clears the recorded failure — the NEXT eval after that carries - // no notice. - presence.disconnect("client-A"); - for (let attempt = 0; attempt < 100 && presence.drainingCount() > 0; attempt++) { - await new Promise((resolve) => setTimeout(resolve, 10)); - } - const afterRetry = await repl(connected, { action: "eval", projectDir: PROJECT, code: '"after retry"' }); - assert.equal(structuredOf(afterRetry).output, "", "the retry cleared the failure — no new notice"); - const ws = (await evalJson(connected, PROJECT, "workspace()")) as { - diagnostics: { childrenClosed: boolean }; - }; - assert.equal(ws.diagnostics.childrenClosed, true, "children closed after the retried drain"); - } finally { - await connected.dispose(); - } -}); - -test("review8: daemon shutdown cleanup runs in FINALLY paths — when broker.dispose REJECTS (its op-end flush retries the retained dirty boundary from the failed drain and fails again), the workspace VM is still disposed and the store still closed (phase-D review round 8)", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const presence = new ReplPresenceLedger(60_000); - const projects = new WorkflowProjectRegistry(okRunner()); - const connected = await connectWithRepl(runner, { presence, projects }); - const tmpPath = join(workflowProjectPaths(PROJECT).rootDir, "repl", "snapshot.bin.tmp"); - try { - const r = await repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'const p = agent("pi/x", "task"); "started"', - }); - assert.ok(!isErrorResult(r), textOf(r)); - await tick(); - const session = runner.last(); - session.hangCancel = true; - session.hangRelease = true; - const projectContext = projects.stores().find((c) => c.projectDir === PROJECT)!; - const state = projectContext.repl!; - const workspace = state.workspace!; - const callStore = state.store.callStore(); - // Obstruct the snapshot write: a DIRECTORY at the tmp path makes the - // atomic write fail (EISDIR). The shutdown drain's op-end flush - // fails → the drain rejects AND the dirty boundary is RETAINED; the - // disposal's own op-end flush RETRIES the same boundary and fails - // again → broker.dispose REJECTS (phase-D review round 8: a - // disposal rejection used to skip workspace.dispose() and - // state.store.close() — the actual VM and call store stayed open - // even though state.workspace was already nulled, and the registry - // swallows the rejection at shutdown, so the cleanup must not - // depend on the disposal resolving). - mkdirSync(tmpPath); - const started = Date.now(); - const teardown = projects.disposeReplStates(300); - const result = await Promise.race([ - teardown.then(() => "done"), - new Promise((resolve) => setTimeout(() => resolve("HUNG"), 3000)), - ]); - const elapsed = Date.now() - started; - assert.equal(result, "done", "daemon shutdown returned within the bound (never hung)"); - assert.ok(elapsed < 2000, `shutdown was bounded: ${elapsed} ms`); - // The cleanup ran in the FINALLY path despite the disposal - // rejection: the VM was ACTUALLY disposed and the store closed. - assert.equal(workspace.isDisposed, true, "the workspace VM was disposed even though broker.dispose rejected"); - assert.equal(callStore.isClosed(), true, "the call store was closed even though broker.dispose rejected"); - assert.equal(state.broker, null, "the broker was disposed"); - assert.equal(state.workspace, null, "the workspace was nulled"); - } finally { - try { - rmSync(tmpPath, { recursive: true, force: true }); - } catch { - // Best-effort cleanup. - } - await connected.dispose(); - } -}); - -test("review8: the shutdown drain FAILS BEFORE releasing sessions — the bounded disposal still sees the hung cancel/release and returns within the remaining bound (phase-D review round 8: the old regression's EISDIR failed only at the op-end flush, AFTER the release phase had already cleared this.sessions, so broker.dispose saw no hung backend and the old unbounded disposal passed)", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const presence = new ReplPresenceLedger(60_000); - const projects = new WorkflowProjectRegistry(okRunner()); - const connected = await connectWithRepl(runner, { presence, projects }); - try { - const r = await repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'const p = agent("pi/x", "task"); "started"', - }); - assert.ok(!isErrorResult(r), textOf(r)); - await tick(); - const session = runner.last(); - session.hangCancel = true; - session.hangRelease = true; - const projectContext = projects.stores().find((c) => c.projectDir === PROJECT)!; - const state = projectContext.repl!; - const workspace = state.workspace!; - // Sabotage the CALL STORE: the bound-expired drain's forced stop - // records the AGENT_CANCELLED completion FIRST (the exactly-once - // discipline) — with the log closed, that record throws (EBADF), so - // the drain FAILS at the forced stop, BEFORE the release phase has - // cleared this.sessions. The disposal therefore still holds the - // hung session and must cancel/release it BOUNDED — an unbounded - // disposal would hang on the exact hung backend the drain had - // already caught. - state.store.close(); - const started = Date.now(); - const teardown = projects.disposeReplStates(300); - const result = await Promise.race([ - teardown.then(() => "done"), - new Promise((resolve) => setTimeout(() => resolve("HUNG"), 3000)), - ]); - const elapsed = Date.now() - started; - assert.equal(result, "done", "daemon shutdown returned within the bound (never hung on the hung cancel/release)"); - assert.ok(elapsed < 2000, `shutdown was bounded: ${elapsed} ms`); - // The disposal still held the busy session (the drain failed BEFORE - // its release phase) and issued its wire calls even though it could - // not await them (the deadline won the race): cancellation is deduplicated - // per active turn, and the release comes from disposal because the drain's - // release phase never ran. - assert.ok(session.cancelCalls <= 1, `the active turn received at most one ACP cancel (got ${session.cancelCalls})`); - assert.equal(session.releases, 1, "the bounded disposal issued the release for the still-registered session"); - // The state was fully torn down — and the VM actually disposed - // (the finally-path cleanup). - assert.equal(workspace.isDisposed, true, "the workspace VM was disposed"); - assert.equal(state.broker, null, "the broker was disposed"); - assert.equal(state.workspace, null, "the workspace was nulled"); - } finally { - await connected.dispose(); - } -}); - -/** A small bounded poll (this file's tests park promises by hand). */ -async function waitFor(predicate: () => boolean, timeoutMs = 5000): Promise { - const start = Date.now(); - while (!predicate()) { - if (Date.now() - start > timeoutMs) throw new Error("waitFor: condition not met in time"); - await new Promise((resolve) => setTimeout(resolve, 10)); - } -} - -test("review8b: a concurrent first touch during a parked restore-time loadSession AWAITS the in-flight reconcile (never bypasses to partially restored state), and daemon teardown during the parked load releases the late session with no stale restore report (phase-D review rejection: ensureReplWorkspace checked state.workspace before state.firstTouch, and doFirstTouch published source/reconcileReport without generation-checking reconcile completion)", async () => { - const PROJECT = freshProject(); - // Phase 1 — seed a stored snapshot with a pending call whose backend - // session is recorded (the restore's re-attach key). - const runner1 = new FakeRunner(); - const first = await connectWithRepl(runner1); - try { - const r = await repl(first, { - action: "eval", - projectDir: PROJECT, - code: 'const p = agent("pi/x", "task"); "started"', - }); - assert.ok(!isErrorResult(r), textOf(r)); - await tick(); - assert.equal(runner1.sessions.length, 1, "the founding session opened"); - assert.ok(existsSync(join(workflowProjectPaths(PROJECT).rootDir, "repl", "snapshot.bin")), "the eval boundary snapshot exists"); - } finally { - await first.dispose(); - } - - // Phase 2 — a fresh daemon: the first touch restores the workspace and - // the restore-time loadSession PARKS (never resolves on its own). - const runner2 = new FakeRunner(); - const presence = new ReplPresenceLedger(60_000); - const projects = new WorkflowProjectRegistry(okRunner()); - const second = await connectWithRepl(runner2, { presence, projects }); - let loadCalls = 0; - let resolveLoad!: () => void; - const parkedLoad = new Promise((resolve) => { - resolveLoad = resolve; - }); - const loadedSessions: FakeSession[] = []; - const originalLoad = runner2.loadSession.bind(runner2); - runner2.loadSession = async (opts) => { - loadCalls++; - await parkedLoad; - const session = await originalLoad(opts); - loadedSessions.push(session); - return session; - }; - try { - // The first touch: restore + reconcile, parked in the re-attach load. - const touch1 = repl(second, { action: "eval", projectDir: PROJECT, code: '"probe 1"' }); - await waitFor(() => loadCalls === 1); - const projectContext = projects.stores().find((c) => c.projectDir === PROJECT)!; - const state = projectContext.repl!; - - // A CONCURRENT first touch must AWAIT the in-flight first-touch - // promise — it must NOT return early through the workspace fast path - // (the old ordering checked state.workspace before state.firstTouch, - // so the concurrent call observed the partially restored workspace - // and evaluated against it). - let touch2Settled = false; - void repl(second, { action: "eval", projectDir: PROJECT, code: '"probe 2"' }).then( - () => { - touch2Settled = true; - }, - () => { - touch2Settled = true; - }, - ); - await new Promise((resolve) => setTimeout(resolve, 60)); - assert.equal(touch2Settled, false, "the concurrent first touch awaited the in-flight restore reconcile (never bypassed to the partially restored workspace)"); - - // Daemon teardown WHILE the restore-time load is parked: the - // shutdown drain force-stops the opening re-attach (the call settles - // durably as AGENT_CANCELLED — it is in the opening-call registry - // now, so the bound's forced stop covers it exactly like an - // openSession), and the bounded disposal runs unlocked at its - // deadline — shutdown returns within the bound. - const started = Date.now(); - await projects.disposeReplStates(300); - const elapsed = Date.now() - started; - assert.ok(elapsed < 2500, `daemon shutdown was bounded while the restore load was parked: ${elapsed} ms`); - assert.equal(state.workspace, null, "the workspace was torn down"); - assert.equal(state.broker, null, "the broker was torn down"); - assert.equal(state.source, null, "no source was published before the reconciliation completed"); - - // The parked load lands AFTER the teardown: the child is released - // exactly once — never registered, never re-issued — and the - // first-touch continuation must NOT write a stale source/report onto - // the torn-down state (the generation check aborts the touch; the - // broker's own disposal fence released the session). - resolveLoad(); - await waitFor(() => loadedSessions.length === 1); - assert.equal(loadedSessions[0].releases, 1, "the late-loaded session was released exactly once"); - assert.equal(loadedSessions[0].prompts.length, 0, "the late-loaded session never prompted"); - assert.equal(runner2.openedWith.length, 0, "no re-issue — no fresh session was opened"); - assert.equal(state.workspace, null, "the workspace stayed torn down"); - assert.equal(state.broker, null, "the broker stayed torn down"); - assert.equal(state.source, null, "no stale restore source on the torn-down state"); - assert.equal(state.reconcileReport, null, "no stale reconcile report on the torn-down state"); - - // The first touch settles LOUDLY (aborted by the teardown) — never a - // successful eval against torn-down state. - const [touch1Result] = await Promise.allSettled([touch1]); - assert.equal(touch1Result.status, "fulfilled", "the abort is surfaced as an error result"); - assert.ok( - isErrorResult((touch1Result as PromiseFulfilledResult).value), - "the aborted first touch is an error result", - ); - assert.ok( - textOf((touch1Result as PromiseFulfilledResult<{ content: unknown[] }>).value).includes("aborted by reset/dispose"), - "the abort names the teardown loudly", - ); - } finally { - await second.dispose(); - } -}); - -test("review-rejection: reset during a parked restore-time loadSession DETACHES the stale first-touch flight — a fresh touch after the reset starts a new workspace instead of awaiting the never-resolving promise forever (phase-D review rejection: reset/dispose left the parked firstTouch in place, and the generation check ran only after broker.reconcile() resolved)", async () => { - const PROJECT = freshProject(); - // Phase 1 — seed a stored snapshot with a pending call whose backend - // session is recorded (the restore's re-attach key). - const runner1 = new FakeRunner(); - const first = await connectWithRepl(runner1); - try { - const r = await repl(first, { - action: "eval", - projectDir: PROJECT, - code: 'const p = agent("pi/x", "task"); "started"', - }); - assert.ok(!isErrorResult(r), textOf(r)); - await tick(); - assert.equal(runner1.sessions.length, 1, "the founding session opened"); - } finally { - await first.dispose(); - } - - // Phase 2 — a fresh daemon whose restore-time loadSession NEVER - // resolves: the first touch parks in the reconcile. - const runner2 = new FakeRunner(); - const presence = new ReplPresenceLedger(60_000); - const projects = new WorkflowProjectRegistry(okRunner()); - const second = await connectWithRepl(runner2, { presence, projects }); - let loadCalls = 0; - let resolveLoad!: () => void; - const parkedLoad = new Promise((resolve) => { - resolveLoad = resolve; - }); - const loadedSessions: FakeSession[] = []; - const originalLoad = runner2.loadSession.bind(runner2); - runner2.loadSession = async (opts) => { - loadCalls++; - await parkedLoad; - const session = await originalLoad(opts); - loadedSessions.push(session); - return session; - }; - try { - // The first touch: restore + reconcile, parked in the re-attach load. - const touch1 = repl(second, { action: "eval", projectDir: PROJECT, code: '"probe 1"' }); - await waitFor(() => loadCalls === 1); - const projectContext = projects.stores().find((c) => c.projectDir === PROJECT)!; - const state = projectContext.repl!; - assert.ok(state.firstTouch !== null, "the first touch is parked in the restore reconcile"); - - // RESET while the touch is parked: the bounded disposal completes, - // and the stale first-touch flight is DETACHED (the old code left it - // in place — every subsequent touch returned the never-resolving - // promise and hung forever). The reset is driven directly with a - // short bound (the `reset()` guest function runs the same path with - // the default shutdown bound). - await resetReplProjectState(state, 300); - assert.equal(state.firstTouch, null, "the stale first-touch flight was detached by the reset"); - assert.equal(state.workspace, null, "the workspace was torn down"); - assert.equal(state.broker, null, "the broker was torn down"); - - // A FRESH touch after the reset starts a NEW first touch (a fresh - // workspace — the repl/ store was cleared) instead of awaiting the - // stale parked flight forever. - const fresh = await Promise.race([ - repl(second, { action: "eval", projectDir: PROJECT, code: "6 * 7" }).then((r) => ({ ok: true as const, r })), - new Promise<{ ok: false }>((resolve) => setTimeout(() => resolve({ ok: false }), 3000)), - ]); - assert.ok(fresh.ok, "the fresh touch after the reset completed — it did not await the stale parked flight forever"); - assert.ok(!isErrorResult(fresh.r), textOf(fresh.r)); - assert.ok(textOf(fresh.r).includes("result: 42"), `the fresh workspace evaluated: ${textOf(fresh.r)}`); - assert.equal(state.source, "fresh", "the fresh touch created a new workspace"); - - // The parked load lands even later: the OLD broker's disposal fence - // releases the child exactly once — never registered, never - // re-issued — and the stale touch aborts loudly (its rejection is - // marked handled by the detach — never an unhandled rejection). - resolveLoad(); - await waitFor(() => loadedSessions.length === 1); - assert.equal(loadedSessions[0].releases, 1, "the late-loaded session was released by the old broker's disposal fence"); - assert.equal(loadedSessions[0].prompts.length, 0, "the late-loaded session never prompted"); - assert.equal(runner2.openedWith.length, 0, "no re-issue — no fresh session was opened"); - const [touch1Result] = await Promise.allSettled([touch1]); - assert.equal(touch1Result.status, "fulfilled", "the stale touch settled when its parked load landed"); - assert.ok( - isErrorResult((touch1Result as PromiseFulfilledResult).value), - "the stale touch aborted as an error result", - ); - assert.ok( - textOf((touch1Result as PromiseFulfilledResult<{ content: unknown[] }>).value).includes("aborted by reset/dispose"), - "the stale touch names the teardown loudly", - ); - } finally { - await second.dispose(); - } -}); diff --git a/packages/mcp-server/test/repl-stdio-relay-worker.test.ts b/packages/mcp-server/test/repl-stdio-relay-worker.test.ts deleted file mode 100644 index 703acde7..00000000 --- a/packages/mcp-server/test/repl-stdio-relay-worker.test.ts +++ /dev/null @@ -1,112 +0,0 @@ -/** - * Phase-F review round 4 pins for the in-process relay worker - * (`repl-stdio-relay-worker.ts`): - * - * - the STREAMING UTF-8 decoder: a multibyte character split across two - * reads survives intact (the old per-chunk `Buffer.toString("utf8")` - * replaced the split character with U+FFFD, so the claimed - * byte-identical MCP forwarding was false for multibyte payloads — - * the built-server repro changed an expected string length), - * - the relay KEY resolution: an interrupt that OMITS projectDir fires - * with the single-project server's own project key, verbatim (the - * repl tool resolves the omitted projectDir to the registry's adopted - * default context, whose projectDir the broker registers as-is); - * an explicit projectDir is realpath'd exactly like the daemon's - * project validation. - * - * Importing the module in the main thread is safe: the stdin pump only - * runs when `parentPort` is present (i.e. inside the worker thread). - */ - -import assert from "node:assert/strict"; -import { mkdtempSync, mkdirSync, rmSync, symlinkSync } from "node:fs"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; -import { test } from "node:test"; - -import { RelayFrameSplitter, relayBreakKey } from "../src/repl-stdio-relay-worker.js"; - -test("the frame splitter decodes a MULTIBYTE character split across reads intact (round 4: per-chunk decoding corrupted it with U+FFFD)", () => { - const lines: string[] = []; - const splitter = new RelayFrameSplitter((line) => lines.push(line)); - // A JSON-RPC frame whose payload contains 4-byte emoji, delivered - // ONE BYTE AT A TIME — every multibyte character straddles reads. - const emoji = "\u{1F600}"; - const code = `"${emoji}${emoji}${emoji}" ('x'.repeat(3)).length; "${emoji}".repeat(200)`; - const frame = JSON.stringify({ - jsonrpc: "2.0", - method: "tools/call", - params: { name: "repl", arguments: { action: "eval", projectDir: "/tmp/w", code } }, - }); - const bytes = Buffer.from(frame, "utf8"); - for (const byte of bytes) splitter.push(Buffer.from([byte])); - splitter.end(); - assert.equal(lines.length, 1, "the split frame is emitted as exactly one line"); - assert.equal(lines[0], frame, "the decoded line is byte-identical to the original frame text"); - assert.ok(!lines[0].includes("\uFFFD"), "no replacement characters anywhere in the decoded frame"); - // The decoded frame is the TRUE payload: the JSON parses, and the - // guest-facing string is verbatim (the round-4 repro: a corrupted - // decode changed an expected JavaScript string length of 40001 to - // 40003). - const parsed = JSON.parse(lines[0]) as { - params?: { arguments?: { code?: string } }; - }; - assert.equal(parsed.params!.arguments!.code, code, "the split multibyte payload decodes verbatim"); -}); - -test("the splitter flushes a final unterminated frame at EOF and emits lines split across arbitrary chunk boundaries", () => { - const lines: string[] = []; - const splitter = new RelayFrameSplitter((line) => lines.push(line)); - const a = '{"jsonrpc":"2.0","method":"ping","id":1}\n'; - const b = '{"jsonrpc":"2.0","method":"ping","id":2}'; // no trailing newline - const stream = Buffer.from(a + b, "utf8"); - // Two frames delivered in awkward halves (the newline falls inside - // the second push, the final frame is unterminated until EOF). - const half = Math.floor(stream.length / 2); - splitter.push(stream.subarray(0, half)); - splitter.push(stream.subarray(half)); - splitter.end(); - assert.deepEqual(lines, [a.trim(), b], "each complete line is emitted; EOF flushes the final unterminated frame"); -}); - -test("the relay key for an OMITTED projectDir is the server's own project key, verbatim (round 4)", () => { - const home = mkdtempSync(join(tmpdir(), "agentprism-repl-relay-key-")); - try { - const realDir = join(home, "real"); - const symDir = join(home, "linked"); - mkdirSync(realDir, { recursive: true }); - symlinkSync(realDir, symDir, "dir"); - // The single-project default context's projectDir is adopted - // VERBATIM (the broker registers it as-is; the tool's omitted- - // projectDir resolution returns `stores()[0].projectDir` raw) — so - // even a symlinked cwd must be posted as-is, not realpath'd. - assert.equal(relayBreakKey(undefined, symDir), symDir, "the omitted projectDir fires with the verbatim default key"); - assert.equal(relayBreakKey(undefined, realDir), realDir); - // No default key available (daemon mode, or no adopted context): - // the call cannot be keyed — the relay is skipped. - assert.equal(relayBreakKey(undefined, undefined), undefined); - assert.equal(relayBreakKey(undefined, ""), ""); - } finally { - rmSync(home, { recursive: true, force: true }); - } -}); - -test("the relay key for an EXPLICIT projectDir is realpath'd exactly like the daemon's project validation", () => { - const home = mkdtempSync(join(tmpdir(), "agentprism-repl-relay-key-")); - try { - const realDir = join(home, "real-project"); - const symDir = join(home, "linked-project"); - mkdirSync(realDir, { recursive: true }); - symlinkSync(realDir, symDir, "dir"); - assert.equal(relayBreakKey(symDir, undefined), realDir, "a symlinked projectDir realpaths to the canonical key"); - assert.equal(relayBreakKey(realDir, undefined), realDir); - // Non-absolute and unresolvable paths cannot be keyed (the server's - // own validation refuses the call). - assert.equal(relayBreakKey("relative/path", undefined), undefined); - assert.equal(relayBreakKey(join(home, "does-not-exist"), undefined), undefined); - assert.equal(relayBreakKey(42, undefined), undefined); - assert.equal(relayBreakKey(null, undefined), undefined); - } finally { - rmSync(home, { recursive: true, force: true }); - } -}); diff --git a/packages/mcp-server/test/repl-stdio-transport.test.ts b/packages/mcp-server/test/repl-stdio-transport.test.ts deleted file mode 100644 index b49e2073..00000000 --- a/packages/mcp-server/test/repl-stdio-transport.test.ts +++ /dev/null @@ -1,98 +0,0 @@ -/** - * Phase-F review round 4 pins for `ReplRelayStdioTransport.send`: - * SEND COMPLETION MEANS FLUSHED — a `write()` that reports - * backpressure (`false`) must not resolve until the stdout `drain` - * event fires, exactly like the `StdioServerTransport` this transport - * replaces (the old fire-and-forget write resolved immediately, - * allowing unbounded buffering against a slow client and violating - * send-completion semantics for all in-process MCP traffic). - * - * Only `send` is exercised here: `start()` spawns the stdin-reader - * worker, which must never run inside the test process (it owns fd 0). - */ - -import assert from "node:assert/strict"; -import { EventEmitter } from "node:events"; -import { test } from "node:test"; - -import { ReplRelayStdioTransport, type ReplRelayStdioSink } from "../src/repl-stdio-transport.js"; - -/** A controllable stdout seam: reports backpressure on demand and lets - * the test fire `drain`/`error` events. */ -class FakeSink extends EventEmitter implements ReplRelayStdioSink { - written: string[] = []; - drainBlocked = false; - - write(chunk: string): boolean { - this.written.push(chunk); - return !this.drainBlocked; - } -} - -function transportFor(sink: FakeSink): ReplRelayStdioTransport { - // The breakUrl source is never consulted without start() — send() - // alone never touches the worker or the channel. - return new ReplRelayStdioTransport( - () => Promise.resolve("http://127.0.0.1:0/break"), - () => undefined, - sink, - ); -} - -test("send resolves immediately when stdout accepts the frame", async () => { - const sink = new FakeSink(); - const transport = transportFor(sink); - const message = { jsonrpc: "2.0" as const, method: "notifications/initialized" }; - let settled = false; - const send = transport.send(message).then(() => { - settled = true; - }); - // No drain event needed: a successful write resolves synchronously - // (well, microtask-wise) — drainBlocked is false, so no listener is - // ever attached. - await send; - assert.equal(settled, true, "the send completed"); - assert.deepEqual(sink.written, [`${JSON.stringify(message)}\n`], "the frame is written verbatim with its newline"); -}); - -test("send WAITS FOR DRAIN when stdout reports backpressure — completion means flushed (round 4)", async () => { - const sink = new FakeSink(); - sink.drainBlocked = true; - const transport = transportFor(sink); - const message = { jsonrpc: "2.0" as const, method: "notifications/cancelled", params: {} }; - let settled = false; - const send = transport.send(message).then(() => { - settled = true; - }); - // Give any (wrong) immediate resolution a chance to surface: the - // write returned false, so the promise must still be pending. - await new Promise((resolvePromise) => setImmediate(resolvePromise)); - assert.equal(settled, false, "the send is pending while the stream is backpressured"); - // The drain event releases it — exactly the SDK semantics. - sink.emit("drain"); - await send; - assert.equal(settled, true, "the send completed once the stream drained"); - assert.deepEqual(sink.written, [`${JSON.stringify(message)}\n`], "the frame was written once"); -}); - -test("backpressure releases only its OWN drain: an earlier drain never resolves a later send early", async () => { - const sink = new FakeSink(); - sink.drainBlocked = true; - const transport = transportFor(sink); - const first = { jsonrpc: "2.0" as const, method: "notifications/initialized" }; - const second = { jsonrpc: "2.0" as const, method: "notifications/initialized" }; - let firstDone = false; - let secondDone = false; - const send1 = transport.send(first).then(() => { - firstDone = true; - }); - const send2 = transport.send(second).then(() => { - secondDone = true; - }); - await new Promise((resolvePromise) => setImmediate(resolvePromise)); - sink.emit("drain"); - await send1; - await send2; - assert.equal(firstDone && secondDone, true, "each send resolves on the drain"); - assert.equal(sink.written.length, 2, "both frames were written"); -}); diff --git a/packages/mcp-server/test/repl-tool.test.ts b/packages/mcp-server/test/repl-tool.test.ts deleted file mode 100644 index 3c38e04e..00000000 --- a/packages/mcp-server/test/repl-tool.test.ts +++ /dev/null @@ -1,1373 +0,0 @@ -/** - * The `repl` tool's surface suite (eval-plane redesign, the roadmap - * bible docs/roadmap/repl-eval-redesign.md §3): the per-project context - * opens the daemon's repl store, attaches the broker's snapshot sink, - * and on FIRST TOUCH restores the stored workspace + reconciles — or - * AUTO-RESETS a refused snapshot (§6.1). Pins: - * - * - a fresh workspace persists across "daemon restarts" (a second server - * over the same HOME restores the VM from the enveloped snapshot), - * - the two-action surface: eval (soft-bound fused pump) and interrupt, - * - the soft-bound eval's finished shape { output, result? } — one call - * when the awaited call settles within the bound — and the honest - * still-running shape { output, running } when the bound elapses, - * - the empty-string eval (the documented idempotent poll) drains what - * settled, - * - a refused stored snapshot AUTO-RESETS: the file is renamed aside - * (`.refused-`, never deleted) and the next eval's output leads - * with the one-line notice naming the file and the reason, - * - the three-way reconcile runs through the tool on restore, - * - interrupt: cancel by id; eval-break without id (refused-idle when - * nothing is running), - * - the §4.5 guest introspection functions: workspace(), agents(), - * reset() — ordinary values, sliceable in the same eval. - */ - -import assert from "node:assert/strict"; -import { existsSync, mkdtempSync, readFileSync, readdirSync, renameSync, rmSync, writeFileSync } from "node:fs"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; -import { test } from "node:test"; -import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; -import { - deserializeSnapshot, - loadShippedWasm, - serializeSnapshot, - SNAPSHOT_FORMAT_VERSION, - wasmSha256Of, - type BrokerLoadSessionOptions, - type BrokerOpenSessionOptions, - type BrokerPromptOptions, - type BrokerRunner, - type BrokerSession, - type BrokerTurn, -} from "@automatalabs/repl-engine"; -import { workflowProjectPaths } from "@automatalabs/workflows"; - -import { createWorkflowServer, renameAsideNeverOverwriting, replToolOutputShape } from "../src/index.js"; -import { WorkflowProjectRegistry } from "../src/project-registry.js"; -import { okRunner, textOf, type Connected } from "./_harness.js"; - -/** The fake held-open ACP session (the broker's structural seam). */ -class FakeSession implements BrokerSession { - readonly sessionId: string; - initializeMeta: Readonly> | undefined; - readonly prompts: Array<{ content: string; resolve: (turn: BrokerTurn) => void; reject: (error: unknown) => void }> = []; - readonly steers: Array<{ content: string; resolve: (outcome: unknown) => void; reject: (error: unknown) => void }> = []; - releases = 0; - cancelCalls = 0; - stopReason = "end_turn"; - readonly completedTexts: string[] = []; - /** The re-attach seam's scripted loaded-turn outcome (null parks it). */ - loadedTurnTextValue: string | null = null; - - constructor(readonly openedWith: BrokerOpenSessionOptions | BrokerLoadSessionOptions) { - this.sessionId = `fake-session-${FakeSession.nextId++}`; - this.initializeMeta = { steering: { supported: true } }; - } - - static nextId = 0; - - prompt(content: string, opts: BrokerPromptOptions = {}): Promise { - return new Promise((resolve, reject) => { - this.prompts.push({ content, resolve, reject }); - opts.onHandoff?.(); - }); - } - - steer(content: string): Promise { - return new Promise((resolve, reject) => { - this.steers.push({ content, resolve, reject }); - }); - } - - awaitCurrentTurn(): Promise { - if (this.loadedTurnTextValue !== null) { - return Promise.resolve({ stopReason: this.stopReason, text: this.loadedTurnTextValue }); - } - return new Promise(() => {}); - } - - cancel(): Promise { - this.cancelCalls++; - for (const pending of this.prompts.splice(0)) { - pending.resolve({ stopReason: "cancelled", text: "" }); - } - return Promise.resolve(); - } - - release(): Promise { - this.releases++; - return Promise.resolve(); - } - - currentTurnText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ""; - } - - finalMessageText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ""; - } - - rawStructuredOutput(): unknown { - return undefined; - } - - completeTurn(text: string): void { - const pending = this.prompts.shift(); - assert.ok(pending, "a prompt turn must be in flight"); - this.completedTexts.push(text); - pending.resolve({ stopReason: this.stopReason, text }); - } -} - -/** The fake runner with the loadSession seam. */ -class FakeRunner implements BrokerRunner { - readonly sessions: FakeSession[] = []; - readonly openedWith: BrokerOpenSessionOptions[] = []; - readonly loadedWith: BrokerLoadSessionOptions[] = []; - /** The scripted loaded-turn outcome (null parks the seam). */ - loadedTurnText: string | null = null; - - listBackends(): string[] { - return ["pi"]; - } - - defaultBackendId(): string { - return "pi"; - } - - async openSession(opts: BrokerOpenSessionOptions): Promise { - const session = new FakeSession(opts); - this.sessions.push(session); - this.openedWith.push(opts); - return session; - } - - async loadSession(opts: BrokerLoadSessionOptions): Promise { - const session = new FakeSession(opts); - session.loadedTurnTextValue = this.loadedTurnText; - this.sessions.push(session); - this.loadedWith.push(opts); - return session; - } - - async dispose(): Promise {} - - last(): FakeSession { - assert.ok(this.sessions.length > 0, "a session must exist"); - return this.sessions[this.sessions.length - 1]; - } -} - -/** A real project directory (resolveProjectDir realpaths it). */ -function freshProject(): string { - return mkdtempSync(join(tmpdir(), "repl-tool-project-")); -} - -/** The repl store's snapshot path for the test project under the harness HOME. */ -function replStorePaths(projectDir: string): { snapshotPath: string; replDir: string } { - const paths = workflowProjectPaths(projectDir); - return { snapshotPath: join(paths.rootDir, "repl", "snapshot.bin"), replDir: join(paths.rootDir, "repl") }; -} - -/** Connect a workflow server with an injected repl runner (single-project mode). */ -async function connectWithRepl( - replRunner: BrokerRunner, - options: { projects?: WorkflowProjectRegistry } = {}, -): Promise { - const server = createWorkflowServer(okRunner(), { replRunner, projects: options.projects }); - const [clientTransport, serverTransport] = InMemoryTransport.createLinkedPair(); - const client = new Client({ name: "mcp-repl-test", version: "0.0.0" }, { capabilities: {} }); - await Promise.all([server.connect(serverTransport), client.connect(clientTransport)]); - return { - client, - server, - runner: replRunner as FakeRunner, - projects: options.projects, - async dispose() { - await client.close(); - await server.close(); - }, - }; -} - -/** Call the repl tool (typed over the raw input). */ -async function repl( - connected: Connected, - input: { action: string; projectDir?: string; code?: string; timeoutMs?: number; id?: string }, -) { - return connected.client.callTool({ name: "repl", arguments: input as Record }); -} - -function structuredOf(res: Awaited>): Record { - return (res as { structuredContent?: Record }).structuredContent ?? {}; -} - -function isErrorResult(res: Awaited>): boolean { - return (res as { isError?: boolean }).isError === true; -} - -async function tick(): Promise { - await new Promise((resolve) => setTimeout(resolve, 0)); -} - -/** Evaluate an expression that returns JSON — the sliceable-introspection - * idiom (§4.5: workspace()/agents() return ordinary values, sliceable in - * the same eval). */ -async function evalJson(connected: Connected, projectDir: string, expression: string): Promise { - const r = await repl(connected, { action: "eval", projectDir, code: `JSON.stringify(${expression})` }); - assert.ok(!isErrorResult(r), textOf(r)); - const sc = structuredOf(r); - assert.ok(typeof sc.result === "string", `the eval resolved with a value: ${JSON.stringify(sc)}`); - return JSON.parse(sc.result as string); -} - -// ── The surface shapes ──────────────────────────────────────────────── - -test("the live MCP description teaches strict steer/queue handle retention and contains no followUp guidance", async () => { - const connected = await connectWithRepl(new FakeRunner()); - try { - const listed = await connected.client.listTools(); - const description = listed.tools.find((tool) => tool.name === "repl")?.description ?? ""; - assert.match(description, /persistent promise-handle/); - assert.match(description, /a\.steer\(text\).*only the currently running turn/); - assert.match(description, /a\.queue\(text\).*distinct FIFO turn/); - assert.match(description, /q\.cancel\(\).*exact turn/); - assert.match(description, /Steering while idle returns/); - assert.ok(!description.includes("followUp"), description); - } finally { - await connected.dispose(); - } -}); - -test("eval returns the finished shape { output, result? } mirrored in structuredContent — one newline-joined output string, no v1 metadata fields", async () => { - const PROJECT = freshProject(); - const connected = await connectWithRepl(new FakeRunner()); - try { - const r = await repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'console.log("line one"); console.error("boom"); 40 + 2', - }); - assert.ok(!isErrorResult(r), textOf(r)); - const sc = structuredOf(r); - // The wire shape is EXACTLY { output, result? } — nothing else. - assert.deepEqual(Object.keys(sc).sort(), ["output", "result"]); - assert.equal(sc.output, 'line one\nerror: boom'); - assert.equal(sc.result, "42"); - assert.ok(textOf(r).includes("result: 42"), textOf(r)); - // The v1 metadata fields are deleted from the wire. - for (const field of ["pending", "completed", "checkpoints", "outputTruncated", "truncated", "referenced", "action", "projectDir", "drained", "timedOut", "workspaces", "dropped"]) { - assert.ok(!(field in sc), `no ${field} on the wire`); - } - } finally { - await connected.dispose(); - } -}); - -test("an empty eval resolves with result \"undefined\" — the documented poll idiom runs as a normal eval", async () => { - const PROJECT = freshProject(); - const connected = await connectWithRepl(new FakeRunner()); - try { - const r = await repl(connected, { action: "eval", projectDir: PROJECT, code: "" }); - assert.ok(!isErrorResult(r), textOf(r)); - const sc = structuredOf(r); - assert.deepEqual(Object.keys(sc).sort(), ["output", "result"]); - assert.equal(sc.output, ""); - assert.equal(sc.result, "undefined"); - } finally { - await connected.dispose(); - } -}); - -test("the output shape models finished and still-running EXACTLY: result and running are mutually exclusive — { output, result } finished, { output, running } bound-elapsed, { output } a thrown eval, and the v1 fields are gone", () => { - const finished = replToolOutputShape.safeParse({ output: "printed", result: "42" }); - assert.equal(finished.success, true, JSON.stringify(finished)); - const running = replToolOutputShape.safeParse({ output: "printed", running: ["c1"] }); - assert.equal(running.success, true, JSON.stringify(running)); - const thrown = replToolOutputShape.safeParse({ output: "Error: boom\n at :1" }); - assert.equal(thrown.success, true, "a thrown eval ships { output } alone"); - const both = replToolOutputShape.safeParse({ output: "", result: "x", running: ["c1"] }); - assert.equal(both.success, false, "result and running together are never a valid eval result"); - const resultOnlyNoOutput = replToolOutputShape.safeParse({ result: "x" }); - assert.equal(resultOnlyNoOutput.success, false, "the output string is required on every eval variant"); - const runningWithInterrupt = replToolOutputShape.safeParse({ output: "", running: ["c1"], interrupt: { outcome: "idle" } }); - assert.equal(runningWithInterrupt.success, false, "an eval variant never carries the interrupt outcome"); - // The error variant carries EXACTLY the bare error key: `error`+ - // `result` and `error`+`running` are invalid, exactly like the - // runtime shapes (§3.1 [C]1 — the published schema mirrors the - // runtime validator). - const errOnly = replToolOutputShape.safeParse({ error: "boom" }); - assert.equal(errOnly.success, true, "the bare error variant"); - const errResult = replToolOutputShape.safeParse({ error: "boom", result: "x" }); - assert.equal(errResult.success, false, "error+result is never a valid result"); - const errRunning = replToolOutputShape.safeParse({ error: "boom", running: ["c1"] }); - assert.equal(errRunning.success, false, "error+running is never a valid result"); - const errOutput = replToolOutputShape.safeParse({ error: "boom", output: "" }); - assert.equal(errOutput.success, false, "error+output is never a valid result"); - // The PUBLISHED schema (what the server advertises through - // `outputSchema`) mirrors the same rule: the error oneOf branch - // excludes every other key. - const published = replToolOutputShape.toJSONSchema(); - const publishedError = ( - published.oneOf as Array<{ title?: string; not?: { anyOf?: Array<{ required?: string[] }> } }> - ).find((branch) => branch.title === "error"); - const excluded = (publishedError?.not?.anyOf ?? []).map((item) => item.required?.[0] ?? ""); - assert.deepEqual( - excluded.sort(), - ["interrupt", "output", "result", "running"], - "the published error branch excludes output, interrupt, result AND running", - ); -}); - -test("the soft-bound eval: everything the code waits on settles within the bound → the FINISHED shape in ONE call (the v1 eval→wait→eval loop fused)", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const connected = await connectWithRepl(runner); - try { - // The eval awaits the subagent; the tool holds the call open pumping - // settlements. The turn completes mid-hold → the SAME call returns - // the finished shape with the completion value's repr. - const held = repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'const p = agent("pi/x", "research task"); const answer = await p; console.log("got it"); answer', - timeoutMs: 5000, - }); - for (let attempt = 0; attempt < 100 && runner.sessions.length === 0; attempt++) { - await new Promise((resolve) => setTimeout(resolve, 10)); - } - assert.equal(runner.sessions.length, 1, "the founding session opened"); - runner.last().completeTurn("the answer"); - const r = await held; - assert.ok(!isErrorResult(r), textOf(r)); - const sc = structuredOf(r); - assert.deepEqual(Object.keys(sc).sort(), ["output", "result"], "the finished shape has no running"); - assert.equal(sc.output, "got it"); - assert.equal(sc.result, "the answer", "the completion value's repr"); - assert.ok(textOf(r).includes("result: the answer"), textOf(r)); - // The value is live in the VM — `_` holds it (the §4.4 result-history - // global). - const underscore = await repl(connected, { action: "eval", projectDir: PROJECT, code: "_" }); - assert.equal(structuredOf(underscore).result, "the answer"); - } finally { - await connected.dispose(); - } -}); - -test("the soft-bound eval: the bound elapses first → the STILL-RUNNING shape { output, running } and a later eval drains what settled", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const connected = await connectWithRepl(runner); - try { - const held = repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'const p = agent("pi/x", "slow task"); const v = await p; "result:" + v', - timeoutMs: 300, - }); - await tick(); - const r = await held; - assert.ok(!isErrorResult(r), textOf(r)); - const sc = structuredOf(r); - assert.deepEqual(Object.keys(sc).sort(), ["output", "running"], "the still-running shape has no result"); - assert.equal(sc.output, ""); - assert.deepEqual(sc.running, ["c1"], "the in-flight call ids"); - assert.ok(textOf(r).includes("running: c1"), textOf(r)); - // The eval CONTINUES server-side: the turn settles after the bound - // and the next eval picks the value up. - runner.last().completeTurn("slow answer"); - const picked = await repl(connected, { action: "eval", projectDir: PROJECT, code: "await p" }); - assert.ok(!isErrorResult(picked), textOf(picked)); - assert.equal(structuredOf(picked).result, "slow answer"); - // `_` holds the late completion too. - assert.equal(structuredOf(await repl(connected, { action: "eval", projectDir: PROJECT, code: "_" })).result, "slow answer"); - } finally { - await connected.dispose(); - } -}); - -test("chain contention: the soft-bound eval still reports the KNOWN in-flight ids when a serialized operation holds the broker through the whole remaining bound (§3.1 [D]3/[C]1 — never an empty running surface)", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const registry = new WorkflowProjectRegistry(okRunner()); - const connected = await connectWithRepl(runner, { projects: registry }); - try { - const held = repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'const p = agent("pi/x", "slow task"); const v = await p; "result:" + v', - timeoutMs: 500, - }); - // A concurrent serialized operation — the client-presence drain's - // yieldful pump loop — holds the broker's chain through the WHOLE - // remaining bound (its pumps interleave sleeps, so the wait's - // deadline expires while the chain stays busy). The wait can never - // read the pending surface: the tool must report the KNOWN ids the - // eval suspended with, never the empty unreadable read. - // - // The held call must have completed its first touch (the broker is - // attached) before the drain starts holding the chain — poll the - // registry instead of a single tick (a loaded parallel suite can - // starve the touch past one event-loop turn). - const context = registry.getOrCreate(PROJECT); - let broker = context.repl?.broker ?? null; - for (let attempt = 0; attempt < 100 && broker === null; attempt++) { - await new Promise((resolve) => setTimeout(resolve, 10)); - broker = context.repl?.broker ?? null; - } - assert.ok(broker, "the touched project state has a broker"); - const draining = broker.drainForDisconnect(3000, () => false); - const r = await held; - assert.ok(!isErrorResult(r), textOf(r)); - const sc = structuredOf(r); - assert.deepEqual(Object.keys(sc).sort(), ["output", "running"], "the still-running shape has no result"); - assert.deepEqual(sc.running, ["c1"], "the KNOWN in-flight ids — the contention must never degrade running to []"); - assert.ok(textOf(r).includes("running: c1"), textOf(r)); - // The drain finishes its bound (it cancels the in-flight call at its - // own forced stop — after the held call already returned). - await draining; - } finally { - await connected.dispose(); - } -}); - -test("the empty-eval poll picks up the LATE COMPLETION VALUE of an eval that exceeded its soft bound — result carries the drained repr, never the poll's own undefined", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const connected = await connectWithRepl(runner); - try { - const held = repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'const p = agent("pi/x", "slow task"); const v = await p; console.log("late:", v); "late-value-" + v', - timeoutMs: 300, - }); - await tick(); - const r = await held; - assert.ok(!isErrorResult(r), textOf(r)); - const sc = structuredOf(r); - assert.deepEqual(Object.keys(sc).sort(), ["output", "running"], "the held call returned the still-running shape"); - assert.deepEqual(sc.running, ["c1"]); - // The turn settles AFTER the bound elapsed — the held call is gone - // and its token-keyed settlement has no reader. The empty eval - // drains the settled continuation AND picks the timed-out eval's - // completion value up as ITS result (§3.1 [C]3). - runner.last().completeTurn("slow answer"); - await tick(); - const poll = await repl(connected, { action: "eval", projectDir: PROJECT, code: "" }); - assert.ok(!isErrorResult(poll), textOf(poll)); - const pollSc = structuredOf(poll); - assert.equal(pollSc.output, "late: slow answer", "the poll drained the late console output"); - assert.equal(pollSc.result, "late-value-slow answer", "the poll picked up the timed-out eval's completion repr"); - // Idempotent: the settlement was claimed once — the next empty eval - // drains nothing new and reports its own undefined. - const again = await repl(connected, { action: "eval", projectDir: PROJECT, code: "" }); - assert.equal(structuredOf(again).output, "", "the second poll drains nothing new"); - assert.equal(structuredOf(again).result, "undefined", "no late settlement left — the poll's own undefined"); - } finally { - await connected.dispose(); - } -}); - -test("the fused pump reports the FINISHED shape the moment the eval's own work settles — an unrelated long-running call from an earlier eval never holds the finished shape to the bound", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const connected = await connectWithRepl(runner); - try { - // An unrelated call started by an EARLIER eval (start-and-don't-await) - // stays in flight for the whole test. - const started = await repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'const slow = agent("pi/x", "long research"); "started"', - }); - assert.ok(!isErrorResult(started), textOf(started)); - for (let attempt = 0; attempt < 100 && runner.sessions.length === 0; attempt++) { - await new Promise((resolve) => setTimeout(resolve, 10)); - } - assert.equal(runner.sessions.length, 1, "the unrelated session opened"); - // THIS eval awaits its OWN call under a long bound. Only its own - // call settles — the held call must return the finished shape - // promptly, not pump until the unrelated call drains. - const heldStartedAt = Date.now(); - const held = repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'const p = agent("pi/x", "my task"); const answer = await p; "mine:" + answer', - timeoutMs: 5000, - }); - for (let attempt = 0; attempt < 100 && runner.sessions.length < 2; attempt++) { - await new Promise((resolve) => setTimeout(resolve, 10)); - } - assert.equal(runner.sessions.length, 2, "the eval's own session opened"); - runner.last().completeTurn("my answer"); - const r = await held; - assert.ok(!isErrorResult(r), textOf(r)); - const sc = structuredOf(r); - assert.deepEqual(Object.keys(sc).sort(), ["output", "result"], "the finished shape — never the still-running shape"); - assert.equal(sc.result, "mine:my answer", "the completion value's repr"); - assert.ok( - Date.now() - heldStartedAt < 2000, - `the finished shape returned promptly (${Date.now() - heldStartedAt} ms), not at the 5 s bound`, - ); - // The unrelated call was untouched — still in flight, and it still - // settles normally later. - assert.equal(runner.sessions.length, 2, "no cancel/reissue of the unrelated call"); - runner.sessions[0].completeTurn("slow result"); - const slowRead = await repl(connected, { action: "eval", projectDir: PROJECT, code: "await slow" }); - assert.equal(structuredOf(slowRead).result, "slow result", "the unrelated call settled normally"); - } finally { - await connected.dispose(); - } -}); - -test("the empty-eval poll (the documented idempotent idiom) drains what settled without re-executing work", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const connected = await connectWithRepl(runner); - try { - // Start-and-don't-await: the eval completes immediately (finished - // shape); the call keeps running server-side. - const started = await repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'const p = agent("pi/x", "task").then((v) => console.log("settled:", v)); "started"', - }); - assert.ok(!isErrorResult(started), textOf(started)); - assert.equal(structuredOf(started).result, "started"); - await tick(); - assert.equal(runner.sessions.length, 1, "the founding session opened"); - // The turn settles AFTER the eval returned. The empty eval drains - // and reports the settled continuation's console output. - runner.last().completeTurn("waited result"); - await tick(); - const poll = await repl(connected, { action: "eval", projectDir: PROJECT, code: "" }); - assert.ok(!isErrorResult(poll), textOf(poll)); - const sc = structuredOf(poll); - assert.equal(sc.output, "settled: waited result", "the poll drained the settled output"); - assert.equal(sc.result, "undefined", "the poll itself completes with undefined"); - // Idempotent: re-sending the empty eval re-executes nothing. - const again = await repl(connected, { action: "eval", projectDir: PROJECT, code: "" }); - assert.equal(structuredOf(again).output, "", "the second poll drains nothing new"); - } finally { - await connected.dispose(); - } -}); - -test("a raised checkpoint renders as an output line; checkpoint.answer in a later eval resolves it", async () => { - const PROJECT = freshProject(); - const connected = await connectWithRepl(new FakeRunner()); - try { - const raised = await repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'const q = checkpoint("what color?"); "asked"', - }); - assert.ok(!isErrorResult(raised), textOf(raised)); - const sc = structuredOf(raised); - assert.ok(sc.output.includes("checkpoint c1: what color?"), `the checkpoint line: ${sc.output}`); - assert.equal(sc.result, "asked"); - const answered = await repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'checkpoint.answer("c1", "blue"); await q', - }); - assert.equal(structuredOf(answered).result, "blue", "the answer resolves the parked promise"); - } finally { - await connected.dispose(); - } -}); - -test("an uncaught eval error renders in output with the §4.6 attribution and the call succeeds", async () => { - const PROJECT = freshProject(); - const connected = await connectWithRepl(new FakeRunner()); - try { - const r = await repl(connected, { action: "eval", projectDir: PROJECT, code: "throw new Error('nope')" }); - assert.ok(!isErrorResult(r), "a throwing eval is honest output, not a tool error"); - const sc = structuredOf(r); - assert.ok(String(sc.output).includes("Error: nope"), `the error rendering: ${sc.output}`); - assert.ok(!("result" in sc), "no completion value for an error"); - } finally { - await connected.dispose(); - } -}); - -// ── Durability: persistence, restore, auto-reset ────────────────────── - -test("repl eval persists to the daemon's per-project store; a later server restores the workspace from the snapshot", async () => { - const PROJECT = freshProject(); - const first = await connectWithRepl(new FakeRunner()); - try { - const r = await repl(first, { action: "eval", projectDir: PROJECT, code: "globalThis.answer = 42; answer + 1" }); - assert.ok(!isErrorResult(r), textOf(r)); - assert.ok(textOf(r).includes("result: 43"), textOf(r)); - // The eval boundary wrote the enveloped snapshot into the repl store - // (next to the workflow state, under workflowHomeDir()/projects//). - const { snapshotPath, replDir } = replStorePaths(PROJECT); - assert.ok(existsSync(replDir), `repl/ dir exists: ${replDir}`); - assert.ok(existsSync(snapshotPath), `snapshot written: ${snapshotPath}`); - const header = readFileSync(snapshotPath).subarray(0, 200).toString("utf8"); - assert.ok(header.includes('"format":"repl-snapshot"'), `enveloped: ${header}`); - assert.ok(header.includes('"wasmSha256":"'), `identity carried: ${header}`); - } finally { - await first.dispose(); - } - - // "Daemon restart": a fresh server over the same HOME (and a fresh - // runner) — the first touch restores the VM from the stored snapshot. - const second = await connectWithRepl(new FakeRunner()); - try { - const r = await repl(second, { action: "eval", projectDir: PROJECT, code: "answer" }); - assert.ok(!isErrorResult(r), textOf(r)); - assert.ok(textOf(r).includes("result: 42"), `state survived the restart: ${textOf(r)}`); - // The restore is visible through the guest introspection surface. - const diag = (await evalJson(second, PROJECT, "workspace().diagnostics")) as { reconcile: unknown }; - assert.ok(diag.reconcile !== null, "the restore's reconcile report lives in diagnostics"); - } finally { - await second.dispose(); - } -}); - -test("format-2 snapshots auto-reset before guest execution and the fresh workspace runs format 3 / guest 0.5", async () => { - const PROJECT = freshProject(); - const first = await connectWithRepl(new FakeRunner()); - try { - const seeded = await repl(first, { - action: "eval", - projectDir: PROJECT, - code: 'globalThis.preRedesignBinding = "must not survive"; 41', - }); - assert.equal(structuredOf(seeded).result, "41", "the real stored snapshot carries user state"); - } finally { - await first.dispose(); - } - - const { snapshotPath, replDir } = replStorePaths(PROJECT); - const currentEnvelope = readFileSync(snapshotPath); - const newline = currentEnvelope.indexOf(0x0a); - assert.ok(newline > 0, "the real snapshot has a newline-terminated envelope header"); - const header = JSON.parse(currentEnvelope.subarray(0, newline).toString("utf8")) as Record; - assert.equal(header.formatVersion, SNAPSHOT_FORMAT_VERSION, "the seed snapshot starts at the running format"); - - // Fabricate the immediately previous format without changing its VM payload. - // The envelope version check must refuse it before restoring or executing old guest code. - const oldEnvelope = Buffer.concat([ - Buffer.from(`${JSON.stringify({ ...header, formatVersion: 2 })}\n`, "utf8"), - currentEnvelope.subarray(newline + 1), - ]); - writeFileSync(snapshotPath, oldEnvelope); - - const second = await connectWithRepl(new FakeRunner()); - try { - const touched = await repl(second, { - action: "eval", - projectDir: PROJECT, - code: 'await sleep(1); [typeof preRedesignBinding, 6 * 7].join(":")', - }); - assert.ok(!isErrorResult(touched), textOf(touched)); - const touchedShape = structuredOf(touched); - const output = touchedShape.output as string; - assert.ok(output.startsWith("REPL workspace auto-reset:"), `the loud notice leads output: ${output}`); - assert.ok(output.includes("snapshot carries format version 2"), `the notice names the old format: ${output}`); - assert.ok( - output.includes(`this engine supports version ${SNAPSHOT_FORMAT_VERSION}`), - `the notice names the running format: ${output}`, - ); - assert.equal(touchedShape.result, "undefined:42", "old bindings are gone and sleep() ran in the fresh 0.5 guest"); - - const entries = readdirSync(replDir); - const refused = entries.filter((name) => name.startsWith("snapshot.bin.refused-")); - assert.equal(refused.length, 1, `the old snapshot was renamed aside exactly once: ${entries.join(", ")}`); - assert.ok(output.includes(refused[0]), `the notice names the refused file: ${output}`); - assert.deepEqual( - readFileSync(join(replDir, refused[0])), - oldEnvelope, - "the refused snapshot bytes were preserved, never deleted or overwritten", - ); - assert.ok(existsSync(snapshotPath), "the fresh workspace persisted a new current-format snapshot"); - - const api = (await evalJson( - second, - PROJECT, - `(() => { - const w = workspace(); - const a = agents(); - return { - workspace: typeof workspace, - workspaceLive: Array.isArray(w.bindings), - agents: typeof agents, - agentsLive: Array.isArray(a), - reset: typeof reset, - resetLive: reset() === undefined, - sleep: typeof sleep, - underscore: _, - }; - })()`, - )) as Record; - assert.deepEqual(api, { - workspace: "function", - workspaceLive: true, - agents: "function", - agentsLive: true, - reset: "function", - resetLive: true, - sleep: "function", - underscore: "undefined:42", - }); - - const afterReset = await repl(second, { action: "eval", projectDir: PROJECT, code: "typeof preRedesignBinding" }); - assert.equal(structuredOf(afterReset).result, "undefined", "reset() was live and opened another fresh format-3 workspace"); - assert.equal(structuredOf(afterReset).output, "", "the refusal notice is emitted exactly once"); - assert.ok(existsSync(join(replDir, refused[0])), "reset() never deletes the renamed-aside snapshot"); - } finally { - await second.dispose(); - } -}); - -test("a pending call with a recorded backend session re-attaches on restore through the tool (reconcile runs at first touch)", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const first = await connectWithRepl(runner); - try { - const r = await repl(first, { action: "eval", projectDir: PROJECT, code: 'const p = agent("pi/x", "task"); "started"' }); - assert.ok(!isErrorResult(r), textOf(r)); - await tick(); - assert.equal(runner.sessions.length, 1, "the founding session opened"); - // The eval boundary snapshot now exists with the call pending in the - // guest registry; the re-attach key was recorded in the store BEFORE - // the prompt (the phase-D contract). - const { snapshotPath } = replStorePaths(PROJECT); - assert.ok(existsSync(snapshotPath)); - } finally { - await first.dispose(); - } - - // Restart: the stored snapshot carries the pending call; the fresh - // runner's loadSession re-attaches it (scripted loaded turn) and the - // SAME guest promise settles exactly once. - const runner2 = new FakeRunner(); - runner2.loadedTurnText = "loaded result"; - const second = await connectWithRepl(runner2); - try { - // The first touch restores + reconciles; the re-attached call settles - // the SAME guest promise exactly once. - const r = await repl(second, { action: "eval", projectDir: PROJECT, code: 'await p.catch((e) => "ERR:" + e.message)' }); - assert.ok(!isErrorResult(r), textOf(r)); - assert.ok(structuredOf(r).result!.includes("loaded result"), textOf(r)); - // §6.2: the reconcile surfacing DEMOTES to diagnostics — the eval - // output carries NO per-call reconciliation lines (only the [C]14 - // aggregate loss notice may ride an eval's output). - assert.equal(structuredOf(r).output, "", "no re-attach line rides the eval output"); - const notes = (await evalJson(second, PROJECT, "workspace().diagnostics.reconcileNotes")) as Array<{ - level: string; - line: string; - }>; - assert.ok( - notes.some((n) => n.level === "info" && n.line.includes("c1") && n.line.includes("re-attached")), - JSON.stringify(notes), - ); - const diag = (await evalJson(second, PROJECT, "workspace().diagnostics")) as { - reconcile: { reattached: string[] } | null; - }; - assert.ok(diag.reconcile !== null && diag.reconcile.reattached.includes("c1"), "the restore re-attached c1"); - assert.equal(runner2.loadedWith.length, 1, "the recorded session was loaded"); - assert.equal(runner2.openedWith.length, 0, "never a fresh session — no re-issue"); - } finally { - await second.dispose(); - } -}); - -test("a corrupted stored snapshot AUTO-RESETS (§6.1): the file is renamed aside (never deleted), the next eval's output leads with the loud notice naming file and reason, and a fresh workspace starts", async () => { - const PROJECT = freshProject(); - const first = await connectWithRepl(new FakeRunner()); - try { - await repl(first, { action: "eval", projectDir: PROJECT, code: "globalThis.doomed = 1" }); - } finally { - await first.dispose(); - } - const { snapshotPath, replDir } = replStorePaths(PROJECT); - assert.ok(existsSync(snapshotPath)); - // Corrupt the stored snapshot (truncate mid-payload). - const bytes = readFileSync(snapshotPath); - writeFileSync(snapshotPath, bytes.subarray(0, Math.floor(bytes.length / 2))); - - // The next daemon's first touch must NOT crash-loop and must NOT - // demand a manual reset: the refused snapshot is renamed aside and a - // fresh workspace starts, with the notice leading the eval's output. - const second = await connectWithRepl(new FakeRunner()); - try { - const r = await repl(second, { action: "eval", projectDir: PROJECT, code: "1 + 1" }); - assert.ok(!isErrorResult(r), textOf(r)); - const sc = structuredOf(r); - const output = sc.output as string; - assert.ok( - output.startsWith("REPL workspace auto-reset:"), - `the notice leads the output: ${output.slice(0, 120)}`, - ); - assert.ok(output.includes("snapshot refused"), output); - // The refused file was RENAMED ASIDE — never deleted. - const entries = readdirSync(replDir); - const refused = entries.filter((name) => name.startsWith("snapshot.bin.refused-")); - assert.equal(refused.length, 1, `exactly one refused snapshot renamed aside: ${entries.join(", ")}`); - assert.ok(output.includes(refused[0]), `the notice names the renamed file: ${output}`); - // The fresh workspace works. - assert.equal(sc.result, "2"); - const fresh = await repl(second, { action: "eval", projectDir: PROJECT, code: "3 + 4" }); - assert.equal(structuredOf(fresh).result, "7"); - // The notice was consumed exactly once — later evals are clean. - assert.equal(structuredOf(fresh).output, ""); - // The fresh workspace persists again (a NEW snapshot was written). - assert.ok(existsSync(snapshotPath), "the fresh workspace re-persisted"); - } finally { - await second.dispose(); - } -}); - -test("a STRUCTURALLY VALID corrupted snapshot (a corrupted in-range VM header that passes every at-rest check) takes the SAME auto-reset path at restore time", async () => { - const PROJECT = freshProject(); - const first = await connectWithRepl(new FakeRunner()); - try { - await repl(first, { action: "eval", projectDir: PROJECT, code: "globalThis.doomed = 1" }); - } finally { - await first.dispose(); - } - const { snapshotPath, replDir } = replStorePaths(PROJECT); - assert.ok(existsSync(snapshotPath)); - // Re-encode the stored snapshot with the VM header's STACK POINTER - // patched to an in-range-but-wrong value (1): the envelope stays fully - // valid, but materializing the VM fails (`RuntimeError: memory access - // out of bounds`) — the corruption class NO at-rest check can see. - const module = await loadShippedWasm(); - const originalBytes = readFileSync(snapshotPath); - const { snapshot } = deserializeSnapshot(originalBytes); - const corrupted = { ...snapshot, stackPointer: 1 }; - writeFileSync(snapshotPath, serializeSnapshot(corrupted, wasmSha256Of(module))); - - const second = await connectWithRepl(new FakeRunner()); - try { - const r = await repl(second, { action: "eval", projectDir: PROJECT, code: "1 + 1" }); - assert.ok(!isErrorResult(r), textOf(r)); - const sc = structuredOf(r); - const output = sc.output as string; - assert.ok(output.startsWith("REPL workspace auto-reset:"), output.slice(0, 120)); - assert.ok( - output.includes("restoring the workspace VM from the snapshot failed") || - output.includes("initializing the restored workspace failed") || - output.includes("memory access out of bounds"), - `names the refusal stage: ${output.slice(0, 300)}`, - ); - const entries = readdirSync(replDir); - assert.equal(entries.filter((name) => name.startsWith("snapshot.bin.refused-")).length, 1, `renamed aside: ${entries.join(", ")}`); - assert.equal(sc.result, "2", "the fresh workspace evaluated"); - } finally { - await second.dispose(); - } -}); - -test("§6.1 [C]13: the auto-reset notice survives a reset() IN THE SAME FIRST EVAL — the notice still leads the output and the renamed-aside refused snapshot is never deleted", async () => { - const PROJECT = freshProject(); - const first = await connectWithRepl(new FakeRunner()); - try { - await repl(first, { action: "eval", projectDir: PROJECT, code: "globalThis.doomed = 1" }); - } finally { - await first.dispose(); - } - const { snapshotPath, replDir } = replStorePaths(PROJECT); - assert.ok(existsSync(snapshotPath)); - const bytes = readFileSync(snapshotPath); - writeFileSync(snapshotPath, bytes.subarray(0, Math.floor(bytes.length / 2))); - - // The FIRST eval after the refused-snapshot auto-reset runs guest - // reset(): the teardown must not erase the pending notice or the - // renamed-aside file (the review finding). - const second = await connectWithRepl(new FakeRunner()); - try { - const r = await repl(second, { action: "eval", projectDir: PROJECT, code: "reset()" }); - assert.ok(!isErrorResult(r), textOf(r)); - const sc = structuredOf(r); - const output = sc.output as string; - assert.ok( - output.startsWith("REPL workspace auto-reset:"), - `the notice leads the output despite the reset(): ${output.slice(0, 120)}`, - ); - const refused = readdirSync(replDir).filter((name) => name.startsWith("snapshot.bin.refused-")); - assert.equal( - refused.length, - 1, - `the renamed-aside refused snapshot survives the reset(): ${readdirSync(replDir).join(", ")}`, - ); - assert.ok(output.includes(refused[0]), `the notice names the renamed file: ${output}`); - // The reset tore the workspace down — the next eval starts fresh, - // the notice was consumed exactly once, and the refused file - // persists even across the fresh workspace's re-persisting. - const fresh = await repl(second, { action: "eval", projectDir: PROJECT, code: "6 * 7" }); - assert.equal(structuredOf(fresh).result, "42", "the fresh workspace works"); - assert.equal(structuredOf(fresh).output, "", "the notice was consumed exactly once"); - assert.equal( - readdirSync(replDir).filter((name) => name.startsWith("snapshot.bin.refused-")).length, - 1, - "the refused snapshot is still renamed aside — never deleted", - ); - } finally { - await second.dispose(); - } -}); - -test("§6.1: the auto-reset CLEARS the call ledger with the snapshot — a nonempty calls.jsonl never leaks into the fresh workspace, whose c1 is minted clean (review finding: the fresh VM restarts ids at c1 and the store's first-wins replay handed the new call the old record and its completion)", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const first = await connectWithRepl(runner); - try { - // c1's record AND its completion land in calls.jsonl before the - // snapshot exists (a real agent call, settled through the tool). - const r = await repl(first, { action: "eval", projectDir: PROJECT, code: 'const p = agent("pi/x", "task"); "started"' }); - assert.ok(!isErrorResult(r), textOf(r)); - await tick(); - assert.equal(runner.sessions.length, 1, "the founding session opened"); - runner.last().completeTurn("OLD ANSWER"); - const settled = await repl(first, { action: "eval", projectDir: PROJECT, code: "await p" }); - assert.equal(structuredOf(settled).result, "OLD ANSWER", "c1 completed before the restart"); - } finally { - await first.dispose(); - } - const { snapshotPath, replDir } = replStorePaths(PROJECT); - const callStorePath = join(replDir, "calls.jsonl"); - assert.ok(existsSync(snapshotPath)); - assert.ok( - existsSync(callStorePath) && readFileSync(callStorePath, "utf8").includes("OLD ANSWER"), - "the call ledger is nonempty", - ); - // Corrupt the stored snapshot: the next daemon's first touch refuses - // and auto-resets — the ledger must go WITH the snapshot. - const bytes = readFileSync(snapshotPath); - writeFileSync(snapshotPath, bytes.subarray(0, Math.floor(bytes.length / 2))); - - const runner2 = new FakeRunner(); - const second = await connectWithRepl(runner2); - try { - const r = await repl(second, { action: "eval", projectDir: PROJECT, code: 'const q = agent("pi/x", "task2"); "fresh-start"' }); - assert.ok(!isErrorResult(r), textOf(r)); - const output = structuredOf(r).output as string; - assert.ok(output.startsWith("REPL workspace auto-reset:"), `the notice leads the output: ${output.slice(0, 120)}`); - // The ledger was cleared with the snapshot: the fresh dispatch - // minted c1 WITHOUT inheriting the old record's completion (the - // first-wins replay kept the old c1 settled on the defect), and - // the log carries only the fresh record. - const agents = (await evalJson(second, PROJECT, "agents()")) as Array<{ callId: string; state: string }>; - assert.equal(agents.length, 1, "one live agent"); - assert.equal(agents[0].callId, "c1", "the fresh ids restart at c1"); - assert.equal(agents[0].state, "running", "the fresh c1 is pending — never the old settled record"); - const log = readFileSync(callStorePath, "utf8"); - assert.ok(!log.includes("OLD ANSWER"), "the old completion is gone from the ledger"); - // The fresh c1 settles with ITS OWN answer, never the old one. - await tick(); - runner2.last().completeTurn("NEW ANSWER"); - const picked = await repl(second, { action: "eval", projectDir: PROJECT, code: "await q" }); - assert.equal(structuredOf(picked).result, "NEW ANSWER", "the fresh c1 settles with its own answer"); - } finally { - await second.dispose(); - } -}); - -test("§6.1 [C]13: renameAsideNeverOverwriting is COLLISION-SAFE — a same-millisecond second refusal bumps a counter suffix instead of overwriting (POSIX renameSync silently replaces an existing destination), and the earlier refused snapshot is never deleted", () => { - const dir = mkdtempSync(join(tmpdir(), "repl-aside-")); - try { - const snapshotPath = join(dir, "snapshot.bin"); - writeFileSync(snapshotPath, "refused-bytes"); - // An earlier refusal already renamed its snapshot aside under the - // SAME millisecond stamp — and the stamp recurs (two refusals in - // one millisecond): the destination must bump, never overwrite. - writeFileSync(`${snapshotPath}.refused-4242`, "first refusal"); - const aside = renameAsideNeverOverwriting(snapshotPath, 4242); - assert.equal(aside, `${snapshotPath}.refused-4242-1`, "the collision bumps a counter suffix"); - renameSync(snapshotPath, aside); - assert.ok(!existsSync(snapshotPath), "the refused snapshot moved aside"); - assert.equal( - readFileSync(`${snapshotPath}.refused-4242`, "utf8"), - "first refusal", - "the earlier refused snapshot is untouched", - ); - assert.equal(readFileSync(aside, "utf8"), "refused-bytes", "the second refusal landed under the bumped name"); - } finally { - rmSync(dir, { recursive: true, force: true }); - } -}); - -test("§6.1 [C]13: TWO auto-resets on one project keep BOTH refused snapshots renamed aside — auto-reset never deletes an earlier aside, even across fresh re-persistence", async () => { - const PROJECT = freshProject(); - const first = await connectWithRepl(new FakeRunner()); - try { - await repl(first, { action: "eval", projectDir: PROJECT, code: "globalThis.doomed = 1" }); - } finally { - await first.dispose(); - } - const { snapshotPath, replDir } = replStorePaths(PROJECT); - const corrupt = (): void => { - const bytes = readFileSync(snapshotPath); - writeFileSync(snapshotPath, bytes.subarray(0, Math.floor(bytes.length / 2))); - }; - corrupt(); - - // Refusal #1: the aside lands, a fresh workspace starts and - // re-persists a NEW snapshot. - const second = await connectWithRepl(new FakeRunner()); - try { - const r1 = await repl(second, { action: "eval", projectDir: PROJECT, code: "1 + 1" }); - assert.ok((structuredOf(r1).output as string).startsWith("REPL workspace auto-reset:"), textOf(r1)); - assert.equal(structuredOf(r1).result, "2"); - assert.ok(existsSync(snapshotPath), "the fresh workspace re-persisted"); - } finally { - await second.dispose(); - } - corrupt(); - - // Refusal #2 (a third daemon): the SECOND aside lands under its own - // name — the first is never overwritten or deleted. - const third = await connectWithRepl(new FakeRunner()); - try { - const r2 = await repl(third, { action: "eval", projectDir: PROJECT, code: "6 * 7" }); - const output2 = structuredOf(r2).output as string; - assert.ok(output2.startsWith("REPL workspace auto-reset:"), `the second refusal notices too: ${output2.slice(0, 120)}`); - const asides = readdirSync(replDir).filter((name) => name.startsWith("snapshot.bin.refused-")); - assert.equal(asides.length, 2, `both refused snapshots renamed aside: ${asides.join(", ")}`); - assert.equal(structuredOf(r2).result, "42"); - } finally { - await third.dispose(); - } -}); - -test("§6.1/§6.2: pending notices survive a THROWING eval — they are consumed only by the first eval result that successfully renders them, never lost on the pump's throwing path (review finding: taking them before the held settlement pump erased them when waitForCalls threw)", async () => { - const PROJECT = freshProject(); - const first = await connectWithRepl(new FakeRunner()); - try { - await repl(first, { action: "eval", projectDir: PROJECT, code: "globalThis.doomed = 1" }); - } finally { - await first.dispose(); - } - const { snapshotPath } = replStorePaths(PROJECT); - const bytes = readFileSync(snapshotPath); - writeFileSync(snapshotPath, bytes.subarray(0, Math.floor(bytes.length / 2))); - - const registry = new WorkflowProjectRegistry(okRunner()); - const connected = await connectWithRepl(new FakeRunner(), { projects: registry }); - try { - // The held eval is the FIRST touch (the auto-reset arms the notice) - // and SUSPENDS: the hold pumps waitForCalls. A CONCURRENT session's - // reset() (driven at the broker seam — the MCP SDK serves one - // transport per server, so the second session speaks through the - // registry's live broker) completes mid-hold and tears the broker - // down, so the held eval's waitForCalls THROWS and it renders NO - // result. The notice must survive that throwing path and lead the - // NEXT successful eval's output. - const held = repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'await new Promise(() => {}); "never"', - timeoutMs: 10_000, - }); - let broker = registry.getOrCreate(PROJECT).repl?.broker ?? null; - for (let attempt = 0; attempt < 100 && broker === null; attempt++) { - await new Promise((resolve) => setTimeout(resolve, 10)); - broker = registry.getOrCreate(PROJECT).repl?.broker ?? null; - } - assert.ok(broker, "the touched project state has a broker"); - const concurrent = await broker.eval("reset()"); - assert.equal(concurrent.result, "undefined", "the concurrent session's reset() ran"); - const r = await held; - assert.ok(isErrorResult(r), "the held eval errored — its workspace was reset mid-hold"); - // The next (fresh) workspace's eval leads with the notice — it was - // never consumed by the throwing eval (the defect: the next eval's - // output was clean, the notice gone). - const next = await repl(connected, { action: "eval", projectDir: PROJECT, code: "1 + 1" }); - assert.equal(structuredOf(next).result, "2"); - const outNext = structuredOf(next).output as string; - assert.ok( - outNext.startsWith("REPL workspace auto-reset:"), - `the notice survived the throwing eval and leads the next successful eval's output: ${outNext.slice(0, 120)}`, - ); - } finally { - await connected.dispose(); - } -}); - -test("§6.2 [C]14: a restore that LOST calls leads the next eval's output with the ONE aggregate notice — the per-call reconcile lines stay in diagnostics", async () => { - const PROJECT = freshProject(); - const runner = new FakeRunner(); - const first = await connectWithRepl(runner); - try { - await repl(first, { action: "eval", projectDir: PROJECT, code: 'const pi = agent("pi/x", "task"); "started"' }); - for (let attempt = 0; attempt < 100 && runner.sessions[0]?.prompts.length !== 1; attempt++) { - await new Promise((resolve) => setTimeout(resolve, 10)); - } - // A steer whose wire call is in flight when the daemon dies is never replayed; - // restore rejects it with steering_interrupted. - const steered = await repl(first, { - action: "eval", - projectDir: PROJECT, - code: 'await pi.steer("deeper")', - timeoutMs: 200, - }); - assert.ok(!isErrorResult(steered), textOf(steered)); - assert.ok("running" in structuredOf(steered), "the steer eval suspended (still-running)"); - } finally { - await first.dispose(); - } - - const second = await connectWithRepl(new FakeRunner()); - try { - const r = await repl(second, { action: "eval", projectDir: PROJECT, code: "1 + 1" }); - assert.ok(!isErrorResult(r), textOf(r)); - const output = structuredOf(r).output as string; - assert.ok( - output.startsWith("restore lost 1 call(s) (c2)"), - `the aggregate loss notice leads the output: ${output.slice(0, 200)}`, - ); - assert.ok( - output.includes("steering_interrupted") || output.includes("interrupted by restart"), - `the recovered steering promise rejects explicitly: ${output}`, - ); - const notes = (await evalJson(second, PROJECT, "workspace().diagnostics.reconcileNotes")) as Array<{ - level: string; - line: string; - }>; - assert.ok( - notes.some((n) => n.level === "warn" && n.line.includes("c2") && n.line.includes("not replayed")), - JSON.stringify(notes), - ); - assert.equal(structuredOf(r).result, "2", "the eval itself ran normally"); - } finally { - await second.dispose(); - } -}); - -// ── Interrupt ───────────────────────────────────────────────────────── - -test("interrupt cancels a call by id; interrupt without an id on an IDLE workspace is the honest refusal; the eval-break signal interrupts a RUNNING eval mid-hold", async () => { - const runner = new FakeRunner(); - const PROJECT = freshProject(); - const connected = await connectWithRepl(runner); - try { - // Cancel by id: the session receives the cancel; the guest promise - // rejects recoverably. - await repl(connected, { action: "eval", projectDir: PROJECT, code: 'const q = agent("pi/x", "task2"); "started"' }); - await tick(); - const interrupted = await repl(connected, { action: "interrupt", projectDir: PROJECT, id: "c1" }); - assert.ok(!isErrorResult(interrupted), textOf(interrupted)); - assert.ok(textOf(interrupted).includes("session/cancel sent"), textOf(interrupted)); - assert.deepEqual(structuredOf(interrupted), { interrupt: { outcome: "cancelled", callId: "c1" } }); - const read = await repl(connected, { action: "eval", projectDir: PROJECT, code: 'await q.catch((e) => "ERR:" + e.message)' }); - assert.ok(String(structuredOf(read).result).includes("cancelled"), textOf(read)); - - // Idle from the start of a FRESH project: no running eval → honest - // refusal, nothing armed; the next eval runs normally. - const otherProject = freshProject(); - const idle = await repl(connected, { action: "interrupt", projectDir: otherProject }); - assert.ok(!isErrorResult(idle), textOf(idle)); - assert.deepEqual(structuredOf(idle), { interrupt: { outcome: "refused-idle" } }); - assert.ok(textOf(idle).includes("no running eval to interrupt"), textOf(idle)); - const r = await repl(connected, { action: "eval", projectDir: otherProject, code: "6 * 7" }); - assert.ok(textOf(r).includes("result: 42"), textOf(r)); - - // The eval-break signal interrupts the RUNNING eval: the fused eval - // holds the call open pumping a suspended eval; the interrupt lands - // mid-hold; when the awaited call settles, the resumed continuation - // (a runaway loop) is broken mid-run by the armed signal. - const eval3 = repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'const p3 = agent("pi/x", "task3"); await p3; while (true) {}', - timeoutMs: 10_000, - }); - await tick(); - await tick(); - const armed = await repl(connected, { action: "interrupt", projectDir: PROJECT }); - assert.ok(!isErrorResult(armed), textOf(armed)); - assert.ok(textOf(armed).includes("interrupting the running eval"), textOf(armed)); - assert.deepEqual(structuredOf(armed), { interrupt: { outcome: "targeted" } }); - // The awaited call settles: the eval's continuation resumes in the - // tool's pump and the armed signal breaks the runaway mid-run. The - // broken eval can never settle — the held eval returns promptly with - // the finished-with-error shape (no result, no running), and the - // interrupted drain is retained in workspace().diagnostics (§6.2). - runner.last().completeTurn("resumed"); - const broken = await eval3; - assert.ok(!isErrorResult(broken), textOf(broken)); - const brokenSc = structuredOf(broken); - assert.ok(!("running" in brokenSc), `not still-running after the break: ${JSON.stringify(brokenSc)}`); - assert.ok(!("result" in brokenSc), `no completion value after the break: ${JSON.stringify(brokenSc)}`); - const diag = (await evalJson(connected, PROJECT, "workspace().diagnostics")) as { drainError: { message: string } | null }; - assert.ok( - diag.drainError !== null && (diag.drainError.message.includes("interrupted") || diag.drainError.message.includes("Job execution error")), - `the interrupted drain is retained in diagnostics: ${JSON.stringify(diag.drainError)}`, - ); - // The signal was consumed by the running eval: the NEXT eval runs - // normally, and the VM stays usable. - const after = await repl(connected, { action: "eval", projectDir: PROJECT, code: "6 * 7" }); - assert.ok(textOf(after).includes("result: 42"), textOf(after)); - const idleAfter = await repl(connected, { action: "interrupt", projectDir: PROJECT }); - assert.deepEqual(structuredOf(idleAfter), { interrupt: { outcome: "refused-idle" } }, "nothing is tracked after the break"); - } finally { - await connected.dispose(); - } -}); - -test("interrupt without an id TERMINATES a running eval suspended on nothing resumable (a never-settling local promise) — the release is reported targeted, the held eval returns promptly with the finished-with-error shape, and refused-idle stays honest only for an idle workspace (§3.2 review finding)", async () => { - const PROJECT = freshProject(); - const connected = await connectWithRepl(new FakeRunner()); - try { - const held = repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'await new Promise(() => {}); "never"', - timeoutMs: 10_000, - }); - await tick(); - await tick(); - // The eval IS running (suspended, `pending: []`) — the no-id - // interrupt must terminate it, never refuse: the tracked - // continuation is released and the outcome is targeted. - const released = await repl(connected, { action: "interrupt", projectDir: PROJECT }); - assert.ok(!isErrorResult(released), textOf(released)); - assert.deepEqual(structuredOf(released), { interrupt: { outcome: "targeted" } }); - assert.ok(textOf(released).includes("terminated outright"), textOf(released)); - // The held eval returns PROMPTLY (no 10 s bound wait) with the - // finished-with-error shape — no result, no running ids. - const r = await held; - assert.ok(!isErrorResult(r), textOf(r)); - const sc = structuredOf(r); - assert.ok(!("result" in sc) && !("running" in sc), `terminated, not still-running: ${JSON.stringify(sc)}`); - // Nothing is running any more: the next no-id interrupt is the - // honest refusal — the ONLY permitted refusal. - const idle = await repl(connected, { action: "interrupt", projectDir: PROJECT }); - assert.deepEqual(structuredOf(idle), { interrupt: { outcome: "refused-idle" } }); - // The workspace stays usable. - const after = await repl(connected, { action: "eval", projectDir: PROJECT, code: "6 * 7" }); - assert.ok(textOf(after).includes("result: 42"), textOf(after)); - } finally { - await connected.dispose(); - } -}); - -// ── The §4.5 guest introspection functions ──────────────────────────── - -test("workspace() returns the plain-data shape (bindings, inFlight, checkpoints, diagnostics) and agents() the live agents — sliceable in the same eval; reset() tears the workspace down", async () => { - const runner = new FakeRunner(); - const PROJECT = freshProject(); - const connected = await connectWithRepl(runner); - try { - const started = await repl(connected, { - action: "eval", - projectDir: PROJECT, - code: 'globalThis.findings = { n: 1 }; globalThis.n = 3; globalThis.research = agent("pi/x", "investigate"); "done"', - }); - assert.ok(!isErrorResult(started), textOf(started)); - await tick(); - // workspace(): plain data — the bindings carry name/type/size/ - // provenance/task and the live-handle status. - const ws = (await evalJson(connected, PROJECT, "workspace()")) as { - bindings: Array<{ name: string; type: string; sizeBytes: number; provenance: string | null; task: string | null; callId?: string; status?: string }>; - inFlight: string[]; - diagnostics: { childrenClosed: boolean }; - }; - const research = ws.bindings.find((b) => b.name === "research"); - assert.ok(research, JSON.stringify(ws.bindings)); - assert.equal(research.type, "agent handle", "the machine-readable type"); - assert.equal(research.callId, "c1", "the stable call id"); - assert.equal(research.status, "pending", "the live-handle status"); - assert.equal(research.task, "investigate", "the task provenance"); - const n = ws.bindings.find((b) => b.name === "n"); - assert.equal(n?.type, "number", "the plain binding type"); - assert.ok((n?.sizeBytes ?? 0) > 0, "every binding carries its size"); - assert.deepEqual(ws.inFlight, ["c1"], "the in-flight ids"); - assert.equal(ws.diagnostics.childrenClosed, false, "children are warm"); - // The intent-plane hygiene rule: metadata only — no value content. - // agents(): the live-agent entries. - const agents = (await evalJson(connected, PROJECT, "agents()")) as Array<{ callId: string; modelSpec: string; state: string }>; - assert.equal(agents.length, 1); - assert.equal(agents[0].callId, "c1"); - assert.equal(agents[0].modelSpec, "pi/x", "the full model spec, verbatim"); - assert.equal(agents[0].state, "running"); - // The handle settles: the status transitions to settled. - runner.last().completeTurn("DUG-UP"); - await tick(); - const picked = await repl(connected, { action: "eval", projectDir: PROJECT, code: "await research" }); - assert.equal(structuredOf(picked).result, "DUG-UP"); - const wsAfter = (await evalJson(connected, PROJECT, "workspace()")) as { - bindings: Array<{ name: string; status?: string }>; - inFlight: string[]; - }; - assert.equal(wsAfter.bindings.find((b) => b.name === "research")?.status, "settled", "the handle status transitioned"); - assert.deepEqual(wsAfter.inFlight, [], "nothing left in flight"); - // reset(): the teardown runs after the eval completes; the next eval - // starts a fresh workspace. - const reset = await repl(connected, { action: "eval", projectDir: PROJECT, code: "reset()" }); - assert.ok(!isErrorResult(reset), textOf(reset)); - const gone = await repl(connected, { action: "eval", projectDir: PROJECT, code: "typeof findings" }); - assert.ok(!isErrorResult(gone), textOf(gone)); - assert.equal(structuredOf(gone).result, "undefined", "the fresh workspace has no bindings"); - } finally { - await connected.dispose(); - } -}); - -// ── The action discriminator ────────────────────────────────────────── - -test("the input is action-discriminated: eval requires code; interrupt rejects code/timeoutMs; the two-action enum rejects everything else", async () => { - const PROJECT = freshProject(); - const connected = await connectWithRepl(new FakeRunner()); - try { - const noCode = await repl(connected, { action: "eval", projectDir: PROJECT }); - assert.ok(isErrorResult(noCode), textOf(noCode)); - assert.ok(textOf(noCode).includes("eval requires a code string"), textOf(noCode)); - const interruptWithCode = await repl(connected, { action: "interrupt", projectDir: PROJECT, code: "1 + 1" }); - assert.ok(isErrorResult(interruptWithCode), textOf(interruptWithCode)); - assert.ok(textOf(interruptWithCode).includes('cannot include code'), textOf(interruptWithCode)); - const interruptWithTimeout = await repl(connected, { action: "interrupt", projectDir: PROJECT, timeoutMs: 100 }); - assert.ok(isErrorResult(interruptWithTimeout), textOf(interruptWithTimeout)); - assert.ok(textOf(interruptWithTimeout).includes('cannot include timeoutMs'), textOf(interruptWithTimeout)); - const evalWithId = await repl(connected, { action: "eval", projectDir: PROJECT, code: "1 + 1", id: "c1" }); - assert.ok(isErrorResult(evalWithId), textOf(evalWithId)); - assert.ok(textOf(evalWithId).includes('cannot include id'), textOf(evalWithId)); - // EVERY key outside the action's exact set is rejected at the wire - // (the strict input schema) and at the discriminator — the deleted - // v1 `refs` parameter and any unknown field fail instead of being - // silently discarded (§3.3 [C]4 / §7). - const evalWithRefs = await repl(connected, { - action: "eval", - projectDir: PROJECT, - code: "", - refs: ["r1"], - } as unknown as { action: string; projectDir: string; code: string }); - assert.ok(isErrorResult(evalWithRefs), textOf(evalWithRefs)); - assert.ok(textOf(evalWithRefs).includes("refs"), `the deleted refs parameter is rejected: ${textOf(evalWithRefs)}`); - const evalWithMystery = await repl(connected, { - action: "eval", - projectDir: PROJECT, - code: "", - mysteryField: 1, - } as unknown as { action: string; projectDir: string; code: string }); - assert.ok(isErrorResult(evalWithMystery), textOf(evalWithMystery)); - assert.ok(textOf(evalWithMystery).includes("mysteryField"), `unknown keys are rejected: ${textOf(evalWithMystery)}`); - const interruptWithRefs = await repl(connected, { - action: "interrupt", - projectDir: PROJECT, - refs: ["r1"], - } as unknown as { action: string; projectDir: string }); - assert.ok(isErrorResult(interruptWithRefs), textOf(interruptWithRefs)); - assert.ok(textOf(interruptWithRefs).includes("refs"), `the interrupt branch rejects refs too: ${textOf(interruptWithRefs)}`); - // The deleted v1 actions refuse at the schema (the wire enum). - for (const dead of ["wait", "status", "reset"]) { - const r = await repl(connected, { action: dead, projectDir: PROJECT }); - assert.ok(isErrorResult(r), textOf(r)); - assert.ok(textOf(r).includes("Input validation error"), `action "${dead}" is deleted: ${textOf(r)}`); - } - // timeoutMs is bounded at [0, 120 000]. - const tooLong = await repl(connected, { action: "eval", projectDir: PROJECT, code: "1", timeoutMs: 120_001 }); - assert.ok(isErrorResult(tooLong), textOf(tooLong)); - assert.ok(textOf(tooLong).includes("Input validation error"), textOf(tooLong)); - } finally { - await connected.dispose(); - } -}); diff --git a/packages/mcp-server/test/workflow-tool.test.ts b/packages/mcp-server/test/workflow-tool.test.ts index 15c1eccb..36a4ff56 100644 --- a/packages/mcp-server/test/workflow-tool.test.ts +++ b/packages/mcp-server/test/workflow-tool.test.ts @@ -34,7 +34,7 @@ test("tool discovery exposes asynchronous lifecycle separately from the dedicate const { client, dispose } = await connect(okRunner(), { listTools: true, uiCapability: "matching" }); try { const { tools } = await client.listTools(); - assert.deepEqual(tools.map(tool => tool.name).sort(), ["repl", "workflow", "workflow-events", "workflow-notifications", "workflow-runs", "workflow_monitor"]); + assert.deepEqual(tools.map(tool => tool.name).sort(), ["workflow", "workflow-events", "workflow-notifications", "workflow-runs", "workflow_monitor"]); const workflow = tools.find(tool => tool.name === "workflow")!; const monitor = tools.find(tool => tool.name === "workflow_monitor")!; assert.equal(field(workflow._meta, "ui"), undefined); diff --git a/packages/pi-acp/README.md b/packages/pi-acp/README.md index d57eb255..fb343711 100644 --- a/packages/pi-acp/README.md +++ b/packages/pi-acp/README.md @@ -85,7 +85,7 @@ normal structured error path. Pi also advertises the `_session/loaded_turn` extension at top-level initialize metadata as `_meta: { loadedTurn: { supported: true } }` — the re-attach arm's AUTHORITATIVE completion -evidence for a session re-opened with `session/load` (the REPL broker's restore path). +evidence for a session re-opened with `session/load` (a restarted host's restore path). `_session/loaded_turn/query { sessionId }` answers whether the loaded session's founding turn is still running right now: `running` while a turn executes in this process (the client then waits for the `_session/loaded_turn/ended` push — sent with the turn's stop reason, or its error, when diff --git a/packages/pi-acp/src/loaded-turn.ts b/packages/pi-acp/src/loaded-turn.ts index a8e08676..07cdefef 100644 --- a/packages/pi-acp/src/loaded-turn.ts +++ b/packages/pi-acp/src/loaded-turn.ts @@ -2,8 +2,8 @@ import { RequestError } from "@agentclientprotocol/sdk"; /** * The `_session/loaded_turn` vendor extension — turn-TERMINAL state for - * loaded sessions (the steering-extension precedent; the REPL broker's - * re-attach arm's authoritative completion evidence). Advertised at + * loaded sessions (the steering-extension precedent; a re-attaching + * host's authoritative completion evidence). Advertised at * initialize via `_meta.loadedTurn.supported === true`; a client seam * against a server without the advertisement degrades guest-visibly * (never settles partial output, never re-issues a possibly-running diff --git a/packages/pi-acp/test/loaded-turn.test.ts b/packages/pi-acp/test/loaded-turn.test.ts index 8e43c969..fbb7e965 100644 --- a/packages/pi-acp/test/loaded-turn.test.ts +++ b/packages/pi-acp/test/loaded-turn.test.ts @@ -1,4 +1,4 @@ -// The `_session/loaded_turn` extension (the REPL broker's re-attach arm's +// The `_session/loaded_turn` extension (a re-attaching host's // authoritative completion evidence): the query answers whether the loaded // session's founding turn is still running right now — `completed` when // the session journal's last message entry is an assistant message (pi diff --git a/packages/repl-engine/CHANGELOG.md b/packages/repl-engine/CHANGELOG.md deleted file mode 100644 index 375f76b9..00000000 --- a/packages/repl-engine/CHANGELOG.md +++ /dev/null @@ -1,735 +0,0 @@ -# @automatalabs/repl-engine - -## 0.4.40 - -### Patch Changes - -- Updated dependencies [71ab48d] - - @automatalabs/acp-agents@3.2.0 - - @automatalabs/shared-types@3.3.0 - - @automatalabs/workflows@6.2.5 - -## 0.4.39 - -### Patch Changes - -- Updated dependencies [4618813] - - @automatalabs/acp-agents@3.1.0 - - @automatalabs/shared-types@3.2.0 - - @automatalabs/workflows@6.2.4 - -## 0.4.38 - -### Patch Changes - -- Updated dependencies [a9267d5] - - @automatalabs/workflows@6.2.3 - -## 0.4.37 - -### Patch Changes - -- Updated dependencies [3b9e249] - - @automatalabs/acp-agents@3.0.2 - - @automatalabs/workflows@6.2.2 - -## 0.4.36 - -### Patch Changes - -- Updated dependencies [f6dddc1] - - @automatalabs/acp-agents@3.0.1 - - @automatalabs/workflows@6.2.1 - -## 0.4.35 - -### Patch Changes - -- Updated dependencies [76bbf8f] - - @automatalabs/acp-agents@3.0.0 - - @automatalabs/shared-types@3.1.0 - - @automatalabs/workflows@6.2.0 - -## 0.4.34 - -### Patch Changes - -- 5311099: Backend-neutral system prompt instructions: one `systemPrompt: { replace?, append? }` option (`SystemPromptOptions`) on `RunOptions`, `InteractiveSessionOptions`, and `AcpAgentOptions`, validated against the routed backend before a session opens and carried on the session `_meta` in each backend's own dialect. - - - **shared-types (breaking):** `RunOptions.baseInstructions` / `developerInstructions` are removed and replaced by `systemPrompt?: SystemPromptOptions` (`replace` swaps the backend's built-in system prompt, `append` adds to it). New `META_KEYS.systemPrompt`, `ClaudeSystemPromptMeta` / `ClaudeCodeSessionMeta.systemPrompt`, and `PiSystemPromptMeta` wire types. - - **acp-agents (breaking):** `AcpAgentOptions.instructions` (`{ base, developer }`) is replaced by `systemPrompt`; `RunOptions` / `InteractiveSessionOptions` / `AcpSessionOptions` / `SessionMetaInputs` drop the Codex-only fields for the same shape. Every backend declares what it carries (`Backend.systemPrompt`, the executable `SYSTEM_PROMPT_SUPPORT` table pinned against the installed adapter dists and the docs): Codex maps `replace` / `append` onto its bare `baseInstructions` / `developerInstructions` keys; **Claude** now drives `claude-agent-acp`'s `_meta.systemPrompt` (a string for `replace`, `{ append }` for `append`, one joined replacement string for both); **pi** sends `_meta.systemPrompt { replace?, append? }` to pi-acp; OpenCode and custom registry backends carry no channel. `assertSystemPromptSupported` (exported) runs in `prepareSession`, the `AcpAgent` constructor, `fork()`, and the cold statics: a field the backend cannot carry, an unknown field, or a blank string is a non-recoverable `SCRIPT_VALIDATION_ERROR` naming the backend — instructions are never silently dropped, on any backend (previously Claude, pi, and OpenCode ignored them). The instructions ride `session/new`, `session/resume`, `session/load`, and `session/fork` (so an id-only fork's reattach carries them too), win over the same key in `meta`, and are inherited by forks. - - **pi-acp:** new session system-prompt channel. `_meta.systemPrompt` on `session/new` / `resume` / `load` / `fork` — a string or `{ replace?, append? }` — becomes pi's `DefaultResourceLoader` overrides (`replace` takes the custom-prompt slot, `append` lands after the operator's append entries); advertised at initialize as `_meta.systemPrompt: { replace: true, append: true }`. A malformed value is rejected with `-32602` / `errorKind: "invalid_system_prompt"` (`data.field` names the offender) before any session state exists. - - **repl-engine / workflows / workflow-engine:** the broker's structural session-options type and the facade barrels follow the seam (`SystemPromptOptions` is re-exported; the README documents the option in place of the Codex-only pair). Workflow scripts' `agent()` option whitelist is unchanged. - -- Updated dependencies [5311099] - - @automatalabs/shared-types@3.0.0 - - @automatalabs/acp-agents@2.0.0 - - @automatalabs/workflows@6.1.1 - -## 0.4.33 - -### Patch Changes - -- Updated dependencies [07505a3] - - @automatalabs/acp-agents@1.3.0 - - @automatalabs/shared-types@2.3.0 - - @automatalabs/workflows@6.1.0 - -## 0.4.32 - -### Patch Changes - -- Updated dependencies [82fc72e] - - @automatalabs/acp-agents@1.2.7 - - @automatalabs/workflows@6.0.1 - -## 0.4.31 - -### Patch Changes - -- Updated dependencies [fef6ac6] -- Updated dependencies [fef6ac6] -- Updated dependencies [fef6ac6] -- Updated dependencies [fef6ac6] - - @automatalabs/shared-types@2.2.0 - - @automatalabs/workflows@6.0.0 - - @automatalabs/acp-agents@1.2.6 - -## 0.4.30 - -### Patch Changes - -- Updated dependencies [efe2c6e] - - @automatalabs/acp-agents@1.2.5 - - @automatalabs/workflows@5.1.1 - -## 0.4.29 - -### Patch Changes - -- Updated dependencies [d5968bd] - - @automatalabs/workflows@5.1.0 - -## 0.4.28 - -### Patch Changes - -- Updated dependencies [3448db1] - - @automatalabs/acp-agents@1.2.4 - - @automatalabs/workflows@5.0.1 - -## 0.4.27 - -### Patch Changes - -- Updated dependencies [954d1be] -- Updated dependencies [954d1be] - - @automatalabs/workflows@5.0.0 - - @automatalabs/shared-types@2.1.0 - - @automatalabs/acp-agents@1.2.3 - -## 0.4.26 - -### Patch Changes - -- Updated dependencies [c1ceaec] - - @automatalabs/shared-types@2.0.0 - - @automatalabs/workflows@4.0.0 - - @automatalabs/acp-agents@1.2.2 - -## 0.4.25 - -### Patch Changes - -- Updated dependencies [6005ed8] - - @automatalabs/acp-agents@1.2.1 - - @automatalabs/workflows@3.1.1 - -## 0.4.24 - -### Patch Changes - -- Updated dependencies [872db50] - - @automatalabs/workflows@3.1.0 - -## 0.4.23 - -### Patch Changes - -- Updated dependencies [b098a93] - - @automatalabs/acp-agents@1.2.0 - - @automatalabs/workflows@3.0.2 - -## 0.4.22 - -### Patch Changes - -- @automatalabs/acp-agents@1.1.3 -- @automatalabs/workflows@3.0.1 - -## 0.4.21 - -### Patch Changes - -- Updated dependencies [e1fd6c2] - - @automatalabs/workflows@3.0.0 - -## 0.4.20 - -### Patch Changes - -- Updated dependencies [18561da] - - @automatalabs/acp-agents@1.1.2 - - @automatalabs/workflows@2.0.1 - -## 0.4.19 - -### Patch Changes - -- Updated dependencies [67ca48d] - - @automatalabs/workflows@2.0.0 - -## 0.4.18 - -### Patch Changes - -- Updated dependencies [3a3932c] - - @automatalabs/acp-agents@1.1.1 - - @automatalabs/workflows@1.1.3 - -## 0.4.17 - -### Patch Changes - -- Updated dependencies [58b4a86] - - @automatalabs/acp-agents@1.1.0 - - @automatalabs/workflows@1.1.2 - -## 0.4.16 - -### Patch Changes - -- @automatalabs/acp-agents@1.0.2 -- @automatalabs/workflows@1.1.1 - -## 0.4.15 - -### Patch Changes - -- Updated dependencies [620c9ca] -- Updated dependencies [620c9ca] -- Updated dependencies [620c9ca] - - @automatalabs/workflows@1.1.0 - - @automatalabs/acp-agents@1.0.1 - - @automatalabs/shared-types@1.1.0 - -## 0.4.14 - -### Patch Changes - -- Updated dependencies [c562237] -- Updated dependencies [c562237] - - @automatalabs/shared-types@1.0.0 - - @automatalabs/workflows@1.0.0 - - @automatalabs/acp-agents@1.0.0 - -## 0.4.13 - -### Patch Changes - -- Updated dependencies [52c7701] - - @automatalabs/acp-agents@0.43.1 - - @automatalabs/workflows@0.58.1 - -## 0.4.12 - -### Patch Changes - -- Updated dependencies [06725fd] - - @automatalabs/shared-types@0.34.0 - - @automatalabs/acp-agents@0.43.0 - - @automatalabs/workflows@0.58.0 - -## 0.4.11 - -### Patch Changes - -- @automatalabs/acp-agents@0.42.1 -- @automatalabs/workflows@0.57.1 - -## 0.4.10 - -### Patch Changes - -- Updated dependencies [1452e15] - - @automatalabs/shared-types@0.33.0 - - @automatalabs/acp-agents@0.42.0 - - @automatalabs/workflows@0.57.0 - -## 0.4.9 - -### Patch Changes - -- Updated dependencies [661d9d1] - - @automatalabs/workflows@0.56.0 - - @automatalabs/acp-agents@0.41.5 - -## 0.4.8 - -### Patch Changes - -- Updated dependencies [2e87092] - - @automatalabs/workflows@0.55.0 - -## 0.4.7 - -### Patch Changes - -- Updated dependencies [7f67500] - - @automatalabs/acp-agents@0.41.4 - - @automatalabs/workflows@0.54.1 - -## 0.4.6 - -### Patch Changes - -- Updated dependencies [6821b31] - - @automatalabs/acp-agents@0.41.3 - - @automatalabs/shared-types@0.32.0 - - @automatalabs/workflows@0.54.0 - -## 0.4.5 - -### Patch Changes - -- Updated dependencies [9ddec60] - - @automatalabs/acp-agents@0.41.2 - - @automatalabs/workflows@0.53.2 - -## 0.4.4 - -### Patch Changes - -- @automatalabs/acp-agents@0.41.1 -- @automatalabs/workflows@0.53.1 - -## 0.4.3 - -### Patch Changes - -- ea0b68c: Make agent configuration fail closed and fully discoverable. Config probes now return effective ACP session modes, including config-option fallback normalization and explicit `null` for unsupported modes; workflow preflight rejects guessed or unadvertised modes before admission. Workflow `agent()` rejects unknown option keys before allocation, while REPL rejects reserved `configOptions.model` with modelSpec-native guidance and preserves independent mode failures instead of falsely blaming carried config keys. Static external MCP resources now accept subscribe/unsubscribe as no-ops. -- Updated dependencies [ea0b68c] -- Updated dependencies [ea0b68c] -- Updated dependencies [ea0b68c] - - @automatalabs/acp-agents@0.41.0 - - @automatalabs/workflows@0.53.0 - -## 0.4.2 - -### Patch Changes - -- @automatalabs/workflows@0.52.1 - -## 0.4.1 - -### Patch Changes - -- Updated dependencies [de4e704] - - @automatalabs/acp-agents@0.40.0 - - @automatalabs/workflows@0.52.0 - -## 0.4.0 - -### Minor Changes - -- 4be0807: Replace the REPL's state-dependent `followUp`/steering behavior with strict active-turn steering and durable queued turns. Agent handles now expose `steer`, `queue`, and `cancel`; `followUp` is removed. `steer` never starts or queues work and resolves only `injected`, `idle`, or `unsupported`. `queue` creates an independently awaitable, addressable FIFO turn on the same ACP session with exact cancellation, persistence, restore, and concurrency semantics. - - Make ACP extension metadata transport transparent. `customCapabilities` metadata gates and the derived steering/loaded-turn capability booleans are removed. Interactive steering returns the complete raw extension response, prompt turns expose their underlying `PromptResponse`, and extension owners interpret raw initialize metadata at the point of use. - - Pi ACP and Codex ACP now implement strict active-turn steering only. Idle or settlement-raced steering returns `promptRequired/noRunningTurn`; steering can no longer create a backend turn. REPL guest snapshots and call ledgers from the previous format are intentionally invalidated and auto-reset without executing old guest code. - -### Patch Changes - -- Updated dependencies [4be0807] - - @automatalabs/acp-agents@0.39.0 - - @automatalabs/workflows@0.51.0 - -## 0.3.4 - -### Patch Changes - -- @automatalabs/acp-agents@0.38.1 -- @automatalabs/workflows@0.50.1 - -## 0.3.3 - -### Patch Changes - -- Updated dependencies [205d110] -- Updated dependencies [205d110] -- Updated dependencies [0cf5bc5] - - @automatalabs/acp-agents@0.38.0 - - @automatalabs/workflows@0.50.0 - - @automatalabs/shared-types@0.31.0 - -## 0.3.2 - -### Patch Changes - -- Updated dependencies [4b27257] -- Updated dependencies [c90fef0] - - @automatalabs/acp-agents@0.37.4 - - @automatalabs/workflows@0.49.0 - -## 0.3.1 - -### Patch Changes - -- Updated dependencies [dfe3c34] -- Updated dependencies [2137490] - - @automatalabs/acp-agents@0.37.3 - - @automatalabs/workflows@0.48.1 - -## 0.3.0 - -### Minor Changes - -- c4c5a09: Redesign the interactive REPL around `eval` and `interrupt`. `eval` now waits up to its soft - bound, returns either `{ output, result }`, `{ output, running }`, or `{ output }`, and supports an - empty-string polling call. Workspace inspection and teardown move into the guest as `workspace()`, - `agents()`, and `reset()`; printing uses the depth-limited repr and `_` retains the previous - completion value. Dispatches beyond the workspace concurrency limit queue in order, follow-up turns - return their answers, invalid backend/options fail at admission, snapshots that cannot be restored - auto-reset with a recovery notice, and reconcile/drain details move under workspace diagnostics. - - This is a breaking removal of the workflow execution `tokenBudget` option, the script-visible - `budget` global, and the per-phase `phase(title, { budget })` option from both - `@automatalabs/workflows` and `@automatalabs/workflow-engine`. Workflow scripts must use explicit - loop bounds; `phase()` now accepts only its title. Agent-count, concurrency, timeout, and inspection - limits remain available. - - ACP assistant message chunks are now joined with a blank line, preventing adjacent chunks from - being concatenated into a single malformed sentence. - -### Patch Changes - -- Updated dependencies [c4c5a09] - - @automatalabs/workflows@0.48.0 - - @automatalabs/acp-agents@0.37.2 - -## 0.2.4 - -### Patch Changes - -- Updated dependencies [216bc1c] - - @automatalabs/acp-agents@0.37.1 - - @automatalabs/workflows@0.47.6 - -## 0.2.3 - -### Patch Changes - -- Updated dependencies [4f18373] - - @automatalabs/acp-agents@0.37.0 - - @automatalabs/shared-types@0.30.0 - - @automatalabs/workflows@0.47.5 - -## 0.2.2 - -### Patch Changes - -- Updated dependencies [471de39] - - @automatalabs/acp-agents@0.36.5 - - @automatalabs/workflows@0.47.4 - -## 0.2.1 - -### Patch Changes - -- Updated dependencies [0c33e65] - - @automatalabs/acp-agents@0.36.4 - - @automatalabs/workflows@0.47.3 - -## 0.2.0 - -### Minor Changes - -- 0ddce7b: repl: emit `console.log` output and eval results up to the result byte budget instead of clamping every string to a 200-char preview. - - A directly emitted top-level string — a `console.log` argument or the eval result — is output the orchestrator asked to see, not a preview of a value's shape, so it is now carried whole up to the byte budget ("200 chars OR the KB max, whichever is greater") rather than head/tail-elided at 200 characters. A subagent's answer comes back whole in one call instead of forcing creative slice-by-slice extraction. The tool-result caps rise to **4000 lines / 50 KB** (from 256 / 10 KB), so a multi-line answer fits; only strings past the budget head/tail-elide (keeping their `$N` ref for the remainder). Nested and property strings are unchanged — they stay preview-short. - -### Patch Changes - -- Updated dependencies [0ddce7b] -- Updated dependencies [0ddce7b] - - @automatalabs/workflows@0.47.2 - -## 0.1.8 - -### Patch Changes - -- Updated dependencies [217ba32] - - @automatalabs/workflows@0.47.1 - -## 0.1.7 - -### Patch Changes - -- Updated dependencies [4a7e4b5] - - @automatalabs/workflows@0.47.0 - -## 0.1.6 - -### Patch Changes - -- Updated dependencies [d4a0682] - - @automatalabs/workflows@0.46.10 - -## 0.1.5 - -### Patch Changes - -- Updated dependencies [7e1f1db] - - @automatalabs/acp-agents@0.36.3 - - @automatalabs/workflows@0.46.9 - -## 0.1.4 - -### Patch Changes - -- Updated dependencies [05af591] - - @automatalabs/acp-agents@0.36.2 - - @automatalabs/workflows@0.46.8 - -## 0.1.3 - -### Patch Changes - -- Updated dependencies [1a2f27d] - - @automatalabs/workflows@0.46.7 - -## 0.1.2 - -### Patch Changes - -- @automatalabs/workflows@0.46.6 - -## 0.1.1 - -### Patch Changes - -- Updated dependencies [db7b927] -- Updated dependencies [c6a896c] - - @automatalabs/acp-agents@0.36.1 - - @automatalabs/workflows@0.46.5 - -## 0.1.0 - -### Minor Changes - -- 6a7ea36: REPL orchestrator phase C: the broker, the append-only call store, and the eval tool-result semantics. - - - **The broker** (`src/broker.ts`) — `Broker.attach(workspace, options)` takes over a workspace's four `__host_*` callbacks (by-name re-registration via the new `Workspace.rehost`; the guest library and its pending-call registry are untouched) and dispatches `agent(modelSpec, task, opts)` as held-open ACP sessions through `@automatalabs/acp-agents` (`openSession`, `keepSession: true`, cwd defaulting to the workspace project dir; `schema` validated by acp-agents' own structured-output ladder driven over the session). **6 concurrent subagents per workspace** (doc-settled; configurable) — over-cap dispatches refuse at dispatch time (recorded in the store, recoverable `ConcurrencyLimitError`). **Steering resolves with what actually happened** — live `_session/steering` injection with the backend's verbatim outcome where the extension is advertised, honest `queued`/`startedNewTurn` next-turn delivery where it is not, `failed` for wire errors (never a hard error), `cancelled`/`idle` for cancel (the cancelled call rejects the RECOVERABLE `AGENT_CANCELLED` — one worker's cancellation never aborts the parallel()/pipeline() owning it). **The eval tool-result shape** — `{ output, outputTruncated, result?, pending, checkpoints, completed }`: previewed console lines (capped 256 lines / 10 KB), the trap-free previewed completion value, pending call ids on suspension (no fabricated value), previewed checkpoint questions, and settled call ids. **Suspended-eval semantics** per transfer lesson 3: continuations resume at settlement like a `.then`; late uncaught rejections surface as error-level console lines in the next tool result (the new `rejectionBridge` eval option). **Checkpoints** per transfer lesson 4: root-mediated answers, recorded before settlement, settling within the answering eval. - - **The call store** (`src/store.ts`) — transfer lesson 1: every outcome recorded by call id BEFORE the guest settlement; `InMemoryCallStore` plus the durable append-only `JsonlCallStore` (fsynced JSON-lines, torn-tail repair with sidecarred fragments, unterminated-but-complete records kept, newline-terminated corruption refused, appends healing to the acknowledged prefix). The pump's record → settle → consume loop is first-wins idempotent on both sides; `Broker.reconcile()` implements the settle-from-store arm of the three-way restore reconciliation — exactly-once settlement across a simulated crash is pinned by tests (live retry and snapshot/restore paths). - - **Engine seams** — `ReplEvalOptions.rejectionBridge` (the uncaught-rejection bridge on the pending arm), the internal eval-with-completion handle seam, `Workspace.rehost`/`snapshot`/`restore` (the raw snapshot seams the daemon layer's identity envelope wraps later). `InteractiveSession` gains additive `currentTurnText()`/`finalMessageText()`/`rawStructuredOutput()` passthroughs so the broker can drive acp-agents' own schema ladder. - - The public type surface stays fully self-contained (structural `BrokerRunner`/`BrokerSession` stand-ins — no acp-agents/quickjs-wasi types in the published declarations; the consumer fixture now exercises the whole phase-C surface). - -- 05a8e0f: REPL orchestrator phase B: the guest-side library, the host bridge, the previewer, and the output caps. - - - **Guest library** (`src/guest/guest-library.ts`) — a fresh TypeScript-authored, version-marked plain script injected at VM creation (`__REPL_GUEST_VERSION`, `Symbol.for("repl.guest")` reconciliation surface). Sandbox globals per the roadmap doc: `agent(modelSpec, task, opts?)` — the doc's own signature, `agent("pi/deepseek-v4-flash-max", "research X")` — where the returned promise IS the live handle, carrying non-enumerable `id`/`followUp`/`steer`/`cancel` (the steering calls resolve with what actually happened, mirroring acp-agents steering outcomes); `checkpoint()`/`checkpoint.answer()` (answer delivery through the `__host_checkpoint` trailing-argument mode); `console.{log,info,warn,error,debug}`; and the pure-JS combinators from `packages/workflows/src/dsl.d.ts` semantics (`parallel`, `pipeline`, `verify`, `judgePanel`, `gate`, `retry`, `loopUntilDry`). `phase()` is deleted and there is NO budget surface — no `budget()` global, no ledger, no caps vocabulary; the host signals non-recoverable failures exclusively through `recoverable: false`. The pending-call registry (entries carry `sessionId` — the founding session id for steering calls — and `modelSpec`, so pending work is fully re-issuable) travels inside snapshots; `ReplVm.restore` + `registerGuestHostCallbacks` make the versioning discipline (host serves older guests; never re-inject over a workspace) concrete and tested. - - **The bridge** (`src/bridge.ts`) — the four `__host_*` callbacks as the realm's entire effect surface: `__host_agent(callId, modelSpec, task, optionsJson)`, `__host_checkpoint(callId, question, optionsJson, answerJson?)`, `__host_agent_steer(callId, sessionId, action, payloadJson)` (both the operation's own registry id and the founding session id cross the bridge, so a pending steer is snapshot-reconcilable), `__host_console(level, payloadJson)`. `GuestCall` owns and disposes every handle it touches — raw `qjs_new_promise` parts (the shim's `newPromise()` Deferred pins the reject function until VM dispose, measured to exhaust a 2 MiB VM after ~5,000 resolved calls), the marshalled value, and the promise handle released via microtask after the trampoline's dup — pinned by 5,000/20,000-call bounded-memory tests. The reconciliation surface (`readGuestSurface`: pending/settle/stats — registry ops use captured intrinsics, immune to `Map.prototype` pollution) pins no guest memory: handles are acquired per call and disposed on the spot. - - **Workspace injection** — `Workspace.create` installs the bridge at VM creation (the doc's discipline; `agent`/`checkpoint`/combinators are never undefined), with a default parking bridge (calls park honestly, console events accumulate) or custom `handlers` per workspace/registry. The workspace exposes the render (`renderRef`), manifest (`inspectBinding`), parked-call and surface seams the `repl` tool layer builds on. - - **The console bridge** — every logged argument frozen IN FULL into a real `$N` global via `structuredClone` (with an iterative marker-copy fallback; deep nesting cannot crash the VM; weak collections/functions/symbols/promises degrade to typed markers), best-effort `{ refs, args }` payload, `console.*` never throws. - - **The previewer** (`src/preview.ts`) — CDP ObjectPreview model per the harness's normative FORMAT.md (imitated): one collapsed level, 8/8/40/200/120 caps, head+tail elision, overflow flag, positional indices, byte-size formatting with the promotion rule, 400-char backstop. Side-effect-free by construction: engine brand checks and own-descriptor reads only, proxies previewed as proxies (incl. `Proxy(revoked)`), typed-array elements via language-guaranteed reads, corrupted key enumeration degrades with `overflow: true` (FORMAT.md §6 — typed arrays included). Forbidden seams stay unwired: symbol descriptions are never read (the bare brand `Symbol`, also for thrown-symbol error messages; a guest that replaces `Symbol.keyFor` cannot forge rendering), `qjs_get_array_buffer`'s raw data pointer is never passed to `qjs_is_exception`, and every heap-returning raw export (`qjs_get_typed_array_buffer`'s backing buffer, `qjs_get_proxy_target`'s exception box) is disposed on every path (3,000-preview bounded-memory test). `renderRefLine`/`renderGlobalLine` render `[$14 · object · 48kB] {…}` lines; accessor-rebound `$N` slots render a sabotage marker (the getter is never invoked). - - **Output caps** (`src/caps.ts`) — 256 lines or 10 KB per tool result, whichever trips first; line-granular, UTF-8-counted with `\n` separators; over-cap content remains reachable through `$N`. - - **Engine additions** — the structured-clone extension is attached to every VM (and to restores); `ReplVm.restore`; the trap-free primitives moved to `src/trapfree.ts` (shared with the previewer); the public type graph stays free of quickjs-wasi types (self-contained `ReplSnapshot`; module-scoped shim access) — verified by the non-DOM `skipLibCheck: false` consumer fixture, extended to the new surface. - - **Pinned engine quirks** — a `value` GETTER on `Object.prototype` empties quickjs-ng's async-eval completion wrapper without running guest code (completions honestly degrade to `{}`); the engine's spec-mandated thenability check fires a polluted `then` getter once per eval before any of our code runs. The previewer itself adds zero getter fires (baseline-count tests). - -- 62c01d5: New engine package for the REPL orchestrator: the QuickJS-in-WASM VM layer — one VM per workspace with workspace-owned lifecycle (create/eval/drain/dispose), eval with top-level await plus the job drain, per-VM `memoryLimit`, per-eval `interruptHandler`, and a structurally enforced trap-free result boundary. `quickjs-wasi` is used as-is including its shipped `quickjs.wasm` binary (pinned exact). This is phase A of the `repl-orchestrator` roadmap; the `repl` MCP tool registration lands in a later phase on top of `WorkspaceRegistry`. - - Review-hardened on top of the initial phase: the own-descriptor read drives the raw `qjs_get_own_property_descriptor` export and never constructs quickjs-wasi's getter-invoking `JSException` (a failing descriptor read is a pinned regression test); settlement drains accept their own per-drain `interruptHandler` so a delayed continuation can't run away unguarded; `WorkspaceRegistry.get` dedupes the in-flight creation promise (exactly one VM under concurrent first touches, with dispose-cancels-in-flight semantics); eval completion is fully synchronous (raw `qjs_promise_result`), making disposal un-raceable and serializing VM operations so interrupt-slot save/restore is concurrency-safe; `WasmModule` is an opaque branded type so `{ wasm: 42 }` is a compile-time error (negative consumer tests); thrown symbols render as the trap-free bare brand `Symbol`, never the fabricated `NaN` (the description is not readable without the forbidden `qjs_get_symbol_description` seam). - -- 529e954: REPL orchestrator phase-E review fixes (engine side): - - - **Global lexical bindings in the workspace manifest**: top-level `let`/`const`/`class` declarations — the roadmap's canonical `const research = agent(...)` state — are now enumerated by the manifest (and by `inspectGlobal`/`inspectBinding`) with structure-only tokens, provenance labels, and live-handle status. Lexical bindings are not global-object properties and no guest API can enumerate them (ECMAScript's global declarative record is non-reflectable); the engine reaches them through the QuickJS context's internal global-var object, located with a self-calibrating scan (the `global_obj`/`global_var_obj` adjacency invariant of the NaN-boxed JSValue encoding, verified against the pinned quickjs-wasi 0.15.1 binary). A layout the scan cannot find refuses with the coded `LexicalEnumerationError` — never silent omission. A lexical binding shadows a same-named global-object property, so the manifest lists ONE binding per name, the lexical view (what the orchestrator's code sees). - - **Lexical provenance**: the provenance registry's attribution pass now covers lexical bindings — the host enumerates them and passes the names as the pass's third argument (no guest-visible surface grows; snapshots whose registry predates the feature skip the merge, the doc's older-library-served-as-is discipline). A name first attributed as a property and later shadowed by a lexical declaration is re-attributed to the eval that created the lexical binding, then stable. - - **`capFinalText`**: the doc's 256-line / 10 KB caps applied to a tool result's FINAL assembled text (console lines, result line, pending/checkpoint/completed sections, timeout notes — everything), with a caller-supplied truncation marker whose own budget is reserved inside the caps so it always ships and the capped result never exceeds the limits. - - New exports: `baselineLexicalKeys` (the fresh-realm lexical baseline, computed and cached like `baselineGlobalKeys`) and `capFinalText`. - -- bd28cd9: The re-attach arm's unobservable-turn degradation and the client-presence drain's two hard edges (phase-D review round 3), plus the manifest's full provenance surface. - - - **The re-attach arm never settles partial output and never re-issues a possibly-running turn.** The loaded-turn seam's rejections are classified three ways (structural markers, so third-party adapter seams can throw the same classes): the still-running class (`LoadedTurnStillRunningError`) — the broker keeps the loaded session attached and the call pending, warns guest-visibly, and RE-ARMS the seam when the rejection is re-armable (a `running` turn past its max-wait bound — a later `_session/loaded_turn/ended` notification or a cancel still settles the call), or resolves a `hold` in-flight outcome when nothing observable will ever arrive (a backend without the `_session/loaded_turn` extension); the failed-at-backend class (`LoadedTurnFailedError`) — a definite outcome, recorded and settled as an ordinary rejection, never re-issued; and the safe-re-issue class (no user message, `interrupted`, a dead process). A third-party `BrokerSession` adapter WITHOUT the `awaitCurrentTurn` seam now keeps the loaded session attached and the call pending (surfaced guest-visibly, cancelable) instead of the old release-and-re-issue. While the broker is draining/disposing, even safe-re-issue rejections hold — a fresh child must never open and run after the last client disconnected. The pump's in-flight outcomes gain the `hold` value (drop the entry without recording/settling). - - **The client-presence drain waits for opening calls** (`openSession` parked — an opening call has no session entry yet, so the old drain returned `true` immediately and let the child open and run after the last client disconnected) and in-flight lazy re-attaches; a parked open that outlives the bound is STOPPED (the late child is closed before it ever prompts, the call settles as the recoverable `AGENT_CANCELLED`, queued steers are dropped with the durable `dropped` marker). - - **The outer drain bound is absolute**: every post-deadline cancel/release await races the remaining time, so a hung backend can never block disconnect/shutdown past the reused session-eviction TTL. - - **The workspace manifest exposes the doc's full provenance surface**: bindings now carry `task` (the founding `agent()` call's task text for `worker cN` and agent-handle bindings, read from the call store, capped at 200 chars) alongside the existing `provenance` label and `provenanceAtMs` wall clock. - -- 2e4bb60: Phase-D review round 5 fixes for the client-presence drain, the lazy re-attach, and settlement provenance: - - - **The drain latch never skips in-flight work**: a fresh agent dispatch and a lazy re-attach start now clear the broker's `drained` latch the moment a child may open (it used to stay set until the open/load RESOLVED). A second disconnect after a reconnect with a parked `openSession` (or a parked lazy load) drains again — the parked open is stopped and the late child is closed before it ever prompts. - - **The drain/disposal generation fence**: the drain deadline and `dispose` bump a generation; an `openSession` or lazy `loadSession` that lands after the bump is released immediately — it never registers a session entry and never prompts (a child can never open or run after the last client disconnected, nor after a reset/dispose). - - **`cancelCall`'s wire phase runs OUTSIDE the serialized operation chain**: the lazy re-attach (and the session cancel) no longer hold the chain, so a hung backend `loadSession` can never delay `drainForDisconnect`'s entry — the documented outer drain bound is effective even then. A consume phase under the chain re-checks the entry and rolls the cancellation marker back on an idle session (a settled turn is a settled turn; queued steers are never dropped by a stale cancel). - - **Per-call settlement provenance**: the settlement pump now delivers ONE ready call at a time, running one drain + one provenance pass per settled call (each with its own settlement boundary). Two simultaneously ready independent continuations producing separate bindings are attributed to their OWN worker and task (`worker c1` / `worker c2` with the matching task text), never a joined batch label. - -- 142a23e: REPL orchestrator phase D, review round 6: the drain's outer bound is absolute for the guest drain too, the bound's forced stop never orphans a pending call, and a client reconnecting mid-drain aborts the drain. - - - **The disconnect bound now bounds the GUEST drains the drain's pumps trigger** (review: a ready settlement resumed the guest continuation through `drainJobs` under the per-eval deadline alone — a runaway continuation near the disconnect deadline could exceed the session-eviction TTL). `pumpUnlocked`/`drain` take an optional remaining-bound deadline and compose it into the quickjs interrupt handler; both of `drainForDisconnect`'s pumps race it, and an interrupted continuation surfaces as a warn-level line in the next tool result. - - **The bound's forced stop never orphans a pending call** (review: a re-attached call whose seam rejected mid-drain resolved `hold`, then the release phase discarded its session — the call stayed pending forever, uncancelable except by reset, because reconcile never runs again on a live workspace). After the bound expires, every call still pending on an attached session is settled with the recoverable `AGENT_CANCELLED` (recorded FIRST, settled into the guest, one bounded drain + settlement boundary) — the same forced-stop vocabulary as a stopped open; a still-observing task's later outcome is a first-wins no-op against the recorded completion. - - **A client reconnecting mid-drain ABORTS the drain** (review: the drain ran to its release phase and closed every child regardless of presence — children must remain warm while any client is connected). `drainForDisconnect(boundMs, shouldAbort?)` consults the abort probe every iteration and before every destructive phase; an abort leaves every child attached and running, keeps the drain latch clear, and returns `false` so the next disconnect drains again. - -- 1b9b23f: REPL orchestrator phase D, review round 10: the bounded drain settles every outstanding restored call, the restore fence detaches its releases, and reset/dispose detach a parked first touch. - - - **The bound's forced stop settles every outstanding restored call** (review: the reconciliation registers calls in the opening-call registry only as its serialized loop reaches them — parked on the FIRST pending call's never-resolving `loadSession`, it never processed the entries behind it, so a bounded disconnect settled only that one call and reported drained while the later registry entries stayed pending and uncancelable; and a load that later landed let the resumed loop initiate SUBSEQUENT loads after the drain/disposal generation bump — children opening and running after the last client disconnected). `drainForDisconnect`'s forced stop now also settles every untracked pending registry entry at the bound: completed-while-down entries from the store (the store arm's semantics, first-wins), agent entries with the recoverable `AGENT_CANCELLED`, steers with the honest `failed`, and pending checkpoints the parked reconcile never reached are re-surfaced into the checkpoint table so answering still works. `reconcileAgentCall` refuses to initiate any load or re-issue while the broker is draining/disposed — the resumed loop settles the recorded completions from the store and opens nothing. Regression: multiple pending restored calls with a never-resolving first load, bounded drain, late landing — no pending entry, no second load, exactly-one release. - - **The restore-time teardown fences detach their best-effort releases** (review: the late-load fences awaited `session.release()` with no deadline — a custom backend with a hung release kept the reconciliation, and with it the daemon's first touch, pending indefinitely, reintroducing the unbounded-teardown defect). The fence releases in `reconcileAgentCall` (both arms), `doLazyReattach` and `runAgentTask`'s stopped-open path are fire-and-forget with catch handlers attached. Regression: a late-landing restore load whose release hangs — the reconciliation completes promptly and the release was issued exactly once. - - **reset/dispose detach a parked first-touch flight** (review: `disposeReplProjectState`/`resetReplProjectState` left `state.firstTouch` in place — the generation check ran only after `broker.reconcile()` resolved, so with a never-resolving restore-time load every subsequent touch returned the stale promise and hung forever). Both teardown paths drop the flight from the state (a fresh touch starts a new first touch) and mark its eventual rejection handled — the stale touch still aborts loudly for its original caller when the parked load lands. Regression: parked restore load → reset → fresh touch completes on a fresh workspace; the late-loaded session is released exactly once and the stale touch aborts naming the teardown. - -- 21f2747: REPL orchestrator phase D, review round 12: the client-presence drain's deadline is absolute against a chain REPLACED mid-wait — the serialized-chain enqueue is atomic with a changed-chain re-check. - - - **The drain bound now survives an operation queued precisely as the prior chain releases** (review rejection: `serialized()`'s deadline path raced ONE chain promise, and after that race won it re-read the mutable `this.opChain` field — an operation enqueued in the microtasks between the chain's release and the re-read chained onto the just-released chain and REPLACED the field, so the drain enqueued behind the NEW operation with no deadline race on it. Reproduced with a pending call: a 20 ms `drainForDisconnect()` took 307 ms; with a replacement op polling another pending call, the drain returned only at the replacement's own 10 s timeout). The post-race path now re-checks the field and, when it changed, re-races the new chain against the REMAINING time — each loop pass only consumes remaining budget, so the total wait can never exceed the deadline plus timer slop no matter how many ops enqueue around a release; and the check-and-enqueue when the field is unchanged run in ONE synchronous block, so no operation can interleave between them. Regression: a drain racing a chain released by a settled call, with a second wait op enqueued mid-wait holding another pending call — the drain returns at its deadline, settles the still-pending call durably, and never waits behind the replacement. - -- 73cc45b: REPL orchestrator phase D, review round 7: a bound-expired openSession settles durably at the drain bound, the drain's outer bound is measured from method entry, and the broker teardown is bounded. - - - **A bound-expired openSession settles DURABLY at the bound** (review: the opening call was only flagged in `stoppedOpens` — it was not recorded, guest-settled, drained or snapshotted until `openSession` resolved, so a parked open that NEVER resolves left the broker reporting drained with the call pending and uncancelable). The drain's forced stop now settles every call still opening with the recoverable `AGENT_CANCELLED` at the bound (recorded FIRST, settled into the guest, one bounded drain + settlement boundary) while RETAINING the `stoppedOpens` late-child fence — an eventual landing still closes the child immediately without prompting, and the late reject is a first-wins no-op against the recorded completion. In-flight steer wire calls the bound cut off (a lazy re-attach whose load never lands, an injection/delivery the release phase is about to cut) settle the honest `failed` the same way, so the drain never reports drained with a pending call of any kind. - - **The drain's outer bound is measured from METHOD ENTRY, before the serialized-chain wait** (review: the clock used to start inside the serialized closure, so a drain queued behind a long operation ran its whole window after the queue wait, and the loop's yield was a fixed 50 ms sleep that could land past the deadline). A deadline already past at chain acquisition skips straight to the forced stop; the loop's yield races the remaining bound. - - **The broker teardown is bounded** (review: `dispose` awaited `cancelSession`, `session.release` and the owned runner's `dispose` with NO deadline — a hung backend could block daemon shutdown and the reset tool indefinitely). `dispose(boundMs)` races every await against the remaining bound, defaulting to `DEFAULT_DISPOSE_BOUND_MS` (5 s, mirroring the daemon's shutdown deadline); a runner-dispose rejection still propagates when it wins the race. - -- af917eb: REPL orchestrator phase D, review round 2: the re-attach arm's completion evidence, the envelope's identity-check ordering, the re-issue branches' refusal cadence, and the daemon wiring. - - - **The still-resumable arm is now fully implemented** (review: a successfully loaded session whose founding turn remained in flight was released and re-issued — duplicated work). The `awaitCurrentTurn` seam on acp-agents' `InteractiveSession` is an OBSERVING wait: after `session/load` resolves, it waits for the update stream to SETTLE (no updates for the loaded-turn settle grace) and only then reads the transcript's trailing content. A turn still running at the backend keeps streaming live chunks after the load response, so the seam KEEPS THE LOADED SESSION ATTACHED and settles from the turn's authoritative completion; the broker arms the call on the seam WITHOUT blocking reconcile (the pump delivers through the same record → settle → consume path as a live call). Re-issue happens only on genuine failure — no user message in the transcript, a released/dead session, a stream settled without a terminal assistant message within the max-wait bound (the never-hang-unobserved backstop), a load failure, or a seam-less third-party adapter — surfaced guest-visibly. - - **ACP message chunks are never completion evidence by themselves** (review: any trailing `agent_message_chunk` used to be treated as proof of completion with a synthesized `end_turn`, so partial output, refusal, cancellation, or truncation could be settled as success). Completion now requires a SETTLED stream (the settle grace) with a trailing assistant message; the resolved text is the turn's real accumulated outcome and the broker's own result-shaping gates (empty-output refusal, schema ladder) still apply. The stop reason stays synthesized `end_turn` — the protocol's replay carries none for a turn this client did not start (documented decision). - - **The envelope's identity check precedes payload interpretation** (review: the payload used to be gunzipped and passed through `QuickJS.deserializeSnapshot()` before the running wasm hash was compared — an incompatible old payload failed as `CORRUPT_PAYLOAD` without naming both hashes). `deserializeSnapshot` now takes `expectedWasmSha256` and refuses `WASM_HASH_MISMATCH` naming both hashes between the header parse and the payload decode; `ReplWorkspaceStore.loadSnapshot` passes the running binary's hash. A mismatched-hash test whose payload cannot be deserialized pins the ordering. - - **Every re-issue branch propagates its refusal cadence** (review: the no-recorded-session and adapter-without-seam re-issue branches discarded `reissueCall()`'s newly-settled flag, so an over-cap refusal skipped the changed-VM settlement drain and its snapshot boundary). Both branches now return the flag; cadence tests cover each. - - **The daemon wiring** (review: `ReplWorkspaceStore` was exported/tested only — the workspace did not survive daemon restarts): `mcp-server` registers the `repl` tool (eval/wait/status/interrupt/reset per the roadmap doc's Surface section) and each daemon project context opens the per-project `repl/` store, attaches the broker's state-changing-boundary sink, and on first touch restores the stored workspace + reconciles (or creates fresh). A stored snapshot that refuses (corrupt/truncated, version bump, wasm-hash mismatch) is CONTAINED: the refusal is surfaced loudly in every result and `reset` clears the store — no crash-loop, no silent data loss. `ReplWorkspaceStore.callStore()` self-heals its directory after `reset()`. - -- af9c9d5: REPL orchestrator phase D: disk persistence — enveloped snapshots, the per-project store, and the restore path's three-way reconciliation. - - - **The identity envelope** (`src/snapshot-envelope.ts`, roadmap doc transfer lesson 5): the shim's own `serializeSnapshot()` output wrapped in a newline-terminated JSON header + gzip. The header carries the format name (`repl-snapshot`), the envelope format version (`SNAPSHOT_FORMAT_VERSION = 1`), the **wasm-binary sha256** and the creation time. A restore whose recorded hash mismatches the running binary REFUSES LOUDLY naming both hashes (`WASM_HASH_MISMATCH` — never a restore into garbage); a version bump refuses naming both versions; corrupt/truncated files refuse with a specific `SnapshotEnvelopeError` code, single-shot, no crash-loop. `wasmSha256Of` hashes raw bytes directly and resolves compiled modules through the registry `loadShippedWasm` populates. - - **The per-project store** (`src/repl-store.ts`): `ReplWorkspaceStore` — a `repl/` subdirectory NEXT TO the workflow state under `workflowHomeDir()/projects//`, reusing `@automatalabs/workflows`' store-layout helpers verbatim (the same helpers the mcp-server project registry uses). Holds `snapshot.bin` (the enveloped snapshot) and `calls.jsonl` (the broker's durable call store). Snapshot writes are atomic (tmp + rename + fsync, best-effort directory fsync); a failed write leaves the previous snapshot untouched. Config knobs (decided names): `SnapshotWriteOptions.debounceBursts` (default true) and `.fsync` (default true), plus `ReplStoreOptions.persistenceRoot`/`env`. - - **The snapshot cadence + debounce**: the broker fires a state-changing boundary after each eval and after each settlement drain that changed VM state (`BrokerOptions.snapshotSink`: `boundary(kind)` per boundary, `flush()` at the end of each serialized operation — the burst boundary). `store.snapshotWriter(workspace, wasm)` debounces one drain burst's boundaries into a single atomic write taken before the operation's promise resolves; the debounced gap is always covered by the call store (settlements are recorded BEFORE they settle, so a restore replays them from the store arm). A drain that changed nothing fires nothing. - - **The restore path with the full three-way reconciliation** (`Broker.reconcile()`): every outstanding call in the in-VM pending-call registry settles exactly one way — completed while down → **settle from the store**; still resumable at the backend → **re-attach** via `runner.loadSession` (capability-gated per acp-agents' `supportsLoadSession` — all four built-ins advertise it per docs/api.md; a custom backend that omits it degrades through the same gate, surfaced guest-visibly as a warn line); lost → **re-issue** under the same call id (store `reissues` counter bumped, the existing guest promise settles exactly once, the concurrency cap applies — over-cap re-issues refuse with the recoverable `ConcurrencyLimitError`). The re-attach keys on the backend session id the store records at session open (`recordAttached` — a new append-only log event, written BEFORE the prompt, so a crash with a turn in flight leaves a restore able to re-attach instead of duplicating). A re-attached call's completion is the loaded session's founding turn, observed through the OPTIONAL `BrokerSession.awaitCurrentTurn` seam — an adapter without it re-attaches the session, then degrades to re-issue, surfaced guest-visibly, never a hanging call. - - **Everything else a restore finds**: pending checkpoints re-surface into the broker's checkpoint table (listed again, answerable through the reconciliation surface); pending steers whose wire call died with the process resolve the honest `failed` with a warn line (their outcome is unknowable; re-injecting would duplicate — queued-but-undelivered steers stay the phase-C queue-rebuild exception); reconcile is idempotent (`isTracked` guard) and adopts store-unknown entries (foreign snapshot / wiped store) so the replay ledger stays complete. - - The public type surface stays fully self-contained (envelope functions return `Uint8Array`; the store's signatures carry no node types; the consumer fixture now exercises the whole phase-D surface under the non-DOM `skipLibCheck: false` configuration). - - Phase D review round 1: the re-attach arm is now REAL through the actual acp-agents adapter — `InteractiveSession.awaitCurrentTurn` observes the completed-while-down turn's final message from the `session/load` replay (tested through a real `AcpAgentRunner` over the fake ACP agent, wire-log proven: load, no re-issue). Review round 2 (see the round-2 changeset) replaced the still-in-flight degradation with the observing-wait semantics: the seam keeps the loaded session attached and settles from the turn's authoritative completion, re-issuing only on genuine unobservability. Reconcile-time refusals (invalid registry options, over-cap re-issue) now participate in the changed-VM bookkeeping: the settlement drain runs and the settlement snapshot boundary fires. A changed-VM settlement drain that FAILS (interrupted continuation) still fires its boundary on the reconcile and pump paths alike — the operation-end flush always has a dirty boundary to persist. The test suite's `setup` helper threads `interruptHandler` through (a silently-dropped handler left a runaway continuation unguarded in the drain-failure regression test). - -- 0c29a86: REPL orchestrator phase F, review round 2: authoritative re-attachment/completion for ALL four built-ins, the out-of-band eval-break relay, and addressable truncation references. - - **acp-agents — the observation path for backends without the `_session/loaded_turn` extension** (the built-in claude and opencode backends today; also the fallback when an extension backend's query wire fails). The old degradation — reject with the non-re-armable `LoadedTurnStillRunningError` so the broker releases the loaded session and re-issues the call — could duplicate a still-running backend turn. The seam now classifies the loaded session's founding turn authoritatively: the post-load continuation watch (`AGENTPRISM_ACP_LOADED_TURN_OBSERVE_MS`, default 1 s — any CONTENT update after the load boundary is live continuation, the still-running signal) plus the replay probe under the CONNECTION-DEATH CONTRACT (live-verified: claude-agent-acp and pi-acp exit on connection close and cancel their turns, `opencode acp` exits on stdin EOF, codex-acp ends/kills the codex process, and their persisted transcripts hold only completed messages — so at restore the founding turn is never still running and the replay's trailing content is authoritative: an assistant message is the turn's terminal message (completed-while-down, settled from the transcript), anything else means it died mid-way (the safe-re-issue class — nothing running, no duplication)). Live continuation flips the classification to the keep-attached wait (bounded, re-armable). The `_session/loaded_turn` extension path is unchanged. Tests pin the seam-less completed/interrupted/still-running classifications end to end through the real adapter. - - **repl-engine — a possibly-running call is never re-issued** (the broker's restore/re-attach arm): every `LoadedTurnStillRunningError` — re-armable and non-re-armable forms alike — keeps the loaded session attached and re-arms the seam on it (the doc's re-attach-to-a-still-running-task arm); re-issue is reserved for the observably-dead classes (interrupted classification, a transcript that never received its prompt, a dead/released session, a third-party adapter with no seam at all). Also new: **the out-of-band eval-break channel** (`EvalBreakChannel` / `createEvalBreakChannel`) — a worker-thread relay with a loopback HTTP break endpoint and a shared-memory (SAB + Atomics) break flag. The interrupt tool's no-id path becomes deliverable to a SYNCHRONOUSLY running eval: a never-yielding eval blocks the daemon's single thread, so the request itself cannot be processed — the relay (a separate thread) arms the flag, and every eval execution's quickjs interrupt handler consumes it mid-run with the arm-after-start rule (a stale break — armed while the workspace was idle — is dropped on first observation and never breaks a later eval). `BrokerOptions.evalBreakChannel` wires it; the broker reports consumed out-of-band breaks to the tool for the honest outcome. - - **mcp-server — the daemon wiring for both**: `run-daemon` creates the channel and advertises its URL in daemon.json (`replBreakUrl`); the stdio shim fires the relay automatically when it forwards a `repl` interrupt without an id (before forwarding, so the break lands while the daemon is wedged); the interrupt tool reports the honest out-of-band outcome (and clears the flag once its own processing owns the break). And **the structured-output cap's continuation references**: the aggregate 10 KB cap previously discarded the tail entries of elided arrays (pending ids, checkpoint questions, completion ids, status metadata) keeping only counts — the omitted values had no address and repeated reads could never recover them. Every elision now snapshots the dropped entries in the workspace's `TruncationRefStore` under a ref id that the `truncated` record carries (`{ elided, ref }`), and a later eval/wait/status call's optional `refs` parameter reads them back under `referenced` (a referenced read is itself capped, chaining fresh refs) — the cap costs reads, never data, for every omitted field. The truncation marker text now names both the `$N` refs and the structured continuation refs. - -- 0baa82c: REPL orchestrator phase-E review round 2 fixes: - - - **The eval-break interrupt targets the RUNNING eval** (`mcp-server` + `repl-engine`): `interrupt` without an id no longer arms a project-wide "next VM execution" boolean. The broker now tracks the suspended eval's completion (retained at suspension, released when the continuation completes or is broken) and `Broker.armEvalBreak()` refuses — an honest "no running eval to interrupt" no-op, nothing armed — when no eval is in flight. When an eval IS in flight, the armed signal is scoped to it and consulted ONLY by settlement drains (the executions that resume suspended-eval continuations), never by a fresh eval's own code or its own job drain — an unrelated eval can neither consume the signal nor be broken by it, and the signal is cleared when its targets settle or the drain it broke reports the interruption (a continuation broken by the quickjs interrupt never settles its engine wrapper — verified against the shipped binary — so the drain-interruption path releases the tracking). An idle workspace's next eval runs normally. - - **Lexical provenance re-attributes on worker settlement** (`repl-engine`): the provenance registry now tracks global lexical bindings by VALUE — the host reads each binding's current value through the internal global-var object and hands it to the pass, which re-attributes a changed value (SameValue) to the operation that produced it. A `let` binding assigned a worker result, or a suspended `const finding = await research` whose continuation assigned the settled value, now reports `via worker cN` with the founding task and the attribution wall clock instead of the declaring eval's label with no task. A registry whose record closure predates the feature degrades to first-sight-only attribution (an older snapshot is served as-is). - - **Size metadata for EVERY manifest binding** (`repl-engine`): the manifest token now carries the byte-size estimate for undefined, null, numbers, booleans, bigints, symbols, functions, plain promises, and agent handles — and the broker-enriched binding exposes it as its own `sizeBytes` field (the doc's name/type/size contract; 0 only for the unreadable accessor/sabotage cases). The `status` tool renders it through the tokens. - -- 1aacc26: REPL orchestrator phase-E review round 3 fixes — the interrupt breaks a currently-executing runaway, the wait never holds the broker chain, workflow calls register project presence, and daemon idleness counts active REPL drains. - - - **The no-id interrupt breaks an EXECUTING runaway eval** (`repl-engine` + `mcp-server`): the eval-break signal is now consulted by EVERY execution that resumes the running eval's continuation — the settlement drains AND a direct eval's own drain (a suspended eval's continuation can be resumed by a SYNCHRONOUS host-callback settlement — `checkpoint.answer` in a later eval — and that execution runs inside the answering eval's own drain, where the old settlement-drain-only signal was blind; the runaway burned the eval deadline instead of being broken by the interrupt). `runEval` composes the signal as the drain-phase handler (`drainInterruptHandler`, a new vm-level option — the eval's own CODE still never consults it, so an unrelated eval's code is never broken), and an eval whose own drain was interrupted releases the tracked running eval exactly like the pump path (`interruptedInDrain` → `noteInterruptedDrain`) — a broken target can never linger as a stale arm target. The tool's result text and docs now state the real semantics: an eval that yields (suspends on a call) is interruptible at its next execution, mid-run; a fully synchronous runaway is bounded by the per-eval wall-clock deadline (the request cannot physically arrive while the single-threaded daemon executes it). - - **`waitForCalls` releases the broker serialization chain between its pumps** (`repl-engine`): the wait used to hold the chain across its whole bounded poll (each sleep included), so a concurrent `interrupt` — `cancelCall` or `armEvalBreak` — queued behind it and could not cancel or break until the wait finished or timed out (up to 120 s), by which point the target could already have completed. Each pump is now its own serialized unit; between pumps other operations interleave, so an interrupt lands promptly mid-wait and the wait's very next pump breaks the armed target mid-run. The target set is captured at entry (with other operations interleaving, "every pending call" can only mean "the calls pending when the wait started"), each pump runs under the REMAINING wait time (the wait's bound is absolute for the guest drain, same posture as the disconnect drain), and a mid-wait pump-drain interruption is honest output in the wait's result. The daemon and engine suites now exercise the interrupt mid-wait and the mid-run break of an eval that keeps executing across drains (a runaway loop over subagent calls), plus a concurrent cancelCall completing during a live wait. - - **The workflow tool registers REPL project presence** (`mcp-server`): the workflow handler resolves the same per-project context the repl tool addresses, and a session that calls it is now registered on the project's repl presence — a workflow-only client B staying connected keeps the workspace warm (children open) when repl-client A disconnects; the drain fires only when the LAST project client of either kind disconnects. A pure-workflow project keeps a stateless repl context (no VM is created — the workspace is materialized only on the first repl tool touch). The presence ledger is now created once per server and shared by both tools. - - **Daemon idleness counts active REPL drains** (`mcp-server`): the idle reaper's busy check now includes `activeReplDrainCount()` (the presence ledger's scheduled/in-flight drains) alongside sessions and workflow runs — a last-client-disconnect drain may legitimately run for the full session-eviction TTL after the final session is gone, and the default idle shutdown can no longer fire mid-drain and replace its bound with the five-second shutdown deadline. In-flight turns are guaranteed to drain to completion under the documented session-eviction-TTL bound. - -- c663a86: REPL orchestrator phase-E review round 3b fixes — the eval-break signal is keyed to the armed target's continuation (unrelated drains neither fire nor consume it), the tool returns the doc's machine-readable shapes as structuredContent with a published output schema, and the bounded wait sleeps only for its remaining budget. - - - **The eval-break signal targets the armed eval's continuation, not whichever drain runs next** (`repl-engine`): the carried defect — the drain-phase interrupt handler was installed on every later eval's drain without checking whether that drain resumed an armed target, so an unrelated finite eval (or an unrelated settlement drain) was interrupted and the interrupted-drain release cleared the target's tracking while its checkpoint stayed pending and uninterruptible. The armed identity is now the union of the armed evals' OWN suspension-time calls (pending at suspension minus the pre-eval baseline — the calls the eval issued itself, whose settlement queues its continuation; a later eval's snapshot never inherits an earlier eval's still-pending calls). Every VM operation maintains a settlement accumulator (`opSettledCalls`, appended by every settlement route and seeded by the pump/reconcile/disconnect drains with the settlements that triggered them), and the signal fires only while that accumulator intersects the armed deps — the currently-executing drain BELONGS to the armed target. An unrelated drain neither fires nor consumes it, and the armed state survives intact. The interrupted-drain release (`noteInterruptedDrain`) is gated the same way: exactly the tracked evals whose own resume keys the interrupted operation settled are released (a deadline-broken resumed runaway releases its tracked eval even when no signal was armed — a stale target would make a later arm target a dead eval), and an unrelated interrupted drain leaves the armed state and every tracked eval intact. A no-id interrupt with NOTHING breakable — no eval in flight, or every in-flight eval suspended on no OWN pending call (a never-settling local promise, or an `await p` on an earlier eval's binding) — refuses and arms nothing. - - **The `repl` tool returns the doc's machine-readable shapes** (`mcp-server`): the carried defect — eval/wait/status were flattened into text-only MCP content with no output schema, mixing guest output and trusted orchestration metadata into one flat string. The tool now publishes an `outputSchema` (the workflow tool's oneOf-branch pattern) and every result carries `structuredContent`: eval/wait return the doc's `{ output, result?, pending, checkpoints, completed }` (plus `outputTruncated` and the wait-only `drained`/`timedOut` flags), status returns the structured workspaces surface (workspace state, the reconcile summary, the workspace MANIFEST with name/token/size/provenance/task per binding, the live agents, the pending ops, child warmth, a retained drain failure), interrupt returns its honest outcome (`targeted`/`refused-idle`/`cancelled`/`idle`/`failed`/`none`), reset the `dropped` acknowledgement, and the refusal paths a structured `error` variant. Guest output and orchestration metadata stay separate fields, and every structured field is bounded metadata (output capped by the broker, checkpoint questions previewed, manifest tokens structure-only). Status checkpoint questions are now previewed in the text surface too (the doc's previewed-question rule). - - **The bounded wait sleeps only for the REMAINING budget** (`repl-engine`): the carried defect — the unconditional 50 ms inter-pump sleep made every sub-50 ms `timeoutMs` take ~51 ms, violating the bounded-wait contract. The sleep is now `min(50, deadline - now)`, matching the disconnect drain's pump discipline. - - **The pending surface reports the WHOLE guest registry** (`repl-engine`): the trap-free reader's generic 256-element array cap silently truncated the guest surface's `pending()` list, and its `[ArrayTruncated]` marker mapped to `undefined` in the broker's id lists — a hole in the structured `pending` field. `readValue` takes an explicit array bound (the preview read is unchanged at 256); the surface read passes `SURFACE_READ_MAX_LEN` (16 384 — the registry is the host's own reconciliation metadata, bounded by VM memory). - -- f04776d: repl phase-E review round 4: the eval-break interrupt is keyed to the calls the running eval AWAITS. The engine now instruments top-level awaits (`await x` → `await __replAwait(x)`, acorn-based, guest library 0.2.0) and attributes each suspended eval's resume keys from the guest's await log — an unawaited sibling call's settlement no longer fires or consumes the armed signal (its own `.then` continuation runs to completion), an eval awaiting an EARLIER eval's binding stays targetable, and the wait tool's serialization-chain acquisition is bounded by its absolute deadline. The pending-call registry and provenance reads are now COMPLETE trap-free reads (no 16 384-element array cap, no 256-property object cap) — the whole registry and every binding's provenance survive eval output and restore reconciliation. The repl tool's input is action-discriminated (exact per-action field sets, extraneous fields rejected), the structured manifest gains machine-readable type + live-handle status/call fields, and guest-derived structured status fields (agent task) are capped at the engine seam. -- f17212a: repl phase-E review round 5: the eval-break interrupt now carries a genuine per-eval CONTINUATION IDENTITY. The guest library (0.3.0) wraps every instrumented top-level await (`await x` → `await this["__replAwait"](x, TOKEN)` — a hygienic seam: the `this` keyword base is unshadowable, and no helper binding is injected into the persistent global lexical record) and sets a continuation lease in the job immediately before the eval's continuation segment; the drain loop mirrors the lease per job, so the armed signal fires only while the armed eval's own continuation executes. An unawaited sibling `.then` registered before the target's await can neither fire nor consume the signal (the carried defect broke the sibling's job and let the target run later unbroken), and indirect waits (`await Promise.all([q])`) are targetable through the promise graph (the 0.2.0 log-only identity refused them). The interrupted-drain release is exact the same way (the interrupted job's lease names the eval). A zero `timeoutMs` wait performs one immediately available state read (idle workspaces drain, pending calls report), the top-level-await instrumenter's injected seam can no longer be shadowed by guest identifiers, the workspace manifest enumerates user bindings that SHADOW or OVERWRITE baseline globals (`const Math = 42` is listed with full metadata and provenance — the provenance registry captures the baseline type tokens and its own intrinsics, so host bookkeeping survives lexical shadows of `Math`/`Object`), the repl tool accepts empty `code` strings (valid JavaScript resolving with `undefined`), and the per-backend steering mechanism table is now a GENERATED artifact gated by a test (`docs/steering-mechanism-table.md`, regenerated from `ACP_EXTENSION_SUPPORT_MATRIX` via `generate:steering-table`). -- d24372f: repl phase-E review round 6: three carried-defect fixes. (1) The eval-break continuation lease is now associated with the ACTUAL CONTINUATION JOB, not the next job: the guest library (0.3.1) registers the lease-setting reaction on the awaited value's WRAPPER promise itself — immediately before the await machinery's own reaction — so the wrapper's settlement queues the lease-setting job directly before the continuation job, and a sibling `q.then(...)` registered after the eval started awaiting `q` can no longer run with the lease set (the 0.3.0 reaction ran on the value's settlement, so the sibling consumed the armed signal and the target's continuation ran later unprotected). (2) The for-await ITERABLE wrap preserves the iterable protocol: the new `__replAwaitIterable(value, token)` global returns an async-iterable wrapper (resolved exactly like `for await` resolves an iterable) whose per-`next()` results are lease-wrapped promises, so `for await (const x of [1, 2])` iterates normally through the broker while a running loop stays breakable mid-iteration; the instrumenter gates for-await sites on the new `supportsIterableLease` surface flag, and `for await (... of await y)` is left unwrapped (its own await is instrumented normally). (3) Same-type baseline-global overwrites are tracked and attributed: the provenance registry captures the ORIGINAL baseline values at creation (they travel inside snapshots and are never updated on attribution) and re-attributes known names on SameValue difference, so `Math = { userOwned: true }` is listed in the workspace manifest with `object` type and full provenance even though the type token never changes; the manifest's changed-binding filter consults the registry's changed-known read alongside the host-side token check. -- 9404d4a: repl phase-E review round 7 (the reviewer's rejection of the previous attempt): five defect fixes. (1) The for-await iterable wrap preserves AsyncFromSyncIterator semantics: `__replAwaitIterable` now awaits and unwraps a SYNC iterator's result VALUE (`for await (const x of [Promise.resolve(1)])` yields `1`, never the promise object — the old wrapper resolved with the raw iterator result, and because the wrapper is an async iterable the machinery used the value as-is), while an async iterator's results pass through untouched. (2) Iterable ACQUISITION errors propagate exactly once: resolving `@@asyncIterator`/`@@iterator` follows GetIterator/GetMethod semantics (a present-but-not-callable `@@asyncIterator` is a TypeError, never a fallback) and a throwing getter runs a single time reporting its ORIGINAL error — the old degrade-to-unwrapped made the machinery acquire the iterable a second time (`boom2` instead of native `boom1`). (3) The instrumentation surface runs on CAPTURED pristine Promise intrinsics (`P`/`PResolve`/`PReject`/`pThen`, bound at installation): replacing `Promise.prototype.then`, overwriting `Promise.resolve`, or shadowing `Promise` lexically cannot change the instrumented `await 40` (still `40`) or skip the continuation-lease setting; the same hardening applies to the host-thenable forwarding in `issueHostCall`, which otherwise silently killed every call settlement under a replaced prototype. (4) Provenance recording reads descriptors off the CAPTURED global object: a top-level lexical `const globalThis = 7` no longer blanks every binding's provenance (`var userValue = 42` reaches the manifest with producer/task/time metadata). (5) The broker's continuation-lease availability check is VERSION-GATED on >= 0.3.1: a restored 0.3.0 library (whose lease-setting reaction still runs on the awaited VALUE's settlement — the carried sibling-reaction interrupt-targeting defect) reports `supportsContinuationLease: true` but is now served WITHOUT instrumentation and the eval-break interrupt refuses honestly — the flag alone re-armed the original defect on a supported older snapshot. Regressions cover every finding at the guest-library and broker boundaries, including a restored-0.3.0 snapshot whose sibling reaction never observes a continuation lease. -- 5f1cdba: repl phase-E review round 8 (the reviewer's rejection of the previous attempt): the two remaining defects fixed. (1) `interrupt { id }` (and the guest handle's `cancel()`) now cancels a call whose `openSession()` is still pending: the `cancelCall` decision's new opening arm fences the call in `stoppedOpens`, settles it DURABLY as the recoverable `AGENT_CANCELLED` (recorded first, guest-settled first-wins, concurrency token released, one drain fires the settlement's guest reactions) and returns `cancelled` — the old decision skipped `openingCalls` entirely, returned `none`, and the eventual open resolved into a prompted, supposedly-interrupted call. A late landing closes the child immediately without ever prompting; a daemon restart settles the call from the store. Regressions at the broker boundary (delayed-open cancel + handle cancel + slot release under a cap of one) and a full daemon regression with a delayed `openSession()`. (2) The doc's 256-line/10 KB tool-result cap now applies to `structuredContent` as an AGGREGATE serialized-size cap, not only to the bounded text: the modelSpec is previewed at the ENGINE seam (head+tail 200 chars, the task bound), and a new `capStructuredResult` pass elides the largest lists (head prefix kept) with an explicit path-keyed `truncated` record of elided counts — elision is never a silent hole (the round-4 registry-read defect) and the wire's serialized structured result always fits the 10 KB bound (a 20,000-character model spec and 16,500 pending ids previously crossed uncapped; the 16,500-checkpoint daemon test now pins the bounded, flagged, size-checked wire plus the full registry surviving in the VM across a restart). -- 3b30612: repl phase-E review round 9 (the reviewer's rejection of the previous attempt): four defects fixed. (1) The opening-call cancellation (`interrupt { id }` on a call whose `openSession()` is still pending) is a settlement drain that changed VM state but never fired the per-settlement provenance pass or the state-changing boundary: `cancelCall`'s opening arm now runs `provenancePass('settlement', [callId])` and `sink.boundary('settlement')` after its drain (the boundary still fires on a `DrainJobError`, mirroring the pump), so the manifest immediately attributes the settlement's continuation bindings to the cancelled worker and the daemon's snapshot writer persists the settled workspace before the interrupt's promise resolves — a kill right after the interrupt (no eval or wait in between) restores the SETTLED snapshot, never the pre-settlement one. (2) The opening-cancel's concurrency-slot release now runs the global queued-delivery kick (`kickQueuedDeliveries`, exactly like every other slot-free transition): a cap-pressure follow-up queued on an idle session starts its delivery turn the moment the opening call is cancelled. (3) The GENERATED steering mechanism table is corrected and re-pinned: the `cancel()`-while-opening case was documented as a no-op returning `failed` while the call continues, contradicting the implementation (which cancels the opening call and returns `cancelled`); the generator's case table and the broker module docs now say the opening call is fenced and settled durably as cancelled, the checked-in artifact is regenerated, and the gate test pins the corrected row (and asserts the stale no-op claim is gone). (4) The generator emitted TWO terminal newlines, so `git diff --check` failed with "new blank line at EOF" on the checked-in artifact; the generator now emits exactly one terminal newline and the gate test pins it. Regressions: broker-boundary tests for the immediate-after-interrupt snapshot/restart with a recording sink (exactly `['settlement']` fired, no intervening eval, provenance + store-arm assertions across the restore), the cap-1 queued-follow-up kick, and a daemon regression that kills the daemon IMMEDIATELY after the interrupt (no eval or wait — the round-8 test masked the defect by performing both before restart) and asserts the restart's reconcile has nothing for the store arm and the manifest provenance traveled inside the interrupt's own snapshot. - -### Patch Changes - -- 149b606: Phase-F review round 1: the re-attach arm's unobservable-completion degradation is replaced - by the doc's honest re-issue fallback — the undocumented fourth reconciliation arm - ("pending until interrupt/reset") is gone. The doc's restore path settles every outstanding - call exactly once through exactly one of the three arms (settle from the store / re-attach / - re-issue); the old `registerUnobservableReattach` path left a successfully re-attached call - permanently pending when the loaded session's founding-turn completion was unobservable, - which is the case for the built-in claude and opencode backends (they do not advertise the - `_session/loaded_turn` extension, per the live-verified `ACP_EXTENSION_SUPPORT_MATRIX`). - Now: - - - A loaded session WITHOUT the `awaitCurrentTurn` seam (a third-party adapter) is released - and the call is re-issued under the same id — the same degradation the catch arm already - used for load failures, surfaced guest-visibly with a warn line naming the reason. - - A NON-re-armable `LoadedTurnStillRunningError` (backend without the extension, or a - failed `_session/loaded_turn/query` wire) degrades the same way: release + re-issue under - the same id. Never settled from a quiet gap (partial output is still never settled), - never left pending. - - The RE-ARMABLE class is unchanged: a `running` turn past the max-wait bound on a backend - that DOES carry the extension keeps the loaded session attached and re-arms the seam — the - doc's second arm (re-attach to a still-running task); a later `_session/loaded_turn/ended` - notification or a cancel still settles the call. - - The drain/disposal fences are unchanged: while the broker is draining or disposed, even - safe-re-issue rejections resolve `hold` — the drain's forced stop settles every - still-pending call DURABLY at its bound (recorded `AGENT_CANCELLED`, guest-settled), so a - drained call is never left pending, and a disposed broker's state is being torn down. - These are now the only `hold` producers left in the pump. - - The seam's rejection messages in acp-agents (`LoadedTurnStillRunningError` text) and the - `awaitCurrentTurn` documentation were re-worded to match (the broker re-issues; the - re-armable form keeps the wait on the attached session); repl-engine module docs, the - package READMEs, and docs/api.md document the degradation and the exhaustive three-arm - contract. Regressions: the seam-absent adapter test and the non-re-armable rejection test - now pin the re-issue path end to end (loaded session released, reissue recorded, fresh - turn settles the SAME guest promise exactly once, warn line names the reason), and the - acp-agents integration test pins the re-worded non-re-armable message. - -- bcede5b: REPL orchestrator phase F, review round 3 — the full-repo verification's carried defects, all closed: - - - **ACP freshness gate green**: the `packages/codex-acp` subtree is re-synced with upstream `agentclientprotocol/codex-acp@main` (ea57892 — the goal-extension `resume` action and the v1.1.11–1.1.13 releases) via a true non-squashed merge commit; the fork's `package.json` version line wins, the package lockfile stays deleted, and the imported upstream head is recorded in the attribution allowlist. - - **The observation path's replay classification is restricted to the verified built-ins** (acp-agents): a CUSTOM backend's quiet observation window is not terminal evidence — its connection-death behavior is not live-verified — so its loaded session stays attached and the seam waits for the authoritative terminal state (the re-armable still-running rejection) instead of settling stale/partial replay or re-issuing a possibly-running call. - - **Non-re-armable seam rejections are never re-invoked** (repl-engine): the broker kept recursing into a seam that rejects with `LoadedTurnStillRunningError` and `rearmable: false`, spinning in an unbounded microtask/warning loop that starved cancellation, drain, and every other task. The broker now keeps the loaded session attached and waits for the terminal state from the session-level `_session/loaded_turn/ended` surface, the call's cancel (settled as the recoverable `AGENT_CANCELLED`), the session's release (the safe-re-issue class), or the drain's forced stop. - - **The interrupt is implemented in the in-process/library mode too** (mcp-server): the single-project server now owns an eval-break channel by default and exposes its relay (`replBreakUrl()`); the stdio transport's stdin reader lives on a worker thread that fires the relay for no-id `repl` interrupts, so a synchronous `while(true)` eval is breakable mid-run exactly like in daemon mode. The relay keys are realpath'd on every fire side (shim and in-process reader), so symlinked or non-normalized projectDirs interrupt correctly. - - **Break targeting has no clock-resolution window** (repl-engine): the eval-break channel now orders arms against execution starts on a shared monotonic arm-sequence counter instead of millisecond `Date.now()` stamps — a break arriving in the same millisecond as the execution start is delivered, never consumed as stale and lost. The channel's slots also GROW on demand (no fixed workspace ceiling) and are released on broker teardown for reuse. - - **The structured-output cap's continuation refs are cumulative, namespaced, and never evicted** (mcp-server): repeated halving of one field chains every dropped chunk into the advertised ref (earlier tails stay addressable); ref ids carry the workspace's project key so a ref from one project can never resolve in another's store; the store retains every ref until `reset` (which now clears it); and the `wait` result variant accepts `referenced` (the handler attached it, the validator forbade it). - - Documentation and the phase-F changeset re-worded: the `repl-engine` dependency line and the shipped-tool status are stated as they are, and the changeset no longer carries the banned marker strings. - -- 1db93d4: REPL orchestrator phase F, review round 4 — the five carried defects from the full-repo verification's clause checklist, all closed with regressions: - - - **The in-process no-id interrupt honors the documented optional `projectDir`** (mcp-server): the single-project `repl` tool resolves an omitted `projectDir` to the server's own adopted project, and the relay transport's stdin-reader worker now fires the out-of-band eval-break with that same key (exposed as `replDefaultProjectDir()` on the server control, wired into the worker's `workerData`). An omitted-`projectDir` interrupt during a synchronous `while(true)` eval previously skipped the relay entirely, ran to the per-eval deadline, and then reported `refused-idle`; the new e2e pins the out-of-band break. - - **Streaming UTF-8 decoding in the relay reader** (mcp-server): the worker now decodes the raw stdin byte stream through a `StringDecoder` (`RelayFrameSplitter`), never per-chunk `Buffer.toString("utf8")` — a multibyte character split across reads used to be replaced with U+FFFD, so the claimed byte-identical MCP forwarding was false (a built-server repro changed an expected string length). Unit tests feed a JSON-RPC frame one byte at a time and pin the verbatim decode. - - **Acknowledged, generation-safe eval-break slot lifecycle** (repl-engine): `EvalBreakChannel.register` now returns a promise the relay worker acknowledges (`{ type: "ack", key, slot, gen }`) only after applying the key→slot mapping, and every serialized broker operation awaits the ack before touching the VM (`runSerialized`) — a first interrupt can no longer 404 against an unapplied mapping and lose the break. Slot assignments carry generations: the worker stamps each arm with the arming key's generation (release order, before the flag), `unregister` clears the flag and invalidates the slot's generation word, the worker clears the flag when a mapping takes a slot over, and `consumeBreak` drops any consumed flag whose generation does not match the consuming key's current one — a stale arm for a released incarnation can never break the workspace that reuses the slot. The channel's worker-message listener is attached only while booting or awaiting acks (Node re-refs the worker port while a message listener is attached; the round-3 code left it attached forever and every server-owning test suite hung on exit). - - **Cumulative truncation refs preserve verbatim order** (mcp-server): the elision record's chained continuation ref now assembles `[...newestDropped, ...priorDropped]` — the halving pass always drops from the current array's tail, so the newest chunk precedes the older ones in the original array. The advertised ref used to concatenate chunks in reverse (`[4…7,2…3]` after two drops instead of the verbatim tail `[2…7]`); the unit test pins head+ref reassembling the original list exactly. - - **`send` completion means flushed** (mcp-server): `ReplRelayStdioTransport.send` now awaits the stdout `drain` event when `write()` reports backpressure, exactly like the `StdioServerTransport` it replaces — the old fire-and-forget write resolved immediately, allowing unbounded buffering against a slow client for all in-process MCP traffic. Unit tests drive a fake stdout seam through backpressure and drain. - -- 4c046ab: repl phase F — full-repo verification (round 2): the ENTIRE monorepo's CI gates pass green - with the phase A–E repl work in place — the frozen-lockfile install, the project-references - build, the monorepo typecheck, every package's test suite (shared-types, codex-acp, - pi-acp, workflow-engine, acp-agents, workflows, agentprism-otel, repl-engine, mcp-server), - the `check:acp-backends-manifest` and attribution gates, and now also the required ACP - dependency freshness gate (`node scripts/check-acp-deps.mjs` — green because this branch - carries the merged maintenance PRs: claude-agent-acp 0.65.0, pi 0.84.0, the - claude-agent-sdk 0.3.223 root override, and the codex-acp upstream syncs with their - attribution allowlist records). - - The clause-by-clause sweep of docs/roadmap/repl-orchestrator.md against the code stands: - npm-shipped `quickjs.wasm` used as-is (no custom wasm build); fresh TypeScript guest library - (no vendored `dsl.js`); a single `repl` tool with the exact five actions; no budget surface - in the guest; snapshots at every state-changing boundary; the per-project `repl/` store - layout; the 6-subagent cap; the 256-line / 10 KB caps on text and structured content alike; - guest-visible steering outcomes; presence-keyed lifecycle with the drain bound reusing the - daemon's session-eviction TTL; plain handles with stable call ids and no canonical path - addressing; no inter-agent communication surface; `Date.now()`/`Math.random()` working - natively (pinned in `vm.test.ts`). No unfinished-work markers remain in the repl - code, and no doc-required behavior is deferred. - -- Updated dependencies [30f3aa5] -- Updated dependencies [bd28cd9] -- Updated dependencies [af917eb] -- Updated dependencies [fac9d5d] -- Updated dependencies [a2a76bc] -- Updated dependencies [0c29a86] -- Updated dependencies [149b606] -- Updated dependencies [bcede5b] - - @automatalabs/acp-agents@0.36.0 - - @automatalabs/workflows@0.46.4 diff --git a/packages/repl-engine/README.md b/packages/repl-engine/README.md deleted file mode 100644 index 013186fb..00000000 --- a/packages/repl-engine/README.md +++ /dev/null @@ -1,1062 +0,0 @@ -# @automatalabs/repl-engine - -The engine package of the **REPL orchestrator** (see -[`docs/roadmap/repl-orchestrator.md`](../../docs/roadmap/repl-orchestrator.md)): a persistent -JavaScript REPL in a capability-free QuickJS-in-WASM VM. One VM per workspace; the workspace -object owns the VM lifecycle (`create` → `eval` → `drainJobs` → `dispose`). The `repl` MCP -tool that registers in `mcp-server` (the daemon wiring below) is built directly on this -engine's `Broker` + `Workspace` + per-project `ReplWorkspaceStore` — with its own tool-level -input/output schemas, an action discriminator (TWO actions: `eval` + `interrupt`), the -soft-bound fused eval pump, and the client-presence drain — -**not** a thin [`WorkspaceRegistry`](#workspace-registry) wrapper; this package is the engine -tier it sits on. - -```ts -import { Workspace } from '@automatalabs/repl-engine'; - -const ws = await Workspace.create('/path/to/project'); -const first = await ws.eval('const findings = [1, 2, 3]; findings.map(x => x * 2)'); -// first = { kind: 'value', value: [2, 4, 6] } — state persists in the VM -const second = await ws.eval('findings.length'); -// second = { kind: 'value', value: 3 } -ws.dispose(); -``` - -## Engine posture - -The runtime shim is [`quickjs-wasi`](https://github.com/vercel-labs/quickjs-wasi) used **as-is, -including the npm package's shipped `quickjs.wasm` binary** — the roadmap doc's mapping table -is followed verbatim, and we never build our own binary. `loadShippedWasm()` resolves the -binary through the package export map and compiles it once per process into a reusable -`WebAssembly.Module`. The engine pins `quickjs-wasi` at an exact version to keep the shipped -`quickjs.wasm` byte-identical across installs — but snapshot compatibility (phase D's envelope + -restore) is enforced on the binary itself, **not** the package version: the envelope records and -compares the `quickjs.wasm` **SHA-256** plus the envelope **format version**. An upgrade that -changes that binary's hash — or a format-version bump — refuses old snapshots loudly (both hashes -named), never restoring them silently; a package bump that ships the same binary keeps old -snapshots restorable. - -- **`memoryLimit` per VM** — passed straight through to `QuickJSOptions.memoryLimit` - (quickjs-wasi built-in). Exceeding it fails allocations with - `InternalError: out of memory` (an `EvalErrorInfo` with `outOfMemory: true`); the VM stays - usable. Default when unconfigured: **64 MiB** (`ReplVm.DEFAULT_MEMORY_LIMIT`) — generous for - data-plane state while still bounding what a single workspace can make the daemon hold. -- **`interruptHandler` per eval and per settlement drain** — quickjs-wasi's `interruptHandler` is a per-VM - create-time option, so the engine composes per-operation semantics on top of the built-in: one - VM-level handler delegates to a per-operation slot that `evalCode` arms for the duration of the - eval **and its drain**, and `drainJobs({ interruptHandler })` arms for the duration of a - standalone settlement drain, then restores. Handlers never leak across operations. Returning - `true` aborts with `InternalError: interrupted` (`EvalErrorInfo.interrupted === true`). Note the - interrupt check is instruction-based (quickjs's built-in check interval), so against a - tiny loop body the handler fires comparatively rarely — that is the shim's native behavior. - **Why the drain takes its own handler:** a suspended eval's handler is removed when the eval - returns, and a settlement drain that later resumes a runaway continuation (a continuation left - queued by an interrupted drain, or resumed by host-side settlement) would run unguarded — the - drain boundary therefore carries its own interrupt signal. - -## Eval semantics - -`ReplVm.evalCode` (and `Workspace.eval`) evaluates with `EvalFlags.ASYNC` — the script-global -REPL mode the harness pinned: bindings persist across evals (`var` lands on `globalThis`, -`let`/`const`/`class`/`function` in the shared global lexical environment), sloppy mode, -completion value = last expression, **top-level `await` accepted**, and **top-level `return` -stays a syntax error** (the parser's "return not in function" check is independent of the -async flag — pinned by test). The eval returns a promise; the engine drains the job queue -(quickjs-wasi's built-in `executePendingJobs()`, with the per-eval interrupt still armed, so a -runaway microtask loop is bounded) and reports one of: -| Outcome | Meaning | -|---|---| -| `{ kind: 'value', value }` | completion promise fulfilled within the drain | -| `{ kind: 'pending' }` | suspended on an unsettled promise — no fabricated value; the continuation resumes at settlement like a `.then` | -| `{ kind: 'error', error }` | threw (synchronously, via a rejected completion promise, or via a job error during the drain — the canonical drain error is the per-eval interrupt firing inside a resumed continuation) | - -The eval promise is fulfilled **synchronously** — the completion is read straight from the -runtime through the raw `qjs_promise_result` export, never through `resolvePromise()` (whose -host promise yields through the microtask queue even when already settled). This makes an eval -structurally un-raceable by `dispose()`: `const p = ws.eval('6*7'); ws.dispose(); await p` -returns `42` (review regression: the yielding completion read crashed on nulled WASM exports). -All VM operations serialize for the same reason, so concurrent evals can never reorder the -interrupt-slot save/restore (review regression: a stale handler stayed armed). - -## Trap-free rendering (from day one) - -Rendering guest state is adversarial territory (roadmap doc transfer lesson R69: a single -`[[Get]]` on the completion wrapper let `Object.prototype.value` pollution hijack every eval -result). This package follows the rule from its first line of engine code, and it is enforced -**structurally** — two quickjs-wasi paths that would violate it are never taken: - -- `QuickJS.evalCode()` wraps synchronous failures (parse errors) in a `JSException` whose - **constructor** performs guest-visible `[[Get]]` reads of `name`/`message`/`stack` on the - guest exception — a getter installed on `SyntaxError.prototype.name` runs during error - construction, before any host `catch`. The engine instead drives the same raw `qjs_eval` - export (through the package's public `_getExports()`/`_writeString()` accessors) and reads - a synchronous exception own-property-descriptor-wise itself; the exception value is freed - immediately after. Adversarial tests pin this: getters on - `SyntaxError.prototype.name/message/stack`, `Error.prototype.name`, and - `TypeError.prototype.name` never fire, and a thrown **proxy** reports a trap-free - `[Proxy]` marker (proxies fire traps on descriptor/prototype reads — every such read is - `isProxy`-guarded, including the prototype of an error whose prototype was replaced with a - proxy via `Object.setPrototypeOf`). -- `JSValueHandle.getOwnPropertyDescriptor()` throws a `JSException` when the C descriptor read - fails (allocation edge) — and that constructor runs the same guest-visible getters. The - engine's descriptor path (`readOwnDataProperty`) never calls it: it drives - `qjs_get_own_property_descriptor` directly, takes a failed read's exception value out of - the runtime and frees it (no `JSException` is ever constructed), and reads the - engine-created descriptor object's own data properties through raw `qjs_get_prop_value`. - A regression test forces every descriptor read to fail C-side and asserts zero guest - getter executions and a still-usable VM. -- `QuickJS.executePendingJobs()` renders a failed job's exception through `exc.toString()` — - a JavaScript string conversion that **executes guest code**. The engine's `drainJobs()` - runs the same built-in pending-job loop (`qjs_is_job_pending` / `qjs_execute_pending_job`, - which is all the built-in is) but reads the exception trap-free and throws a - `DrainJobError` carrying `EvalErrorInfo`; `evalCode` converts that into the error outcome. -- The engine-created `{ value }` completion wrapper is unwrapped via - **own-property-descriptor reads** (`getOwnPropertyDescriptor`), never `[[Get]]`. -- Completion values and error info are read the same way: own **data** properties only, - accessors skipped (**and their `get`/`set` handles disposed — a leaked accessor handle - pins guest memory**; review measured a 1 MiB VM exhausting after ~3,128 accessor-valued - completions), proxies and branded objects (`[Promise]`, `[Date]`, `[Map]`, …) rendered as - markers, depth ≤ 4, ≤ 256 properties per level, cycle-guarded. This shallow read is the - conservative seed of the ObjectPreview rendering a later phase owns. -- Error names come from the error prototype's own `name` data property when instances carry - none (quickjs-ng stores `name` on the prototype) — still trap-free; a guest-installed - accessor or proxy prototype is skipped and the name falls back to `'Error'`. -- **No handle is ever leaked from a failed path**: the exception values of failed evals, failed - jobs, and failed descriptor reads are disposed in `finally` blocks, and accessor - descriptors' `get`/`set` handles are disposed on the spot — long-lived VMs must not - accumulate guest memory from error paths (both leaks were measured during adversarial - review and are pinned by bounded-memory regression tests). -- Error rendering converts **symbols** natively (the bare brand, FORMAT.md §5.7): a - thrown `Symbol('x')` reports `Symbol`, never the fabricated `NaN` the default - number conversion produced. The description is deliberately NOT read — the raw - `qjs_get_symbol_description` export invokes guest `Symbol.keyFor` (FORMAT.md §1.1), - a forbidden seam, so `Symbol(x)` is unimplementable trap-free, not merely - unimplemented (review regression, pinned by test). - -The published type graph is also self-contained: the public options take `WasmInput` — a -locally declared stand-in for `WebAssembly.Module | BufferSource` (`ArrayBuffer | -ArrayBufferView | WasmModule`, see `src/types.ts`) — because the repo's tsconfig has no DOM -lib and the ambient declarations the package compiles against are source-only (never -published; the package ships `dist` only). `WasmModule` is **opaque/branded**: its only -producer is `loadShippedWasm()`, so accidental values (`{ wasm: 42 }`, a plain object, a -string) are compile-time errors — pinned by `@ts-expect-error` negative cases in the -consumer fixture (review regression: `WasmModule` used to be an empty interface that -satisfied every non-null value). Custom WASM is accepted as raw bytes (`ArrayBuffer` / -`ArrayBufferView`). A consumer check with the repo's non-DOM lib and -`skipLibCheck: false` is part of the test suite (`test/public-types.test.ts`). - -## The guest library and the bridge (phase B) - -At VM creation the host installs the **guest-side library** — a version-marked plain script -evaluated exactly once in the realm — plus the four `__host_*` callbacks that are the realm's -entire effect surface. The library is this package's fresh implementation (not a vendor of the -harness's `guest/dsl.js`); its source is `src/guest/guest-library.ts`, its version is -`GUEST_LIBRARY_VERSION` (marker global `__REPL_GUEST_VERSION`), and its semantics follow the -roadmap doc's DSL split: only a sliver needs host effects, everything else is pure JS. - -### Sandbox globals - -- `agent(modelSpec, task, options?) → Promise` — the delegation primitive, per the roadmap - doc's own example (`agent("pi/deepseek-v4-flash-max", "research X")`). `modelSpec` is - the backend-routing spec; `task` the worker's prompt; `options` (structured-output - schema, cwd, backend config) cross the bridge as JSON. The returned promise **is** the - live handle: it may sit in a variable across evals, and it carries own non-enumerable - handle methods `queue(prompt, opts?)` / `steer(prompt, opts?)` / `cancel()`, plus `id` - (the stable call id `"c1"`, … used by `interrupt` and reported by `agents()`). `steer` - targets only an ACP prompt currently in flight and resolves `injected`, `idle`, or - `unsupported`; it never starts or queues work. `queue` synchronously returns a distinct - promise-handle with its own `id` and exact `cancel()`, then resolves with that FIFO turn's answer. -- `checkpoint(question, options?) → Promise` and `checkpoint.answer(callId, value) → boolean` - — the data plane interrupting the intent plane. The answer enters the data plane only - through `checkpoint.answer` (the `__host_checkpoint` trailing-argument answer mode); it - returns whether a pending checkpoint with that id was answered. -- `console.{log,info,warn,error,debug}` — the bridge: ONE joined line per call — the - arguments' §4.4 reprs joined with a single space (strings passed directly print whole; - objects/arrays render to depth 2, 20 entries per level, nested strings head-limited at - 200 chars). The line itself is forwarded as `{ line }` to `__host_console` without a - byte ceiling. -- `parallel` / `pipeline` / `verify` / `judgePanel` / `gate` / `retry` / `loopUntilDry` — - pure JavaScript layered on `agent()`, following `packages/workflows/src/dsl.d.ts` semantics. - A rejection with `recoverable: false` halts the surrounding orchestration; any other - rejection is recoverable (a `null` slot in `parallel`/`pipeline`, reported via - `console.warn`). Resource limits are server configuration, invisible to the guest; the - host's non-recoverable signal is exclusively `recoverable: false`. - -### Guest library ⇄ host contract - -| Function | Meaning | -|---|---| -| `__host_agent(callId, modelSpec, task, optionsJson)` | Kick off one worker run against the backend routed by `modelSpec`. May return a thenable (the bridge's `GuestCall` promise) — the guest chains onto it — or `undefined` (settle later via the surface). | -| `__host_checkpoint(callId, question, optionsJson, answerJson?)` | Question mode: three arguments, like `__host_agent`. Answer mode: a PRESENT fourth argument (the JSON-encoded answer) — the host settles the pending checkpoint and returns a boolean synchronously; nothing new pends. | -| `__host_agent_queue(callId, sessionId, payloadJson)` | Create one durable queued public turn on the founding session. | -| `__host_agent_steer(callId, sessionId, payloadJson)` | Strict transient control of the ACP prompt currently in flight. | -| `__host_agent_cancel(callId, sessionId)` | Cancel the session's current public turn. | -| `__host_queue_cancel(callId, queueCallId)` | Cancel exactly one queued-turn handle. | -| `__host_console(level, payloadJson)` | The console bridge, called synchronously with the rendered `{ line }` payload (one joined line per call). | - -Settlement is first-wins idempotent by call id, through two always-valid routes: the live -`GuestCall` (a promise created via the raw `qjs_new_promise` export whose parts the call -owns and disposes completely — the TS analogue of the Rust reference broker's -`new_promise_raw`/`Deferred`; the shim's `newPromise()` Deferred is deliberately not used -because it pins the reject-function handle until VM dispose, measured to exhaust a 2 MiB -VM after ~5,000 resolved calls) or the **reconciliation surface** after a restore. The -surface — `globalThis[Symbol.for("repl.guest")]`, read host-side via -`readGuestSurface(vm)` — exposes `version`, `pending()` (verbatim details for re-issuing -lost work, including `sessionId` — the founding session id for steering calls — and -`modelSpec`), `settle(callId, outcome, value)` and `stats()`; it is frozen, its binding -non-configurable, and its registry operations use captured intrinsics, so `Map.prototype` -pollution cannot corrupt settlement. The returned surface object pins NO guest memory: -every handle it needs is acquired per call and disposed on the spot. The pending-call -registry lives in the library's closure and **travels inside snapshots**; on restore the -host re-registers the four callbacks by name (`registerGuestHostCallbacks`) and -reconciles — the library itself is never re-evaluated (idempotence guard). - -**Eval-await tracking — the continuation lease (version 0.3.1)** — the eval-break - targeting seam (the `interrupt` tool's no-id arm): the library defines - `__replAwait(value, token)` — the global the host's `instrumentTopLevelAwaits` rewrite - inserts around every TOP-LEVEL `await` of an eval (`await x` → - `await this["__replAwait"](x, TOKEN)`; the `this` base is the engine's global-object - binding for the script's async wrapper, so the injected expression names no - shadowable identifier — the phase-E review round-5 hygiene regression: the old - instrumenter's guest-resolvable `__replAwait` identifier was shadowable by a lexical - declaration, changing program semantics). With a token the awaited value is WRAPPED in - a fresh promise. The CONTINUATION LEASE (the writable `__replLease` accessor global) - is set by a reaction registered on the WRAPPER itself — BEFORE the await machinery - registers its own reaction on the same wrapper — so the wrapper's settlement queues - the lease-setting job DIRECTLY BEFORE the machinery job that runs the eval's - continuation segment: the job after the lease-setting reaction IS the segment, and - NO job queued between the awaited value's settlement and the wrapper's settlement can - run with the lease set (round-6 rejection: the 0.3.0 reaction ran on the awaited - VALUE's settlement, so a sibling `q.then(...)` registered after the eval started - awaiting `q` ran between the lease set and the continuation, consumed the armed - signal, and the target's continuation ran later unprotected — the lease is - associated with the actual continuation job, not the next job). The host's drain - loop reads the lease between jobs: a job that starts with a lease set IS the armed - eval's continuation, and the lease is cleared after the segment ends — the armed - signal's genuine per-eval identity. An unawaited sibling `.then` registered before - the target's await runs first in the settlement drain (before the lease-setting - reaction) and can neither fire nor consume the signal; an indirect wait (`await - Promise.all([q])`) is targetable through the promise graph (the 0.2.0 log-only - targeting refused it); a never-settling local promise is refused at arm time (no - pending host call can ever resume it). The surface's `supportsContinuationLease` - reports the capability. For-await loops ride the same discipline through a second - global, `__replAwaitIterable(value, token)` (0.3.1): the instrumenter wraps every - top-level `for await (... of )` ITERABLE in it, and the wrap returns an - ASYNC-ITERABLE — never a promise — that sets the lease per iteration, so the loop - iterates exactly like the un-instrumented program (`for await (const x of [1, 2])` - works — the 0.3.0 wrap returned a promise and made every loop throw `TypeError: not - a function`, the round-6 rejection) and stays breakable mid-iteration. The surface's - `supportsIterableLease` gates the for-await sites: a snapshot carrying the 0.3.0 - library is served as-is with its for-await sites left unwrapped (native semantics, - no mid-loop targeting — the honest degradation). The broker's continuation-lease - availability check is VERSION-GATED on ≥ 0.3.1 (round-7 decision): a restored 0.3.0 - copy reports `supportsContinuationLease: true` but its lease-setting reaction still - runs on the awaited VALUE's settlement — the sibling-reaction defect — so the host - serves it WITHOUT instrumentation and the eval-break interrupt refuses honestly - (the flag alone would re-arm the original defect on a supported older snapshot; - see `Broker.continuationLeaseAvailable`). The instrumentation surface is also - hardened against guest Promise sabotage (round-7 decision): `__replAwait` / - `__replAwaitIterable` (and the host-thenable forwarding in `issueHostCall` — - without it, a replaced `Promise.prototype.then` silently killed every settlement) - mirror values through CAPTURED pristine intrinsics (`P`/`PResolve`/`PReject`/`pThen` - — bound at installation, before any guest code runs), so replacing - `Promise.prototype.then`, overwriting `Promise.resolve`, or shadowing `Promise` - lexically cannot change the instrumentation's semantics (the instrumented - `await 40` stays `40`) or skip the continuation-lease setting. The for-await wrap - also follows GetIterator/GetMethod acquisition semantics exactly once (an - observable/throwing `@@asyncIterator` getter runs a single time and its ORIGINAL - error propagates — the old degrade-to-unwrapped made the machinery acquire a - second time) and passes SYNC-iterable results through - AsyncFromSyncIteratorContinuation (the result VALUE is awaited and unwrapped — - `for await (const x of [Promise.resolve(1)])` yields `1`, never the promise - object). A snapshot carrying the - 0.1.0/0.2.0 library is served as-is (the version-compatibility - rule below): the host skips the instrumenter on it and the eval-break interrupt - degrades to the honest refusal (the 0.2.0 log-only targeting is the rejected - settled-call-ids identity). The transform is a pure source rewrite at exact AST - boundaries (acorn; nested function bodies are never touched — an await inside a - `.then` callback or a combinator thunk belongs to its own continuation, not the - eval's; `for await (... of await y)` needs no iterable wrap — the right expression's - own await is instrumented normally and the loop iterates the unwrapped value) and - injects nothing but the call sites (no helper binding — a top-level - `const` would persist in the realm's global lexical record and redeclare on the loop - idiom). - -**Version compatibility** (the doc's evolution disciplines): the library is versioned with the -workspace, not the host — a host must serve any snapshot whose resident library is the same or -an older version, the host-call surface is append-only (new optional trailing arguments = -minor; new `__host_*` names = major), and the host discovers the resident version through the -surface rather than assuming. `ReplVm.restore` exists so the evolution discipline is testable -now: state, the pending-call registry and the version marker survive a snapshot/restore round trip. - -### The console bridge and the §4.4 repr - -Every `console.*` call renders **ONE joined output line**: the arguments' reprs joined -with a single space (§4.4 [D]). The repr is depth-limited and predictable, with no byte -ceiling on the rendered stream (the Python posture: an agent can flood its own context by -printing something enormous; accepted and documented): - -- strings passed **directly** to `console.log`, and a string **completion value**, print - **whole** — they are the output the orchestrator asked for; -- objects/arrays render to **depth 2**; deeper levels render as `{…}` / `[…]`; -- collections render their first **20 entries** per level, then `… +N more`; -- **nested** strings (inside a collection) render head-limited at **200 chars**; -- everything deeper/longer is reached by evaluating a narrower expression — the values are - alive in the VM; slicing is the API. The sole result-history global is `_` (the previous - eval's completion value, IPython-style); bindings are the memory. - -The repr is generated **side-effect-free by construction** (engine brand checks only, -own-property-descriptor reads only, proxies detected first and previewed *as* proxies, -cycle-guarded via a per-render `seen` set, every argument rendered under its own guard — -`console.*` NEVER throws by contract). The forbidden seams stay unwired: symbol descriptions -are never read (`qjs_get_symbol_description` invokes guest `Symbol.keyFor` — FORMAT.md §1.1), -so symbols render as the bare brand `Symbol` everywhere, including thrown-symbol error -messages, and `qjs_get_array_buffer`'s raw data pointer is never passed to -`qjs_is_exception` (a guest-controlled buffer must not be able to forge a failed read). - -### Unbounded guest output - -Console lines and completion reprs ship verbatim. The previewer is also used internally for -bounded metadata tokens such as manifest entries and checkpoint/task previews; those 200-character -metadata renderings do not alter guest output. - -## The workspace (phase B) - -`Workspace.create` installs the guest bridge at VM creation — the doc's injection discipline: -`agent`/`checkpoint`/the combinators are live from the first eval, never undefined. `options.handlers` -may supply custom bridge handlers; the default is a **parking bridge** (agent/checkpoint/steer calls -park — they pend in the guest registry, visible through `surface()`/`parkedCalls()`, and stay -unsolved until a later phase attaches real backends; parking never fabricates a result — console -events accumulate in `consoleEvents()`). The one deliberate exception is `checkpoint.answer`: -answering a parked question settles the matching pending checkpoint first-wins, so the data plane -can interrupt the intent plane even with no backends attached. The workspace also exposes the -rendering seam -(`renderRef`, `inspectBinding`) and the reconciliation surface (`surface()`) the `repl` tool layer -builds on. A later phase that wires real backends swaps handlers via `registerGuestHostCallbacks` -(the same re-registration the restore path uses) — the broker does exactly that through -`Workspace.rehost`. `Workspace.snapshot()` / `Workspace.restore` are the raw snapshot -seams; the identity envelope (wasm hash + format version + gzip) is the daemon layer's wrap -(`ReplWorkspaceStore`). - -## The broker (phase C) - -`Broker.attach(workspace, options)` takes over a workspace's four `__host_*` callbacks (by-name -re-registration — the guest library and its pending-call registry are untouched) and implements -the doc's broker contract against real ACP sessions through `@automatalabs/acp-agents`: - -- **`agent(modelSpec, task, opts)` dispatches a held-open ACP session** — the runner's - `openSession` with the routing grammar, model spec and per-call `cwd` (default: the workspace's - project directory). The guest option bag is exactly `{ schema, cwd, configOptions, mode }` — - any other key refuses the call (`recoverable: false`). `schema` is a - JSON Schema object validated by acp-agents' own structured-output ladder (`resolveStructuredOutput` - driven over the session: convert/check, native + prose extraction, re-prompt, `SCHEMA_NONCOMPLIANCE` - — the one divergence from `run()`: the client-hosted StructuredOutput MCP capture tool is not - injected on the interactive path). Sessions stay open for the workspace's lifetime (the - live-handle contract) and are opened with `keepSession: true`, so the ACP session persists on the - backend for the restore path's lazy re-attach. -- **Six concurrent subagents per workspace** (doc-settled; `maxConcurrentAgents` configurable — - server configuration, invisible to the guest). The limit counts live work: unsettled agent calls - plus active queued turns. Founding calls and queued turns share one admission sequence; the - oldest eligible item takes the next free slot while an ineligible old item never blocks another - session — never a rejection, matching the workflow - engine's semantics (`parallel(items.map(...))` never loses work). -- **Strict steering is transient active-turn control.** With an ACP prompt in flight and the exact - raw initialize advertisement, the broker sends one `_session/steering` request with - `idleBehavior: "promptRequired"`; otherwise it sends nothing and resolves `idle` or `unsupported`. - Transport/server failures reject. Malformed responses and `startedNewTurn` are fatal protocol - violations; no path converts steering into future work. `queue()` is the only future-turn API: - every call owns a durable FIFO record, answer, cancellation target, and ordinary `session/prompt`. -- **The append-only call store** (`src/store.ts`, transfer lesson 1): every call's outcome is - recorded by call id BEFORE it is settled into the guest. `InMemoryCallStore` for tests and - ephemeral hosts; `JsonlCallStore` is the durable append-only JSON-lines file — every mutation - one fsynced line, torn-tail repair on open (fragment sidecarred then truncated; unterminated- - but-complete records kept; newline-terminated corruption refused), and appends heal to the - acknowledged prefix after a failed write. The pump's delivery loop is record → settle → - consume, with both sides first-wins idempotent — a crash between the store write and the guest - settlement is healed by the next delivery, exactly once (pinned by the simulated-crash tests, - including the snapshot/restore + `reconcile()` path). -- **The eval tool-result seam** (`Broker.eval` → `{ output, kind, result?, evalToken?, pending, - checkpoints, completed }`): output lines (console events rendered through the previewer — ONE - joined line per console call, non-log levels prefixed `warn:`/`error:`/… — never capped), the - previewed completion value when the eval resolved (trap-free, from the live completion - handle), the pending call ids when it suspended (no fabricated value), the raised - checkpoints (previewed questions), the call ids this operation settled (checkpoint answers - deliberately excluded — an answered id leaves the `checkpoints` list), and `kind` naming the - outcome (`value`/`error`/`pending`) with the continuation `evalToken` the fused-eval pump - attributes swept settlements with. The §3.1 wire shape `{ output, result?, running? }` is - assembled by the tool phase over this seam. Eval errors render as - `Name: message` lines in `output`. -- **Suspended-eval semantics** (transfer lesson 3): top-level `await` accepted; an eval whose - completion resolves within its drain reports the previewed value; a suspension returns - immediately with the pending call ids; the continuation resumes at settlement like a `.then` - (its output lands in the next tool result); a late uncaught rejection surfaces as an - error-level console line in the next tool result (the VM's rejection bridge, armed by the - broker); top-level `return` stays a syntax error. -- **Checkpoints** (transfer lesson 4): `checkpoint(question)` parks a promise and records the - dispatch; the question appears in the tool result's `checkpoints` list previewed through the - top-level string rule (quoted, head+tail elided past 200 chars — guest-chosen text never - crosses unbounded); `checkpoint.answer(id, value)` in a later eval records the answer and - settles the parked promise within that eval — root-mediated by construction, first-wins, and - the answer's continuation output lands in the delivering eval's own tool result. - -The broker's public type surface is fully self-contained (structural `BrokerRunner`/ -`BrokerSession` stand-ins — no acp-agents or quickjs-wasi types leak into the published -declarations; verified by the consumer fixture). `Broker.eval`/`pump`/`reconcile`/`dispose` -serialize, so overlapping tool calls can never interleave settlement bookkeeping. - -## Decisions for spec-owed details - -These are the decisions this phase (the broker/engine tier) made where the roadmap doc left -room. The later phases that build on them — the daemon wiring and the `repl` MCP tool — have -since shipped (roadmap phase E; see [Daemon wiring](#daemon-wiring-phase-d-in-mcp-server) -below). - -- **Default memory limit: 64 MiB per VM** (configurable per workspace and per registry). -- **Per-eval and per-drain interrupts composed over the built-in per-VM handler** (see - Engine posture) — this is the only composition quickjs-wasi's API allows, and it keeps the - whole interrupt mechanism on the built-in `qjs_set_interrupt_handler` path. A standalone - settlement drain arms its own handler because the suspended eval's handler is gone. -- **`` as the default eval filename** for guest stack traces. -- **Eval completion is synchronous**: the completion value is read through the raw - `qjs_promise_result` export instead of the shim's `resolvePromise()` (whose host promise - yields even when already settled). This makes `dispose()` structurally un-raceable and - serializes all VM operations, which in turn makes the interrupt-slot save/restore - concurrency-safe (an `opDepth` reentrancy guard makes the serialization invariant - structural). -- **Drain errors are authoritative eval errors**: when a drained job throws (interrupt-in-job - is the canonical case), the eval reports that error; the guest exception has already been - consumed and cleared by the drain loop, so the VM stays usable. The drain is the built-in - pending-job loop, but the failed job's exception is read trap-free (see Trap-free - rendering) and thrown as `DrainJobError` — never rendered through `toString()`. -- **A failed eval's exception value is freed immediately** (in a `finally`), and accessor - descriptors' `get`/`set` handles are disposed on the spot — long-lived VMs must not - accumulate guest memory from error paths (both leaks were measured during adversarial - review and are pinned by bounded-memory regression tests). -- **The public wasm surface uses self-contained, branded types** (`WasmInput`/`WasmModule` - from `src/types.ts`) instead of the DOM-lib `BufferSource`/`WebAssembly.Module` names, so - the published declarations compile under the repo's non-DOM lib with `skipLibCheck: false` - — and `WasmModule` is opaque, so only `loadShippedWasm()` can produce one (custom wasm - goes in as raw bytes). -- **The registry dedupes the in-flight creation promise**: concurrent first-touches of one - project key share a single creation, so exactly one VM is instantiated per workspace (the - first caller's options win). `dispose` during an in-flight create cancels it — the created - VM is torn down without materializing, the waiting `get` rejects, and a later `get` - starts fresh. -- **Primitive error rendering follows native conversions for every primitive type**, with - symbols as the one deliberate exception: a thrown symbol renders the bare brand `Symbol` - (FORMAT.md §5.7) — its description is not readable trap-free, because reading it reaches - `qjs_get_symbol_description`, which invokes guest `Symbol.keyFor` (FORMAT.md §1.1). A - guest that replaces `Symbol.keyFor` must not be able to forge error rendering (pinned by - test). -- **Realpath validation of `projectDir`** is deliberately NOT here: that is the daemon's - project-registry concern (the `repl` tool's phase); the registry keys by the string it is - given. - -Phase C decisions (the broker, the call store, the eval tool-result semantics): - -- **The broker dispatches held-open sessions** (`runner.openSession`), not one-shot `run()` - calls: the reusable-handle queue contract requires a session - that outlives the call. Sessions are opened with `keepSession: true` (the ACP session persists - on the backend for the restore path's re-attach) and stay open while any MCP client is - connected to the project; on last-client disconnect the daemon drives the client-presence - drain (`drainForDisconnect` — in-flight turns drain to completion, then idle children - close); the next eligible queued turn re-attaches its recorded session lazily. Idle steering - and idle cancellation never reattach. `schema` calls - drive acp-agents' own `resolveStructuredOutput` over the session (`tryNative` = raw - structured output, else the generic parse-final-JSON dialect) — the one divergence from - `run()`: the client-hosted StructuredOutput MCP capture tool is not injected on the - interactive path. -- **The concurrency limit counts active public turns.** Founding calls and queued turns share one - workspace admission sequence; the oldest eligible item gets a free slot, while pending queue - items, idle sessions, steering, and cancellation consume no extra slot. -- **Steering and queueing are separate.** `steer()` is strict active-prompt control and returns - `injected`, `idle`, or `unsupported`; transport/protocol failures reject and no prompt fallback - exists. `queue()` creates one addressable FIFO public turn whose answer and schema repair remain - under its own call id. `cancel()` returns `cancelled` or `idle`; the selected public turn rejects - recoverably with `AGENT_CANCELLED`. -- **The store records refused calls too** (dispatched + rejected with the refusal error): - without the record, a restore would re-issue a call that was deliberately refused. -- **`completed` excludes checkpoint answers** (the harness's pump convention): an answered id - leaves the `checkpoints` list — that is its visibility; `completed` reports delegated work - (pump deliveries + dispatch-time refusals). -- **`result` is the FORMAT.md collapsed rendering** of the completion value (the bare body — - `42`, `"hello"`, `{a: 1, …}`), previewed from the live completion handle through the - previewer's own trap-free machinery (the engine's internal eval-with-completion seam returns - the unwrapped value handle; the published type graph stays clean). Eval errors render as a - plain `Name: message` line in `output` (the harness's "thrown-exception message" convention; - late uncaught top-level rejections are the `error:`-prefixed console-bridge lines). -- **Checkpoint questions cross previewed** via the previewer's top-level string rule - (`stringDescription`: quoted, head+tail elided past 200 chars) — the harness's R74 rule; - the id stays exact. -- **The broker serializes its async operations** (eval/pump/reconcile/dispose share one - promise chain): two overlapping tool calls can never interleave settlement bookkeeping or - the eval's pump-before-eval ordering. The pump delivers ready outcomes one at a time - (record → settle → consume), keeping a failed delivery staged for the next pump — both the - store write and the guest settlement are first-wins idempotent, so the retry settles exactly - once. -- **`Workspace.snapshot()`/`Workspace.restore` are the raw snapshot seams** (the daemon layer - wraps the identity envelope later); `Workspace.rehost` is the by-name callback re-registration - the broker uses to take a workspace over — the same re-registration the restore path uses. - -Phase D decisions (snapshots + restore; see also the "Snapshots and durability" section): - -- **The envelope is a JSON header line + gzip of the shim's own `serializeSnapshot()` output** - (its versioned QJSS binary with extension metadata) — the doc's "serializeSnapshot() output - wrapped in the identity envelope" is followed verbatim; gzip is the shim-documented - compression choice (JS runtimes decompress it natively). The header carries format name + - format version + wasm sha256 + createdAtMs; the restore path compares the recorded hash - against `wasmSha256Of` of the binary it restores with and REFUSES LOUDLY naming both - hashes. The format version is a second refusal axis (version-bump test included). -- **`loadShippedWasm` records the shipped binary's hash against the compiled module** — - `wasmSha256Of(module)` resolves through that registry; a module the engine did not load - cannot be hashed (bytes are not recoverable from the compiled form) and refuses loudly - (pass raw bytes instead). -- **The repl store reuses `@automatalabs/workflows`' store-layout helpers verbatim** - (`workflowProjectPaths` — the mcp-server project registry's own helpers), so the store key - derives from the project directory exactly as the workflow engine's and one project has - one repl store. Files: `repl/snapshot.bin` + `repl/calls.jsonl`. -- **Atomic writes are tmp + rename + fsync** (fixed-name `.tmp`, single-writer discipline; - best-effort directory fsync); a failed write removes the tmp and throws, leaving the - previous snapshot untouched; the store directory self-heals on write after a `reset()`. -- **Restore-time corruption is contained in the same refusal family** - (`SnapshotRestoreError`, code `RESTORE_CORRUPT` — a `SnapshotEnvelopeError` subclass, so - the daemon's single containment catch covers the whole load path): the envelope's decode - checks now include pointer-BOUNDS validation (runtime/context/stack pointers must be - integers strictly inside the snapshot memory — a corrupted in-range VM header like - `contextPtr: 0xfffffff0` refuses as `CORRUPT_PAYLOAD` at decode, before any VM exists), - and a payload that passes every at-rest check yet cannot be materialized (a header - patched to a wrong-but-in-bounds value, a guest surface that cannot be rehosted, a - provenance registry that cannot bootstrap) refuses from `Workspace.restore` naming the - underlying failure — after DISPOSING the partially created VM. The daemon records the - refusal as stable state (later touches surface it without re-attempting the restore; - `reset` clears it) — never a raw `RuntimeError` retry loop into garbage. -- **The safe-re-issue fence is re-checked after every awaited release** (`reissueReattached` - and the reconcile catch arm): the loaded session's `release()` can park past the - client-presence drain's bound (or a disposal's generation bump), during which the drain's - forced stop settles the call durably and reports `isDrained`; a re-issue that resumed - after the release would record a reissue and open a FRESH child post-drain. The - generation captured at entry is re-checked after the await — a fenced landing holds the - call (no reissue recorded, nothing opened; the call stays as the drain/disposal left it). -- **The debounce is boundary-in/burst-out**: the broker fires `boundary(kind)` per - doc-defined boundary (after each eval; after each settlement drain that changed VM state) - and `flush()` at the end of each serialized operation; the store's `snapshotWriter` - debounces the burst into one atomic write. `SnapshotWriteOptions.debounceBursts` (default - true) and `fsync` (default true) are the decided knob names; `ReplStoreOptions.persistenceRoot`/ - `env` override the workflow home. -- **The re-attach arm keys on a store-recorded backend session id** (`recordAttached`, - written at session open BEFORE the prompt — a new append-only log event; overwrites on - re-issue so a later restore re-attaches the CURRENT session). The capability gate is the - runner's own `loadSession` (acp-agents' `supportsLoadSession` — a custom backend that - omits it degrades through the same gate, surfaced guest-visibly). -- **`BrokerSession.awaitCurrentTurn` is REAL on the acp-agents adapter** (the loaded - session's founding-turn completion; `InteractiveSession.awaitCurrentTurn`, phase-D - review round 1: the seam used to be absent, so every built-in backend loaded, released, - and re-issued). Its completion evidence is the **`_session/loaded_turn` vendor - extension** (phase-D review round 3: the quiet-grace heuristic — a settled stream with - a trailing assistant chunk treated as completion, which durably settled an assistant - PARTIAL as a completed-while-down turn when the next live chunk arrived later — and the - blind re-issue fallback, which duplicated a still-running backend turn, were both - rejected; an AUTHORITATIVE terminal channel is required). `session/load` obliges the - agent to replay the entire persisted conversation and only then resolve the load; the - runner marks the LOAD BOUNDARY synchronously after the response, and the seam then - asks `_session/loaded_turn/query` whether the founding turn is still running RIGHT - NOW. The backend answers one of three terminal classifications: (1) `completed` — the - turn observably completed while the host was down, so the replay's trailing assistant - message is its FINAL message and the seam resolves immediately with the REAL - accumulated text (`stopReason` synthesized `end_turn` — the protocol's replay carries - none; the broker's result-shaping gates still apply); (2) `interrupted` — the turn - ended without a terminal assistant message and no turn is running, so the seam rejects - with the SAFE-RE-ISSUE class (nothing to duplicate); (3) `running` — the turn is still - executing at the backend, so the seam KEEPS THE LOADED SESSION ATTACHED and waits for - the authoritative `_session/loaded_turn/ended` notification (a quiet gap is only a - progress-stream gap, never terminal evidence), absorbing the live update stream and - settling with the turn's REAL accumulated text at the notification, bounded by - `AGENTPRISM_ACP_LOADED_TURN_MAX_WAIT_MS` (default 15 min — the "never hang - unobserved" backstop). A backend WITHOUT the `_session/loaded_turn` extension — the - built-in **claude** and **opencode** backends today — is classified by the seam's - **observation path** instead (phase-F review round 2: the old degradation released the - loaded session and blindly re-issued, which can duplicate a still-running backend turn - — re-issue is now reserved for the observably-dead classes). The observation path is - the post-load **continuation watch** (any content update after the load boundary is - live continuation — the authoritative still-running signal, which flips to the - keep-attached wait) plus the replay probe under the **connection-death contract**: the - built-in ACP servers terminate in-flight turns when the client connection closes - (live-verified) and their persisted transcripts hold only completed messages, so at - restore the founding turn is never still running — the replay's trailing assistant - message is the turn's terminal message (`completed`-while-down, settled from the - replay), and anything else means it died mid-way (the safe-re-issue class, no - duplication possible). A `running` turn past the max-wait bound rejects with - `LoadedTurnStillRunningError`: the **re-armable** form re-arms the seam on the - still-attached session (a later notification or a cancel still settles the call), and - the **non-re-armable** form (a third-party seam that can NEVER observe the terminal - state) is NOT re-invoked — the broker keeps the loaded session attached and waits for - the terminal state from the session's own ended notification, the call's cancel, the - session's release, or the client-presence drain's forced stop (a possibly-running call - is never re-issued). A turn that failed at the backend rejects with - `LoadedTurnFailedError` (a definite outcome, settled as an ordinary rejection, never - re-issued); everything else (no user message in the transcript, `interrupted`, a dead - process) is the safe-re-issue class. A handle that was never load-marked rejects - immediately (without the boundary the completion is not observable and the seam never - guesses). The broker arms the re-attached call on the seam WITHOUT blocking reconcile: - reconcile returns immediately, the pump delivers the completion through the same - record → settle → consume path as a live call. Only a third-party `BrokerSession` - adapter WITHOUT the seam at all re-attaches the session and then degrades through the - re-issue fallback — the seam absence is a capability omission. -- **Backend identity/pool routing is persisted** (phase-D review round 2): the store - records the model spec VERBATIM (including the guest's `"default"` sentinel) AND the - RESOLVED backend id at session open (`recordAttached` — a backend id doubles as a model - routing spec). The restore's re-attach, the lazy re-attach, and re-issues all route by - the recorded pin — never by the CURRENT configured default, so a changed default - across a restart can never load or re-issue on the wrong backend and miss a - still-resumable original session. -- **Queued turns re-attach lazily.** After the client-presence drain, the next eligible - queue head loads its recorded backend session through `loadSession`; failure rejects the entire - lane with `session_reattach_failed`, never opens a blank replacement. Idle steering and idle - cancellation perform no load. Concurrent queue admissions share one reattachment. -- **The client-presence drain** (`Broker.drainForDisconnect`, phase-D review round 2): - in-flight turns DRAIN TO COMPLETION **within the bound** (each settlement boundary - snapshots — a turn that finishes in time is never cancelled), bounded by the spec-owed - concrete bound — the daemon's `REPL_DRAIN_BOUND_MS` (`AGENTPRISM_REPL_DRAIN_BOUND_MS`, - default 2 h; it originally reused the session-eviction TTL, since decoupled so dead - clients are collected promptly; a turn that OVERRUNS the bound is force-cancelled — the honest - bounded teardown, settled as the recoverable `AGENT_CANCELLED`), then every idle child - closes (`keepSession` keeps the backend - sessions re-openable; pending queued turns remain durable against their founding - session ids). The workspace and broker stay alive; the next eligible queue head lazily - re-attaches. The drain WAITS for calls - still OPENING (`openSession` parked — an opening call has no session entry yet, so a - drain that considered only registered busy sessions returned `true` immediately and - let the child open and run after the last client disconnected) and in-flight lazy - re-attaches, and a parked open that outlives the bound is STOPPED (the late child is - closed before it ever prompts, the call settles as the recoverable `AGENT_CANCELLED`, - and queued turns fail explicitly when no reusable session was established); the outer bound is ABSOLUTE — every - post-deadline cancel/release await races the remaining time, so a hung backend can - never block disconnect/shutdown past the eviction TTL. -- **The per-eval wall-clock deadline** (`BrokerOptions.evalTimeoutMs`, default 30 s, - `AGENTPRISM_REPL_EVAL_TIMEOUT_MS`; phase-D review round 2): every eval and settlement - drain runs under a deadline enforced by the quickjs interrupt handler, COMPOSED with - the configured signal handler — so a runaway eval can never hang the workspace - forever: the deadline ALWAYS bounds it (a synchronous eval blocks the event loop before - a later interrupt request could arm the signal, so the deadline, not the signal, is the - last-resort bound; phase F adds the out-of-band relay that breaks a synchronous runaway - before the deadline). This is distinct from the interactive no-id `interrupt`, which - breaks a *yielding* eval promptly but honestly **refuses** (`refused-idle`) the cases it - cannot key a resumption to — a never-settling local promise, or a restored older guest - that predates the continuation-lease seam. The VM stays usable after an interruption. -- **The workspace manifest** (`Broker.workspaceManifest()`, phase-D review round 2; the - doc's status surface): top-level USER bindings (fresh-realm baseline set difference — - the baseline is captured once per process from a throwaway VM provisioned exactly like - a real workspace, and the engine-versioned library never grows the realm's baseline) - with structure-only tokens (`{2 keys} · 1.2kB`, `string · 10B`, `number`, `Array(3) · - …` — metadata, never content: no value fragments, no nested names), provenance labels - (`via eval N` / `via worker cN` / `session restore` — from the in-realm provenance - registry, which is HOST policy (bootstrap-installed with the baseline as its `known` - set) so it travels inside snapshots without touching the guest library; the - maintenance pass runs after every eval and settlement drain, trap-free descriptor - reads only, sanitized at render), and live-handle status (`agent handle · - pending|settled · call cN` — the call id maps to the task and timestamps in the - store) and the doc's full provenance surface — `task` (the founding `agent()` call's - task text for `worker cN` and handle bindings, limited to 200 chars) and - `provenanceAtMs` (the attribution wall clock; phase-D review round 3: bindings used - to carry only the label and an internal timestamp). The sole result-history global is `_` - (the previous eval's completion value, §4.4). -- **Pending steering is never replayed.** A steering operation interrupted by restart rejects - recoverably with `details.reason: "steering_interrupted"`; re-injecting would duplicate control. - First-class queue records are restored independently: an unhanded queue head remains eligible, - while a handed-off turn requires authoritative terminal classification and is never blindly resent. - Pending checkpoints re-surface into the broker's checkpoint table - (`PendingCheckpoint.call` is null on that path; answers settle through the reconciliation - surface). Reconcile is idempotent (an `isTracked` guard never re-attaches/re-issues twice) - and adopts store-unknown entries (foreign snapshot / wiped store) so the replay ledger - stays complete. Re-issues respect the concurrency limit (additional re-issues QUEUE for the - next free slot — never a rejection). - -Phase E review round 3 decisions (the carried review's three defects, as re-verified in -round 5): - -- **The eval-break signal is keyed to the armed target's CONTINUATION, not to whichever drain runs next.** The carried defect: the drain-phase interrupt handler was installed on every later eval's drain without checking whether that drain resumed an armed target — an unrelated finite eval B (or an unrelated settlement drain) consumed the signal and the interrupted-drain release cleared the target's tracking while its checkpoint stayed pending and uninterruptible. The armed identity is the target's CONTINUATION TOKEN (round 5): the guest library's `__replAwait(value, token)` wrap sets the continuation lease to the eval's token in the job immediately before the eval's continuation segment, the drain loop mirrors the lease per job, and the signal fires only while the executing JOB holds an armed token — the executing job IS the target's continuation. An unrelated drain — and an unrelated JOB inside a drain that settled a target's call (an unawaited sibling `.then` registered before the target's await runs first, before the lease-setting reaction: it can neither fire nor consume the signal) — leaves the armed state intact; an indirect wait (`await Promise.all([q])`) is targetable through the promise graph (round 5's regressions). The interrupted-drain release (`releaseInterruptedEval`) is exact the same way: the interrupted job's lease names the eval whose continuation was actually executing — exactly that eval is released (a deadline-broken resumed runaway releases its tracked eval even when no signal was armed — a stale target would make a later arm target a dead eval); an unrelated interrupted drain leaves the armed state and every tracked eval intact. A no-id interrupt with NOTHING BREAKABLE — no eval in flight, or every in-flight eval suspended with NO pending host call (a never-settling local promise — no execution can ever resume it; a suspended eval's continuation is always queued by a pending call's settlement, directly or through any promise chain) — REFUSES and arms nothing. -- **The bounded wait sleeps only for the remaining time**: `waitForCalls`'s inter-pump sleep is `min(50, deadline - now)` (the carried defect: the unconditional 50 ms sleep made every sub-50 ms `timeoutMs` take ~51 ms, violating the bounded-wait contract). The disconnect drain's pumps already did this; the wait now matches. A zero `timeoutMs` still performs ONE immediately available state read (round 5's regression: the chain acquisition used to return unacquired with the deadline already past, so an idle workspace reported `drained: false` and a pending call's surface read as empty). -- **The pending surface reports the WHOLE guest registry**: the trap-free reader's generic 256-element array limit once shortened the guest surface's `pending()` list, and its marker mapped to `undefined` in the broker's id list. `readValue` still bounds general preview reads (default 256); host-owned metadata surfaces (`readValueComplete` — the pending registry, the await log, and the provenance registry's `read()` result) read the complete arrays/objects because they are the frozen guest library's own metadata, bounded by VM memory like the metadata itself. - -Phase E review round 6 decisions (the carried review's three defects): - -- **The lease is associated with the ACTUAL CONTINUATION JOB, not the next job.** The carried defect: the 0.3.0 lease-setting reaction ran on the awaited VALUE's settlement (inside the job that resolved the wrapper), so a sibling `q.then(...)` registered AFTER the eval started awaiting `q` ran between the lease set and the continuation — the drain attributed the lease to the SIBLING job, fired the armed signal on it, and the target's continuation ran later unprotected (repro: `siblingDone: false`, then `targetDone: true`). The 0.3.1 reaction is registered on the WRAPPER promise itself, immediately before the await machinery's own reaction: the wrapper's settlement queues [lease-setting, machinery] adjacently, so the job after the lease-setting job IS the continuation, and no job queued between the value's settlement and the wrapper's settlement can run with the lease set. Regression: a deferred sibling reaction registered after `await q` completes (`await deferred` resolves `sibling:resumed`) while the target's own continuation is the job broken mid-run. -- **The for-await iterable wrap preserves the iterable protocol.** The carried defect: the 0.3.0 instrumenter wrapped every top-level `for await` iterable in `__replAwait`, whose promise result made `for await (const x of [1, 2])` throw `TypeError: not a function` instead of iterating. The 0.3.1 surface adds `__replAwaitIterable(value, token)`: an ASYNC-ITERABLE wrapper (resolved exactly like `for await` resolves an iterable — `@@asyncIterator` then `@@iterator`; a promise iterable throws the same TypeError) whose per-`next()` results are lease-wrapped promises (registered before the machinery's own reactions), so the loop iterates natively and remains breakable mid-iteration. `for await (... of await y)` is not wrapped at all (the right expression's own await is instrumented normally). The instrumenter gates the for-await sites on the new `supportsIterableLease` surface flag; a 0.3.0 snapshot's loops run unwrapped (native semantics, honest degradation). Regressions: array/async-generator/awaited-iterable iteration through the broker, and a mid-loop break. -- **Same-type baseline-global overwrites are tracked and attributed.** The carried defect: baseline-global rebinding was detected only when the value's TYPE TOKEN changed, so `Math = { userOwned: true }` (both values objects) stayed absent from the manifest with no provenance. The provenance registry now captures the ORIGINAL baseline VALUES at creation (descriptor reads in the pristine realm; they travel inside snapshots and are never updated on attribution) and tracks the last-attributed value per known name: the record pass re-attributes on SameValue difference (a second same-type rebind re-attributes to its own eval; a pre-snapshot rebind is not re-attributed by the first post-restore pass), and the registry's read reports the changed-known list (current value no longer SameValue to the ORIGINAL baseline, or token changed) which the manifest's filter consults alongside the host-side token check. In-place mutation of a rebound value still does not re-attribute (the documented stance). Regression: `Math = { userOwned: true }` is listed with `object` type and `eval 1` provenance. - -Phase B decisions (the guest library, bridge, previewer): - -- **`repl.guest` as the surface key** (`Symbol.for("repl.guest")`, marker global - `__REPL_GUEST_VERSION`) — a fresh namespace for this product's own library (the harness's - `agentprism.guest`/`__AGENTPRISM_GUEST_VERSION` are its sibling project's). -- **Four host callbacks**: `__host_agent`, `__host_checkpoint`, `__host_console`, and - `__host_agent_steer`. The steering callback carries the handle methods - (`queue`/`steer`/`cancel`) as a new host-callback name in the initial major. -- **`agent(modelSpec, task, opts?)` carries the model spec as a first-class argument** — the - roadmap doc's own signature (`agent("pi/deepseek-v4-flash-max", "research X")`). The spec - crosses the bridge to `__host_agent` verbatim and is recorded in the pending-call registry - entry (`modelSpec`) so a restore can re-issue the call against the same routing. -- **Steering payloads are `{ prompt, options }` JSON** (or `null` for cancel) — the host - interprets them; the guest passes the settlement value (the steering outcome) through - verbatim, mirroring the outcome values `acp-agents` surfaces in its steering events. -- **A pending steer is snapshot-reconcilable**: `__host_agent_steer` receives the operation's - OWN registry id first (the settlement key) and the founding session id second, and the - registry entry records both (`id` + `sessionId`) in the pending manifest — the host can - durably settle (by registry id) or re-issue (to the session) a pending steer after a - restore (review regression: the entry used to omit the founding id). -- **Combinator model specs**: `verify`/`judgePanel` spawn their reviewers/graders through - `agent("default", …)` — the DSL options are exactly `{ reviewers, threshold, lens }` - and `{ judges, rubric }` (packages/workflows/src/dsl.d.ts); there is no per-call model - option (an invented `opts.model` was removed in review). The `"default"` sentinel is - host-routed to the configured default backend (mirrors dsl.d.ts, where reviewers - inherit the run's default model when none is given). -- **The handle is the promise**: `agent()` returns the promise itself with own non-enumerable - `id`/`queue`/`steer`/`cancel` — started-not-awaited handles come free with top-level - await, per the doc (`const research = agent(...)`; end the eval; check in next call). No - `agent.start`/`agent.continue` variants (the doc does not carry them; `queue` is the - continuation vector). -- **Non-recoverable = `recoverable: false` exclusively.** -- **`retry` mirrors the workflow engine exactly**: without `until`, the FIRST attempt's - result is returned (`workflow.ts`: `if (!opts.until || opts.until(last)) return last` — - "stopping early once `until(result)` holds" holds trivially when there is no predicate); - with `until`, attempts run until the predicate holds or `attempts` are exhausted, and the - last result is then returned for the caller to inspect. Review regression: the guest used - to run every attempt without `until`, diverging from the repository DSL. -- **`loopUntilDry` dedupes within rounds too** (the harness dedupes across rounds only) — - "collecting fresh (deduped by `key`) items" is honored completely; the default key degrades - to a safe string for non-serializable items instead of throwing. -- **The repr handles hostile values without ever throwing** (the §4.4 repr renders each - console argument under its own guard): proxies render - *as* proxies, cycles are preserved by a per-render `seen` set, and an unstringifiable - value degrades to the `[unstringifiable]` marker — `console.*` NEVER throws by contract, - so a hostile value cannot take down guest code. (Pinned by the repr tests.) -- **`GuestCall` owns and disposes every handle it touches** (the Rust broker's Deferred - discipline): the marshalled value is disposed after settling, both resolving functions are - disposed at settlement (raw `qjs_new_promise` parts — the shim's `newPromise()` pins the - reject function until VM dispose, measured to exhaust a 2 MiB VM after ~5,000 resolved - calls), and the promise handle is released via microtask once the host-callback trampoline - has dupped it (the shim's host_call path never frees the host-side original). Pinned by - 5,000-call / 20,000-call bounded-memory tests. -- **A throwing handler disposes every deferred part** — `GuestCall.dispose()` frees the raw - promise handle (synchronously — on the throwing path the trampoline never dups a return - value) and both resolving functions, without settling, and every `__host_*` question-mode - maker wraps its handler call in try/catch: the documented synchronous-refusal path can no - longer strand the unused raw promise/resolver handles (review regression: ~3 JSValues + - heap boxes leaked per refusal — 30,000 rejected calls filled a 2 MiB VM and the next - normal agent call failed with `Error: null`; pinned by the 30,000-refusals - bounded-memory test, which then re-registers working handlers and proves the VM is - healthy). Answer mode mints no `GuestCall`, so a throw there has nothing to dispose. -- **The console path and the argument gatherers use captured intrinsics.** The library is - evaluated exactly once at VM creation, before any guest code can run, so it captures what - it needs then: a bound copy of `Array.prototype.slice` (created via - `Function.prototype.call.bind` at installation — no property lookups at call time, so - replacing either prototype method with a throwing function cannot make `console.*` or - `pipeline()` throw; `console.*` NEVER throws by contract; review regression, pinned by - test). -- **`readGuestSurface` returns a surface that pins no guest memory**: every handle is - acquired per call and disposed on the spot (review regression: the surface used to capture - three owned function handles in closures with no disposal contract). -- **The console payload is `{ line }`** — the guest renders ONE joined line per call - (§4.4) and forwards it verbatim. -- **`ReplSnapshot` is a self-contained structural stand-in** for the shim's `Snapshot` type, - so the public `ReplVm.restore` declaration stays checkable by a non-DOM `skipLibCheck: - false` consumer; snapshots produced through the shim satisfy it without conversion. -- **The internal shim is reached through a module-scoped map** (`getVmShim`, private to the - package) — the published type graph never names a quickjs-wasi type (verified by the - consumer fixture). -- **Engine quirk pinned**: a `value` GETTER on `Object.prototype` makes quickjs-ng's - async-eval completion wrapper come out empty (engine-internal, guest-code-free — the - getter never fires, verified by counter); eval completions honestly degrade to `{}` under - that pollution, and the trap-free fallback never fabricates a value. Similarly, the - engine's spec-mandated thenability check fires a polluted `then` getter once per eval — - before any of our code runs; the previewer itself adds zero getter fires (pinned by - baseline-count tests). - -## Snapshots and durability (phase D) - -Disk persistence is v1 scope (the roadmap doc's §Snapshots): the workspace survives daemon -restarts — the property that makes a "persistent REPL" trustworthy. Three cooperating pieces: - -- **The identity envelope** (`src/snapshot-envelope.ts`, transfer lesson 5): the shim's own - `serializeSnapshot()` output wrapped in a JSON header line + gzip — the header carries the - format name (`repl-snapshot`), the envelope format version (`SNAPSHOT_FORMAT_VERSION = 2`), - the **wasm-binary sha256** (`wasmSha256Of` — raw bytes hash directly; a compiled module - hashes through the registry `loadShippedWasm` populates) and the creation time. A restore - whose recorded hash mismatches the running binary REFUSES LOUDLY naming both hashes - (`WASM_HASH_MISMATCH`) — never a silent restore into garbage; a version bump refuses - naming both versions (`VERSION_MISMATCH`); corrupt/truncated files refuse with a specific - `SnapshotEnvelopeError` code, single-shot, no crash-loop. -- **The per-project store** (`src/repl-store.ts`): `ReplWorkspaceStore` lives in a `repl/` - subdirectory NEXT TO the workflow state under `workflowHomeDir()/projects//`, reusing - `@automatalabs/workflows`' store-layout helpers verbatim (the same helpers the mcp-server - project registry uses — one project, one repl store). It holds `snapshot.bin` (the - enveloped snapshot) and `calls.jsonl` (the broker's durable `JsonlCallStore`). Snapshot - writes are atomic (tmp + rename + fsync, best-effort directory fsync — a kill at any - moment leaves either the old complete snapshot or the new one). -- **The snapshot cadence + debounce**: the broker fires a state-changing boundary after each - eval and after each settlement drain that changed VM state (`BrokerOptions.snapshotSink`: - `boundary(kind)` per boundary, `flush()` at the end of each serialized operation — the - burst boundary). The daemon wires it to `store.snapshotWriter(workspace, wasm)`, which - debounces one drain burst's boundaries into a single atomic write taken before the - operation's promise resolves (a broker eval that pumps settlements and then drains the - eval is ONE write). The debounced gap is always covered by the call store — settlements - are recorded BEFORE they settle, so a restore replays them from the store arm. Config - knobs (decided names): `SnapshotWriteOptions.debounceBursts` (default true) and - `SnapshotWriteOptions.fsync` (default true), plus `ReplStoreOptions.persistenceRoot` / - `env` for the workflow-home root. -- **The restore path with the full three-way reconciliation** (transfer lesson 1): - `Broker.reconcile()` reads the in-VM pending-call registry and settles every outstanding - call exactly one way — completed while down → **settle from the store**; still resumable - at the backend → **re-attach** via `runner.loadSession` (the capability gate is the - runner's own, per acp-agents' `supportsLoadSession` — all four built-ins advertise it per - docs/api.md; a custom backend that omits it degrades through the same gate, surfaced - guest-visibly as a warn line); lost → **re-issue** under the same call id (reissues - counter bumped, the existing guest promise settles exactly once, the concurrency cap - applies). The re-attach keys on the backend session id the store recorded at session - open (`recordAttached`, written BEFORE the prompt — a crash with a turn in flight leaves - a restore able to re-attach instead of duplicating) and routes by the store's RECORDED - backend id (never the current configured default). A re-attached call's completion is - the loaded session's founding turn, observed through the REAL - `BrokerSession.awaitCurrentTurn` seam on acp-agents' `InteractiveSession` — the - `_session/loaded_turn` extension's authoritative terminal classification (a `completed` - answer settles from the replay immediately; an `interrupted` answer re-issues safely; - a `running` turn is KEPT ATTACHED and settles only from the `_session/loaded_turn/ended` - notification — a quiet gap is never settled, a still-running turn is never re-issued; - a built-in backend without the extension is classified by the observation path instead - (the post-load continuation watch plus the connection-death replay probe — never a - blind possibly-running re-issue, never a permanent pending hold); a turn that failed at - the backend settles as a definite rejection). Pending checkpoints re-surface (answerable - across a restore, through the reconciliation surface); pending steering rejects - `steering_interrupted` and is never replayed; first-class queue records restore under their - handoff markers and authoritative loaded-turn evidence. Reconcile is - idempotent: a repeated reconcile never re-attaches or re-issues twice. Reconcile-time - re-issue refusals (invalid registry options, the concurrency cap — including the - no-recorded-session and adapter-without-seam branches) settle the guest and participate - in the changed-VM drain + settlement boundary. - -## Daemon wiring (phase D, in `mcp-server`) - -The `repl` MCP tool is registered in `mcp-server` and wired to the daemon's project model -(the roadmap doc's Surface section): one persistent VM per `projectDir` context. The -per-project context opens this phase's store — `repl/` under -`workflowHomeDir()/projects//` — and on FIRST TOUCH either restores the stored -workspace (enveloped snapshot → `Workspace.restore` → broker attach → the three-way -`reconcile()`) or creates a fresh one (SINGLE-FLIGHT: concurrent first touches share one -in-flight promise — one VM and broker per project, the single-writer persistence model); -the broker's state-changing-boundary sink is attached so every eval and every settlement -drain that changed VM state persists. A stored snapshot that REFUSES on first touch -(corrupt/truncated, format-version bump, or a wasm-hash mismatch naming both hashes) is -CONTAINED and AUTO-RESET (§6.1): the refused file is renamed aside -(`snapshot.bin.refused-` — NEVER deleted, auto-reset must not be silent data -destruction) and a fresh workspace starts; the next eval's `output` leads with a loud -one-line notice naming the file and the reason. The daemon never crash-loops. The -workspace therefore survives daemon restarts: this is the production wiring the phase-D -review demanded (`ReplWorkspaceStore` used to be exported/tested only). - -Every `repl` result also carries the doc's MACHINE-READABLE shape as -`structuredContent`: the published `outputSchema` (the workflow tool's -oneOf-branch pattern) mirrors the TWO actions exactly — eval as -`{ output, result? }` (finished), `{ output, running: [ids] }` -(bound elapsed), or the bare `{ output }` of a thrown eval (the §4.6 -rendering — `result` and `running` are mutually exclusive), interrupt -as its honest outcome (`targeted` / `refused-idle` / -`cancelled` / `idle` / `failed`), and the error variant carrying a -bare structured error string (`{ error }` — exactly one key, mirroring -the runtime validator). `output` is one unbounded, newline-joined string: console lines -(one per call), raised checkpoint lines (`checkpoint c9: `), -uncaught-error renderings (§4.6), and the §6.1/§6.2 one-line notices -(auto-reset; a restore that lost calls or a drain failure that lost -state). Reconcile summaries and retained drain errors leave the result -surface entirely — they live under `workspace().diagnostics` (§6.2). - -## Client presence and the drain (phase D, in `mcp-server`) - -The doc's client-presence policy is wired in full. The daemon's session registry measures -liveness by connection presence and now SIGNALS last-connection-closed -(`SessionRegistry.onLastConnectionClosed`); a per-daemon `ReplPresenceLedger` maps MCP -sessions to the repl projects they touched. The ledger is **shared with the `workflow` -tool** — a session that only ran `workflow` calls against a project is present on it too, -so a repl client's disconnect never drains a workspace while a workflow-only client is -still connected. A project whose client set becomes EMPTY is DRAINED: in-flight subagent -turns drain to completion — `Broker.drainForDisconnect` pumps until no session has a turn -running, each settlement boundary snapshots, so "close the laptop while two researchers -run" ends with the findings durable in the workspace — bounded by the SPEC-OWED concrete -bound, the daemon's `REPL_DRAIN_BOUND_MS` (`AGENTPRISM_REPL_DRAIN_BOUND_MS`, default 2 h; -originally the session-eviction TTL, since decoupled; the runner's -own runaway protections already bound individual turns — the bound is the outer ceiling; a -turn that overruns the bound is force-cancelled, the honest bounded teardown), and then -every idle child closes (`keepSession` keeps the backend sessions re-openable). A client -that **reconnects mid-drain ABORTS it** — `drainForDisconnect` re-checks presence every -iteration and before every destructive phase, so the children stay warm while any client -is connected. The workspace and broker stay alive; on the next client connect -`queue`/`steer`/`cancel` on a settled handle RE-ATTACHES the recorded backend session -lazily via the capability matrix (`Broker.canLazyReattach`/`lazyReattach` — the runner's -own `loadSession` gate. Pending queued turns survive the drain durably against their -founding session ids; the next eligible queue head reattaches and delivers them in FIFO order. -At daemon shutdown every workspace drains with the shutdown deadline before the -broker teardown. - -## The generated steering mechanism table - -The per-backend steering mechanism table (the doc's spec-owed decision: "the table is -documentation generated from the capability probes") is a GENERATED ARTIFACT: -[`docs/steering-mechanism-table.md`](docs/steering-mechanism-table.md) is produced from -the live capability probes in `@automatalabs/acp-agents`'s -`ACP_EXTENSION_SUPPORT_MATRIX` (see `src/steering-table.ts`), and -`test/steering-table.test.ts` GATES it — the suite regenerates the document and fails -when the checked-in file drifts from the probes. Regenerate with -`pnpm --filter @automatalabs/repl-engine generate:steering-table`. The matrix is documentation -only: runtime steering parses each session's raw initialize metadata. Unadvertised active -steering is `unsupported`, idle steering is `idle`, and neither path queues a prompt. - -## Development - -```sh -pnpm build # tsc -b -pnpm typecheck # tsc --noEmit -pnpm test # tsx --test (deterministic, credential-free) -``` - -The test suite pins the doc-required behaviors: eval round-trip, drain of microtasks/jobs, -memory-limit enforcement, interrupt breaking a runaway eval with the VM still usable after, -top-level-await acceptance, top-level-`return` rejection, pending-suspension with no -fabricated value, trap-free completion reads under `Object.prototype` pollution — plus the -adversarial regressions: no guest getter runs during synchronous parse failures, rejected -completions, drain failures, or **failing descriptor reads** (the raw descriptor path never -constructs quickjs-wasi's getter-invoking `JSException`); thrown proxies and proxy -prototypes report trap-free markers; thrown symbols report the bare brand `Symbol`, never -`NaN`; -standalone settlement drains arm their own interrupt handler and break delayed runaway -continuations; `dispose()` cannot race an in-flight eval; concurrent evals never leak -interrupt handlers; the registry instantiates exactly one VM under concurrent first touches -and cancels in-flight creates on dispose; 20,000 consecutive syntax errors and 20,000 -accessor-valued completions leave a 1 MiB VM healthy; and a non-DOM `skipLibCheck: false` -consumer compiles the published declarations — including `@ts-expect-error` negative cases -pinning the opaque `WasmModule` boundary. - -Phase B pins the guest library and bridge: install/version-marker/idempotence (re-eval and -re-install are no-ops), agent -round trips with the model-spec signature and JSON options, rejections normalizing to Errors -carrying `code`/`recoverable`, the live handle (`id`/`queue`/`steer`/`cancel`, -non-enumerable, steering addressed to the founding session id), synchronous host-refusal -rejection, started-not-awaited settlement through a later standalone drain, the checkpoint -question→answer flow across evals (with `false` for unknown/answered ids and a TypeError for -non-JSON answers), every combinator over a mocked `__host_agent` (parallel order/null -slots/non-recoverable halts, pipeline stages and slot semantics, retry attempts and `until`, -gate feedback loops, loopUntilDry dedupe/emptiness/maxRounds and circular-safe keys, verify -votes and dropped reviewers routed through the `"default"` sentinel (no `opts.model` in the -DSL options), judgePanel mean scores and stable tie-breaks), the reconciliation surface -(pending/settle/stats with `sessionId` and -`modelSpec` on entries, first-wins idempotence, `Map.prototype` pollution immunity via -captured intrinsics, no pinning of guest memory), steering snapshot-reconciliation (the -pending steer entry carries both ids; settle works by registry id across a restore), handle -hygiene (5,000 resolved agent calls leave a 2 MiB VM healthy; 5,000 parked agent calls leave a -3 MiB VM healthy — parked registry entries are live for the VM's lifetime, so their honest -footprint fills a 2 MiB limit to 99.9%, a knife-edge where any library evolution tips it; the -3 MiB limit keeps ~70% headroom while a per-call leak still cannot hide; 30,000 synchronous -host refusals — agent, steer and checkpoint — leave a 2 MiB VM healthy, with a normal agent -call still completing afterwards), the captured-intrinsic regressions (`console.*`/ -`pipeline` keep working when -`Array.prototype.slice` and `Function.prototype.call` are replaced by throwing functions), -host-serving-an-older-library (a workspace whose resident library is v0.0.1 installs, works, -and keeps its resident version under the current host's install path), snapshot -travel (state, the pending-call registry and version marker survive; callbacks re-register -by name; new -calls mint fresh ids), and the console payload shapes (the `{ line }` payload — one joined -line per call). The workspace suite pins the phase-B injection: -a created workspace exposes the DSL, accumulates console events, parks calls, and serves the -surface/render/manifest seams. Phase C adds the store and broker suites: the call-store -semantics (in-memory first-wins dispatch/completion idempotence and unknown-id refusal; the -JSONL replay with first-wins across reopens; the torn-tail repair discipline — an -unterminated unparseable tail is sidecarred and truncated, an unterminated-but-complete -record is kept with its terminator restored, newline-terminated corruption anywhere is a -hard error, a partial append heals to the acknowledged prefix — and the missing-file open), -and the broker against a fake runner/session: the eval tool-result shapes (resolved with the -previewed value, suspended with the pending call ids and no fabricated value, rejected with -the error line, top-level `return` as a syntax error), the continuation-at-settlement flow, -the late-uncaught-rejection error line in the next tool result, the schema ladder (validated -extraction, re-prompt, `SCHEMA_NONCOMPLIANCE`) and `AGENT_EMPTY_OUTPUT`, exactly-once -settlement (a simulated crash between the store write and the guest settlement — both the -live pump retry and the snapshot/restore + reconcile settle-from-store arm, with the guest -continuation firing exactly once), the checkpoint round trip (raised → previewed question → -answered in a later eval → settlement within that eval, unknown/answered ids report false, -the answer recorded before settlement), steering outcome visibility (extension backend: live -injection with the backend's verbatim outcome and wire failures resolving `failed`; -no-extension backend: `queued` at enqueue and next-turn delivery, delivery-failure warn -lines; idle sessions start new turns with and without the extension; cancel → `cancelled` + -the call's `AGENT_CANCELLED` rejection, idle cancel → `idle`), the concurrency limit -(additional dispatches QUEUE in dispatch order — never a rejection; slot release on -settlement), the §3.1 fused pump (the eval-token settlement short-circuits the target set — -an unrelated pending call never holds the finished shape to the bound), trap-free result -rendering (accessor completions render `(...)` and never fire; the `Object.prototype.value` -pollution cannot hijack the result line), and the console rendering (one joined line per -`console.*` call with level prefixes). The consumer fixture -exercises the whole phase-C public surface (`Broker`, the store classes, the self-contained -`BrokerRunner`/`BrokerSession` stand-ins, the eval-result types) under the non-DOM -`skipLibCheck: false` configuration. The previewer suite pins the FORMAT.md rules (primitives incl. -`-0`/exponent forms, string head+tail elisions and escaping, functions, errors with -own-data-only names, promise states, arrays with holes/named props/overflow, plain objects -with positional indices and accessor `(...)`, branded objects and typed arrays with expando -overflow, proxies incl. revoked, property-level shorthand tokens, the 400-char backstop, -byte-size formatting with the promotion rule) and the §4.4 repr rules (depth 2, 20 entries -per level, nested strings head-limited at 200 chars, direct strings whole — the completion -and console reprs), the trap-freedom -guarantees (hostile getters on `Object.prototype`/`Array.prototype` never fire — including -the `Object.prototype.value` pollution case; proxy traps never fire; a guest that replaces -`Symbol.keyFor` cannot forge -thrown-symbol rendering; the byte-size estimate is bounded and cycle-safe), the FORMAT.md §6 -degradation (a corrupted key materialization lists nothing and flags overflow — typed arrays -included), and bounded-memory previews (3,000 revoked-proxy and typed-array previews leave a -2 MiB VM healthy). - -Phase D adds the snapshot + restore suites: `snapshot-envelope.test.ts` (the envelope round -trip — serialize → deserialize → restore with state intact, gzip actually compressing — -`wasmSha256Of` byte/module/foreign-module behaviors, the version-bump refusal naming BOTH -versions, the format-name refusal, and corrupt/truncated header + payload refusals), -`repl-store.test.ts` (the `repl/` subdirectory layout under -`workflowHomeDir()/projects//`, the write/load round trip with the call store -coexisting, the wasm-hash-mismatch refusal NAMING BOTH HASHES, the version-bump refusal -through the store, corrupted/truncated handling — loud single-shot failure with the store -immediately usable, no crash-loop — a failed write leaving the previous snapshot untouched -and removing the tmp, the boundary-in/burst-out debounce (one atomic write per drain burst, -`debounceBursts: false` writing per boundary), and `reset()` teardown) and `restore.test.ts` -(the full restore flow with all three reconciliation arms against mock backends — settle -from the store / re-attach via `loadSession` with the capability gate / re-issue under the -same call id with the reissues counter bumped and the guest promise settling exactly once — -the custom-backend-without-the-capability degradation surfaced guest-visibly, the lost- -session degradation, reconcile idempotence, over-cap re-issue QUEUED in dispatch order -(never a `ConcurrencyLimitError` rejection), checkpoint re-surfacing + answering across a -restore, in-flight -steers resolving the honest `failed`, the state-changing-boundary cadence (after each eval -and each settlement drain that changed VM state; nothing for an empty drain; the -reconcile-time refusal branches — the invalid-options refusal settles the guest and fires -the boundary too (the over-cap re-issue QUEUES instead, settling nothing), and a changed-VM -drain that FAILS still fires its -boundary, on the reconcile and pump paths alike), the end-to-end debounce through the -per-project store, and the re-attach arm through the REAL acp-agents adapter (a real -`AcpAgentRunner` + `InteractiveSession` over the fake ACP agent, driven by the -`_session/loaded_turn` extension: a completed-while-down call re-attaches and settles -from the loaded session's replay with no re-issue, wire-log proven — including the -`_session/loaded_turn/query` on the wire; a still-running turn settles ONLY from the -authoritative `_session/loaded_turn/ended` notification — an assistant PARTIAL whose next -live chunk arrives later than any quiet grace is never durably settled, and the -still-running turn is never re-issued (no fresh session ever opens); an `interrupted` -turn re-issues immediately; a built-in backend WITHOUT the extension is classified by the -observation path — a completed-while-down turn settles from the replay with no re-issue -and no extension query on the wire, under the connection-death contract). Phase-D -review round 2 adds `review2.test.ts` (the persisted backend routing pin — restore and -re-issue route by the recorded backend id, never the current default; the -`drainForDisconnect` policy — turns drain to completion with settlement boundaries, the -bound cancels an over-bound turn as the recoverable `AGENT_CANCELLED`, and pending queued -turns remain durable; the -workspace manifest — structure-only tokens, provenance labels, live-handle status, -metadata-never-content asserted hard; the per-eval wall-clock deadline breaking a -currently-running runaway eval with the VM usable after; the provenance passes; the §6.2 -demotion — reconcile surfacing lines retained under `workspace().diagnostics.reconcileNotes`, -never the eval output) and the -mcp-server `repl-review2.test.ts` (the soft-bound eval's finished/still-running shapes, the -empty-eval poll, the single-flight first touch, the last-client-disconnect drain + lazy -re-attach, the eval-timeout env knob). The consumer fixture exercises the whole -phase-D public surface (envelope functions, `ReplWorkspaceStore`, the snapshot sink, the -manifest/provenance/drain surfaces, the extended report/seam types) under the non-DOM -`skipLibCheck: false` configuration. diff --git a/packages/repl-engine/docs/steering-mechanism-table.md b/packages/repl-engine/docs/steering-mechanism-table.md deleted file mode 100644 index 8185b109..00000000 --- a/packages/repl-engine/docs/steering-mechanism-table.md +++ /dev/null @@ -1,32 +0,0 @@ -# Per-backend steering mechanism table - - - -The installed-distribution inventory: - -| Backend | `_session/steering` | Strict steering behavior | -|---|---|---| -| claude | advertised (probed: claude) | strict active-turn injection via `session.steer()` | -| codex | advertised (probed: codex) | strict active-turn injection via `session.steer()` | -| opencode | NOT advertised | unsupported (no steering wire request) | -| pi | advertised | strict active-turn injection via `session.steer()` | -| custom backend | whatever its raw initialize metadata advertises | strict active-turn injection when advertised; unsupported otherwise | - -Runtime steering availability is read from the session's raw initialize metadata only: -`initializeMeta.steering.supported === true`. The distribution matrix above is documentation, -not a runtime router. - -| Case | Wire behavior | Result | -|---|---|---| -| ACP prompt in flight; raw steering advertised | one strict `_session/steering` request with `idleBehavior: "promptRequired"` | `injected`, or `idle` for `promptRequired` | -| ACP prompt in flight; raw steering not advertised | no request | `unsupported` | -| no ACP prompt in flight, including opening/extraction/repair gaps | no request | `idle` | -| steering transport/server failure | no prompt fallback | rejects `AGENT_EXECUTION_ERROR` | -| malformed response or `startedNewTurn` | cancel + fatal session lane | rejects non-recoverably | - -Future work is always explicit: `handle.queue(prompt)` creates a distinct, durable FIFO public -turn and the broker sends it through ordinary `session/prompt` only when it reaches the queue -head. Queueing never uses `_session/steering` or a backend-native queue. diff --git a/packages/repl-engine/package.json b/packages/repl-engine/package.json deleted file mode 100644 index 9100ef99..00000000 --- a/packages/repl-engine/package.json +++ /dev/null @@ -1,53 +0,0 @@ -{ - "name": "@automatalabs/repl-engine", - "version": "0.4.40", - "license": "Apache-2.0", - "engines": { - "node": ">=22" - }, - "repository": { - "type": "git", - "url": "git+https://github.com/agentprism/agentprism-workflows.git", - "directory": "packages/repl-engine" - }, - "type": "module", - "main": "./dist/index.js", - "types": "./src/index.ts", - "exports": { - ".": { - "types": "./src/index.ts", - "import": "./dist/index.js", - "default": "./dist/index.js" - } - }, - "files": [ - "dist" - ], - "publishConfig": { - "access": "public", - "main": "./dist/index.js", - "types": "./dist/index.d.ts", - "exports": { - ".": { - "types": "./dist/index.d.ts", - "import": "./dist/index.js", - "default": "./dist/index.js" - } - } - }, - "scripts": { - "build": "tsc -b", - "typecheck": "tsc --noEmit", - "test": "tsx --test \"test/**/*.test.ts\"", - "prepublishOnly": "tsc -b", - "generate:steering-table": "tsx scripts/generate-steering-table.ts" - }, - "dependencies": { - "@automatalabs/acp-agents": "workspace:*", - "@automatalabs/shared-types": "workspace:*", - "@automatalabs/workflows": "workspace:*", - "acorn": "^8.17.0", - "quickjs-wasi": "3.3.1", - "typebox": "1.3.2" - } -} diff --git a/packages/repl-engine/scripts/generate-steering-table.ts b/packages/repl-engine/scripts/generate-steering-table.ts deleted file mode 100644 index c27c6ef9..00000000 --- a/packages/repl-engine/scripts/generate-steering-table.ts +++ /dev/null @@ -1,16 +0,0 @@ -/** - * Regenerate the checked-in per-backend steering mechanism table (the - * roadmap doc's generated documentation — see `src/steering-table.ts`): - * `pnpm --filter @automatalabs/repl-engine generate:steering-table`. - * The gate test (`test/steering-table.test.ts`) fails when the checked-in - * document drifts from what this script produces. - */ - -import { writeFile } from 'node:fs/promises'; -import { fileURLToPath } from 'node:url'; - -import { generateSteeringMechanismTable } from '../src/steering-table.js'; - -const outPath = fileURLToPath(new URL('../docs/steering-mechanism-table.md', import.meta.url)); -await writeFile(outPath, generateSteeringMechanismTable(), 'utf8'); -console.log(`wrote ${outPath}`); diff --git a/packages/repl-engine/src/await-instrument.ts b/packages/repl-engine/src/await-instrument.ts deleted file mode 100644 index 1ad742ca..00000000 --- a/packages/repl-engine/src/await-instrument.ts +++ /dev/null @@ -1,302 +0,0 @@ -/** - * The top-level-await instrumenter — the guest-side half of the eval-break - * targeting discipline (the `interrupt` tool's no-id arm): the broker - * rewrites `await ` into `await this["__replAwait"](, TOKEN)` - * for every TOP-LEVEL await of an eval, where `TOKEN` is the eval's own - * continuation token. - * - * The guest library's `__replAwait(value, token)` WRAPS the awaited value - * in a fresh promise whose settling reaction — the job that runs - * IMMEDIATELY BEFORE the eval's continuation segment — sets the - * CONTINUATION LEASE to the eval's token (see `guest-library.ts`). The - * broker's drain loop reads the lease between jobs: a job that starts - * with a lease set IS the armed eval's continuation, so the eval-break - * interrupt fires only while THAT execution runs. This is the armed - * target's genuine CONTINUATION IDENTITY (phase-E review rejection round - * 5): not the calls the eval awaited, not the calls it created — the - * execution itself. - * - * Consequences, pinned by regressions: - * - * - an eval that awaits a call it created (`const c1 = agent(...); await - * c1`) is targetable — the wrap's reaction is queued on c1's promise; - * - an eval that awaits an EARLIER eval's binding (`await p` where `p` - * was created by a previous eval) is targetable the same way — the - * wrap does not care where the promise came from; - * - an UNAWAITED sibling reaction registered BEFORE the await - * (`q.then(sibling)` then `await q`) runs FIRST in the settlement - * drain — before the lease-setting reaction — so the sibling job can - * neither fire the signal nor consume it; the armed state survives - * and the target's own continuation (the job after the lease-setting - * reaction) is the one broken mid-run (the carried review defect: the - * signal was keyed to settled call ids, so the sibling job consumed - * it and the target ran later unbroken); - * - INDIRECT awaits are targetable: `await Promise.all([q])` wraps the - * combinator's promise, whose settlement queues the eval's - * continuation exactly like a direct call's — the identity is the - * promise graph, not a logged call-id list (the carried review - * defect: the 0.2.0 log-based targeting refused indirect waits). - * - * ## Hygiene (phase-E review rejection round 5) - * - * The injected code must never change the guest program's semantics. The - * 0.2.0 instrumenter inserted the guest-resolvable identifier - * `__replAwait` at every site, so a guest lexical declaration shadowed - * it: `{ const __replAwait = () => 7; globalThis.seen = await - * Promise.resolve(42); }` yielded 7 instead of 42. The 0.3.0 transform - * is hygienic by construction: - * - * - the injected base is the `this` KEYWORD at the eval's top level — - * the engine invokes the script's async wrapper with the realm's - * global object as its `this` (verified against the shipped binary), - * so `this["__replAwait"]` resolves the library's global without - * naming any identifier the guest could shadow; `this` is a keyword — - * no declaration, in any scope, can shadow it; - * - no capture line and no helper binding are injected: a top-level - * `const` in an eval persists in the realm's global lexical record, - * so a helper declaration would (a) redeclare on the loop idiom - * (re-running identical code — `SyntaxError: redeclaration`) and (b) - * leak a binding into the workspace manifest. The direct - * `this["__replAwait"]` form injects nothing but the call sites. - * - * The rewrite is restricted to awaits in the eval'd script's TOP-LEVEL - * body — the engine evaluates the script with top-level-await semantics - * (see vm.ts), so only awaits directly in that body queue THE EVAL's - * continuation. Awaits inside NESTED function bodies (a `.map(async x => - * await ...)` callback, a `parallel()` thunk, a `for await` inside a - * helper) belong to their own continuations: wrapping them would - * attribute the wrong execution and let the signal break an unrelated - * one — the exact false positive the discipline forbids. The library's - * own combinators are deliberately NOT wrapped: an eval that awaits - * `parallel([...])` awaits the combinator's RESULT promise, and the wrap - * rides that promise's settlement like any other — the combinator's - * internals need no special handling. - * - * The instrumenter is a pure source transform driven by acorn (already a - * monorepo dependency — mcp-server and workflow-engine use it): parse the - * script, walk the AST, collect `(argument.start, node.end)` pairs of - * every top-level AwaitExpression (plus the iterable of every top-level - * `for await`), and splice the call at those exact AST boundaries. - * Boundaries are tokenization-safe by construction: the original parse - * already resolved every `await` expression's extent, so no token can be - * split. A parse failure returns the code UNCHANGED — the VM reports the - * syntax error with the original source (positions preserved; the - * instrumenter never shifts lines). - * - * The broker gates the instrumenter on the workspace's library carrying - * the 0.3.0 continuation-lease surface (`surface.supportsContinuation - * Lease`): a restored snapshot with the 0.1.0/0.2.0 library is served - * as-is and simply gets no instrumentation (the eval-break interrupt - * degrades to the honest refusal — the 0.2.0 log-only targeting is the - * rejected settled-call-ids identity). The for-await ITERABLE sites are - * gated separately on the 0.3.1 iterable-lease surface - * (`surface.supportsIterableLease` — `instrumentTopLevelAwaits`'s - * `wrapIterables` option): a 0.3.0 snapshot's for-await wrap returned a - * promise (breaking every `for await` loop), so those sites are left - * UNWRAPPED on a 0.3.0 copy — the loop runs natively, and only the - * loop's mid-iteration eval-break targeting is lost (the honest - * degradation). `for await (... of await y)` needs no iterable wrap at - * all: the right expression's own top-level await is instrumented - * normally (the loop iterates the unwrapped value), so the site is - * skipped — wrapping the AwaitExpression itself would hand the - * iterable wrap a promise. - */ - -import { parse } from 'acorn'; - -/** How many distinct source strings the transform cache holds (evals - * re-run identical code — the workspace's loop idiom — so caching the - * parse is a real win; the bound keeps a pathological caller from - * growing the cache unboundedly). */ -const INSTRUMENT_CACHE_MAX = 256; - -/** One source's cached parse plan: the await sites are token- and - * eval-independent — only the per-eval TOKEN varies, so the splice is - * re-derived per call from this plan. */ -interface InstrumentPlan { - sites: AwaitSite[]; -} - -const cache = new Map(); - -/** Cache sentinel for an un-instrumentable source (a parse failure, or a - * script with no top-level await). */ -const MISS = Symbol('instrument-miss'); - -/** Function-like AST nodes whose bodies own their continuations: the - * instrumenter never descends into them (an await inside one is not a - * top-level await — see the module docs). */ -const FUNCTION_NODE_TYPES = new Set([ - 'FunctionDeclaration', - 'FunctionExpression', - 'ArrowFunctionExpression', - 'StaticBlock', -]); - -/** - * Rewrite every TOP-LEVEL `await ` of the script into - * `await this["__replAwait"](, TOKEN)` (and every top-level - * `for await (... of )` into - * `for await (... of this["__replAwaitIterable"](, TOKEN))` — - * the iterable wrap is the 0.3.1 surface and is applied only when - * `opts.wrapIterables` is set (the broker gates it on the resident - * library's `supportsIterableLease`; a 0.3.0 snapshot's for-await sites - * stay unwrapped — native semantics, the honest degradation). Returns - * the original code unchanged when there is nothing to rewrite or the - * code does not parse (the VM reports the syntax error). - */ -export function instrumentTopLevelAwaits( - code: string, - token: string, - opts: { wrapIterables?: boolean } = {}, -): string { - const plan = planFor(code); - if (plan === undefined || plan.sites.length === 0) return code; - const enabled = opts.wrapIterables ? plan.sites : plan.sites.filter((site) => site.kind === 'await'); - if (enabled.length === 0) return code; - const tokenJson = JSON.stringify(token); - // All insertions, applied right-to-left by position: sites NEST (an - // outer await's range contains its argument's awaits — `await (await - // a, await b)`), so a range-splice of the outer site would cut - // through the already-shifted interior. Point insertions at exact AST - // boundaries are position-safe at any nesting depth (each site's - // `start` and `end` are distinct positions; when two sites share a - // position — `for await (const x of a ?? await y)` — the doubled - // insertion is harmless: both wrappers still surround their - // expression, and the shared-position close order (inner first in - // the text — the inner close is applied last at the position) keeps - // the outer wrap's argument an EXPRESSION, evaluated before the outer - // call: `__replAwaitIterable(a ?? await __replAwait(y, T), T)`). The - // one shape this cannot express — `for await (const x of await y)`, - // where the iterable IS the awaited expression — is skipped at - // collection time (see `findAwaitSites`). - const insertions: Array<{ pos: number; text: string }> = []; - for (const site of enabled) { - insertions.push({ - pos: site.start, - text: site.kind === 'iterable' ? 'this["__replAwaitIterable"](' : 'this["__replAwait"](', - }); - insertions.push({ pos: site.end, text: `, ${tokenJson})` }); - } - insertions.sort((a, b) => b.pos - a.pos); - let out = code; - for (const insertion of insertions) { - out = out.slice(0, insertion.pos) + insertion.text + out.slice(insertion.pos); - } - return out; -} - -/** The cached parse plan for a source (see `InstrumentPlan`). Returns - * undefined when the source does not parse or has no top-level await. */ -function planFor(code: string): InstrumentPlan | undefined { - const cached = cache.get(code); - if (cached !== undefined) return cached === MISS ? undefined : cached; - - let plan: InstrumentPlan | undefined; - try { - const sites = findAwaitSites(code); - if (sites.length > 0) plan = { sites }; - } catch { - // Parse failure — the engine reports the syntax error; never - // instrument what we cannot parse (a half-rewritten script would - // produce a confusing double error). - plan = undefined; - } - - if (cache.size >= INSTRUMENT_CACHE_MAX) cache.clear(); - cache.set(code, plan === undefined ? MISS : plan); - return plan; -} - -/** Parse the script and collect every top-level await site (see the - * module docs for the top-level rule). */ -function findAwaitSites(code: string): AwaitSite[] { - // `allowAwaitOutsideFunction` matches the engine's top-level-await - // semantics: `await` is the await OPERATOR everywhere in the script, - // exactly as the VM treats it. - const ast = parse(code, { - ecmaVersion: 'latest', - sourceType: 'script', - allowAwaitOutsideFunction: true, - }) as unknown as AcornNode; - - const sites: AwaitSite[] = []; - visit(ast, false, sites); - return sites; -} - -/** Recursive AST walk collecting await sites. `inFunction` is true - * inside any nested function body — awaits there are skipped entirely - * (their continuations are not the eval's). */ -function visit(node: AcornNode | null | undefined, inFunction: boolean, sites: AwaitSite[]): void { - if (node === null || node === undefined || typeof node !== 'object' || typeof node.type !== 'string') return; - - if (node.type === 'AwaitExpression') { - if (!inFunction && node.argument !== null && typeof node.argument === 'object') { - sites.push({ start: node.argument.start, end: node.end, kind: 'await' }); - } - visit(node.argument, inFunction, sites); - return; - } - if (node.type === 'ForOfStatement') { - // `for await (const x of y)`: the eval suspends on the ITERABLE's - // iterator — wrapping the iterable rides the iteration's first - // suspension like any other awaited value. The wrap must preserve - // the iterable protocol (the 0.3.1 `__replAwaitIterable` returns an - // ASYNC-ITERABLE, never a promise — phase-E review rejection round - // 6: the 0.3.0 `__replAwait` wrap made `for await (const x of [1, - // 2])` throw `TypeError: not a function`). When the iterable IS an - // AwaitExpression (`for await (const x of await y)`), the site is - // SKIPPED: the right expression's own top-level await is - // instrumented separately (the loop then iterates the unwrapped - // value, exactly like the un-instrumented program) — wrapping the - // awaited expression itself would hand the iterable wrap a promise - // (the machinery evaluates the instrumented `await` BEFORE the - // wrapper call only when the await sits INSIDE the argument, which - // a top-level AwaitExpression right cannot be). - if (node.await === true && node.right !== null && typeof node.right === 'object') { - if (node.right.type !== 'AwaitExpression') { - sites.push({ start: node.right.start, end: node.right.end, kind: 'iterable' }); - } - } - visit(node.left, inFunction, sites); - visit(node.right, inFunction, sites); - return; - } - if (FUNCTION_NODE_TYPES.has(node.type)) return; - - for (const key of Object.keys(node)) { - if (key === 'type' || key === 'start' || key === 'end' || key === 'loc' || key === 'range') continue; - const child = (node as unknown as Record)[key]; - if (Array.isArray(child)) { - for (const item of child) { - visit(item as AcornNode, inFunction, sites); - } - } else if (child !== null && typeof child === 'object' && typeof (child as AcornNode).type === 'string') { - visit(child as AcornNode, inFunction, sites); - } - } -} - -interface AwaitSite { - /** Where to insert the wrap call's open (the awaited expression's - * start). */ - start: number; - /** Where to insert `, "TOKEN")` (the awaited expression's end). */ - end: number; - /** Which seam the site rides: a top-level `await` (the `__replAwait` - * promise wrap) or a `for await` ITERABLE (the `__replAwaitIterable` - * async-iterable wrap — gated on the 0.3.1 iterable-lease surface). */ - kind: 'await' | 'iterable'; -} - -/** The acorn AST node shape the walker needs (a structural subset). */ -interface AcornNode { - type: string; - start: number; - end: number; - await?: boolean; - argument?: AcornNode | null; - left?: AcornNode | null; - right?: AcornNode | null; -} diff --git a/packages/repl-engine/src/bridge.ts b/packages/repl-engine/src/bridge.ts deleted file mode 100644 index 2a8540a6..00000000 --- a/packages/repl-engine/src/bridge.ts +++ /dev/null @@ -1,1234 +0,0 @@ -/** - * The host side of the guest-library bridge. - * - * Installs the `__host_*` callbacks the guest library consumes (the - * realm's entire effect surface — the dispatch table stays almost - * embarrassingly small by design), evaluates the library once at VM - * creation, and exposes the host's doors back into the realm: the - * reconciliation surface (post-restore settlement), the trap-free realm - * slot reader, and the console-event channel. - * - * Settlement contract (see the package README's "Guest library ⇄ host - * contract"): each `__host_agent` / `__host_checkpoint` (question mode) / - * `__host_agent_steer` callback creates a `GuestCall` — a quickjs-wasi - * `Deferred` whose promise handle is returned into the realm as the - * thenable the guest chains onto — and hands it to the handler. The - * handler settles it (`resolve`/`reject`) whenever its work completes, - * then drains; alternatively the host returns nothing and settles later - * through `readGuestSurface().settle(...)`. Both routes converge on the - * guest's idempotent settle-by-call-id; the first settlement wins. - * `checkpoint.answer` is the synchronous answer mode: a PRESENT fourth - * argument to `__host_checkpoint` means answer delivery, and the callback - * returns the handler's boolean synchronously (no registry entry is - * minted — nothing new pends). - * - * Every callback path is defensive: protocol violations (non-string - * arguments) throw, which the shim turns into a guest-thrown error that - * the guest library converts into a call rejection — the documented - * "synchronous host refusal" path. A throwing handler behaves the same. - * - * The public type graph stays free of quickjs-wasi types (a consumer with - * a non-DOM lib and `skipLibCheck: false` must type-check the published - * declarations cleanly); the shim is reached through `getVmShim`, and - * `GuestCall`'s internals are private. - */ - -import { JSValueHandle, type HostFunction, type QuickJS } from 'quickjs-wasi'; - -import { - GUEST_SURFACE_KEY, - HOST_AGENT, - HOST_AGENTS, - HOST_CHECKPOINT, - HOST_CONSOLE, - HOST_DEFAULT_BACKEND, - HOST_QUEUE, - HOST_QUEUE_CANCEL, - HOST_RESET, - HOST_SESSION_CANCEL, - HOST_SLEEP, - HOST_STEER, - HOST_WORKSPACE, - buildGuestLibrarySource, -} from './guest/guest-library.js'; -import { getVmShim, type ReplVm } from './vm.js'; -import type { EvalErrorInfo } from './errors.js'; -import { getPropRaw, hasOwnRaw, readOwnDataProperty, readValue, readValueComplete, takeAndFreeException, type QuickJSExports } from './trapfree.js'; - -/** The console levels the guest bridge emits. */ -export type ConsoleLevel = 'log' | 'info' | 'warn' | 'error' | 'debug'; - -/** - * One console event crossing the bridge: `line` is the ONE joined line - * the guest rendered for this call (the arguments' §4.4 reprs joined - * with a single space — the per-argument `$N` capture system is - * deleted). The guest computed it in the realm; the handler may render - * it verbatim. - */ -export interface ConsoleEvent { - level: ConsoleLevel; - /** The rendered line (one per console.* call). */ - line: string; -} - -/** - * A host call in flight: wraps a promise created through the raw - * `qjs_new_promise` export whose parts (promise + resolve/reject - * functions) the call owns and disposes completely. - * - * The handler settles the call with a host value (`resolve`) or a host - * error value (`reject`); both are marshalled into the realm (plain data, - * errors, arrays, objects — never guest code). Settlement is first-wins: a - * second resolve/reject is a no-op, matching the guest's own idempotent - * settle-by-call-id. The caller must run a job drain after settling for - * the guest's continuations to fire. - * - * Handle ownership (mirrors the Rust reference broker's Deferred - * discipline): the marshalled value handle handed to `resolve`/`reject` - * is disposed here after the call — the engine does not consume it — and - * settling disposes BOTH resolving functions. This deliberately does NOT - * use the shim's `newPromise()` Deferred, whose reject-function handle is - * pinned in the VM's `_ownedHandles` set until VM dispose even when the - * promise is resolved — measured at ~2 objects + 7 heap boxes per call, - * which exhausted a 2 MiB VM after roughly 5,000 sequential resolved - * agent calls (review regression, pinned by test). - * - * The promise handle itself is returned into the realm once (the - * host-callback trampoline dups it and the shim's host_call path never - * frees the host-side original), then released through `releaseToRealm`, - * which defers the dispose to a microtask so it runs only after the - * trampoline's synchronous dup. - */ -export class GuestCall { - /** The owning VM (private — elided from the published declarations). */ - private readonly vm: ReplVm; - /** The raw deferred; its `handle` is the guest promise. */ - private readonly deferred: { - handle: JSValueHandle; - resolve(value: JSValueHandle): void; - reject(value: JSValueHandle): void; - /** Free both resolving functions without settling (see `dispose`). */ - dispose(): void; - }; - private settled = false; - /** True once the promise handle was released to the realm. */ - private released = false; - - constructor(vm: ReplVm) { - this.vm = vm; - const shim = getVmShim(vm) as QuickJS; - this.deferred = newRawDeferred(shim); - guestCallHandles.set(this, this.deferred.handle); - } - - /** - * Dispose every part this call still owns, WITHOUT settling: the raw - * promise handle (when it was never released to the realm) and both - * resolving functions (when no settlement consumed them). This is the - * throwing-handler path — a handler that refuses by throwing leaves the - * call unsettled and its promise never returned into the realm, so - * nothing the call owns is reachable afterwards. Without this, every - * refusal leaked the raw promise plus its two resolving functions - * (~3 JSValues + heap boxes per call — measured: 30,000 rejected - * calls filled a 2 MiB VM and the next agent call failed with - * `Error: null`; review regression, pinned by test). - * - * Idempotent, and safe in every order with resolve/reject/ - * releaseToRealm: after dispose the call reads as settled (nothing can - * settle it), a released promise handle is left to its queued microtask - * dispose, and the deferred's own dispose is a no-op once settlement - * consumed the functions. - */ - dispose(): void { - this.settled = true; - if (!this.released) { - const handle = guestCallHandles.get(this); - if (handle !== undefined) { - guestCallHandles.delete(this); - // Safe to free synchronously on this path: when the handler - // threw, the host-call trampoline never dupped a return value - // (there was none), so no other reference to this promise exists - // host-side. (`releaseToRealm` defers only because the trampoline - // dups the pointer AFTER a successful callback returns.) - handle.dispose(); - } - } - this.deferred.dispose(); - } - - /** - * Resolve the call with a host value (marshalled into the realm). The - * promise's reactions fire on the next job drain. - */ - resolve(value: unknown): void { - this.settle((shim) => { - const valueHandle = marshalValue(shim, value); - try { - this.deferred.resolve(valueHandle); - } finally { - // The raw call borrows the value; the caller owns it and must - // release it (Rust reference: the broker disposes the marshalled - // value after settling). - valueHandle.dispose(); - } - }); - } - - /** - * Reject the call with a host error value (an Error, or a plain - * `{ message, name?, code?, recoverable? }` object — the guest - * normalizes both). The promise's reactions fire on the next job drain. - */ - reject(error: unknown): void { - this.settle((shim) => { - const valueHandle = marshalValue(shim, error); - try { - this.deferred.reject(valueHandle); - } finally { - valueHandle.dispose(); - } - }); - } - - /** - * Release this call's host-side reference to the realm promise. Called - * by the host-function wrappers after the promise handle has been - * returned into the realm: the trampoline dups the returned pointer - * synchronously after the callback returns, so the dispose is deferred - * to a microtask — it runs only once that dup has happened. After this - * the handle must not be used (settlement goes through the deferred's - * resolve/reject functions, which are independent of the promise - * handle). - */ - releaseToRealm(): void { - if (this.released) return; - this.released = true; - const handle = guestCallHandles.get(this); - if (handle === undefined) return; - queueMicrotask(() => handle.dispose()); - } - - /** True once the call has been settled (first-wins). */ - get isSettled(): boolean { - return this.settled; - } - - private shim(): QuickJS { - return getVmShim(this.vm) as QuickJS; - } - - private settle(apply: (shim: QuickJS) => void): void { - if (this.settled) return; // first settlement wins, like the guest registry - this.settled = true; - apply(this.shim()); - } -} - -/** - * Create a promise through the raw `qjs_new_promise` export and return a - * deferred over the three owned parts — the TS analogue of the Rust - * reference broker's `new_promise_raw`/`Deferred` (which settles by - * calling the resolving function and then disposes BOTH functions, plus - * the promise handle at `releaseToRealm` time). `dispose()` frees the - * resolving functions without settling — the throwing-handler path, where - * a call is abandoned before its promise is ever returned to the realm - * (see `GuestCall.dispose`). See `GuestCall` for why the shim's - * `newPromise()` Deferred is not used. - */ -function newRawDeferred(shim: QuickJS): { - handle: JSValueHandle; - resolve(value: JSValueHandle): void; - reject(value: JSValueHandle): void; - dispose(): void; -} { - const e = shim._getExports() as QuickJSExports; - const resolveOut = e.wasm_malloc(4); - const rejectOut = e.wasm_malloc(4); - let promise: JSValueHandle | undefined; - let resolveFn: JSValueHandle | undefined; - let rejectFn: JSValueHandle | undefined; - try { - const promisePtr = e.qjs_new_promise(resolveOut, rejectOut); - const view = new DataView(e.memory.buffer); - const resolvePtr = view.getUint32(resolveOut, true); - const rejectPtr = view.getUint32(rejectOut, true); - promise = new JSValueHandle(shim, promisePtr); - resolveFn = new JSValueHandle(shim, resolvePtr); - rejectFn = new JSValueHandle(shim, rejectPtr); - } finally { - e.wasm_free(resolveOut); - e.wasm_free(rejectOut); - } - - let settled = false; - const freeFunctions = (): void => { - resolveFn?.dispose(); - rejectFn?.dispose(); - resolveFn = undefined; - rejectFn = undefined; - }; - const settleWith = (fn: JSValueHandle, value: JSValueHandle): void => { - if (settled) return; - settled = true; - try { - // Raw `qjs_call` with one borrowed argument (mirrors the shim's own - // callFunctionRaw, which is private). The result — including an - // exception result, whose runtime exception is taken out and freed - // — is disposed here; the engine never consumes the argument. - const argv = e.wasm_malloc(4); - let resultPtr: number; - try { - new DataView(e.memory.buffer).setUint32(argv, value.ptr, true); - resultPtr = e.qjs_call(fn.ptr, shim.undefined.ptr, 1, argv); - } finally { - e.wasm_free(argv); - } - const result = new JSValueHandle(shim, resultPtr); - try { - if (e.qjs_is_exception(result.ptr) !== 0) { - takeAndFreeException(e, shim); - } - } finally { - result.dispose(); - } - } finally { - // Settling consumes both resolving functions (Rust reference: - // `Deferred::dispose` releases every part the deferred still owns). - freeFunctions(); - } - }; - - return { - handle: promise!, - resolve: (value) => settleWith(resolveFn!, value), - reject: (value) => settleWith(rejectFn!, value), - dispose: () => { - // Free the resolving functions without calling them. A settled - // deferred's functions are already freed (settleWith's finally); - // the `settled` flag makes a post-dispose resolve/reject a no-op, - // so a handler that (pathologically) kept a reference to the call - // and settles it later can never call into freed memory. - settled = true; - freeFunctions(); - }, - }; -} - -// Registered by the constructor above; read by `guestCallHandle` so the -// public `GuestCall` class never names a quickjs-wasi type. -const guestCallHandles = new WeakMap(); - -/** The guest promise handle of a call (bridge-internal). */ -function guestCallHandle(call: GuestCall): JSValueHandle { - return guestCallHandles.get(call)!; -} - -/** - * Marshal a host value into the realm. `hostToHandle` handles primitives, - * arrays, plain objects, errors and buffers; a value it cannot marshal (a - * local symbol) falls back to a guest Error carrying the string form — a - * rejection must never throw host-side. The returned handle is owned by - * the caller, who must dispose it after handing it to the deferred - * (promise settlement dups its own reference). - */ -function marshalValue(shim: QuickJS, value: unknown): JSValueHandle { - try { - return shim.hostToHandle(value); - } catch (err) { - return shim.newError(err instanceof Error ? err : String(err)); - } -} - -/** The host's handlers for the four guest calls. */ -export interface GuestBridgeHandlers { - /** - * An `agent(modelSpec, task, options?)` call. `modelSpec` is the - * backend-routing spec (`"pi/deepseek-v4-flash-max"`), `task` the - * worker's prompt; `optionsJson` is the JSON-encoded options bag - * (or `null` when none were given). Settle `call` when the worker's - * result (final text, or the schema-validated object when the options - * carried a schema) is ready. - */ - agent( - call: GuestCall, - callId: string, - modelSpec: string, - task: string, - optionsJson: string | null, - ): void; - /** - * A checkpoint question (question mode: `call` is a fresh `GuestCall`, - * `answerJson` is `null`) or answer delivery (answer mode: `call` is - * `null`, `answerJson` is the JSON-encoded answer). In answer mode the - * handler settles the ORIGINAL pending checkpoint (through its own - * records) and returns a boolean — truthy iff a checkpoint with that id - * was pending when the call was made. Nothing new pends in answer mode. - */ - checkpoint( - call: GuestCall | null, - callId: string, - question: string | null, - optionsJson: string | null, - answerJson: string | null, - ): boolean | void; - /** Create one durable future turn on the founding session. */ - queue(call: GuestCall, callId: string, sessionId: string, payloadJson: string | null): void; - /** Control only the ACP prompt currently in flight on the founding session. */ - steer( - call: GuestCall, - callId: string, - sessionId: string, - payloadJson: string | null, - ): void; - /** Cancel the current public turn on a reusable agent/session handle. */ - cancelSession(call: GuestCall, callId: string, sessionId: string): void; - /** Cancel exactly one queued-turn handle. */ - cancelQueue(call: GuestCall, callId: string, queueCallId: string): void; - /** - * A console event (log/info/warn/error/debug): the guest-rendered ONE - * line per call (see `ConsoleEvent`). A throw becomes a guest error - * inside the library's own swallow-guard — console never breaks guest - * code by contract. - */ - console(event: ConsoleEvent): void; - /** - * A `sleep(ms)` call: settle `call` from a HOST-side timer (the VM - * itself stays timer-free). The promise resolves undefined after the - * host timer fires; the guest's continuation resumes at the next - * settlement drain. - */ - sleep(call: GuestCall, ms: number): void; - /** - * A `workspace()` call: return the JSON-encoded workspace value - * (`{ bindings, inFlight, checkpoints, diagnostics }` — see the - * roadmap doc's §4.5 shape). The guest parses it into a plain value. - */ - workspace(): string; - /** - * An `agents()` call: return the JSON-encoded array of live-agent - * entries (`{ callId, modelSpec, task, state, supportsSteering, - * queuedTurns }`). - */ - agents(): string; - /** - * A `reset()` call: mark the teardown request. The host tears the - * workspace down AFTER the current eval completes; the call itself - * returns nothing meaningful. - */ - reset(): void; - /** - * The host's configured DEFAULT backend id (a registered segment, - * served synchronously) — the guest library's verify/judgePanel - * combinators resolve their reviewer/grader model spec through it - * (§4.7: the DSL options carry no per-call model, so the workers - * inherit the run's default model; §4.1: the spec is a real - * registered backend, validated at admission like any `agent()` - * call). Return `undefined` when no backend registry is attached - * (the parking bridge): the combinators then reject honestly. - */ - defaultBackend(): string | undefined; -} - -/** - * Install the guest bridge on a fresh VM: register the four host - * callbacks, expose them as realm globals, and evaluate the guest library - * once. Re-injecting over a workspace that already carries the library is - * a no-op (the resident version stays authoritative for the life of the - * workspace — the doc's rule: never re-inject over a workspace). - */ -export async function installGuestBridge(vm: ReplVm, handlers: GuestBridgeHandlers): Promise { - if (readGuestSurface(vm) !== undefined) return; // never re-inject - const shim = getVmShim(vm) as QuickJS; - const callbacks = makeCallbacks(vm, handlers); - for (const [name, fn] of callbacks) { - const fnHandle = shim.newFunction(name, fn); - shim.setProp(shim.global, name, fnHandle); - fnHandle.dispose(); - } - // The library is a plain script with no top-level await; the eval and - // its drain complete synchronously (the returned promise is already - // fulfilled — `await` only unwraps). - const outcome = await vm.evalCode(buildGuestLibrarySource(), { filename: '' }); - if (outcome.kind === 'error') { - throw new GuestLibraryInstallError(outcome.error); - } -} - -/** - * Re-register the four host callbacks by name on a VM restored from a - * snapshot. This is the quickjs-wasi restore discipline: the library (and - * its pending-call registry) travels inside the snapshot; only the - * host-side name → callback map must be re-attached (the guest function - * values already exist in the restored memory). Do NOT re-evaluate the - * library — and do NOT call this on a fresh VM (the guest function values - * do not exist yet; that is `installGuestBridge`'s job). - */ -export function registerGuestHostCallbacks(vm: ReplVm, handlers: GuestBridgeHandlers): void { - const shim = getVmShim(vm) as QuickJS; - for (const [name, fn] of makeCallbacks(vm, handlers)) { - shim.registerHostCallback(name, fn); - } -} - -/** The (name, host function) pairs the bridge installs. */ -function makeCallbacks(vm: ReplVm, handlers: GuestBridgeHandlers): Array<[string, HostFunction]> { - return [ - [HOST_AGENT, makeAgentHostFunction(vm, handlers)], - [HOST_CHECKPOINT, makeCheckpointHostFunction(vm, handlers)], - [HOST_QUEUE, makeQueueHostFunction(vm, handlers)], - [HOST_STEER, makeSteerHostFunction(vm, handlers)], - [HOST_SESSION_CANCEL, makeSessionCancelHostFunction(vm, handlers)], - [HOST_QUEUE_CANCEL, makeQueueCancelHostFunction(vm, handlers)], - [HOST_CONSOLE, makeConsoleHostFunction(vm, handlers)], - [HOST_SLEEP, makeSleepHostFunction(vm, handlers)], - [HOST_WORKSPACE, makeWorkspaceHostFunction(vm, handlers)], - [HOST_AGENTS, makeAgentsHostFunction(vm, handlers)], - [HOST_RESET, makeResetHostFunction(vm, handlers)], - [HOST_DEFAULT_BACKEND, makeDefaultBackendHostFunction(vm, handlers)], - ]; -} - -/** - * The `__host_agent` shape: mint a `GuestCall`, hand it to the handler, - * return its promise handle into the realm. The guest chains onto the - * returned thenable; settlement happens whenever the handler settles the - * call (then a drain). - * - * A handler that throws synchronously (the documented refusal path) - * leaves the call unsettled and its promise never returned into the - * realm: dispose every owned part BEFORE the shim converts the throw - * into a guest error (which the guest library turns into a call - * rejection), then re-throw — otherwise each refusal strands the raw - * promise plus both resolving functions (measured: repeated refusals - * corrupt the VM until a normal agent call fails with `Error: null`; - * review regression, pinned by the 30,000-refusals bounded-memory test). - */ -function makeAgentHostFunction(vm: ReplVm, handlers: GuestBridgeHandlers): HostFunction { - return function (this: JSValueHandle, ...args: JSValueHandle[]): JSValueHandle { - const callId = requireString(args[0], HOST_AGENT, 'callId'); - const modelSpec = requireString(args[1], HOST_AGENT, 'modelSpec'); - const task = requireString(args[2], HOST_AGENT, 'task'); - const optionsJson = optionalString(args[3]); - const call = new GuestCall(vm); - try { - handlers.agent(call, callId, modelSpec, task, optionsJson); - } catch (err) { - call.dispose(); - throw err; - } - // The trampoline dups the returned pointer after this callback - // returns; release the host-side reference once that has happened. - call.releaseToRealm(); - return guestCallHandle(call); - }; -} - -/** - * Queue, steer, session-cancel, and queue-cancel are distinct callbacks. - * Keeping them separate prevents a state-dependent compatibility handler - * from turning steering into a prompt or a cancellation into queue-wide - * mutation. - */ -function makeSessionPayloadHostFunction( - vm: ReplVm, - hostName: string, - handler: (call: GuestCall, callId: string, sessionId: string, payloadJson: string | null) => void, -): HostFunction { - return function (this: JSValueHandle, ...args: JSValueHandle[]): JSValueHandle { - const callId = requireString(args[0], hostName, 'callId'); - const sessionId = requireString(args[1], hostName, 'sessionId'); - const payloadJson = optionalString(args[2]); - const call = new GuestCall(vm); - try { - handler(call, callId, sessionId, payloadJson); - } catch (err) { - call.dispose(); - throw err; - } - call.releaseToRealm(); - return guestCallHandle(call); - }; -} - -function makeQueueHostFunction(vm: ReplVm, handlers: GuestBridgeHandlers): HostFunction { - return makeSessionPayloadHostFunction(vm, HOST_QUEUE, handlers.queue.bind(handlers)); -} - -function makeSteerHostFunction(vm: ReplVm, handlers: GuestBridgeHandlers): HostFunction { - return makeSessionPayloadHostFunction(vm, HOST_STEER, handlers.steer.bind(handlers)); -} - -function makeCancelHostFunction( - vm: ReplVm, - hostName: string, - handler: (call: GuestCall, callId: string, targetId: string) => void, -): HostFunction { - return function (this: JSValueHandle, ...args: JSValueHandle[]): JSValueHandle { - const callId = requireString(args[0], hostName, 'callId'); - const targetId = requireString(args[1], hostName, 'targetId'); - const call = new GuestCall(vm); - try { - handler(call, callId, targetId); - } catch (err) { - call.dispose(); - throw err; - } - call.releaseToRealm(); - return guestCallHandle(call); - }; -} - -function makeSessionCancelHostFunction(vm: ReplVm, handlers: GuestBridgeHandlers): HostFunction { - return makeCancelHostFunction(vm, HOST_SESSION_CANCEL, handlers.cancelSession.bind(handlers)); -} - -function makeQueueCancelHostFunction(vm: ReplVm, handlers: GuestBridgeHandlers): HostFunction { - return makeCancelHostFunction(vm, HOST_QUEUE_CANCEL, handlers.cancelQueue.bind(handlers)); -} - -/** - * The `__host_checkpoint` shape, with the answer mode: a PRESENT fourth - * argument (the JSON-encoded answer) flips the call into answer delivery — - * the handler's boolean is returned synchronously and no `GuestCall` is - * minted (nothing new pends; a snapshot can never capture an answer in - * flight). - */ -function makeCheckpointHostFunction(vm: ReplVm, handlers: GuestBridgeHandlers): HostFunction { - const shim = getVmShim(vm) as QuickJS; - return function (this: JSValueHandle, ...args: JSValueHandle[]): JSValueHandle { - const callId = requireString(args[0], HOST_CHECKPOINT, 'callId'); - if (args.length >= 4) { - // Answer mode. - const answerJson = requireString(args[3], HOST_CHECKPOINT, 'answerJson'); - const answered = handlers.checkpoint(null, callId, null, null, answerJson); - return answered ? shim.true : shim.false; - } - const question = optionalString(args[1]); - const optionsJson = optionalString(args[2]); - const call = new GuestCall(vm); - try { - handlers.checkpoint(call, callId, question, optionsJson, null); - } catch (err) { - // Same disposal discipline as `__host_agent` (see there): a - // throwing checkpoint handler must not strand the raw promise and - // its resolving functions. (Answer mode mints no GuestCall, so a - // throw there has nothing to dispose — it propagates as the - // documented protocol-violation guest error.) - call.dispose(); - throw err; - } - call.releaseToRealm(); - return guestCallHandle(call); - }; -} - -/** - * The `__host_console` shape: parse the payload JSON and dispatch. Never - * throws host-side for malformed payloads (the guest never sends one; a - * malformed payload is dropped). - */ -function makeConsoleHostFunction(vm: ReplVm, handlers: GuestBridgeHandlers): HostFunction { - const shim = getVmShim(vm) as QuickJS; - return function (this: JSValueHandle, ...args: JSValueHandle[]): JSValueHandle { - const level = optionalString(args[0]); - const payloadJson = optionalString(args[1]); - if (level !== null && payloadJson !== null && isConsoleLevel(level)) { - let payload: unknown; - try { - payload = JSON.parse(payloadJson); - } catch { - payload = undefined; - } - if (isConsolePayload(payload)) { - handlers.console({ level, line: payload.line }); - } - } - return shim.undefined; - }; -} - -function isConsoleLevel(level: string): level is ConsoleLevel { - return level === 'log' || level === 'info' || level === 'warn' || level === 'error' || level === 'debug'; -} - -function isConsolePayload(value: unknown): value is { line: string } { - if (typeof value !== 'object' || value === null) return false; - const v = value as { line?: unknown }; - return typeof v.line === 'string'; -} - -/** - * The `__host_sleep` shape: same pattern as `__host_agent` — mint a - * `GuestCall`, hand it to the handler (which settles it from a host-side - * timer), return its promise handle into the realm. `ms` is validated - * (a non-number is a guest protocol violation, like the other host - * functions' string validations). - */ -function makeSleepHostFunction(vm: ReplVm, handlers: GuestBridgeHandlers): HostFunction { - return function (this: JSValueHandle, ...args: JSValueHandle[]): JSValueHandle { - const ms = args[0]; - if (ms === undefined || !ms.isNumber) { - throw new TypeError(`${HOST_SLEEP}: ms must be a number (guest protocol violation)`); - } - const call = new GuestCall(vm); - try { - handlers.sleep(call, ms.toNumber()); - } catch (err) { - // Same disposal discipline as `__host_agent` (see there): a - // throwing sleep handler must not strand the raw promise and its - // resolving functions. - call.dispose(); - throw err; - } - call.releaseToRealm(); - return guestCallHandle(call); - }; -} - -/** - * The `__host_workspace` shape: the handler's JSON string is returned - * synchronously into the realm (a string handle — the guest parses it). - * A throwing handler propagates as the documented protocol-violation - * guest error. - */ -function makeWorkspaceHostFunction(vm: ReplVm, handlers: GuestBridgeHandlers): HostFunction { - const shim = getVmShim(vm) as QuickJS; - return function (this: JSValueHandle): JSValueHandle { - return shim.newString(handlers.workspace()); - }; -} - -/** - * The `__host_agents` shape: same synchronous JSON-string contract as - * `__host_workspace`. - */ -function makeAgentsHostFunction(vm: ReplVm, handlers: GuestBridgeHandlers): HostFunction { - const shim = getVmShim(vm) as QuickJS; - return function (this: JSValueHandle): JSValueHandle { - return shim.newString(handlers.agents()); - }; -} - -/** - * The `__host_reset` shape: the handler marks the teardown request and - * returns nothing. A throwing handler propagates as the documented - * protocol-violation guest error. - */ -function makeResetHostFunction(vm: ReplVm, handlers: GuestBridgeHandlers): HostFunction { - const shim = getVmShim(vm) as QuickJS; - return function (this: JSValueHandle): JSValueHandle { - handlers.reset(); - return shim.undefined; - }; -} - -/** - * The `__host_default_backend` shape: the handler's default backend id - * string is returned synchronously (undefined when no registry is - * attached — the parking bridge; the guest's verify/judgePanel then - * reject honestly). A throwing handler propagates as the documented - * protocol-violation guest error. - */ -function makeDefaultBackendHostFunction(vm: ReplVm, handlers: GuestBridgeHandlers): HostFunction { - const shim = getVmShim(vm) as QuickJS; - return function (this: JSValueHandle): JSValueHandle { - const backend = handlers.defaultBackend(); - if (backend === undefined) return shim.undefined; - return shim.newString(backend); - }; -} - -function requireString(arg: JSValueHandle | undefined, hostFn: string, what: string): string { - if (arg === undefined || !arg.isString) { - throw new TypeError(`${hostFn}: ${what} must be a string (guest protocol violation)`); - } - return arg.toString(); -} - -function optionalString(arg: JSValueHandle | undefined): string | null { - if (arg === undefined || !arg.isString) return null; - return arg.toString(); -} - -/** A guest-library install failure (the library script failed to evaluate). */ -export class GuestLibraryInstallError extends Error { - /** Trap-free error info from the failed evaluation. */ - readonly info: EvalErrorInfo; - - constructor(info: EvalErrorInfo) { - super(`Guest library install failed: ${info.name}: ${info.message}`); - this.name = 'GuestLibraryInstallError'; - this.info = info; - } -} - -// ──────────────────────────────────────────────────────────────────────── -// The reconciliation surface and realm-slot access -// ──────────────────────────────────────────────────────────────────────── - -/** One entry of the guest's pending-call manifest. */ -export interface GuestSurfaceEntry { - id: string; - /** `"agent"` | `"checkpoint"` | `"queue"` | `"steer"` | `"cancel"`. */ - kind: string; - /** Verbatim prompt / question / action. */ - detail: string | null; - /** Verbatim options JSON string, or null. */ - optionsJson: string | null; - /** Realm `Date.now()` at issue time. */ - createdAt: number; - /** - * The id the host addresses this call by: the call's own id for agent/ - * checkpoint calls, the FOUNDING session id for steering calls — the - * correlation a restore needs to settle (by `id`) or re-issue (to the - * session) a pending steer. - */ - sessionId: string; - /** The agent call's backend-routing spec (null for other kinds). */ - modelSpec: string | null; -} - -/** The name of the guest library's continuation-lease accessor global - * (see `readContinuationLease`). */ -export const GUEST_LEASE_GLOBAL = '__replLease'; - -/** - * Read the guest library's CONTINUATION LEASE token (a string) or - * `undefined` — the host side of the eval-break targeting seam (see - * `ReplJobLease` in vm.ts): the drain loop reads it between jobs, and a - * job that starts with a lease set IS the armed eval's continuation - * segment. The lease global is a NON-CONFIGURABLE accessor installed by - * the library itself; its getter is the library's frozen closure - * (trusted host-installed code, never guest-authored), so invoking it - * between VM operations is safe. Best-effort: any failure reads as - * `undefined` (the targeting degrades to the honest refusal). - */ -export function readContinuationLease(vm: ReplVm): string | undefined { - const shim = getVmShim(vm) as QuickJS; - let key: JSValueHandle | undefined; - let value: JSValueHandle | undefined; - try { - key = shim.newString(GUEST_LEASE_GLOBAL); - value = shim.getProp(shim.global, key); - if (value.isUndefined) return undefined; - if (!value.isString) return undefined; - return value.toString(); - } catch { - return undefined; - } finally { - key?.dispose(); - value?.dispose(); - } -} - -/** Clear the guest library's continuation lease (see - * `readContinuationLease`). Called by the drain loop at drain start and - * after every lease-carrying job; the library's setter is its own - * frozen closure. Best-effort: a failing clear leaves a stale lease - * that the next drain's start-clear retries. */ -export function clearContinuationLease(vm: ReplVm): void { - const shim = getVmShim(vm) as QuickJS; - let key: JSValueHandle | undefined; - try { - key = shim.newString(GUEST_LEASE_GLOBAL); - shim.setProp(shim.global, key, shim.undefined); - } catch { - // Best-effort (see the doc comment). - } finally { - key?.dispose(); - } -} - -/** - * The trap-free TYPE TOKEN of a realm global slot: the data value's - * `typeof` (read through the own property descriptor, never a `[[Get]]` - * — an accessor is never invoked), `'accessor'` for an accessor-rebound - * slot, `'absent'` for a missing one. The manifest's baseline-change - * detector: a user rebinding of a baseline global (`Math = 42`) changes - * the token from the fresh-realm baseline's, so the binding is listed - * (the phase-E review rejection: the baseline filter hid overwritten - * built-ins). - */ -export function readRealmSlotTypeToken(vm: ReplVm, name: string): string { - const shim = getVmShim(vm) as QuickJS; - const global = shim.global; // cached singleton — do not dispose - const e = shim._getExports(); - const keyHandle = shim.newString(name); - let descPtr: number; - try { - descPtr = e.qjs_get_own_property_descriptor(global.ptr, keyHandle.ptr); - } finally { - keyHandle.dispose(); - } - if (descPtr === 0) return 'absent'; - const desc = new JSValueHandle(shim, descPtr); - try { - if (e.qjs_is_exception(desc.ptr) !== 0) { - const excPtr = e.qjs_get_exception(); - if (excPtr !== 0) new JSValueHandle(shim, excPtr).dispose(); - return 'absent'; - } - if (!hasOwnRaw(e, shim, desc.ptr, 'value')) { - getPropRaw(e, shim, desc.ptr, 'get')?.dispose(); - getPropRaw(e, shim, desc.ptr, 'set')?.dispose(); - return 'accessor'; - } - const valueProp = getPropRaw(e, shim, desc.ptr, 'value'); - if (valueProp === undefined) return 'absent'; - try { - return valueProp.typeof; - } finally { - valueProp.dispose(); - } - } finally { - desc.dispose(); - } -} - -/** - * The host's door back into the guest's pending-call registry — the - * post-restore reconciliation surface - * (`globalThis[Symbol.for("repl.guest")]`). `pending`/`settle`/`stats` - * execute the guest library's own frozen closure functions (never - * guest-authored code — the surface object is frozen and its global - * binding is non-configurable), and the member reads are trap-free. - */ -export interface GuestSurface { - /** The resident guest library version (equals `__REPL_GUEST_VERSION`). */ - version: string; - /** True when this library copy carries the 0.2.0 eval-await tracking - * surface (`__replAwait`/`awaitLog`/`promiseCallIds`). False remains - * the defensive fallback for direct/raw restores carrying an older - * library. Stored pre-v2 snapshots never reach this fallback: their - * older envelope format is refused and auto-reset on first touch. */ - supportsAwaitTracking: boolean; - /** True when this library copy carries the 0.3.0 CONTINUATION-LEASE - * surface (`__replAwait(value, token)` + the `__replLease` accessor - * global): the eval-break interrupt's genuine per-eval identity — the - * drain loop reads the lease between jobs, and the armed signal fires - * only while the armed eval's continuation executes. False on a - * 0.1.0/0.2.0 snapshot: the host serves it as-is, skips the - * instrumenter, and the interrupt refuses honestly (the 0.2.0 - * log-only targeting is the rejected settled-call-ids identity). */ - supportsContinuationLease: boolean; - /** True when this library copy carries the 0.3.1 ITERABLE-LEASE - * surface (`__replAwaitIterable` — the for-await iterable wrap that - * preserves the iterable protocol while setting the continuation - * lease per iteration). False on a 0.3.0 copy (whose for-await wrap - * returned a promise and broke every `for await` loop): the host - * leaves that snapshot's for-await sites unwrapped — native loop - * semantics, no mid-loop eval-break targeting (the honest - * degradation). */ - supportsIterableLease: boolean; - /** Manifest of every pending host call, oldest first. */ - pending(): GuestSurfaceEntry[]; - /** - * Settle a pending call by id. Returns true iff an entry was pending; - * false for unknown/already-settled ids (idempotent). The caller must - * drain afterwards so continuations fire. - */ - settle(callId: string, outcome: 'resolve' | 'reject', value: unknown): boolean; - /** Counters for diagnostics and the workspace manifest. */ - stats(): { version: string; callSeq: number; logSeq: number; pendingCalls: number }; - /** The awaits logged since the host last took them, oldest first - * (call-id strings); the log is cleared by the take. The eval-break - * targeting seam — the entries between two operation boundaries are - * the awaits of the operations' own code plus any continuations - * their drains resumed. Absent on 0.1.0 library copies (the host - * guards on `supportsAwaitTracking`). */ - awaitLogTake?(): string[]; -} - -/** - * Read the reconciliation surface from a VM. Returns `undefined` when the - * guest library is not installed (a bare VM, or a host that has not yet - * injected it). - * - * The returned surface object pins NO guest memory: it is plain data plus - * closures over the VM. Every handle it needs (the surface object itself, - * the member functions) is acquired per call and disposed on the spot — - * a long-lived surface must not accumulate handles (review: the previous - * shape captured three owned function handles in closures with no - * disposal contract). - */ -export function readGuestSurface(vm: ReplVm): GuestSurface | undefined { - const shim = getVmShim(vm) as QuickJS; - const symbol = shim.newSymbolFor(GUEST_SURFACE_KEY); - let surfaceHandle: JSValueHandle | undefined; - try { - surfaceHandle = shim.getProp(shim.global, symbol); - if (surfaceHandle.isUndefined || !surfaceHandle.isObject) return undefined; - - const versionHandle = readOwnDataProperty(surfaceHandle, 'version'); - let version = 'unknown'; - if (versionHandle !== undefined) { - try { - if (versionHandle.isString) version = versionHandle.toString(); - } finally { - versionHandle.dispose(); - } - } - - // Presence check once: the three functions must exist for the surface - // to be usable. The handles are only touched here — the closures below - // re-acquire per call (see `callSurfaceFunction`). - const pendingHandle = readOwnDataProperty(surfaceHandle, 'pending'); - const settleHandle = readOwnDataProperty(surfaceHandle, 'settle'); - const statsHandle = readOwnDataProperty(surfaceHandle, 'stats'); - const complete = - pendingHandle !== undefined && - settleHandle !== undefined && - statsHandle !== undefined && - pendingHandle.isFunction && - settleHandle.isFunction && - statsHandle.isFunction; - pendingHandle?.dispose(); - settleHandle?.dispose(); - statsHandle?.dispose(); - if (!complete) return undefined; - - // The 0.2.0 eval-await tracking seam: `supportsAwaitTracking` (a - // static boolean) and the optional `awaitLogTake` function. A - // snapshot carrying the 0.1.0 library lacks both — the surface - // reports `false` and the host degrades (no await instrumenter, no - // eval-break targeting). - const trackingHandle = readOwnDataProperty(surfaceHandle, 'supportsAwaitTracking'); - let supportsAwaitTracking = false; - if (trackingHandle !== undefined) { - try { - supportsAwaitTracking = trackingHandle.isBool && trackingHandle.toBoolean(); - } finally { - trackingHandle.dispose(); - } - } - // The 0.3.0 continuation-lease seam: `supportsContinuationLease` (a - // static boolean). Absent on 0.1.0/0.2.0 copies — the host serves - // the snapshot as-is and the eval-break interrupt refuses honestly - // (no instrumentation on it). - const leaseHandle = readOwnDataProperty(surfaceHandle, 'supportsContinuationLease'); - let supportsContinuationLease = false; - if (leaseHandle !== undefined) { - try { - supportsContinuationLease = leaseHandle.isBool && leaseHandle.toBoolean(); - } finally { - leaseHandle.dispose(); - } - } - // The 0.3.1 iterable-lease seam: `supportsIterableLease` (a static - // boolean). Absent on 0.1.0/0.2.0/0.3.0 copies — the instrumenter - // leaves their for-await iterables unwrapped (native semantics). - const iterableLeaseHandle = readOwnDataProperty(surfaceHandle, 'supportsIterableLease'); - let supportsIterableLease = false; - if (iterableLeaseHandle !== undefined) { - try { - supportsIterableLease = iterableLeaseHandle.isBool && iterableLeaseHandle.toBoolean(); - } finally { - iterableLeaseHandle.dispose(); - } - } - const awaitTakeHandle = readOwnDataProperty(surfaceHandle, 'awaitLogTake'); - const hasAwaitTake = awaitTakeHandle !== undefined && awaitTakeHandle.isFunction; - awaitTakeHandle?.dispose(); - - return { - version, - supportsAwaitTracking, - supportsContinuationLease, - supportsIterableLease, - pending: () => callSurfaceFunction(vm, 'pending') as GuestSurfaceEntry[], - settle: (callId, outcome, value) => callSurfaceSettle(vm, callId, outcome, value), - stats: () => callSurfaceFunction(vm, 'stats') as ReturnType, - ...(supportsAwaitTracking && hasAwaitTake - ? { awaitLogTake: () => callSurfaceFunction(vm, 'awaitLogTake') as string[] } - : {}), - }; - } finally { - symbol.dispose(); - surfaceHandle?.dispose(); - } -} - -/** - * Raw `qjs_call` (the shim's private `callFunctionRaw` drives the same - * export): invoke a guest function with borrowed arguments and return the - * raw result pointer (owned by the caller). The arguments are written into - * a wasm argv array exactly like the shim's own path; they are NOT - * consumed by the call. - */ -function callRaw(e: QuickJSExports, shim: QuickJS, fnPtr: number, args: JSValueHandle[]): number { - const argc = args.length; - let argvPtr = 0; - if (argc > 0) { - argvPtr = e.wasm_malloc(argc * 4); - const view = new DataView(e.memory.buffer); - for (let i = 0; i < argc; i++) { - view.setUint32(argvPtr + i * 4, args[i].ptr, true); - } - } - try { - return e.qjs_call(fnPtr, shim.undefined.ptr, argc, argvPtr); - } finally { - if (argvPtr !== 0) e.wasm_free(argvPtr); - } -} - -/** - * Acquire the surface and one of its member functions, call it with the - * given arguments, and read the result trap-free. Every handle is - * disposed on every path — nothing is retained by the caller. - * - * The functions are the library's frozen closures (no guest-authored code - * can be substituted), and settle's arguments are pre-validated so the - * library functions cannot throw; the raw call is still checked for an - * exception result and the runtime exception is taken out and freed — - * never routed through quickjs-wasi's `JSException` constructor (which - * performs guest-visible `[[Get]]` reads of name/message/stack on the - * exception value). - */ -function callSurfaceFunction( - vm: ReplVm, - member: 'pending' | 'stats' | 'awaitLogTake', -): unknown { - const shim = getVmShim(vm) as QuickJS; - const e = shim._getExports() as QuickJSExports; - const symbol = shim.newSymbolFor(GUEST_SURFACE_KEY); - let surfaceHandle: JSValueHandle | undefined; - let fn: JSValueHandle | undefined; - try { - surfaceHandle = shim.getProp(shim.global, symbol); - if (surfaceHandle.isUndefined || !surfaceHandle.isObject) return undefined; - fn = readOwnDataProperty(surfaceHandle, member); - if (fn === undefined || !fn.isFunction) return undefined; - const result = new JSValueHandle(shim, callRaw(e, shim, fn.ptr, [])); - try { - if (e.qjs_is_exception(result.ptr) !== 0) { - takeAndFreeException(e, shim); - return undefined; - } - if (result.isUndefined) return undefined; - // The pending-call registry and the await log are the host's own - // reconciliation/targeting metadata (call ids, kinds, verbatim - // options — created by the frozen guest library, never by guest - // code), not guest content: the read is COMPLETE (no array-length - // or object-key cap — phase-E review round 3: the 16 384-element - // array cap silently truncated the pending registry and its - // `[ArrayTruncated]` marker leaked into the broker's id lists as - // an `undefined` hole; `readValueComplete` lifts both caps, so - // `pending` reports the WHOLE registry, bounded like the metadata - // itself by the VM's memory). - return readValueComplete(result); - } finally { - result.dispose(); - } - } finally { - fn?.dispose(); - surfaceHandle?.dispose(); - symbol.dispose(); - } -} - -function callSurfaceSettle( - vm: ReplVm, - callId: string, - outcome: 'resolve' | 'reject', - value: unknown, -): boolean { - // Pre-validate host-side: the guest's own settle throws a TypeError for - // an invalid outcome, and surfacing that through the shim's callFunction - // would construct a `JSException` (whose constructor performs - // guest-visible [[Get]] reads of name/message/stack on the guest - // exception — a polluted Error.prototype.name getter would fire). With - // the outcome validated here, the library function cannot throw. - if (outcome !== 'resolve' && outcome !== 'reject') { - throw new TypeError('settle(callId, outcome, value): outcome must be "resolve" or "reject"'); - } - const shim = getVmShim(vm) as QuickJS; - const e = shim._getExports(); - const symbol = shim.newSymbolFor(GUEST_SURFACE_KEY); - let surfaceHandle: JSValueHandle | undefined; - let settleFn: JSValueHandle | undefined; - let callIdHandle: JSValueHandle | undefined; - let outcomeHandle: JSValueHandle | undefined; - let valueHandle: JSValueHandle | undefined; - try { - surfaceHandle = shim.getProp(shim.global, symbol); - if (surfaceHandle.isUndefined || !surfaceHandle.isObject) return false; - settleFn = readOwnDataProperty(surfaceHandle, 'settle'); - if (settleFn === undefined || !settleFn.isFunction) return false; - callIdHandle = shim.newString(callId); - outcomeHandle = shim.newString(outcome); - valueHandle = marshalValue(shim, value); - const result = new JSValueHandle( - shim, - callRaw(e, shim, settleFn.ptr, [callIdHandle, outcomeHandle, valueHandle]), - ); - try { - if (e.qjs_is_exception(result.ptr) !== 0) { - takeAndFreeException(e, shim); - return false; - } - if (!result.isBool) return false; - return result.toBoolean(); - } finally { - result.dispose(); - } - } finally { - callIdHandle?.dispose(); - outcomeHandle?.dispose(); - valueHandle?.dispose(); - settleFn?.dispose(); - surfaceHandle?.dispose(); - symbol.dispose(); - } -} - -/** - * How a realm global slot resolved, trap-free. The `$N` store is the - * agent's own workspace — guest code CAN redefine a slot as an accessor; - * the resolver reads the slot through its own property descriptor, never - * a `[[Get]]` (an accessor is never invoked). - */ -export type RealmSlot = { kind: 'data' } | { kind: 'accessor' } | { kind: 'absent' }; - -/** - * Resolve a realm global slot trap-free (own-descriptor read on - * `globalThis` only): `data` when the slot holds a value, `accessor` when - * guest code rebound it to a getter (observing it would execute guest - * code — render a marker instead), `absent` when it does not exist. - */ -export function readRealmSlot(vm: ReplVm, name: string): RealmSlot { - const shim = getVmShim(vm) as QuickJS; - const global = shim.global; // cached singleton — do not dispose - const e = shim._getExports(); - const keyHandle = shim.newString(name); - let descPtr: number; - try { - descPtr = e.qjs_get_own_property_descriptor(global.ptr, keyHandle.ptr); - } finally { - keyHandle.dispose(); - } - if (descPtr === 0) return { kind: 'absent' }; - const desc = new JSValueHandle(shim, descPtr); - try { - if (e.qjs_is_exception(desc.ptr) !== 0) { - // Failed read: take the exception out and free it; the slot reads as - // absent rather than throwing through JSException. - const excPtr = e.qjs_get_exception(); - if (excPtr !== 0) new JSValueHandle(shim, excPtr).dispose(); - return { kind: 'absent' }; - } - // Data vs accessor via `hasOwnProperty` on the descriptor object (raw): - // `getPropRaw` alone returns an undefined-valued handle for a plain - // miss, which would misread an accessor as data. - if (hasOwnRaw(e, shim, desc.ptr, 'value')) { - getPropRaw(e, shim, desc.ptr, 'value')?.dispose(); - return { kind: 'data' }; - } - // Accessor: never invoke; free the owned get/set handles. - getPropRaw(e, shim, desc.ptr, 'get')?.dispose(); - getPropRaw(e, shim, desc.ptr, 'set')?.dispose(); - return { kind: 'accessor' }; - } finally { - desc.dispose(); - } -} diff --git a/packages/repl-engine/src/broker.ts b/packages/repl-engine/src/broker.ts deleted file mode 100644 index 304c6bc1..00000000 --- a/packages/repl-engine/src/broker.ts +++ /dev/null @@ -1,6657 +0,0 @@ -/** - * Persistent REPL broker. - * - * The broker owns two deliberately separate control planes for each reusable ACP session: - * - * - strict `steer()` is transient control of the ACP prompt currently in flight. It parses only - * raw `initializeMeta.steering.supported === true`, serializes control requests per lane, sends - * `idleBehavior: "promptRequired"`, and never starts or queues a prompt. Idle/unadvertised - * steering resolves `idle`/`unsupported`; malformed or `startedNewTurn` responses are fatal - * protocol violations. - * - `queue()` creates a first-class future public turn with its own call id, durable store record, - * promise, cancellation target, answer/schema repair, and workspace admission sequence. Queue - * heads run FIFO per session through ordinary `session/prompt`; no backend-native queue or - * steering extension implements them. - * - * A session lane explicitly tracks its active turn, prompt-in-flight boundary, queued turns, - * steering-control FIFO, cancellation fence, and usable/fatal/released lifecycle. Founding calls - * and queued turns share the global concurrency scheduler; the oldest eligible admission wins and - * ineligible work never blocks another session. Steering and cancellation consume no extra slot. - * - * The append-only call store records before guest settlement. Queue admission, handoff and - * cancellation markers make restore bounded and explicit: unhanded queue work remains eligible; - * handed-off work is never blindly resent and requires authoritative loaded-turn evidence; - * unresolved steering rejects `steering_interrupted` and is never replayed. Format-2 snapshots are - * refused by the format-3 envelope before guest execution and the daemon's existing auto-reset path - * renames them aside and clears their ledger. - * - * Cancellation targets one selected public turn. Pending queue cancellation sends no ACP request; - * active cancellation fences late output, sends at most one cancel, and keeps the lane blocked until - * the prompt settles or the 5-second fatal escalation fires. Turn-local failures allow later queue - * items; session/process loss, reattach failure, persistence failure, cancellation timeout, and - * steering protocol violations reject the whole lane without opening a blank replacement session. - * - * Unrelated REPL semantics remain unchanged: persistent QuickJS bindings, top-level await, - * checkpoint delivery, trap-free rendering, continuation-targeted eval break, provenance, - * boundary snapshots, client-presence drain, and record -> settle -> consume idempotence. - */ - -import type { JSValueHandle } from 'quickjs-wasi'; -import { - AcpAgentRunner, - LoadedTurnFailedError, - isLoadedTurnFailedError, - isLoadedTurnStillRunningError, - parseFinalJson, - resolveStructuredOutput, - type StructuredSession, -} from '@automatalabs/acp-agents'; -import { isWorkflowError, WorkflowError, WorkflowErrorCode } from '@automatalabs/shared-types'; -import { isAbsolute } from 'node:path'; - -import type { GuestBridgeHandlers, GuestCall, GuestSurfaceEntry } from './bridge.js'; -import { instrumentTopLevelAwaits } from './await-instrument.js'; -import { formatByteSize, headTailDescription, renderCompletionLine } from './preview.js'; -import { InMemoryCallStore, type CallOutcome, type CallRecord, type CallStore } from './store.js'; -import { DrainJobError, type ReplEvalOptions, type ReplEvalOutcome, type ReplJobLease } from './vm.js'; -import { Workspace } from './workspace.js'; -import type { EvalBreakChannel } from './eval-break-channel.js'; -import type { EvalErrorInfo } from './errors.js'; - -// ──────────────────────────────────────────────────────────────────────── -// Public types (all self-contained — the published declaration graph must -// stay free of acp-agents / quickjs-wasi types, per the package's -// consumer-fixture discipline) -// ──────────────────────────────────────────────────────────────────────── - -/** The outcome surface steering operations settle with (see module docs). */ -export type SteeringOutcomeValue = 'injected' | 'idle' | 'unsupported'; -export type CancelOutcomeValue = 'cancelled' | 'idle'; - -/** Options for opening one subagent session (the broker's structural - * subset of the runner's `InteractiveSessionOptions`). */ -export interface BrokerOpenSessionOptions { - /** The model spec — routed by the runner's own grammar. */ - model?: string; - /** The structured-output contract (a JSON Schema object) — folded into - * the backend's native schema channels by the runner's session - * (session/new `_meta` where the backend carries it there; the - * per-turn `_meta` forward and the in-band prompt contract come from - * the same value inside `InteractiveSession.prompt`). */ - schema?: unknown; - /** Agent-advertised ACP session mode id (strict confinement lever). */ - mode?: string; - /** ACP session config options, applied verbatim in sorted id order. */ - configOptions?: Record; - /** Coarse tier consulted only when `model` is unset. */ - tier?: string; - /** Absolute working directory for `session/new`. */ - cwd: string; - /** Event/telemetry label stamped onto this session's ACP events. */ - label?: string; - /** Correlation id stamped into `session/new` `_meta`. */ - runId?: string; - /** Generic session-scoped ACP `_meta` passthrough. */ - meta?: Record; - /** Backend-neutral system prompt instructions (`replace` / `append`): the - * structural twin of shared-types' `SystemPromptOptions`, spelled out here - * so the published type graph stays self-contained (public-types test); - * the runner refuses them before open when the backend carries neither. */ - systemPrompt?: { replace?: string; append?: string }; - /** Tool allow-list used by the headless permission auto-responder. */ - toolNames?: string[]; - /** Tool deny-list, applied after the allow-list. */ - disallowedToolNames?: string[]; - /** Skip the release-time session/close so the ACP session stays - * re-openable (the broker always passes true). */ - keepSession?: boolean; - /** Keep accumulated text/history after each turn (the broker always - * passes true — the schema ladder and status need the final message). */ - retainSessionLog?: boolean; -} - -/** Per-turn options (the broker's structural subset of the runner's). */ -export interface BrokerPromptOptions { - /** Generic turn-scoped ACP `_meta` passthrough. */ - promptMeta?: Record; - /** Handoff acknowledgment: the session invokes this exactly when the - * prompt has passed every preflight check (released session, aborted - * signal, prompt-in-flight) and is being handed to the underlying ACP - * session/prompt — the point of no return. The broker records its - * queued-turn `delivered` marker inside this callback: a marker - * recorded here can never precede the backend handoff (review - * regression: the marker used to be recorded when the prompt promise - * was CREATED, so an async pre-handoff rejection — released session, - * aborted signal, prompt-in-flight — produced a non-null marker for a - * steer the backend never saw, and reconcile then skipped that - * never-delivered steer permanently). */ - onHandoff?: () => void; -} - -/** One completed interactive turn (the broker's structural stand-in for - * the runner's `InteractiveTurn`). */ -export interface BrokerTurn { - readonly stopReason: string; - readonly text: string; - readonly response?: unknown; -} - -/** Options for re-attaching an existing backend session (the restore - * path's re-attach arm; structural subset of acp-agents' - * `ReattachSessionOptions` — the same open options as - * `BrokerOpenSessionOptions` plus the required backend `sessionId`). */ -export interface BrokerLoadSessionOptions extends BrokerOpenSessionOptions { - /** The existing backend session id to re-attach (ACP `session/load`). */ - sessionId: string; -} - -/** A held-open ACP session (structural subset of the runner's - * `InteractiveSession` — what the broker drives). */ -export interface BrokerSession { - readonly sessionId: string; - /** The RESOLVED backend id this session belongs to (the re-attach - * routing pin — a backend id doubles as a model routing spec, so a - * restore or lazy re-attach routes by it instead of re-resolving the - * model spec against the current default backend; phase-D review - * round 2). Optional for third-party adapters: when absent, the - * persisted backendId stays null and routing falls back to the - * persisted model spec. */ - readonly backendId?: string; - /** Complete initialize response metadata. Extension support is parsed - * strictly by the REPL at the point of use. */ - readonly initializeMeta?: Readonly>; - /** Send one prompt turn. Only one turn may be in flight at a time. */ - prompt(content: string, opts?: BrokerPromptOptions): Promise; - /** Inject content into the in-flight turn via `_session/steering`. */ - steer(content: string, opts?: BrokerPromptOptions): Promise; - /** Cancel the active turn (ACP `session/cancel`). */ - cancel(): Promise; - /** Release the ACP session and close its dedicated process (the - * session stays re-openable on the backend when it was opened with - * `keepSession: true`). Idempotent. */ - release(): Promise; - /** The latest turn's assistant text. */ - currentTurnText(): string; - /** The latest turn's assistant text with the §5 chunk joiner — EVERY - * assistant message chunk joins with "\n\n" (the bible's [C]12 fold: - * multi-chunk replies gain the separator instead of gluing, and - * narration chunks stay apart from answer chunks). REAL on the - * acp-agents adapter (`SessionHandle.foldedTurnText`); OPTIONAL for - * third-party adapters — the broker degrades to `currentTurnText()` - * (the adapter's own fold) when absent. */ - foldedTurnText?(): string; - /** The latest turn's FINAL assistant message (schema extraction). */ - finalMessageText(): string; - /** The backend's native structured output for the latest turn, if any. */ - rawStructuredOutput(): unknown; - /** - * The loaded session's founding-turn completion — the re-attach arm's - * task source. REAL on the acp-agents adapter - * (`InteractiveSession.awaitCurrentTurn`), whose completion evidence is - * the `_session/loaded_turn` vendor extension (phase-D review round 3: - * an AUTHORITATIVE terminal channel — the quiet-grace heuristic and - * the blind re-issue were rejected): right after the `session/load` - * response the seam asks `_session/loaded_turn/query` whether the - * founding turn is still running, and the backend's answer is the - * classification — `completed` (the replay's trailing assistant message - * is the turn's FINAL message; resolves immediately with the real - * accumulated text, `stopReason` synthesized `end_turn`), `interrupted` - * (ended without a terminal message, nothing running — the - * SAFE-RE-ISSUE rejection class), or `running` (the loaded session - * stays ATTACHED and the seam waits for the authoritative - * `_session/loaded_turn/ended` notification — a quiet gap is only a - * progress-stream gap, never terminal evidence — bounded by the - * max-wait backstop). A backend WITHOUT the extension (the built-in - * claude and opencode backends today) is classified by the seam's - * OBSERVATION path instead (phase-F review round 2): the post-load - * continuation watch plus the replay probe under the connection-death - * contract — never a possibly-running re-issue. A `running` turn past - * the max-wait bound rejects with the `LoadedTurnStillRunningError` - * (the broker re-arms the seam on the still-attached session for BOTH - * forms — a possibly-running call is never re-issued); a turn that - * failed at the backend rejects with `LoadedTurnFailedError` (a - * definite outcome, settled as a rejection, never re-issued); - * everything else (no user message, `interrupted`, a dead process) is - * the safe-re-issue class (observably dead — re-issue cannot - * duplicate). OPTIONAL for third-party `BrokerSession` adapters: an - * adapter without the seam still re-attaches the session, then - * degrades through the re-issue fallback — never a permanent hold - * (re-attachment itself is unavailable there). A seam that rejects - * with the still-running class and `rearmable: false` (it can NEVER - * observe the terminal state) is NOT re-invoked: the broker keeps the - * loaded session attached and waits for the terminal state from the - * surfaces below (`loadedTurnEndedState`/`subscribeLoadedTurnEnded`, - * `released`, the call's cancel, or the client-presence drain's - * forced stop). - */ - awaitCurrentTurn?(): Promise; - /** The loaded session's recorded founding-turn terminal state (the - * `_session/loaded_turn/ended` notification, when the backend pushed - * one — a seam-less backend that sends it anyway). The - * non-re-armable settlement wait's observability surface; OPTIONAL - * for third-party adapters that cannot observe it (the wait then - * settles only on a cancel, the drain's forced stop, or the - * session's release). */ - loadedTurnEndedState?(): { stopReason?: string; error?: { name: string; message: string } } | null; - /** Watch the loaded-turn-ended channel (fires immediately for a - * notification that already arrived). Returns the unsubscribe - * thunk. OPTIONAL like `loadedTurnEndedState`. */ - subscribeLoadedTurnEnded?(listener: () => void): () => void; - /** Resolve when the session is released (its dedicated process died - * or was disposed) — the non-re-armable settlement wait's release - * watch. OPTIONAL for third-party adapters that cannot expose it. */ - released?(): Promise; -} - -/** The runner seam the broker drives (structural subset of - * `AcpAgentRunner` — tests inject fakes). */ -export interface BrokerRunner { - /** Ids of every configured backend (built-ins plus registered custom - * agents) — the admission-validation vocabulary for the model spec's - * backend segment (§4.1: an unknown segment rejects synchronously, - * naming the segment and enumerating the known backends — never a - * silent route to the default backend). REQUIRED: every runner must - * publish its registry — validation runs on every dispatch path. The - * real acp-agents runner exposes `listBackends()`. */ - listBackends(): string[]; - /** The configured DEFAULT backend id (a REGISTERED segment) — the - * host's own routing for an omitted model. The broker serves it to - * the guest library (`__host_default_backend`), where the - * verify/judgePanel combinators resolve their reviewer/grader spec - * through it (§4.7 — the workers inherit the run's default model as - * a REAL registered segment; the v1 reserved 'default' sentinel that - * bypassed registry validation is deleted). The real acp-agents - * runner exposes `defaultBackendId()`. */ - defaultBackendId(): string; - /** A backend's STATIC config-option vocabulary, when its adapter - * publishes one (§4.1: `configOptions` keys validate at admission - * against the resolved backend's known vocabulary WHERE IT IS - * KNOWABLE). Returning `undefined` means the vocabulary is genuinely - * dynamic (agent-advertised at initialize — the built-ins) and the - * [C]5 fallback applies: the late error MUST name the offending key. - * Returning an array means the vocabulary is known: an unknown key - * rejects synchronously naming the key and the valid alternatives. */ - knownConfigOptionIds?(backendId: string): string[] | undefined; - openSession(opts: BrokerOpenSessionOptions): Promise; - /** - * Re-attach an existing backend session (ACP `session/load`) — the - * restore path's re-attach arm. Capability-gated per acp-agents - * (`negotiateCapabilities().supportsLoadSession` — all four built-in - * backends advertise it per docs/api.md): a backend that does not - * advertise the capability rejects before any wire request (the - * "same gate" a custom backend degrades through), and a lost/deleted - * session rejects with the backend's error. Either way the broker - * degrades to re-issue, surfaced guest-visibly. - */ - loadSession(opts: BrokerLoadSessionOptions): Promise; - dispose(): Promise; -} - -/** The eval tool-result shape. NOTE: the eval-plane redesign's §3.1 wire - * shape (`{ output: string, result?, running? }`) is assembled by the - * tool phase — this engine seam carries the pieces: `output` lines are - * the §4.4 reprs (one joined line per console.* call), - * raised-checkpoint lines, and the §4.6 uncaught-error renderings, with - * NO output caps applied; `kind` names the eval's outcome; `result` is - * the completion value's §4.4 repr when the eval resolved; `evalToken` - * is the eval's continuation token — the tool's fused-eval pump passes - * it back to `waitForCalls`, which attributes settlements swept during - * its pumps to exactly that eval. The v1 wire fields - * (`pending`/`checkpoints`/`completed`/`outputTruncated`) are deleted - * from the WIRE; `pending` stays on this internal seam because the - * pump's drained/target bookkeeping and the engine's own tests read it. */ -export interface ReplEvalResult { - /** Rendered output lines for this operation (one line per console.* - * call, checkpoint lines, error renderings) — NOT capped: the engine - * stops applying output caps to guest output (the redesign's §7; the - * Python posture — an agent CAN flood its own context). */ - output: string[]; - /** The eval's outcome: `value` — resolved (its repr in `result`); - * `error` — threw (the §4.6 rendering is in `output`); `pending` — - * suspended on a host call. For a wait result the kind reports the - * suspended eval the wait's pumps swept: `value`/`error` when that - * eval's continuation completed during the pumps, `pending` when it - * is still in flight. */ - kind: 'value' | 'error' | 'pending'; - /** The previewed completion value when the eval resolved (FORMAT.md - * collapsed rendering, trap-free); absent when the eval suspended or - * threw — EXCEPT a wait result, whose `result` is the suspended - * eval's completion repr when its continuation completed during the - * wait's pumps. */ - result?: string; - /** The eval's continuation token (`e`) — the fused-eval seam: pass - * it to `waitForCalls` so the wait attributes swept settlements to - * THIS eval (a concurrent client's eval can never steal the - * attribution). Absent on wait results. */ - evalToken?: string; - /** Pending call ids (the whole guest registry, in order) — non-empty - * exactly when the eval suspended (or when other work is in flight). */ - pending: string[]; - /** Checkpoints raised and still awaiting an answer. */ - checkpoints: CheckpointSummary[]; - /** Call ids settled into the guest by this operation: the pump's - * deliveries plus dispatch-time refusals during the eval. Checkpoint - * answers are deliberately excluded (an answered id leaves the - * `checkpoints` list — that is its visibility). */ - completed: string[]; -} - -/** One pending checkpoint as the tool result carries it: the question - * PREVIEWED through the top-level string rule (quoted, head+tail - * elided past 200 chars — guest-chosen text never crosses into the - * intent plane verbatim and unbounded; the id stays exact). */ -export interface CheckpointSummary { - id: string; - question: string; -} - -/** A pending checkpoint as the broker tracks it (raw question — the - * broker's internal table, not the tool-result surface). */ -export interface CheckpointInfo { - id: string; - question: string; - optionsJson: string | null; - raisedAtMs: number; -} - -/** One live subagent as `status` carries it. */ -export interface LiveAgentInfo { - /** The founding call id (the session's steering address). */ - callId: string; - modelSpec: string; - task: string; - state: 'opening' | 'running' | 'queued' | 'idle'; - supportsSteering: boolean; - queuedTurns: number; -} - -/** What the store-arm reconciliation did with each pending guest call. */ -export interface ReconcileReport { - /** Completed while down → settled now from the store. */ - settledFromStore: string[]; - /** Still resumable at the backend → re-attached via `loadSession` - * (including calls this broker already tracks — a repeated - * reconcile never re-attaches or re-issues twice). */ - reattached: string[]; - /** Lost → re-issued under the same call id (fresh session, the - * reissues counter bumped, the outcome settling the existing guest - * promise exactly once). */ - reissued: string[]; - /** Pending calls whose outcome is unknowable: steers whose wire call - * died with the process (settled `failed` with a warn line), and - * re-issue refusals (a corrupt registry entry, or the concurrency - * cap exhausted at restore). */ - failedLost: string[]; - /** Pending checkpoints re-surfaced into the broker's checkpoint - * table (answerable again across the restore). */ - requeuedCheckpoints: string[]; - /** Calls neither settled nor re-attached nor re-issued. Empty once - * all three arms ran (kept for report-shape compatibility with the - * store-only reconcile). */ - leftPending: string[]; - /** Queued-for-next-turn steers the store showed as accepted but - * undelivered (completion `queued`, no delivered/dropped marker) — - * re-queued against their founding sessions (or held for the - * session's next open), so a crash between enqueue and delivery - * loses nothing. Delivered and dropped steers are never replayed - * (the markers are first-wins). */ - reQueuedUndelivered: string[]; -} - -/** What kind of state-changing boundary fired (the doc's snapshot - * cadence: after each eval, and after each settlement drain that - * changed VM state). */ -export type SnapshotBoundaryKind = 'eval' | 'settlement'; - -/** One manifest binding (the broker-enriched form; see - * `Broker.workspaceManifest`). */ -export interface WorkspaceManifestBinding { - name: string; - /** Structure-only token — for an agent handle, `agent handle · - * pending|settled · call · ` (the live-handle status - * appended from the call store; the id maps to the task and - * timestamps). Every token carries the binding's byte size (phase-E - * review rejection: the token used to omit size for primitives, - * null, functions and plain promises). */ - token: string; - /** The machine-readable structure-only type label (`string`, - * `number`, `object`, `array`, `agent handle`, … — see preview.ts's - * `manifestTypeLabel`): the structured manifest's type field, so a - * structured consumer never has to parse the token (phase-E review - * round 4: the type used to live only inside the formatted token). */ - type: string; - /** The trap-free byte-size estimate of the binding's value — the - * doc's manifest contract is name, type, AND size for every - * top-level binding; exposed as its own field so a structured - * consumer never has to parse the token (phase-E review rejection: - * the broker exposed no separate size field). 0 only for the - * unreadable accessor/sabotage cases. */ - sizeBytes: number; - /** The stable call id when the binding is an agent handle; null - * otherwise (the engine reports it — the broker no longer embeds - * the call id only inside the token string; phase-E review round - * 4: the call id used to be discarded from the structured - * surface). */ - handleCallId: string | null; - /** The LIVE-HANDLE STATUS of an agent-handle binding, read from the - * call store: `pending` while the founding call is unsettled, - * `settled` once it completed (phase-E review round 4: the status - * used to be embedded only in the human token and was absent from - * the structured surface). Null for non-handle bindings. */ - handleStatus: 'pending' | 'settled' | null; - /** The sanitized provenance label (`eval 3`, `worker c2`, `session - * restore`), or null when untracked. */ - provenance: string | null; - /** Wall clock of the provenance attribution (ms since epoch). */ - provenanceAtMs: number | null; - /** The task text behind a worker provenance (`worker c1` → the - * founding agent() call's task, read from the call store) or an - * agent-handle binding's founding call — the doc's "from what task" - * provenance half. Null when the provenance is not worker-shaped or - * the store record is missing. Capped at 200 chars (head+tail) so - * the manifest stays bounded metadata. */ - task: string | null; -} - -/** The broker-enriched workspace manifest (`Broker.workspaceManifest`). */ -export interface WorkspaceManifestReport { - bindings: WorkspaceManifestBinding[]; - /** The `$N` log-ref globals as a range. */ - logs: { first: number | null; last: number | null; count: number }; - /** The provenance registry's snapshot-durable eval counter. */ - evalSeq: number; - /** In-flight host-task call ids (dispatch order). */ - inFlight: string[]; - /** Pending checkpoints (raw questions — the tool result previews). */ - checkpoints: CheckpointInfo[]; -} - -/** The state-changing-boundary sink (see the module docs' "The - * state-changing-boundary sink" section): `boundary(kind)` fires at - * every doc-defined boundary; `flush()` fires at the end of each - * serialized broker operation (the burst boundary) so a debouncing - * writer coalesces one drain burst's boundaries into one write. */ -export interface SnapshotSink { - /** A state-changing boundary occurred. */ - boundary(kind: SnapshotBoundaryKind): void; - /** The current burst ended — flush any debounced write now. */ - flush(): void; -} - -/** Options for attaching a broker to a workspace. */ -export interface BrokerOptions { - /** The ACP runner (structural subset of `AcpAgentRunner`; fakes for - * tests). Defaults to a bare `AcpAgentRunner` owned by this broker - * (disposed with it); hosts with a backend registry pass their own - * configured runner and own its lifetime (the broker still RELEASES - * every session it opened before dropping its state). */ - runner?: BrokerRunner; - /** The append-only call store. Defaults to `InMemoryCallStore`. */ - store?: CallStore; - /** The concurrency cap: max concurrent subagents per workspace - * (doc-settled default 6). Counts unsettled agent calls plus sessions - * running a queued turn (a queued-turn delivery or a §4.2 - * queued turn turn on a settled handle). The cap gates turn starts as - * well as dispatches: a dispatch above it queues in dispatch order - * (§4.1 — never a rejection), and an idle-session queued that - * would exceed it queues for the next free slot (its promise stays - * pending until the delivery runs). Validated as an - * integer in - * `[1, DEFAULT_MAX_CONCURRENT_AGENTS]`: invalid values (NaN, - * fractional, < 1) throw at attach time; values ABOVE the doc-settled - * ceiling are clamped to it (the ceiling is absolute — a - * misconfigured host can never open a seventh subagent). */ - maxConcurrentAgents?: number; - /** Per-eval and per-settlement-drain interrupt handler (a runaway - * guest continuation stays bounded). Also the DEFAULT interrupt - * handler for evals: a direct eval that runs away is interrupted - * with this signal unless the caller passes a per-eval handler. When - * omitted, the broker's own wall-clock EVAL DEADLINE bounds every - * eval and drain (the harness's eval guard — the doc's "break a - * runaway eval (the quickjs interrupt handler)": a runaway eval is - * interrupted by the quickjs interrupt handler once it exceeds the - * budget, the VM stays usable, and the currently-running eval can - * never hang the workspace forever — see `evalTimeoutMs`). */ - interruptHandler?: () => boolean; - /** The per-eval wall-clock deadline in ms (the harness's eval guard; - * phase-D review round 2: the interrupt tool's armed signal alone - * could only break the NEXT VM execution, because a synchronous - * runaway eval blocks the event loop before a later MCP request can - * arm it — the deadline makes the CURRENTLY running eval always - * breakable through the quickjs interrupt handler). Applied to every - * eval and every settlement drain; a `null`/`0` value disables the - * deadline. Default `DEFAULT_EVAL_TIMEOUT_MS` (30 000). */ - evalTimeoutMs?: number; - /** The OUT-OF-BAND eval-break channel (phase-F review round 2): the - * interrupt tool's no-id path deliverable to a SYNCHRONOUSLY running - * eval. A never-yielding eval blocks the daemon's single thread, so - * the interrupt request itself cannot be processed — the channel's - * worker thread receives the break (via its loopback HTTP endpoint) - * and sets a shared-memory flag that every eval execution's quickjs - * interrupt handler consumes mid-run (see `eval-break-channel.ts`). - * The probe is composed into every execution — fresh evals AND - * settlement drains — with the arm-after-start rule: a break armed - * after the execution began breaks it; a stale break (armed while - * the workspace was idle) is dropped on first observation and never - * breaks a later eval. The daemon's interrupt handling clears the - * flag once it processes the request (the continuation-targeted - * signal owns the break from then on). */ - evalBreakChannel?: EvalBreakChannel; - /** The state-changing-boundary sink (see the module docs): the - * daemon wires it to `ReplWorkspaceStore.snapshotWriter(workspace, - * wasm)` so every doc-defined boundary — after each eval, after - * each settlement drain that changed VM state — persists the - * workspace, with one drain burst's boundaries debounced into a - * single atomic write. */ - snapshotSink?: SnapshotSink; -} - -// ──────────────────────────────────────────────────────────────────────── -// Internal shapes -// ──────────────────────────────────────────────────────────────────────── - -/** One in-flight host task (an agent call or a steering operation): the - * outcome promise plus a readiness flag flipped by a microtask when the - * promise settles — the pump's poll-shaped readiness probe. */ -interface InFlightTask { - callId: string; - /** `agent` tasks release their session's concurrency token and start - * queued-turn delivery when the pump delivers them; `steer` tasks - * do not. */ - kind: 'agent' | 'queue' | 'steer' | 'cancel'; - /** `resolve`/`reject` deliver a record → settle → consume outcome; - * `hold` (the re-attach arm's unobservable-turn degradation) deletes - * the in-flight entry WITHOUT recording or settling — the call stays - * pending, the session stays attached (cancelable), and the broker - * surfaces the condition guest-visibly. */ - promise: Promise<{ outcome: 'resolve' | 'reject' | 'hold'; value: unknown }>; - done: boolean; -} - -/** The broker's per-session state. */ -interface SessionLaneState { - activeTurnId: string | null; - promptInFlight: boolean; - queuedTurnIds: string[]; - steeringControlIds: string[]; - steeringInFlight: boolean; - laneState: 'opening' | 'usable' | 'fatal' | 'released'; - cancellingTurnId: string | null; - cancellationTimer: ReturnType | null; -} - -interface SessionEntry { - session: BrokerSession; - /** The founding call id (the session's steering address). */ - callId: string; - modelSpec: string; - task: string; - /** The RESOLVED backend id (the session's own when it advertises one, - * else the admission-validated model-spec segment) — the §4.6 error - * attribution for queued turn turns on this session. */ - backendId: string; - initializeMeta: Readonly> | undefined; - /** True once the founding call settled (resolved or rejected). */ - callSettled: boolean; - /** True when the founding/current public turn was explicitly cancelled. */ - callCancelled: boolean; - /** Waiters woken when the entry's cancel flag flips (the - * non-re-armable settlement wait's cancel signal — see - * `markCancelled`). One-shot listeners, removed on fire. */ - readonly cancelWaiters: Set<() => void>; -} - -/** One first-class future public turn. */ -interface QueuedTurn { - callId: string; - sessionId: string; - prompt: string; - promptMeta?: Record; - call: GuestCall | null; - admissionSequence: number; - state: 'pending' | 'active' | 'cancelling' | 'settled'; - cancelRequested: boolean; -} - -interface SteeringControl { - callId: string; - sessionId: string; - targetTurnId: string; - prompt: string; - promptMeta?: Record; - call: GuestCall | null; -} - -/** A pending checkpoint as the broker tracks it (the GuestCall is the - * settlement target — kept separate from the agent/steer call table so - * an answer can never settle a call that shares the id space; NULL on - * the restore path, where the checkpoint re-surfaced from the in-VM - * registry and answers settle through the reconciliation surface). */ -interface PendingCheckpoint { - callId: string; - call: GuestCall | null; - question: string; - optionsJson: string | null; - raisedAtMs: number; -} - -/** The validated guest options bag (§4.1: the guest may pass EXACTLY - * `{ schema, cwd, configOptions, mode }` — any other key rejects - * synchronously listing the valid keys). */ -interface ParsedAgentOptions { - schema?: Record; - cwd?: string; - configOptions?: Record; - mode?: string; -} - -/** The exact option keys the guest may pass (§4.1). */ -const AGENT_OPTION_KEYS = new Set(['schema', 'cwd', 'configOptions', 'mode']); -/** The human-readable valid-keys list for admission errors. */ -const AGENT_OPTION_KEYS_TEXT = 'schema, cwd, configOptions, mode'; - -/** The exact option keys a steering payload may carry. */ -const STEER_OPTION_KEYS = new Set(['promptMeta']); - -/** Active prompt cancellation must terminate or quarantine the lane. */ -const CANCELLATION_SETTLEMENT_BOUND_MS = 5_000; - -/** Default per-workspace concurrency cap (the doc-settled limit). */ -export const DEFAULT_MAX_CONCURRENT_AGENTS = 6; - -/** Default per-eval wall-clock deadline in ms (the harness's eval guard; - * see `BrokerOptions.evalTimeoutMs`). */ -export const DEFAULT_EVAL_TIMEOUT_MS = 30_000; - -/** Default teardown ceiling in ms for `Broker.dispose` (spec-owed - * decision: the daemon's shutdown deadline is 5 s — the engine's own - * default mirrors it, so a hung backend can never block reset/shutdown - * past this bound even when the caller passes no explicit bound; the - * daemon's shutdown path shares ONE deadline across the drain and the - * disposal and passes the remaining time instead). */ -export const DEFAULT_DISPOSE_BOUND_MS = 5_000; - -/** The error-code vocabulary the broker rejects with (shared-types' - * `WorkflowErrorCode`, kept in lockstep with the runner's own errors). */ -const CODE = WorkflowErrorCode; - -// ──────────────────────────────────────────────────────────────────────── -// The broker -// ──────────────────────────────────────────────────────────────────────── - -/** - * The broker: wires a workspace's guest bridge to real ACP sessions - * (see the module docs for the full contract). Attach it to take over a - * workspace: - * - * ```ts - * const ws = await Workspace.create(projectDir); - * const broker = await Broker.attach(ws, { runner }); - * const result = await broker.eval('const pi = agent("pi/deepseek-v4-flash-max", "research X")'); - * ``` - * - * Operations serialize (eval/pump/reconcile/dispose run one at a time), - * so two overlapping tool calls can never interleave their settlement - * bookkeeping. - */ -export class Broker { - /** The workspace this broker drives. */ - readonly workspace: Workspace; - /** The configured concurrency cap. */ - readonly maxConcurrentAgents: number; - - private readonly runner: BrokerRunner; - private readonly ownsRunner: boolean; - private readonly callStore: CallStore; - private readonly interruptHandler: (() => boolean) | undefined; - private readonly evalTimeoutMs: number; - private readonly evalBreakChannel: EvalBreakChannel | undefined; - /** The workspace's eval-break slot REGISTRATION ACK (phase-F review - * round 4): resolves once the relay worker applied the key→slot - * mapping; every serialized operation awaits it before touching the - * VM (`runSerialized`). Rejects when the worker dies before - * acknowledging — operations swallow the rejection and degrade to - * the per-eval deadline bound. */ - private readonly evalBreakReady: Promise; - private readonly sink: SnapshotSink | undefined; - private readonly consoleBuffer: Array<{ level: string; line: string }> = []; - private readonly sessions = new Map(); - /** Explicit lane state exists from founding-call admission through release. */ - private readonly lanes = new Map(); - /** Lazy re-attaches in flight (a settled handle's queued turn/steer/cancel - * loading its recorded backend session) — deduped per founding call id - * so concurrent steers share one load. */ - private readonly pendingReattaches = new Map>(); - private readonly checkpoints = new Map(); - /** Live GuestCalls by call id — the pump's settlement targets. */ - private readonly deferreds = new Map(); - /** In-flight host tasks (agent calls and steering ops). */ - private readonly inFlight = new Map(); - /** Unsettled agent call ids — one concurrency token each. */ - private readonly agentSlots = new Set(); - /** §4.1: dispatches QUEUED above the concurrency cap, in dispatch - * order — never a rejection. Each entry carries everything the - * dispatch needs; `kickDispatchQueue` starts them as slots free. - * Restore-time RE-ISSUES queue through the same structure (a lost - * call re-issued above a tightened cap stays PENDING in the guest - * registry until the kick — §4.1's queue-above-cap applies to every - * dispatch path, never a `ConcurrencyLimitError`). */ - private readonly dispatchQueue: Array< - | { - kind: 'dispatch'; - call: GuestCall; - callId: string; - modelSpec: string; - task: string; - optionsJson: string | null; - parsed: ParsedAgentOptions; - admissionSequence: number; - } - | { - kind: 'reissue'; - entry: GuestSurfaceEntry; - parsed: ParsedAgentOptions; - reason: string; - report: ReconcileReport; - } - > = []; - /** The cached known-backend vocabulary (see `knownBackends`). */ - private knownBackendsCache: string[] | undefined; - private readonly queuedTurns = new Map(); - private readonly steeringControls = new Map(); - private admissionSequence = 0; - /** Live `sleep(ms)` calls, keyed by host-minted tracking ids (not - * guest call ids — sleeps never enter the guest registry or the call - * store). */ - private readonly sleepCalls = new Map(); - private sleepSeq = 0; - /** The `reset()` request (see the eval handler): the teardown runs - * after the current eval completes. SCOPED to the eval being - * executed: cleared at the start of every eval op, so an unrelated - * later eval never inherits an earlier eval's request (the review - * defect: the workspace-global flag inserted EVERY later suspending - * eval into `resetOwningCompletions`, delaying the teardown behind - * unrelated suspended evals — the workspace kept running guest code - * after the reset-owning eval completed). A reset() called from a - * RESUMED continuation is attributed to the reset-calling eval's - * completion wrapper immediately, through the executing job's - * continuation token (see the reset handler) — the flag is only the - * code-phase snapshot. */ - private resetRequested = false; - /** The reset-requesting evals' retained SUSPENDED completions (a - * reset() owes its teardown AFTER the eval completes — a reset eval - * that suspended tears down only once its continuation settles; the - * sweep observes the settlement and flips `resetDue`). */ - private readonly resetOwningCompletions = new Set(); - /** The teardown owed by a completed reset eval, flipped by the sweep - * (or by `eval` for an in-call completion) and consumed by the - * serialized-op post-hook (`resetIfDue`) — OUTSIDE the chain slot - * (the disposal acquires the chain itself). */ - private resetDue = false; - /** The retained last settlement-drain error (workspace().diagnostics). - * — the §6.2 demotion: drain failures leave the eval result surface - * entirely and live under the diagnostics field. */ - private retainedDrainError: { name: string; message: string; atMs: number } | null = null; - /** The retained last reconcile summary (workspace().diagnostics — - * the §6.2 demotion). */ - private lastReconcileReport: ReconcileReport | null = null; - /** The retained per-call reconciliation lines (§6.2): the re-attach / - * re-issue / refusal / lost-steer surfacing v1 wrote into the - * console buffer demotes to workspace().diagnostics with the - * reconcile summary — ordinary reconciliation is diagnostics-only, - * and the eval result surface carries ONLY the [C]14 aggregate loss - * notice (never per-call reconciliation lines). Replaced at each - * reconcile. */ - private reconcileNotes: { level: 'info' | 'warn'; line: string; atMs: number }[] = []; - /** The fused-eval seam: settlements of suspended evals swept during - * the pumps of the CURRENT operation, keyed by the settled eval's - * continuation token (`e`). A wait's render reads its caller's - * token and reports that eval's completion (kind + result repr); - * the entry is consumed on read. Token-keyed so a concurrent - * client's eval can never steal another wait's attribution. */ - private readonly sweptEvalSettlements = new Map< - string, - { kind: 'value' | 'error'; result?: string } - >(); - /** Active queued public turns — one concurrency token each. */ - private readonly queueSlots = new Set(); - /** Call ids settled synchronously at dispatch (refusals) since the - * last eval result — reported in that eval's `completed`. */ - private readonly syncSettled: string[] = []; - - /** Completion wrappers of SUSPENDED evals whose continuation is still - * in flight (see `eval`): the wrapper settles when the continuation - * completes or is broken — the broker's "an eval is running" probe. - * This is the interrupt tool's eval-break TARGET surface (phase-E - * review rejection: the no-id interrupt used to arm a project-wide - * "next VM execution" boolean with no notion of a running eval, so - * an idle workspace's next eval — or an unrelated drain — consumed - * it; the doc's "break a runaway eval" requires a tracked, - * targetable running eval). Handles are owned by the broker; - * released by `sweepActiveEvals` when they settle and by `dispose`. - * A suspended eval stacks alongside earlier ones (each suspension - * retains its own wrapper). */ - private readonly activeEvalCompletions = new Set(); - /** The per-eval CONTINUATION TOKEN (`e1`, `e2`, …): minted per eval - * (see `runEval`), embedded in the instrumented code's - * `__replAwait(value, token)` calls, and attributed to the eval's - * completion wrapper when it SUSPENDS. The token is the eval-break - * signal's armed-target identity (phase-E review rejection round 5: - * the signal used to be keyed to settled call ids — the calls the - * target awaited — so an unawaited sibling `.then` job running - * before the target's continuation consumed it, and indirect waits - * (`await Promise.all([q])`) were refused entirely): the guest - * library's wrap-settling reaction sets the CONTINUATION LEASE to - * this token immediately before the eval's continuation segment, the - * drain loop mirrors the lease per job (see `jobLease`), and the - * signal fires only while the mirror holds an armed token — the - * executing job IS the armed eval's continuation. */ - private readonly evalTokens = new Map(); - /** The token of the eval currently being run (see `eval`/`runEval`: - * `runEval` mints it before the VM execution; `eval` attributes it - * to the completion wrapper when the eval suspends). */ - private lastEvalToken: string | undefined; - /** True while `runEval` is executing (its code phase and its own - * drain) — the reset handler's attribution discriminator: a job - * whose continuation token is NOT the current eval's while `runEval` - * is active is a RESUMED SUSPENDED eval (attributed through its - * completion wrapper); outside `runEval` every leased job is a - * resumed eval (the stale `lastEvalToken` of a now-suspended eval - * must never route through the per-eval flag). */ - private inRunEval = false; - /** The per-eval continuation-token mint counter. */ - private evalTokenSeq = 0; - /** The per-job continuation-lease seam (see `ReplJobLease` in vm.ts): - * the drain loop reads the guest library's lease before each job - * into `jobLease.cell.current` and clears it after a lease-carrying - * job. The interrupt handler (consulted DURING a job) reads the - * mirror; the interrupted-drain release reads it after the drain - * throws. Read/clear ride the workspace's lease seams (the guest - * library's `__replLease` accessor — trusted library code). */ - private readonly jobLease: ReplJobLease = { - read: () => this.workspace.readContinuationLease(), - clear: () => this.workspace.clearContinuationLease(), - cell: { current: undefined }, - }; - /** The eval-break signal (the interrupt tool's no-id arm): consulted - * ONLY by executions that resume suspended-eval continuations — the - * settlement drains (`drain`) and a direct eval's own drain phase - * (`runEval` composes the same handler) — NEVER by a fresh eval's - * own code (`runEval` does not compose it for the code phase): an - * unrelated eval can neither consume the signal nor be broken by - * it. The signal fires only while the currently-executing job is - * one of the armed targets' continuation segments — the job's lease - * (see `jobLease`) holds one of the armed tokens (phase-E review - * round 3/5: the carried defect's handler fired on whichever drain - * ran next — or whichever JOB ran first in a drain that settled a - * target's call, breaking an unawaited sibling's continuation and - * clearing the arm before the target ran). Consumed on first - * observation (the quickjs interrupt polls constantly, so the next - * target continuation execution after arming breaks mid-run). */ - private evalBreakArmed = false; - /** The OUT-OF-BAND break probe's execution marker (phase-F review - * round 2, see `evalBreakProbe`; round 3: the wall-clock start was - * replaced by the channel's monotonic ARM-SEQUENCE marker — a total - * order across the worker and this thread, so a break armed in the - * same millisecond as the execution's start can never be consumed - * as stale and lost): the arm-sequence counter's value the moment - * the CURRENT execution began — a fresh eval's code phase or a - * settlement drain. The probe consumes the channel's break flag - * only when its arm sequence exceeds this marker (the arm-after- - * start rule). Zero when no channel is wired (the probe is then - * absent anyway). */ - private currentExecutionStartSeq = 0; - /** How many out-of-band breaks were CONSUMED by an executing eval - * (a break that actually broke a running eval). */ - outOfBandBreakCount = 0; - /** The wall-clock moment of the most recent CONSUMED out-of-band - * break (see `consumeOutOfBandBreakReport`): the honest outcome - * record for the interrupt tool when the daemon was blocked. */ - private lastOutOfBandBreakAtMs: number | null = null; - /** The arming-time active-eval set the eval-break signal is scoped - * to: when every target settles (or is released), the signal is - * cleared with them — a signal whose target no longer exists must - * never leak into a later execution. */ - private evalBreakTargets = new Set(); - /** The armed targets' continuation TOKENS (see `evalTokens`): the - * eval-break signal's firing identity — the handler fires only - * while the current job's lease is one of these (phase-E review - * round 5). */ - private evalBreakTokens = new Set(); - /** The cached continuation-lease capability probe (see - * `continuationLeaseAvailable`): whether the workspace's guest - * library carries the 0.3.0 lease surface. Cached — the library - * never changes within a broker's lifetime; `undefined` until first - * probed. */ - private leaseCapabilityCached: boolean | undefined; - /** The cached iterable-lease capability probe (see - * `iterableLeaseAvailable`): whether the workspace's guest library - * carries the 0.3.1 `__replAwaitIterable` surface. Cached like - * `leaseCapabilityCached`; `undefined` until first probed. */ - private iterableLeaseCapabilityCached: boolean | undefined; - /** True once the client-presence drain released every child (see - * `drainForDisconnect`): the workspace stays live, and later - * queued turn/steer/cancel on a settled handle lazily re-attach the - * recorded backend session. */ - private drained = false; - /** True while a client-presence drain or a dispose is in progress (and - * stays true after a drain — the drained broker owns no attached - * sessions until a new open). The re-attach arm keys on it: a seam - * rejection while the broker is draining/disposing must NOT re-issue - * (a fresh child would open and run after the last client - * disconnected — the drain defect); the call is left pending and - * surfaced guest-visibly instead. */ - private draining = false; - /** Agent calls whose `openSession` is still in flight (no session entry - * exists yet — the session may appear at any moment). The client- - * presence drain waits for these exactly like busy sessions: a call - * blocked in openSession is still in flight, and draining past it - * would let the child open and run after the last client - * disconnected (phase-D review round 3). */ - private readonly openingCalls = new Set(); - /** Opening calls the drain's bound forced to STOP (see - * `drainForDisconnect`): when the parked `openSession` eventually - * lands, the session is released immediately (never prompts), the - * call settles as the recoverable `AGENT_CANCELLED`, and queued - * steers are dropped with the durable `dropped` marker — the child - * never runs after the drain. */ - private readonly stoppedOpens = new Set(); - /** The disposal/drain GENERATION (phase-D review round 5): bumped when - * the client-presence drain's bound expires and when the broker is - * disposed. In-flight `openSession` calls and lazy re-attaches capture - * the generation when they START; when they land after a bump, the - * child session is released immediately — it never registers and - * never prompts (a child must never open or run after the last - * client disconnected, nor after a reset/dispose). */ - private generation = 0; - private disposed = false; - private opChain: Promise = Promise.resolve(); - - private constructor(workspace: Workspace, options: BrokerOptions) { - this.workspace = workspace; - // Validate the cap: an integer in [1, DEFAULT_MAX_CONCURRENT_AGENTS]. - // Non-integer / NaN / < 1 values are programming errors at attach time - // (loud throw); values ABOVE the doc-settled ceiling are clamped to it - // — the six-per-workspace maximum is absolute, so a misconfigured host - // can never open a seventh subagent (review regression: a config of 7 - // used to open seven sessions). - const rawCap = options.maxConcurrentAgents ?? DEFAULT_MAX_CONCURRENT_AGENTS; - if (typeof rawCap !== 'number' || !Number.isInteger(rawCap) || rawCap < 1) { - throw new Error( - `Broker: maxConcurrentAgents must be an integer in [1, ${DEFAULT_MAX_CONCURRENT_AGENTS}] ` + - `(got ${String(rawCap)})`, - ); - } - this.maxConcurrentAgents = Math.min(rawCap, DEFAULT_MAX_CONCURRENT_AGENTS); - this.callStore = options.store ?? new InMemoryCallStore(); - this.interruptHandler = options.interruptHandler; - this.evalTimeoutMs = options.evalTimeoutMs ?? DEFAULT_EVAL_TIMEOUT_MS; - this.evalBreakChannel = options.evalBreakChannel; - // The workspace's slot is registered up front so the worker's HTTP - // endpoint knows the key before any eval can run. The registration - // is ACKNOWLEDGED (phase-F review round 4): every serialized - // operation awaits the ack before touching the VM (see - // `runSerialized`), so a no-id interrupt can never 404 against a - // key→slot mapping the worker has not applied yet — the old - // fire-and-forget registration left a window where the first - // interrupt's out-of-band break was lost and the eval ran to the - // per-eval deadline. - this.evalBreakReady = - options.evalBreakChannel?.register(this.workspace.projectDir) ?? Promise.resolve(); - this.sink = options.snapshotSink; - this.ownsRunner = options.runner === undefined; - this.runner = options.runner ?? new AcpAgentRunner(); - } - - /** - * Attach a broker to a workspace: re-register the four `__host_*` - * callbacks with the broker's handlers (the same by-name - * re-registration the restore path uses — the guest library and its - * pending-call registry are untouched, and every subsequent guest call - * routes to the broker). Works on a live workspace (replacing the - * parking bridge) and on a restored one. - */ - static async attach(workspace: Workspace, options: BrokerOptions = {}): Promise { - const broker = new Broker(workspace, options); - workspace.rehost(broker.makeHandlers()); - return broker; - } - - /** The guest-bridge handlers (see `GuestBridgeHandlers`). */ - makeHandlers(): GuestBridgeHandlers { - return { - agent: (call, callId, modelSpec, task, optionsJson) => { - this.onAgent(call, callId, modelSpec, task, optionsJson); - }, - checkpoint: (call, callId, question, optionsJson, answerJson) => { - return this.onCheckpoint(call, callId, question, optionsJson, answerJson); - }, - queue: (call, callId, sessionId, payloadJson) => { - this.onQueue(call, callId, sessionId, payloadJson); - }, - steer: (call, callId, sessionId, payloadJson) => { - this.onSteer(call, callId, sessionId, payloadJson); - }, - cancelSession: (call, callId, sessionId) => { - this.onSessionCancel(call, callId, sessionId); - }, - cancelQueue: (call, callId, queueCallId) => { - this.onQueueCancel(call, callId, queueCallId); - }, - console: (event) => { - this.consoleBuffer.push(event); - }, - // The eval-plane helpers (§4.5/§4.7): sleep settles from a host - // timer; workspace()/agents() serve the §4.5 plain-value shapes; - // reset() marks the teardown consumed after the current eval; - // defaultBackend() serves the runner's configured default backend - // id to the guest library's verify/judgePanel (§4.7). - sleep: (call, ms) => { - this.onSleep(call, ms); - }, - workspace: () => this.workspaceJson(), - agents: () => this.agentsJson(), - reset: () => { - // The request belongs to the eval whose code/continuation is - // EXECUTING. A RESUMED SUSPENDED eval's continuation (the - // continuation IS the eval's tail — `await agent(...); - // reset()`) is attributed to that eval's retained completion - // wrapper NOW, through the executing job's continuation token - // (the lease mirror — set by the drain loop while the - // continuation segment runs): the teardown then depends ONLY - // on the reset-calling eval's completion (the sweep flips - // `resetDue` when it settles) and never leaks into an - // unrelated later eval's snapshot (the review defect's shape). - // The CURRENT eval's request — its code phase (the mirror is - // clean there, see `runEval`) or its own continuation in its - // own drain (a job carrying THIS eval's token) — takes the - // per-eval flag instead: the eval op's snapshot attributes it - // (suspension → owning set, completion → `resetDue`). A legacy - // workspace without the lease surface reads an empty mirror - // and takes the flag path (its snapshots take the §6.1 - // auto-reset path on first touch anyway). - const token = this.jobLease.cell.current; - if (token !== undefined && (!this.inRunEval || token !== this.lastEvalToken)) { - for (const [completion, evalToken] of this.evalTokens) { - if (evalToken === token) { - this.resetOwningCompletions.add(completion); - break; - } - } - return; - } - this.resetRequested = true; - }, - defaultBackend: () => this.runner.defaultBackendId(), - }; - } - - /** - * The tool-result eval: settle what can be settled (pump), run the - * script (top-level-await semantics, the uncaught-rejection bridge - * armed), drain, and report the doc's shape. The pump runs FIRST so an - * eval that awaits a call which completed earlier resolves in-eval - * with its value ("an eval whose promise resolves within the drain - * reports the value"); a suspended eval's continuation resumes at a - * later settlement drain and its output lands in the next tool result. - */ - async eval(code: string, options: ReplEvalOptions = {}): Promise { - return this.serialized(async () => { - this.assertAlive(); - let completed: string[]; - const pumped = await this.pumpUnlocked(); - completed = pumped.settled; - // The pump's per-call settlement boundaries already fired inside - // `pumpUnlocked` (one per settled call's continuation drain); the - // sink's burst bookkeeping coalesces them with the eval's own - // boundary into one write at the operation's flush. A pump drain - // failure is RETAINED under workspace().diagnostics (§6.2 — it - // leaves the eval result surface; the already-settled call ids - // are still reported, and the VM stays usable). - // reset() (§4.5): the pump already swept the retained suspended - // evals (its end-of-pump sweep reads every settlement the pump's - // drains ran) — a reset-owning eval that COMPLETED during the - // pump flipped `resetDue`. Its teardown runs NOW, before this - // eval's submitted code: the reset-owning eval completed, so - // later guest code must never run against the doomed workspace - // (the review probe: `reset(); await sleep(10)` followed by an - // eval returning its own result before the disposal). The - // disposal runs with an already-expired bound — the deadline - // path's unlocked body — because the eval op itself holds the - // serialization chain (the default disposal would enqueue behind - // the operation that is awaiting it). - if (this.resetDue) { - await this.resetIfDue(0); - } - // reset() (§4.5) SCOPING: the request belongs to THIS eval — the - // snapshot below captures only what this eval's execution - // requested. The scope starts clean (an earlier suspended eval's - // continuation may have called reset() during the pump's drains; - // that request was attributed to ITS completion wrapper through - // the continuation-token seam in the reset handler) so an - // UNRELATED later eval that suspends can never join the owning - // set (the review defect: the workspace-global flag inserted - // every later suspending eval into `resetOwningCompletions`, - // delaying the teardown behind unrelated suspended evals — the - // workspace kept running guest code after the reset-owning eval - // completed). - this.resetRequested = false; - const { outcome, completion, interruptedInDrain } = this.runEval(code, options); - const resetByThisEval = this.resetRequested; - this.resetRequested = false; - if (interruptedInDrain === true) { - // The eval's OWN drain was interrupted (the armed eval-break - // signal's target resumed by a synchronous host-callback - // settlement — a checkpoint answer — inside this eval's drain, - // or the per-eval deadline): the interrupted continuation's - // engine wrapper NEVER settles (the quickjs interrupt aborts - // the async job without rejecting its promise — verified - // against the shipped binary), so the tracked "running eval" - // can only be released HERE — exactly like the pump path's - // release (phase-E review rejection round 2: the old signal was - // consulted only by settlement drains, and without this release - // a broken target stayed tracked forever, making a later - // eval-break arm target a dead eval). The release is EXACT - // (phase-E review rounds 3/5): the interrupted job's - // continuation lease (see `jobLease`) names the eval whose - // continuation was actually executing — an unrelated - // interrupted drain — THIS eval's own completion jobs bounded - // by the per-eval deadline, with no tracked continuation - // running — releases nothing and leaves the eval-break armed - // state intact. - this.releaseInterruptedEval(); - } - // The eval's own provenance pass: bindings this eval created or - // rebound are attributed to `eval N` (the registry's snapshot- - // durable counter). - this.provenancePass('eval'); - // The active-eval sweep FIRST reads late completions (a previous - // suspended eval settled during this operation's pump/drain): - // its value becomes `_` here, BEFORE this eval's own `_` write - // below (this eval is the most recent one — its value wins). - this.sweepActiveEvals(); - // The §4.4 result-history global: `_` holds the previous eval's - // completion value, IPython-style — the sole replacement for the - // deleted `$N` capture globals. Set after every eval that - // RESOLVED with a value — an undefined completion (an empty - // poll) makes `_` undefined: the previous eval's completion - // value IS undefined (the review probe: `42`, then `""`, then - // `_` must read undefined, never the stale 42). An error or a - // suspension leaves `_` unchanged, like IPython's. The set - // borrows the completion handle the render below still owns. - if (outcome.kind === 'value' && completion !== undefined) { - try { - this.workspace.setGlobal('_', completion as JSValueHandle); - } catch { - // A failed `_` write must not fail the eval that produced the - // value — the result line still renders. - } - } - // A SUSPENDED eval's completion wrapper is retained as the - // active-eval probe (the interrupt tool's eval-break target - // surface): the wrapper is pending while the eval's continuation - // is in flight and settles when the continuation completes or is - // broken — `sweepActiveEvals` releases it at the next operation - // (reading its fulfilled value into `_` first). - // A RESOLVED eval's completion handle is owned by `render` (it - // previews and disposes it); an error outcome carries none. The - // eval's CONTINUATION TOKEN is attributed alongside (see - // `evalTokens`): the token `runEval` minted and embedded in the - // instrumented code's `__replAwait(value, token)` calls — the - // eval-break signal's armed-target identity. The token is - // attributed at suspension only (a resolved eval needs no - // identity); it is only meaningful when the workspace's library - // carries the 0.3.0 continuation-lease surface (the arm refuses - // otherwise). - if (outcome.kind === 'pending' && completion !== undefined) { - this.activeEvalCompletions.add(completion as JSValueHandle); - if (this.lastEvalToken !== undefined) { - this.evalTokens.set(completion as JSValueHandle, this.lastEvalToken); - } - // reset() (§4.5) owes its teardown after THIS eval completes — - // a reset eval that SUSPENDED retains its completion in the - // owning set; the sweep flips `resetDue` when it settles. ONLY - // this eval's own wrapper joins (the per-eval scope above): an - // unrelated suspended eval never gates the teardown. - if (resetByThisEval) { - this.resetOwningCompletions.add(completion as JSValueHandle); - } - } - // reset() (§4.5): an eval that called reset() and COMPLETED - // within this call owes the teardown NOW (no owning completion - // outstanding). The disposal itself runs after the operation — - // the serialized-op post-hook (`resetIfDue`), OUTSIDE the chain - // slot — so this eval's result ships first. - if (resetByThisEval && outcome.kind !== 'pending' && this.resetOwningCompletions.size === 0) { - this.resetDue = true; - } - // The pump's deliveries first, then this eval's own synchronous - // settlements (dispatch-time refusals). - completed = [...completed, ...this.syncSettled.splice(0)]; - const result = this.render(outcome, completion, completed); - // The eval's state-changing boundary (the doc's cadence: after - // each eval). The operation-end flush coalesces it with the - // pump's settlement boundary into one debounced write. - this.sink?.boundary('eval'); - return result; - }); - } - - /** - * The settlement pump: poll every in-flight host task, deliver the - * ready outcomes (record → settle → consume, per the exactly-once - * discipline), drain the guest once, and return the settled call ids - * in settlement order. A delivery failure keeps the outcome staged - * (both the store write and the guest settlement are first-wins - * idempotent), so the next pump retries it — a crash between the - * store write and the guest settlement is healed, not doubled. A drain - * failure (a guest continuation threw) propagates as `DrainJobError` - * with the VM left usable and the already-settled ids still reported. - */ - async pump(): Promise { - return this.serialized(async () => { - const { settled, drainError } = await this.pumpUnlocked(); - // The per-call settlement boundaries fired inside `pumpUnlocked` - // (one per settled call's continuation drain). - if (drainError !== undefined) throw drainError; - return settled; - }); - } - - /** - * The three-way post-restore reconciliation (the roadmap doc's restore - * path, step 3): read the guest registry's pending calls and settle - * every outstanding call EXACTLY one way — - * - * - completed while down → settle from the store (whatever the kind), - * - still resumable at the backend → re-attach via `runner.loadSession` - * (capability-gated per acp-agents; see the module docs' "The - * restore path" section), - * - observably lost founding work may re-issue under the same call id; - * unresolved steering rejects `steering_interrupted` without replay. - * - * Pending checkpoints re-surface into the broker's checkpoint table. - * First-class queue records rebuild independently: unhanded turns remain - * eligible and handed-off turns require authoritative classification. - * Drains once when any guest settlement happened, so - * snapshot-carried continuations fire before this returns — and the - * drain's state change fires the settlement boundary. - */ - async reconcile(): Promise { - return this.serialized(async () => { - this.assertAlive(); - const surface = this.workspace.surface(); - if (surface === undefined) { - throw new Error('Broker: cannot reconcile — the guest surface is not installed'); - } - const report: ReconcileReport = { - settledFromStore: [], - reattached: [], - reissued: [], - failedLost: [], - requeuedCheckpoints: [], - leftPending: [], - reQueuedUndelivered: [], - }; - // §6.2: this reconcile's per-call surfacing lines are retained - // under diagnostics (replaced per reconcile, like the summary). - this.reconcileNotes = []; - let changedVm = false; - for (const entry of surface.pending()) { - const record = this.callStore.lookup(entry.id); - const completion = record?.completion; - if (completion !== null && completion !== undefined) { - const settled = surface.settle(entry.id, completion.outcome, completion.value); - if (settled) { - report.settledFromStore.push(entry.id); - changedVm = true; - } - continue; - } - if (entry.kind === 'checkpoint') { - // A question still awaiting its answer: re-surface it (the - // checkpoint analogue of re-attachment — there is no backend - // task to find). - this.requeueCheckpoint(entry, record); - report.requeuedCheckpoints.push(entry.id); - continue; - } - if (entry.kind === 'queue') { - // First-class queues rebuild after every pending registry entry - // has been inspected. A queue is never interpreted as steering. - continue; - } - if (entry.kind === 'steer') { - if (this.settleSteerInterrupted(entry)) { - changedVm = true; - } - report.failedLost.push(entry.id); - continue; - } - if (entry.kind === 'cancel') { - if (this.refuseReconciled( - entry, - 'cancel', - executionError('cancellation operation was interrupted by restart', 'cancellation_interrupted', true), - `cancel ${entry.id}: interrupted by restart`, - )) changedVm = true; - report.failedLost.push(entry.id); - continue; - } - if (entry.kind === 'agent') { - // Agent call: the re-attach / re-issue arms. Returns whether a - // guest entry was newly settled (a reconcile-time refusal - // mutates the VM and must participate in the changed-VM drain - // and its settlement snapshot — review regression: refusals - // used to settle the guest without the boundary). - if (await this.reconcileAgentCall(entry, record, report)) { - changedVm = true; - } - continue; - } - // An unrecognized kind (a foreign snapshot from a library version - // this host does not speak): refuse loudly — settled + recorded + - // surfaced — never re-issued into the agent machinery. - if ( - this.refuseReconciled( - entry, - 'agent', - new Error(`pending call ${entry.id} has unrecognized kind ${JSON.stringify(entry.kind)} — this host cannot serve it`), - `unrecognized pending call kind ${JSON.stringify(entry.kind)}`, - ) - ) { - changedVm = true; - } - report.failedLost.push(entry.id); - } - report.reQueuedUndelivered = this.rebuildQueuedTurns(); - if (changedVm) { - // The settlement drain fires snapshot-carried continuations. The - // state-changing boundary is the doc's cadence (after each - // settlement drain that changed VM state) and it fires EVEN when - // the drain fails: the settlements landed (the VM changed) and - // the operation-end flush must have a dirty boundary to persist - // them (review regression: an interrupted drain used to skip the - // boundary, so the operation-end flush had nothing to write and - // a kill lost the settlements). The drain itself performs the - // interrupted-drain release when it ran a tracked eval's - // continuation (the interrupted job's continuation lease — see - // `releaseInterruptedEval`). The DrainJobError still propagates - // — the caller reports it like the pump does. - let drainError: DrainJobError | undefined; - try { - this.drain(); - } catch (error) { - if (error instanceof DrainJobError) { - drainError = error; - } else throw error; - } - // The reconciliation's provenance pass: bindings the reconciled - // settlements' continuations created are attributed to the settled - // call ids (a pre-provenance restore's own sweep ran inside - // `Workspace.restore`). - this.provenancePass('settlement', [ - ...report.settledFromStore, - ...report.failedLost, - ]); - this.sink?.boundary('settlement'); - if (drainError !== undefined) { - // §6.2: the reconcile drain failure DEMOTES to - // workspace().diagnostics.drainError — the settlements landed - // and the state-changing boundary above persists them, so - // nothing was lost and the first touch resolves with its - // report instead of failing outside the eval result contract. - // The failure never rides the eval output surface (the [C]14 - // one-line notice is reserved for drains that LOST state — the - // tool layer's client-presence drain rethrow). - this.retainedDrainError = { - name: drainError.info.name, - message: drainError.info.message, - atMs: now(), - }; - } - } - // §6.2: the reconcile summary DEMOTES to workspace().diagnostics - // (retained; it never rides the eval result surface). - this.lastReconcileReport = report; - return report; - }); - } - - /** - * Rebuild the per-session delivery queues from the store: every steer - * record with a `queued` completion and NO delivered/dropped marker is - * undelivered-at-crash — its payload and founding session id are in the - * store, so delivery can be replayed exactly once. A steer whose - * founding call was CANCELLED is not re-queued (its queue was dropped - * when the cancel landed — the dropped marker covers the normal drop - * path; this is the belt-and-braces check for a crash between the - * cancel and the drop record). Returns the re-queued steer call ids. - */ - private rebuildQueuedTurns(): string[] { - const restored: string[] = []; - for (const record of this.callStore.all()) { - if (record.kind !== 'queue' || record.completion !== null) continue; - const sessionId = record.foundingCallId; - const payload = parseTurnPayload(record.optionsJson); - if (sessionId === null || payload === null || this.queuedTurns.has(record.callId)) continue; - const lane = this.lanes.get(sessionId) ?? this.newLane( - this.callStore.lookup(sessionId)?.completion === null ? 'opening' : 'released', - ); - this.lanes.set(sessionId, lane); - if (!lane.queuedTurnIds.includes(record.callId) && lane.activeTurnId !== record.callId) { - lane.queuedTurnIds.push(record.callId); - } - this.queuedTurns.set(record.callId, { - callId: record.callId, - sessionId, - prompt: payload.prompt, - promptMeta: payload.promptMeta, - call: null, - admissionSequence: record.admissionSequence, - state: record.handoffAtMs === null ? 'pending' : 'active', - cancelRequested: false, - }); - restored.push(record.callId); - if (record.handoffAtMs !== null) this.restoreHandedOffQueue(record.callId); - else if (lane.laneState === 'released') this.scheduleQueueReattach(sessionId); - } - this.scheduleAdmissions(); - return restored; - } - - // ── The restore path: re-attach / re-issue arms ────────────────────── - - /** One pending agent call's reconcile: re-attach when a backend - * session is recorded (capability-gated through the runner's own - * `loadSession` — the same gate a custom backend without - * `session/load` degrades through), re-issue when it is lost. See - * the module docs' "The restore path" section. Returns whether a - * guest entry was newly settled (a reconcile-time refusal mutates - * the VM and must participate in the changed-VM drain and its - * settlement boundary). */ - private async reconcileAgentCall( - entry: GuestSurfaceEntry, - record: CallRecord | undefined, - report: ReconcileReport, - ): Promise { - if (this.isTracked(entry.id)) { - // Already live under this broker (a repeated reconcile, or a call - // this pass already re-attached/re-issued): duplicating the task - // would double-poll the session. The registry's first-wins settle - // makes any replay a no-op anyway. - report.reattached.push(entry.id); - return false; - } - if (this.draining || this.disposed) { - // A PARKED restore-time load resumed after the client-presence - // drain force-stopped (or after disposal): the drain already - // settled EVERY outstanding call at its bound — including the - // registry entries this serialized loop had not reached yet (see - // `drainForDisconnect`'s forced stop) — and a disposed broker - // must never open a child. Never initiate a NEW load or re-issue - // from the resumed loop: a fresh child must not open and run - // after the last client disconnected, nor after disposal - // (phase-D review rejection: the generation fence covered only - // the parked load itself, so a load that landed after the - // drain/disposal bump let the reconciliation loop initiate - // SUBSEQUENT loads for the registry entries behind it). The call - // stays pending (leftPending — the state owning it is being torn - // down or drained). - report.leftPending.push(entry.id); - return false; - } - let parsed: ParsedAgentOptions; - try { - parsed = this.parseAgentOptions(entry.optionsJson); - } catch (error) { - // A corrupt options bag (a hostile/foreign registry entry): the - // same refusal a live dispatch would have produced — recorded, - // settled, surfaced. The §4.6 attribution rides it too: the - // call's recorded spec names its resolved backend (the same - // stamp a live admission refusal with a resolved segment gets). - if (entry.modelSpec !== null && entry.modelSpec !== '') { - const segment = backendSegment(entry.modelSpec); - if (this.knownBackends().includes(segment)) { - (error as { replBackend?: string }).replBackend = segment; - } - } - const settled = this.refuseReconciled(entry, 'agent', error, 're-issue refused (invalid options)'); - report.failedLost.push(entry.id); - return settled; - } - const sessionId = record?.sessionId ?? null; - if (sessionId === null) { - // The founding session never opened (or its record predates the - // attachment log): there is nothing at the backend to re-attach. - // The re-issue may itself refuse (the concurrency cap) — that - // refusal settles the guest, so its newly-settled flag propagates - // into the changed-VM bookkeeping (review regression: this branch - // used to drop the flag, skipping the settlement drain and its - // snapshot boundary when the re-issue was refused). - return this.reissueCall(entry, parsed, 'no resumable backend session was recorded', report); - } - // The restore-time re-attach is covered by the OPENING-CALL registry - // (phase-D review rejection: a parked restore-time loadSession used to - // be invisible to the client-presence drain and to dispose). The drain - // now WAITS for the load exactly like an openSession (a parked restore - // load is in-flight work — the child may open and run after the last - // client disconnected) and force-stops it DURABLY at the bound - // (recorded AGENT_CANCELLED, guest-settled, drained, snapshotted — - // never an orphaned pending call), while the GENERATION captured at - // START fences the late landing: a load that resolves after the - // broker was disposed (or after the drain force-stopped) is released - // immediately — never registered (a late landing must not leak the - // session or repopulate liveAgents on a torn-down broker) and never - // re-issued (a fresh child must not open or prompt after disposal). - const generation = this.generation; - this.openingCalls.add(entry.id); - let loaded: BrokerSession | undefined; - try { - // The re-attach routing: the store's RECORDED backend id pins the - // original backend (a backend id doubles as a model routing spec), - // falling back to the recorded model spec verbatim — never the - // current configured default (phase-D review round 2: a changed - // default across a restart used to load on the wrong backend and - // miss the still-resumable original session). - const model = - record?.backendId ?? - (entry.modelSpec ?? undefined); - const session = await this.runner.loadSession({ - sessionId, - model, - schema: parsed.schema as never, - cwd: parsed.cwd ?? this.workspace.projectDir, - configOptions: parsed.configOptions, - mode: parsed.mode, - label: `repl:${entry.id}`, - runId: entry.id, - keepSession: true, - retainSessionLog: true, - }); - loaded = session; - this.openingCalls.delete(entry.id); - // The disposed/drain fence (see above): the drain's force-stop - // marked the call stopped (and settled it durably) when the bound - // expired with the load parked; disposal bumped the generation. - const stoppedByDrain = this.stoppedOpens.delete(entry.id); - if (stoppedByDrain || this.disposed || this.generation !== generation) { - if (!stoppedByDrain) { - // Not already settled by the drain's force-stop: the broker was - // disposed while the load was parked. The child is closed - // immediately, and the call stays pending in the guest - // (leftPending — the state owning it is being torn down anyway; - // it is never settled from a quiet gap and never re-issued). - this.reconcileNote( - 'info', - `call ${entry.id}: restore re-attach of backend session ${sessionId} landed after the broker ` + - `was disposed — the child was closed without registering`, // eslint-disable-line max-len - ); - report.leftPending.push(entry.id); - } - // The teardown-fence release is DETACHED, never awaited - // (phase-D review rejection: the fence used to await - // `session.release()` with no deadline — a custom backend with a - // hung release kept the reconciliation — and with it the daemon's - // first touch — pending indefinitely, reintroducing the - // unbounded-teardown defect). The child's close is best-effort - // here: the drain/disposal already returned at its bound, and a - // parked release must not hold the resumed reconcile. - void Promise.resolve(session.release()).catch(() => undefined); - loaded = undefined; - return false; - } - const awaitTurn = session.awaitCurrentTurn; - if (awaitTurn === undefined) { - // A THIRD-PARTY BrokerSession adapter without the seam (the real - // acp-agents adapter has it): the loaded session's founding-turn - // completion is unobservable to this host. Phase-F review: the - // doc's three reconciliation arms are exhaustive — a call must - // settle exactly once through settle-from-the-store / re-attach / - // re-issue, never through an undocumented fourth arm that parks - // it until interrupt/reset. The seam absence is a capability - // omission, and the doc's rule for a capability-omitting backend - // is "re-issue is the honest fallback, surfaced guest-visibly": - // the catch arm below releases the loaded session (best-effort — - // the re-issue opens its own fresh session) and re-issues the - // call under the same id. The old keep-attached-and-pending arm - // (`registerUnobservableReattach`) is deleted: it left every - // re-attached call on a seam-less backend — including the - // built-in claude and opencode backends, which do not advertise - // `_session/loaded_turn` — permanently pending across a crash. - throw new Error( - 'the loaded session exposes no awaitCurrentTurn seam — its founding-turn completion is ' + - 'unobservable; re-issue is the honest fallback', - ); - } - // The seam (REAL on acp-agents' InteractiveSession): an OBSERVING - // wait — it resolves with the founding turn when the session/load - // replay's update stream settles with a trailing assistant message - // (a turn that ended while the daemon was down has its final - // message in the replay), keeps the session ATTACHED while a - // still-running turn keeps streaming live chunks after the load - // response (settling from its authoritative completion — phase-D - // review: this case used to be rejected, releasing the loaded - // session and re-issuing a call whose turn was still running), and - // rejects only when the outcome is genuinely unobservable (no user - // message, a released/dead session, or a stream settled without a - // terminal assistant message within the max-wait bound). The call - // is ARMED on the seam WITHOUT blocking reconcile: a still-running - // founding turn may take minutes, so reconcile returns immediately - // and the pump delivers the completion when the seam settles — the - // same record → settle → consume path as a live call. A seam - // rejection degrades to re-issue INSIDE the task (releasing the - // loaded session first), surfaced guest-visibly — a re-attached - // call can never hang unobserved. - this.registerReattached(entry, parsed, session); - report.reattached.push(entry.id); - return false; - } catch (error) { - this.openingCalls.delete(entry.id); - // The disposed/drain fence for a load that FAILED while parked - // (mirrors the try arm): a force-stopped call was already settled - // by the drain; on a disposed broker the call stays pending — never - // a re-issue (a fresh child must not open after disposal). - const stoppedByDrain = this.stoppedOpens.delete(entry.id); - if (stoppedByDrain || this.disposed || this.generation !== generation) { - if (loaded !== undefined) { - // The teardown-fence release is DETACHED, never awaited (the - // same boundless-release family as the try arm above — phase-D - // review rejection: a hung release must not keep the resumed - // reconciliation pending forever). - void Promise.resolve(loaded.release()).catch(() => undefined); - loaded = undefined; - } - if (!stoppedByDrain) report.leftPending.push(entry.id); - return false; - } - // The capability gate (a backend without session/load), a - // lost/deleted session, a wire failure, or the seam's rejection - // (founding-turn outcome unobservable): release the loaded session - // when one was obtained (best-effort — the re-issue opens its own - // fresh session) and re-issue is the honest fallback, surfaced - // guest-visibly (a warn line in the next tool result). The re-issue - // may itself refuse (the concurrency cap) — that refusal settles - // the guest, so its newly-settled flag propagates into the - // changed-VM bookkeeping (review regression: the catch used to - // drop it, skipping the settlement boundary). - if (loaded !== undefined) { - await Promise.resolve(loaded.release()).catch(() => undefined); - } - // The disposed/drain fence RE-CHECKED after the awaited release - // (phase-D review rejection — the same late-fence family as - // `reissueReattached`): the release can park past the drain's - // bound or a disposal's generation bump, and the drain's forced - // stop then settles the call durably (the opening-call pass — - // recorded AGENT_CANCELLED, guest-settled, drained, snapshotted) - // and reports `isDrained` while the release is still parked. A - // late re-issue would open a fresh child after the broker - // reported drained; the re-check holds instead — the call stays - // as the drain settled it (the stopped-open marker is consumed - // here when the forced stop landed during the parked release). - const stoppedByDrainLate = this.stoppedOpens.delete(entry.id); - if (stoppedByDrainLate || this.disposed || this.generation !== generation) { - if (!stoppedByDrainLate) report.leftPending.push(entry.id); - return false; - } - return this.reissueCall( - entry, - parsed, - loaded === undefined - ? `backend session ${sessionId} not resumable (${toRejectionValue(error).message})` - : `backend session ${sessionId} loaded, but its founding turn's outcome is not observable (${toRejectionValue(error).message})`, - report, - ); - } - } - - /** Register a successfully re-attached session and ARM the call's - * completion on the loaded session's founding turn — the seam runs as - * an in-flight task (reconcile does NOT block on a still-running - * turn), delivered by the same record → settle → consume pump as a - * live call. The call holds a concurrency token until the pump - * delivers it, exactly like a live call. A seam that can never - * observe the founding turn (absent on a third-party adapter) degrades - * INSIDE the task to a re-issue under the same call id — the honest - * fallback when re-attachment itself is unavailable (phase-F review - * round 2: the seam-less BUILT-INS no longer take this path — the - * seam classifies their loaded turns authoritatively through the - * observation path, and every possibly-running call stays attached; - * this degradation is reserved for the observably-dead classes and - * for third-party adapters whose sessions expose no seam at all). */ - private registerReattached(entry: GuestSurfaceEntry, parsed: ParsedAgentOptions, session: BrokerSession): void { - const sessionEntry: SessionEntry = { - session, - callId: entry.id, - modelSpec: entry.modelSpec ?? '', - task: entry.detail ?? '', - initializeMeta: initializeMetaOf(session), - callSettled: false, - callCancelled: false, - backendId: session.backendId ?? backendSegment(entry.modelSpec ?? ''), - cancelWaiters: new Set(), - }; - const lane = this.lanes.get(entry.id) ?? this.newLane('usable'); - lane.laneState = 'usable'; - lane.activeTurnId = entry.id; - lane.promptInFlight = true; - this.lanes.set(entry.id, lane); - this.sessions.set(entry.id, sessionEntry); - this.watchSessionRelease(sessionEntry); - this.agentSlots.add(entry.id); - this.drained = false; // children are warm again - this.reconcileNote('info', `call ${entry.id}: re-attached to backend session ${session.sessionId}`); - const taskPromise = this.runReattachedTask(entry.id, sessionEntry, parsed); - this.trackInFlight(entry.id, 'agent', taskPromise); - } - - /** The re-attached call's task: observe the loaded session's founding - * turn through the seam (the observing wait — a still-running turn is - * kept attached and settles from its authoritative terminal - * notification), then shape the result (schema ladder or the - * empty-output gate). A seam REJECTION is classified three ways - * (phase-D review round 3, amended phase-F review): - * - * - the still-running class (`LoadedTurnStillRunningError`): the turn - * may still be running and its terminal state is unobservable — - * NEVER settle a quiet gap and NEVER re-issue a possibly-running - * call. The broker KEEPS THE LOADED SESSION ATTACHED and re-arms - * the seam on it for BOTH the re-armable form (a `running` turn - * past its max-wait bound on a backend that carries the extension) - * and the non-re-armable form (a third-party seam that can never - * observe the terminal state) — a later terminal notification — or - * a cancel — still settles the call, and the drain's forced stop - * settles it DURABLY at its bound (phase-F review round 2: the - * non-re-armable form used to release the loaded session and - * re-issue the call, which can duplicate a still-running backend - * turn; re-issue is now reserved for the observably-dead classes - * below). - * - the failed-at-backend class (`LoadedTurnFailedError`): the turn - * RAN and failed — a definite outcome, settled as an ordinary - * rejection (never re-issued, never settled as success). - * - the safe-re-issue class (anything else — no user message in the - * transcript, an `interrupted` answer, a dead process): re-issued - * under the same id through the ordinary dispatch path. While the - * broker is draining/disposing, even these resolve `hold` — a - * fresh child must never open and run after the last client - * disconnected (the drain's forced stop settles every still-pending - * call DURABLY at its bound, so a drained call is never left - * pending; a disposed broker's state is being torn down). - * - * Result-shaping failures AFTER the turn resolved (stop-reason gate, - * empty output, schema ladder) settle as ordinary rejections, exactly - * like a live call — never a re-issue. */ - private runReattachedTask( - callId: string, - entry: SessionEntry, - parsed: ParsedAgentOptions, - ): Promise<{ outcome: 'resolve' | 'reject' | 'hold'; value: unknown }> { - return (async (): Promise<{ outcome: 'resolve' | 'reject' | 'hold'; value: unknown }> => { - let turn: BrokerTurn; - const awaitTurn = entry.session.awaitCurrentTurn; - if (awaitTurn === undefined) { - // Unreachable — `registerReattached` is only called after the seam - // was checked — but a structural guard keeps the optional seam - // honest for third-party adapters: degrade to re-issue exactly - // like every other unobservable completion (phase-F review: a - // call must settle through one of the doc's three arms — never a - // permanent hold). - return this.reissueReattached( - callId, - entry, - parsed, - new Error('the loaded session exposes no awaitCurrentTurn seam — re-issue is the honest fallback'), - ); - } - try { - turn = await awaitTurn.call(entry.session); - } catch (error) { - if (isLoadedTurnStillRunningError(error)) { - // The turn may still be running at the backend and its terminal - // state is unobservable: never settle partial output, and never - // re-issue a possibly-running call. The broker KEEPS THE LOADED - // SESSION ATTACHED — the doc's second reconciliation arm, - // re-attach to a still-running task. The RE-ARMABLE form (a - // `running` turn past the max-wait bound on an extension- - // carrying backend): re-arm the seam on the still-attached - // session — a later ended notification — or a cancel — still - // settles the call. The NON-RE-ARMABLE form (a third-party seam - // that can never observe the terminal state) is NOT re-invoked: - // an immediate recursive re-arm would spin in an unbounded - // microtask/warning loop (each rejection cycles instantly), - // starving cancellation, drain, and every other task — the - // broker instead waits for the terminal state from the - // remaining authority surfaces (see - // `waitForNonRearmableSettlement`). The drain's forced stop - // settles a still-pending call durably at its bound either way - // (phase-F review round 2: the non-re-armable form used to - // release the loaded session and re-issue the call — a - // still-running backend turn would have been duplicated; the - // seam's own observation path now classifies the seam-less - // built-ins authoritatively, and re-issue is reserved for the - // observably-dead classes below). - if (error.rearmable === false) { - this.reconcileNote( - 'warn', - `call ${callId}: ${toRejectionValue(error).message} — the seam can never observe the terminal state; ` + - `the loaded session stays attached and the call settles on a cancel, the backend's ended ` + - `notification, the session's release, or the client-presence drain`, // eslint-disable-line max-len - ); - return this.waitForNonRearmableSettlement(callId, entry, parsed); - } - this.reconcileNote('warn', `call ${callId}: ${toRejectionValue(error).message} — re-armed on the attached session`); - return this.runReattachedTask(callId, entry, parsed); - } - if (isLoadedTurnFailedError(error)) { - // The founding turn RAN and failed at the backend: a definite - // outcome — settle it as an ordinary rejection (record → settle - // → consume), never a re-issue and never a success. - return { outcome: 'reject', value: toRejectionValue(error) }; - } - if (this.draining || this.disposed) { - // The broker's own teardown released the session (or the seam - // rejected mid-drain): re-issuing would open a fresh child after - // the last client disconnected. The call is NOT left pending - // forever: the drain's forced stop settles every still-pending - // call DURABLY at its bound (recorded AGENT_CANCELLED, - // guest-settled), and a disposed broker's state is being torn - // down. Surfaced guest-visibly. - this.reconcileNote( - 'warn', - `call ${callId}: ${toRejectionValue(error).message} — the broker is draining; the call stays ` + - `pending (never re-issued after the last client disconnected)`, // eslint-disable-line max-len - ); - return { outcome: 'hold', value: undefined }; - } - return this.reissueReattached(callId, entry, parsed, error); - } - try { - this.assertNormalStopReason(turn.stopReason, callId); - const value = - parsed.schema !== undefined - ? await this.resolveStructuredOutput(entry, parsed) - : this.finalTextOf(turn.text, entry.callId); - return { outcome: 'resolve', value }; - } catch (error) { - return { outcome: 'reject', value: toRejectionValue(error) }; - } - })().finally(() => { - const lane = this.lanes.get(callId); - if (lane !== undefined && lane.activeTurnId === callId) lane.promptInFlight = false; - }); - } - - /** The NON-RE-ARMABLE still-running settlement wait (see - * `runReattachedTask`'s seam-rejection classification): a third-party - * seam that rejects with `LoadedTurnStillRunningError` and - * `rearmable: false` can NEVER observe the loaded session's founding- - * turn terminal state, so re-invoking it is pointless — the immediate - * recursive re-arm would spin in an unbounded microtask/warning loop - * (each rejection cycles instantly), starving cancellation, drain, - * and every other task. The broker KEEPS THE LOADED SESSION ATTACHED - * (a possibly-running call is never re-issued, never settled from a - * quiet gap) and waits — zero polling, one-shot subscriptions — for - * the terminal state from the remaining authority surfaces: - * - * - the session's own `_session/loaded_turn/ended` notification (the - * `loadedTurnEndedState`/`subscribeLoadedTurnEnded` surfaces — a - * seam-less backend that pushes the notification anyway): a turn - * that ended with an error settles as a rejection (the - * `LoadedTurnFailedError` class — a definite outcome, never - * re-issued); one that ended with a response resolves with the - * accumulated text (the stop-reason gate applies, exactly like a - * live call); - * - a cancel of the call: settled as the recoverable `AGENT_CANCELLED` - * (the interrupt tool works on a held call); - * - the session's release: its dedicated process died, which is lane-fatal; - * - the client-presence drain: the forced stop settles every - * still-pending call DURABLY at the bound (recorded `AGENT_CANCELLED`, - * guest-settled); while draining/disposed, the wait holds — the - * call stays as the drain/disposal left it. - * - * The wait is a bounded task exactly like the re-armable seam's: it - * holds the call's in-flight entry, and a later settlement is - * first-wins against the drain's recorded completion. */ - private async waitForNonRearmableSettlement( - callId: string, - entry: SessionEntry, - parsed: ParsedAgentOptions, - ): Promise<{ outcome: 'resolve' | 'reject' | 'hold'; value: unknown }> { - const session = entry.session; - // The release watch (the session's process death/disposal): created - // once and raced every cycle. Absent on adapters that cannot expose - // it — the wait then settles only on the remaining signals. The - // flag is set exactly once (a released session stays released), so - // the race can never spin on a resolved watch. - let released = false; - const releaseWatch = (session.released?.() ?? new Promise(() => undefined)).then(() => { - released = true; - }); - for (;;) { - if (this.draining || this.disposed) { - // The drain/disposal fences: the forced stop settled (or is - // about to settle) the call durably at the bound — never a - // re-issue after the last client disconnected, and never a - // settlement from a torn-down state. - this.reconcileNote( - 'warn', - `call ${callId}: the held re-attach's settlement wait was cut off by the client-presence drain (or ` + - `the broker was disposed) — the call stays as the drain/disposal left it`, // eslint-disable-line max-len - ); - return { outcome: 'hold', value: undefined }; - } - if (released) { - const error = executionError( - `session ${callId}: ACP session was released while awaiting the loaded turn`, - 'session_released', - false, - ); - this.markLaneFatal(callId, error); - return { outcome: 'hold', value: undefined }; - } - if (entry.callCancelled) { - // A cancel landed on the held call: settle it as the recoverable - // `AGENT_CANCELLED` — the interrupt tool's contract, exactly like - // a live call's cancellation. - const value = toRejectionValue( - new WorkflowError(`call ${callId} was cancelled`, CODE.AGENT_CANCELLED, { - recoverable: true, - }), - ); - return { outcome: 'reject', value }; - } - const ended = session.loadedTurnEndedState?.() ?? null; - if (ended !== null) { - if (ended.error !== undefined) { - // The turn RAN and failed at the backend: a definite outcome — - // settled as an ordinary rejection, never re-issued and never - // settled as success. - return { - outcome: 'reject', - value: toRejectionValue( - new LoadedTurnFailedError( - `the loaded session's founding turn failed at the backend: ${ended.error.message}`, - ), - ), - }; - } - try { - this.assertNormalStopReason(ended.stopReason ?? 'end_turn', callId); - return { outcome: 'resolve', value: this.finalTextOf(this.finalTurnText(session), callId) }; - } catch (error) { - return { outcome: 'reject', value: toRejectionValue(error) }; - } - } - // One-shot signal promises: the ended notification (immediately - // for a notification that already arrived — the state was checked - // above, so only the in-between race can land here), the cancel - // flag, and the release watch. Each wake re-runs the checks; the - // wait never spins. - const endedNotification = new Promise((resolve) => { - const off = session.subscribeLoadedTurnEnded?.(() => { - off?.(); - resolve(); - }); - }); - const cancelSignal = new Promise((resolve) => { - if (entry.callCancelled) { - resolve(); - return; - } - const wake = () => { - entry.cancelWaiters.delete(wake); - resolve(); - }; - entry.cancelWaiters.add(wake); - }); - await Promise.race([endedNotification, cancelSignal, releaseWatch]); - } - } - - /** The safe-re-issue degradation (inside the re-attached task) — the - * observably-dead classes ONLY (phase-F review round 2): a seam - * rejection that proves nothing is running at the backend — the - * interrupted classification (the replayed transcript's trailing - * content is not an assistant message and no live continuation - * followed the load), a transcript that never received its prompt, a - * dead/released session, or a third-party adapter whose session - * exposes no seam at all. A possibly-running call NEVER reaches this - * path: the still-running class keeps the loaded session attached and - * re-arms the seam. Release the loaded session (best-effort — the - * re-issue opens its own fresh session), record the reissue (counter - * bumped), surface the reason guest-visibly, and re-dispatch the SAME - * call id through the ordinary dispatch path. The call's concurrency - * token is reused (it was held for the re-attached wait and never - * left the slot), so the workspace's concurrent-subagent total never - * grows. Steers queued - * against the re-attached session are handed to the fresh session - * (the dispatch path merges `pendingSteers` into its entry's queue). - * - * The drain/disposal fence is checked by the CALLER before this path - * is entered AND re-checked HERE after the awaited release (phase-D - * review rejection: the release can park past the client-presence - * drain's bound — or past a disposal — and the drain's forced stop - * settles the call durably and reports `isDrained` while the release - * is still parked; the old code resumed into a post-drain re-issue - * that recorded a reissue and opened a FRESH child after the broker - * reported drained). The generation is captured at entry so the - * re-check is exact; a fenced landing holds the call (its outcome - * stays as the drain/disposal left it — settled, or pending on a - * torn-down state) and never records a reissue, never opens. */ - private async reissueReattached( - callId: string, - entry: SessionEntry, - parsed: ParsedAgentOptions, - error: unknown, - ): Promise<{ outcome: 'resolve' | 'reject' | 'hold'; value: unknown }> { - const generation = this.generation; - const releasingLane = this.lanes.get(callId); - if (releasingLane !== undefined) releasingLane.laneState = 'released'; - await Promise.resolve(entry.session.release()).catch(() => undefined); - // The fence RE-CHECK after the awaited release (see above): the - // release may have parked past the drain's bound, during which the - // forced stop recorded + settled the call and the drain reported - // drained, or past a disposal's generation bump. Re-issuing now - // would open a fresh child after the last client disconnected (or - // on a torn-down broker) — the call stays as the drain left it. - if (this.lanes.get(callId)?.laneState === 'fatal') { - return { outcome: 'hold', value: undefined }; - } - if (this.draining || this.disposed || this.generation !== generation) { - this.reconcileNote( - 'warn', - `call ${callId}: ${toRejectionValue(error).message} — the loaded session's release outlived the ` + - `client-presence drain (or the broker was disposed); the call stays as the drain/disposal left it, ` + - `never re-issued after the last client disconnected`, // eslint-disable-line max-len - ); - return { outcome: 'hold', value: undefined }; - } - const lane = this.lanes.get(callId); - if (lane !== undefined) lane.laneState = 'opening'; - this.callStore.recordReissued(callId, now()); - this.reconcileNote( - 'warn', - `call ${callId}: re-attached session ${entry.session.sessionId} released (${toRejectionValue(error).message}) — re-issued`, // eslint-disable-line max-len - ); - // The re-issue opens a fresh session: covered by the opening-call - // registry like any dispatch. - this.openingCalls.add(callId); - return this.runAgentTask(callId, entry.modelSpec, entry.task, parsed, this.recordedBackendId(callId)); - } - - /** The recorded backend routing pin of a call, if any (the store's - * `backendId` — recorded at session open). Re-issues and lazy - * re-attaches route by it. */ - private recordedBackendId(callId: string): string | null { - return this.callStore.lookup(callId)?.backendId ?? null; - } - - /** - * Can this founding call id be lazily re-attached? A store record that - * is a SETTLED agent call (its completion is recorded — the handle's - * call is done, the session outlives it by the live-handle contract) - * with a recorded backend session id. A call still opening is handled - * by the queued-while-opening arm, a pending call by reconcile; this - * arm exists for the doc's lazy re-attach of settled handles after a - * drain (or a restore that left settled calls unattached). - */ - private canLazyReattach(sessionId: string): boolean { - if (this.isTracked(sessionId)) return false; - const record = this.callStore.lookup(sessionId); - if (record === undefined || record.kind !== 'agent') return false; - if (record.completion === null || record.sessionId === null) return false; - return true; - } - - /** - * The lazy re-attach (deduped per founding call id): load the store's - * recorded backend session for a SETTLED call through the runner's own - * `loadSession` — capability-gated exactly like the restore path's - * re-attach arm (a custom backend without `session/load` degrades - * through the same gate, surfaced guest-visibly as a warn line) — and - * register it as the call's live session (settled, idle, its pending - * steers merged). Returns undefined when the load failed or the record - * cannot serve one; the caller settles the honest `failed` outcome. - * Concurrent lazy re-attaches of one session share a single load. - */ - private lazyReattach(sessionId: string): Promise { - const existing = this.pendingReattaches.get(sessionId); - if (existing !== undefined) return existing; - // A lazy re-attach warms a child again: the drain latch is stale the - // moment the load starts (phase-D review round 5: the latch used to - // stay set until the load RESOLVED, so a disconnect while the load - // was parked skipped the drain and the re-attached child could run - // after the last client disconnected). - this.drained = false; - const promise = this.doLazyReattach(sessionId).finally(() => { - if (this.pendingReattaches.get(sessionId) === promise) this.pendingReattaches.delete(sessionId); - }); - this.pendingReattaches.set(sessionId, promise); - return promise; - } - - private async doLazyReattach(sessionId: string): Promise { - const record = this.callStore.lookup(sessionId); - if (record === undefined || record.kind !== 'agent' || record.completion === null || record.sessionId === null) { - return undefined; - } - // The drain/disposal generation captured at START: when the load - // lands after the drain's bound expired (or after a dispose/reset), - // the loaded child is released immediately — it must never register - // or prompt (phase-D review round 5: the drain cleared - // `pendingReattaches` but the in-flight load resolved afterward and - // registered a warm child that could run after the last client - // disconnected). - const generation = this.generation; - let parsed: ParsedAgentOptions; - try { - parsed = this.parseAgentOptions(record.optionsJson); - } catch { - // A corrupt options bag: nothing was steered (the failed outcome). - return undefined; - } - try { - // The routing pin: the recorded backend id (a backend id doubles as - // a model routing spec), falling back to the recorded model spec - // verbatim — never the current configured default (phase-D review - // round 2). - const model = - record.backendId ?? - (record.modelSpec ?? undefined); - const session = await this.runner.loadSession({ - sessionId: record.sessionId, - model, - schema: parsed.schema as never, - cwd: parsed.cwd ?? this.workspace.projectDir, - configOptions: parsed.configOptions, - mode: parsed.mode, - label: `repl:${sessionId}`, - runId: sessionId, - keepSession: true, - retainSessionLog: true, - }); - if (this.disposed || this.generation !== generation) { - // The drain's bound expired (or the broker was disposed) while - // the load was in flight: the child is closed immediately — it - // never registers and never prompts (nothing runs after the last - // client disconnected / after disposal). The caller settles the - // honest `failed` outcome. - this.warnLine( - 'info', - `call ${sessionId}: lazy re-attach of backend session ${record.sessionId} landed after the ` + - `client-presence drain (or disposal) — the child was closed without registering`, // eslint-disable-line max-len - ); - // Detached, never awaited: the drain/disposal already returned at - // its bound, and a hung release must not hold the re-attach task - // (the same boundless-release family as the restore fence). - void Promise.resolve(session.release()).catch(() => undefined); - return undefined; - } - const entry: SessionEntry = { - session, - callId: sessionId, - modelSpec: record.modelSpec ?? '', - task: record.detail, - initializeMeta: initializeMetaOf(session), - callSettled: true, - callCancelled: false, - backendId: session.backendId ?? backendSegment(record.modelSpec ?? ''), - cancelWaiters: new Set(), - }; - const lane = this.lanes.get(sessionId) ?? this.newLane('usable'); - lane.laneState = 'usable'; - this.lanes.set(sessionId, lane); - this.sessions.set(sessionId, entry); - this.watchSessionRelease(entry); - this.drained = false; // children are warm again - this.warnLine('info', `call ${sessionId}: lazily re-attached to backend session ${session.sessionId}`); - return entry; - } catch (error) { - this.warnLine( - 'warn', - `call ${sessionId}: lazy re-attach of backend session ${record.sessionId} failed ` + - `(${toRejectionValue(error).message})`, - ); - return undefined; - } - } - - /** Re-issue a lost call under the SAME call id: the store records the - * reissue (counter bumped), a fresh session opens through the - * ordinary dispatch path, and the outcome settles the existing guest - * promise via the reconciliation surface. A store-unknown entry - * (foreign snapshot / wiped store) is adopted first so the replay - * ledger stays complete. §4.1: an over-cap re-issue QUEUES in - * dispatch order for the next free slot — the call stays PENDING in - * the guest registry (never a `ConcurrencyLimitError` rejection; the - * workflow engine's queue-above-cap semantics cover every dispatch - * path). The queued entry reports as re-issued (it is being - * re-issued — not left pending, not lost). Returns whether a guest - * entry was newly settled (always false now — queueing settles - * nothing). */ - private reissueCall( - entry: GuestSurfaceEntry, - parsed: ParsedAgentOptions, - reason: string, - report: ReconcileReport, - ): boolean { - if (this.callStore.lookup(entry.id) === undefined) this.adoptEntry(entry, 'agent'); - this.dispatchQueue.push({ kind: 'reissue', entry, parsed, reason, report }); - report.reissued.push(entry.id); - this.scheduleAdmissions(); - return false; - } - - /** The re-issue dispatch body (shared by the live path and the queued - * path): record the reissue, register the concurrency token, and - * start the agent task under the SAME call id (the original backend - * routing pin). */ - private startReissue( - entry: GuestSurfaceEntry, - parsed: ParsedAgentOptions, - reason: string, - report: ReconcileReport, - ): void { - const lane = this.lanes.get(entry.id) ?? this.newLane('opening'); - lane.laneState = 'opening'; - lane.activeTurnId = entry.id; - this.lanes.set(entry.id, lane); - this.callStore.recordReissued(entry.id, now()); - this.agentSlots.add(entry.id); - this.reconcileNote('warn', `call ${entry.id}: ${reason} — re-issued`); - // The re-issue opens a fresh session: the opening-call registry covers - // it like any dispatch (the drain waits for opens, not just sessions). - this.openingCalls.add(entry.id); - const taskPromise = this.runAgentTask( - entry.id, - entry.modelSpec ?? '', - entry.detail ?? '', - parsed, - this.recordedBackendId(entry.id), - ); - this.trackInFlight(entry.id, 'agent', taskPromise); - void report; - } - - /** A reconcile-time dispatch refusal (invalid registry options, or an - * unrecognized pending-call kind): record dispatched-rejected FIRST - * (a refused call - * is never re-issued again), settle, and surface the reason. Returns - * whether the guest entry was newly settled (a refusal mutates the - * VM and its caller must propagate the change into the changed-VM - * drain and its settlement boundary). (§4.1: over-cap re-issues are - * NOT refused — they queue via `reissueCall`.) */ - private refuseReconciled(entry: GuestSurfaceEntry, kind: 'agent' | 'queue' | 'steer' | 'cancel', error: unknown, warn: string): boolean { - if (this.callStore.lookup(entry.id) === undefined) this.adoptEntry(entry, kind); - const value = toRejectionValue(error); - this.recordCompletion(entry.id, { outcome: 'reject', value, completedAtMs: now() }); - const newlySettled = this.settleIntoGuest(entry.id, 'reject', value); - this.reconcileNote('warn', `call ${entry.id}: ${warn}`); - return newlySettled; - } - - /** A steering request is never replayed after restore: its target - * turn boundary is gone, so the only honest result is the recoverable - * steering_interrupted rejection. */ - private settleSteerInterrupted(entry: GuestSurfaceEntry): boolean { - if (this.callStore.lookup(entry.id) === undefined) this.adoptEntry(entry, 'steer'); - const value = toRejectionValue( - executionError( - `steer ${entry.id}: operation was interrupted by restart and was not replayed`, - 'steering_interrupted', - true, - ), - ); - this.recordCompletion(entry.id, { - outcome: 'reject', - value, - completedAtMs: now(), - }); - const newlySettled = this.settleIntoGuest(entry.id, 'reject', value); - this.reconcileNote( - 'warn', - `steer ${entry.id}: was interrupted by restart and was not replayed`, - ); - return newlySettled; - } - - /** Re-surface a pending checkpoint into the broker's checkpoint table - * (its question + options travel inside the snapshot). The restored - * checkpoint has no live `GuestCall` — answers settle through the - * reconciliation surface (`settleCheckpoint`). */ - private requeueCheckpoint(entry: GuestSurfaceEntry, record: CallRecord | undefined): void { - if (record === undefined) this.adoptEntry(entry, 'checkpoint'); - this.checkpoints.set(entry.id, { - callId: entry.id, - call: null, - question: entry.detail ?? '', - optionsJson: entry.optionsJson, - raisedAtMs: record?.dispatchedAtMs ?? now(), - }); - } - - /** Settle a checkpoint's answer: through its live `GuestCall` when it - * has one, through the reconciliation surface when it re-surfaced - * from a restore (`call` is null). The answering eval's own drain - * fires the continuation either way. */ - private settleCheckpoint(callId: string, call: GuestCall | null, outcome: 'resolve' | 'reject', value: unknown): void { - if (call !== null) { - if (outcome === 'resolve') call.resolve(value); - else call.reject(value); - return; - } - this.settleIntoGuest(callId, outcome, value); - } - - /** Adopt a registry entry the store has never seen (foreign snapshot / - * wiped store): record its dispatch from the entry's verbatim detail - * + optionsJson (+ modelSpec — the re-attach routing source), so the - * replay ledger stays complete (completions, re-issues and attachment - * records can all be written against it). */ - private adoptEntry(entry: GuestSurfaceEntry, kind: 'agent' | 'checkpoint' | 'queue' | 'steer' | 'cancel'): void { - const admittedAtMs = now(); - this.callStore.recordDispatched({ - callId: entry.id, - kind, - detail: entry.detail ?? '', - optionsJson: entry.optionsJson, - modelSpec: kind === 'agent' ? entry.modelSpec : null, - backendId: null, - foundingCallId: kind === 'queue' || kind === 'steer' || kind === 'cancel' ? entry.sessionId : null, - admittedAtMs, - admissionSequence: ++this.admissionSequence, - dispatchedAtMs: admittedAtMs, - reissues: 0, - completion: null, - sessionId: null, - queuedAtMs: null, - handoffAtMs: null, - cancelledAtMs: null, - }); - } - - /** Is this call already tracked by this broker (a live in-flight task, - * a live session, or a live deferred)? The reconcile arms' idempotence - * guard: a tracked call is never re-attached or re-issued twice. */ - private isTracked(callId: string): boolean { - return ( - this.inFlight.has(callId) || - this.sessions.has(callId) || - this.deferreds.has(callId) || - this.queuedTurns.has(callId) || - this.steeringControls.has(callId) - ); - } - - /** A broker-authored console line (live-operation surfacing): - * rendered in the next tool result with its level prefix. */ - private warnLine(level: 'info' | 'warn', message: string): void { - this.consoleBuffer.push({ level, line: message }); - } - - /** A restore/reconcile-machinery line (§6.2): the re-attach / - * re-issue / refusal / lost-steer surfacing — retained under - * workspace().diagnostics.reconcileNotes with the reconcile summary, - * NEVER pushed into the console buffer. Ordinary reconciliation is - * diagnostics-only; only the [C]14 LOSS cases surface in the eval - * output, as the tool's single aggregate notice. */ - private reconcileNote(level: 'info' | 'warn', message: string): void { - this.reconcileNotes.push({ level, line: message, atMs: now() }); - } - - /** - * Cancel one subagent call by its founding call id — the `interrupt` - * tool's engine-side path (the guest handle's `cancel()` funnels - * through the same session cancel; it additionally settles the guest - * steer call). A turn in flight is cancelled (the call then rejects - * with the recoverable `CancelledError` at the next pump); an idle - * session is a no-op. A call whose session is still OPENING (the - * `openSession` is in flight — a delayed backend, a parked open) is - * fenced and settled DURABLY as cancelled right here (see - * `stopOpeningCall`). After the client-presence drain released every - * child, a SETTLED handle's recorded backend session is re-attached - * lazily (the doc: queued turn/steer/cancel re-attach the subagent - * session lazily via the capability matrix) and cancelled if a turn - * is running there; an idle loaded session is the honest no-op. - * Returns the outcome the tool renders: `cancelled` | `idle` | - * `failed` | `none` (no session to act on). - */ - async cancelCall(callId: string): Promise<'cancelled' | 'idle' | 'failed' | 'none'> { - const decision = await this.serialized(async () => { - this.assertAlive(); - const queued = this.queuedTurns.get(callId); - if (queued !== undefined && queued.state !== 'settled') { - return { kind: 'turn' as const, sessionId: queued.sessionId, turnId: callId }; - } - const lane = this.lanes.get(callId); - if (lane === undefined) return { kind: 'none' as const }; - if (lane.laneState === 'opening' && !lane.promptInFlight) { - return { kind: 'founding-opening' as const }; - } - if (lane.activeTurnId === null) return { kind: 'idle' as const }; - return { kind: 'turn' as const, sessionId: callId, turnId: lane.activeTurnId }; - }); - if (decision.kind === 'none') return 'none'; - if (decision.kind === 'idle') return 'idle'; - if (decision.kind === 'founding-opening') { - return this.serialized(async () => - this.cancelFoundingBeforeSession(callId, 'interrupt', true) ? 'cancelled' : 'none', - ); - } - const outcome = await this.requestTurnCancellation(decision.sessionId, decision.turnId); - return outcome.outcome === 'resolve' ? outcome.value as 'cancelled' | 'idle' : 'failed'; - } - - /** Every pending guest call (the registry manifest) — the `status` - * seam. */ - pendingCalls(): GuestSurfaceEntry[] { - this.assertAlive(); - return this.workspace.surface()?.pending() ?? []; - } - - /** - * Arm the eval-break signal — the `interrupt` tool's no-id path (the - * roadmap doc: "break a runaway eval (the quickjs interrupt - * handler)"). Returns `false` ONLY when NO eval is in flight — the - * workspace is idle, there is no continuation to break, and NOTHING - * is armed (phase-E review rejection: the old project-wide boolean - * was armed regardless, so an idle workspace's next eval — or an - * unrelated drain — consumed it before the intended continuation). - * `refused-idle` is honest ONLY then: a running eval is never - * refused. - * - * A running eval the signal cannot target — one suspended on nothing - * resumable (no pending host call AND no pending sleep: a - * never-settling local promise, so no execution can ever queue its - * continuation), one whose resident library predates the - * continuation-lease surface, or a defensive token-less suspension — - * is TERMINATED instead: its tracked completions are RELEASED (the - * eval is no longer running) and the token-keyed fused-eval seam - * records an error settlement so a concurrent wait pumping it - * reports the finished-with-error shape promptly (§3.2: an - * interrupt must terminate/release every running eval — arming dead - * weight was the phase-E round-3 refusal rule, and refusing was the - * review defect: the eval was still running, so it was neither a - * break nor an honest idle refusal). A pending SLEEP keeps an eval - * armable: its host timer's settlement drain resumes the - * continuation exactly like a host call's, so the armed signal - * breaks it mid-run there. - * - * When an eval IS in flight (a suspended eval whose continuation will - * run at a later execution — a settlement drain, or a direct eval's - * own drain when a synchronous host-callback settlement like - * `checkpoint.answer` resumes it), the signal is armed and SCOPED to - * the arming-time active evals: it is consulted ONLY by the - * executions that resume those evals' continuations (settlement - * drains and direct evals' own drains — `runEval` composes it as the - * drain-phase handler — phase-E review rejection round 2), never by - * a fresh eval's own code or an unrelated eval's code, so an - * unrelated eval can neither consume the signal nor be broken by it; - * the first target continuation execution after arming breaks - * mid-run (the quickjs interrupt handler), and the signal is - * consumed on that observation. - * When every arming-time target settles (completed or broken), the - * signal is cleared with them — it never leaks into a later - * execution. - * - * The signal is keyed to the armed target's CONTINUATION — the - * eval's continuation TOKEN (phase-E review rounds 3/5, the carried - * review's defects): the guest library's wrap-settling reaction sets - * the continuation lease to the token immediately before the target - * eval's continuation segment, and the signal fires only while the - * executing JOB holds an armed token — never on whichever drain (or - * whichever JOB) runs next. An unawaited sibling `.then` registered - * before the target's await runs first in the settlement drain — - * before the lease-setting reaction — so it can neither fire nor - * consume the signal; an indirect wait (`await Promise.all([q])`) - * is targetable through the promise graph. - * - * With MULTIPLE concurrently suspended evals (each suspension retains - * its own completion), the first continuation execution after arming - * is broken — honest ambiguity, the same stance as the provenance - * batch labels; the doc's model runs one eval per tool call, where - * the armed signal targets exactly the running eval. - */ - async armEvalBreak(): Promise { - return this.serialized(async () => { - this.assertAlive(); - this.sweepActiveEvals(); - if (this.activeEvalCompletions.size === 0) return false; - // The 0.3.1+ continuation-lease surface is the targeting seam - // (version-gated — a restored 0.3.0 snapshot reports the flag but - // its lease-setting reaction still runs on the awaited VALUE's - // settlement, the carried sibling-reaction defect; phase-E review - // rejection round 7): a workspace whose resident library predates - // it (a restored 0.1.0/0.2.0/0.3.0 snapshot) cannot key the signal - // to an eval's continuation — the 0.2.0 log-only targeting is the - // rejected settled-call-ids identity. Such an eval is RELEASED - // (see the module-level arm doc — never refused). - // - // Resumability (phase-E review round 3, amended for the §4.7 - // sleep helper): a suspended eval's continuation can only ever be - // resumed by the settlement of a pending host call OR a pending - // `sleep` host timer (a promise resolved by guest code alone - // would have settled within the eval's own drain — a genuinely - // SUSPENDED eval awaits one of those, directly or through any - // promise chain). With the registry EMPTY AND no sleep pending no - // execution can ever resume a tracked continuation — arming would - // be dead weight that lingers until reset — so the running eval - // is TERMINATED (released) instead: an interrupt must never - // refuse a running eval. (The converse is deliberately not - // required: the continuation identity is the promise graph, so - // `await Promise.all([q])` is targetable through q even though - // the awaited value is not itself a registry promise — phase-E - // review round 5.) - // - // The armed identity: the targets' continuation TOKENS (see - // `evalTokens`). A tracked eval without a token (a suspension - // the instrumenter never covered — a defensive corner) is not - // targetable: released, never refused. - const tokens = new Set(); - let targetable = this.continuationLeaseAvailable() && - (this.pendingIds().length > 0 || this.sleepCalls.size > 0); - if (targetable) { - for (const completion of this.activeEvalCompletions) { - const token = this.evalTokens.get(completion); - if (token === undefined) { - targetable = false; - break; - } - tokens.add(token); - } - } - if (!targetable) { - this.releaseUntargetableEvals(); - return true; - } - this.evalBreakArmed = true; - this.evalBreakTargets = new Set(this.activeEvalCompletions); - this.evalBreakTokens = tokens; - return true; - }); - } - - /** - * §3.2: TERMINATE every running eval the eval-break signal cannot - * target (see `armEvalBreak` — nothing resumable, a pre-0.3.1 - * resident library, or a defensive token-less suspension). The - * tracked completions are released exactly like a broken - * continuation's (see `releaseInterruptedEval`): each token-keyed - * fused-eval seam records an error settlement so a concurrent wait - * pumping the eval returns the finished-with-error shape promptly - * instead of polling to its bound, a reset() the released eval - * requested still owes its teardown, and the eval-break signal's - * armed state is cleared with the targets it can no longer have. - */ - private releaseUntargetableEvals(): void { - for (const completion of [...this.activeEvalCompletions]) { - const evalToken = this.evalTokens.get(completion); - this.activeEvalCompletions.delete(completion); - this.evalTokens.delete(completion); - this.evalBreakTargets.delete(completion); - if (this.resetOwningCompletions.delete(completion) && this.resetOwningCompletions.size === 0) { - this.resetDue = true; - } - if (evalToken !== undefined) { - // The released eval can never settle (its continuation will - // never run — or, for a pre-0.3.1 library, was never keyed), - // so the seam records the termination the same way a broken - // continuation's release does. - this.sweptEvalSettlements.set(evalToken, { kind: 'error' }); - } - completion.dispose(); - } - this.evalBreakArmed = false; - this.evalBreakTargets = new Set(); - this.evalBreakTokens = new Set(); - } - - /** - * One pass over the retained suspended-eval completions: a completion - * that SETTLED (its continuation completed, or was broken) is - * released. The eval-break signal is scoped to its arming-time - * targets: when every target settled, the signal is cleared with them - * — a signal whose target no longer exists must never leak into a - * later execution (a settled target can no longer be broken, and an - * armed signal would otherwise fire at the next unrelated drain). - * Runs at the start of every serialized operation — the only moments - * between operations, since a drain that settles a continuation can - * only run inside one. Trap-free: a raw promise-state read per - * retained handle. - * - * The §4.4 result-history seam rides the sweep: a settled completion - * IS the previous eval — its FULFILLED value is read trap-free into - * `_` (a rejected completion leaves `_` unchanged, like IPython's). - * A settled completion owed by a reset-requesting eval flips - * `resetDue` — the teardown runs after the operation (the - * serialized-op post-hook), once the eval actually completed. - */ - private sweepActiveEvals(): void { - if (this.activeEvalCompletions.size === 0 && !this.evalBreakArmed) return; - const settled = new Set(); - for (const completion of this.activeEvalCompletions) { - if (completion.promiseState !== 0) settled.add(completion); - } - if (this.evalBreakArmed && this.evalBreakTargets.size > 0) { - let anyLive = false; - for (const target of this.evalBreakTargets) { - // A target still in the active set is still pending; one in - // `settled` was just released (its state already read — skip - // re-reading a disposed handle). - if (!settled.has(target) && target.promiseState === 0) { - anyLive = true; - break; - } - } - if (!anyLive) { - this.evalBreakArmed = false; - this.evalBreakTargets = new Set(); - this.evalBreakTokens = new Set(); - } - } - for (const completion of settled) { - const token = this.evalTokens.get(completion); - this.activeEvalCompletions.delete(completion); - this.evalTokens.delete(completion); - this.evalBreakTargets.delete(completion); - if (this.resetOwningCompletions.delete(completion) && this.resetOwningCompletions.size === 0) { - // The reset-requesting eval completed (or, with several - // outstanding, the LAST one did): the teardown is owed once - // this operation ends. - this.resetDue = true; - } - // The previous eval's completion value becomes `_` (a rejected - // completion reads undefined — `_` stays unchanged, the error - // already rendered through the rejection bridge). - try { - const value = this.workspace.readRetainedCompletion(completion) as JSValueHandle | undefined; - if (value !== undefined) { - // The fused-eval seam (tool phase): record THIS eval's - // completion under its continuation token, with the §4.4 - // repr — the tool's wait reports it as the finished shape's - // `result` when the eval completed within the bound. The - // repr renders BEFORE the handle is consumed by the `_` - // write (which dups it) and disposed. - if (token !== undefined) { - try { - this.sweptEvalSettlements.set(token, { kind: 'value', result: renderCompletionLine(value) }); - } catch { - this.sweptEvalSettlements.set(token, { kind: 'error' }); - } - } - try { - this.workspace.setGlobal('_', value); - } catch { - // A failed `_` write must not fail the operation. - } - value.dispose(); - } else if (token !== undefined) { - // A REJECTED completion (the eval errored late): the error - // already rendered through the rejection bridge into the - // console buffer — the seam records the error outcome so the - // wait reports the finished-with-error shape. - this.sweptEvalSettlements.set(token, { kind: 'error' }); - } - } catch { - // Best-effort bookkeeping: a hostile completion shape must not - // break the operation. - } - completion.dispose(); - } - } - - /** A guest execution that resumes a suspended eval's continuation was - * INTERRUPTED (the eval-break signal's consumption, or the per-eval - * deadline bounding a runaway continuation): the execution broke a - * suspended eval's continuation, and the interrupted continuation's - * engine wrapper NEVER settles (the quickjs interrupt aborts the - * async job without rejecting its promise — verified against the - * shipped binary), so the tracked "running eval" can only be - * released HERE. The callers are the pump path's drain-failure arm - * AND the direct-eval path (an eval's own drain interrupted — - * `runEval` reports `interruptedInDrain`; phase-E review rejection - * round 2). - * - * The release is EXACT (phase-E review rounds 3/5): the interrupted - * job's CONTINUATION LEASE (see `jobLease`) names the eval whose - * continuation was actually executing — the drain loop read the - * guest lease into the mirror before the job, and no further job - * ran after the drain threw. Exactly the tracked eval(s) holding - * that token are released. An interrupted job with NO lease — an - * unrelated drain (a later finite eval's own drain bounded by the - * deadline, a settlement of a call no tracked eval awaits) — - * releases NOTHING and leaves the eval-break armed state intact: - * the target's continuation was never running, so it is still - * breakable at its next execution (the carried review's defect: the - * old release cleared the armed signal while the target's - * checkpoint stayed pending and uninterruptible). The armed signal - * is cleared only when the released eval was an armed target and no - * target remains (the handler already consumed the flag when IT - * fired; a deadline break leaves the arm in place for any surviving - * target). The released handles are disposed here (between - * operations — the drain's exception already unwound). - */ - private releaseInterruptedEval(): void { - const token = this.jobLease.cell.current; - if (token === undefined) return; - let released = false; - for (const completion of [...this.activeEvalCompletions]) { - if (this.evalTokens.get(completion) === token) { - const evalToken = this.evalTokens.get(completion)!; - this.activeEvalCompletions.delete(completion); - this.evalTokens.delete(completion); - this.evalBreakTargets.delete(completion); - // A broken eval is finished (its continuation never settles): - // a reset() it requested owes its teardown all the same (the - // eval will never complete any other way). - if (this.resetOwningCompletions.delete(completion) && this.resetOwningCompletions.size === 0) { - this.resetDue = true; - } - // The fused-eval seam: the broken eval can NEVER settle (the - // quickjs interrupt aborts the async job without rejecting its - // promise), so the tool's wait must not poll it to the bound — - // record the break under its continuation token; the wait - // reports the finished-with-error outcome and the held call - // returns promptly. - this.sweptEvalSettlements.set(evalToken, { kind: 'error' }); - completion.dispose(); - released = true; - } - } - if (released && this.evalBreakTargets.size === 0) { - this.evalBreakArmed = false; - this.evalBreakTokens = new Set(); - } - } - - /** The eval-break signal's interrupt handler: consulted by the - * executions that resume suspended-eval continuations — the - * settlement drains (`drain`) AND a direct eval's own drain - * (`runEval` composes it as the drain-phase handler — a continuation - * resumed by a synchronous host-callback settlement like - * `checkpoint.answer` executes inside the answering eval's drain, - * where the phase-E review rejection round 2's old settlement-drain- - * only signal was blind). A fresh eval's OWN CODE still never - * consults it (an unrelated eval's code can neither consume the - * signal nor be broken by it — the phase-E review rejection). - * Consumed on first observation: the quickjs interrupt polls it - * constantly, so the first target continuation execution after - * arming breaks mid-run. The signal fires ONLY while the currently- - * executing JOB is one of the armed targets' continuation segments — - * the job's lease (see `jobLease`, set by the drain loop before the - * job) holds one of the armed tokens (phase-E review rounds 3/5, - * the carried review's defects): an unrelated drain — and an - * unrelated JOB inside a drain that settled a target's call (an - * unawaited sibling `.then` registered before the target's await - * runs FIRST, before the lease-setting reaction; one registered - * AFTER the target's await runs after the wrapper's settlement but - * still BEFORE the lease-setting job — the lease is set only by the - * reaction registered on the WRAPPER itself, immediately before the - * await machinery's own reaction, so no other job can run with it - * set — phase-E review rejection round 6) — neither fires - * nor consumes it, and the armed state stays intact for the - * target's actual continuation. Returns `undefined` while nothing - * is armed (the composition drops it). */ - private evalBreakHandler(): (() => boolean) | undefined { - if (!this.evalBreakArmed) return undefined; - return () => { - if (!this.evalBreakArmed) return false; - const lease = this.jobLease.cell.current; - if (lease === undefined) return false; - if (this.evalBreakTokens.has(lease)) { -this.evalBreakArmed = false; - return true; - } - return false; - }; - } - - /** Every pending checkpoint, oldest first (raw questions — the tool - * result's `checkpoints` field carries the previewed form). */ - pendingCheckpoints(): CheckpointInfo[] { - return [...this.checkpoints.values()].map((c) => ({ - id: c.callId, - question: c.question, - optionsJson: c.optionsJson, - raisedAtMs: c.raisedAtMs, - })); - } - - /** Every live subagent session — the `status` seam. The guest-derived - * `task` is previewed (head+tail capped at 200 chars, the same bound - * as the manifest's task surface) at the ENGINE seam so EVERY - * consumer — the bounded text renderer AND the structured status — - * is bounded: the tool's structuredContent must respect the doc's - * output limits (phase-E review rejection: the structured status - * used to copy the raw task, so a guest could push an unbounded - * task string through `structuredContent` while only the text - * content was capped). */ - liveAgents(): LiveAgentInfo[] { - const entries: LiveAgentInfo[] = [...this.lanes.entries()].flatMap(([callId, lane]) => { - const entry = this.sessions.get(callId); - const record = this.callStore.lookup(callId); - if ( - record?.kind !== 'agent' || - (lane.laneState === 'fatal' && entry === undefined && lane.queuedTurnIds.length === 0) - ) return []; - return [{ - callId, - // The modelSpec ships VERBATIM (§7: the engine retains 200-char - // metadata formatting ONLY for manifest tokens, checkpoint - // questions and task previews — `agents()` is the §4.5 - // guest-visible plain data, and the full spec must stay - // recoverable; the review defect capped it at 200 chars). The - // task preview keeps its 200-char bound (a retained preview). - modelSpec: entry?.modelSpec ?? record.modelSpec ?? '', - task: (entry?.task ?? record.detail).length > 200 ? headTail(entry?.task ?? record.detail, 200) : (entry?.task ?? record.detail), - state: lane.laneState === 'opening' - ? 'opening' - : lane.activeTurnId !== null - ? 'running' - : 'idle', - supportsSteering: advertisesSteering(entry?.initializeMeta), - queuedTurns: lane.queuedTurnIds.filter((id) => this.queuedTurns.get(id)?.state === 'pending').length, - } satisfies LiveAgentInfo]; - }); - for (const [callId, turn] of this.queuedTurns) { - if (turn.state === 'settled') continue; - const entry = this.sessions.get(turn.sessionId); - const founding = this.callStore.lookup(turn.sessionId); - entries.push({ - callId, - modelSpec: entry?.modelSpec ?? founding?.modelSpec ?? '', - task: turn.prompt.length > 200 ? headTail(turn.prompt, 200) : turn.prompt, - state: turn.state === 'active' || turn.state === 'cancelling' ? 'running' : 'queued', - supportsSteering: advertisesSteering(entry?.initializeMeta), - queuedTurns: 0, - }); - } - return entries; - } - - /** The `workspace()` guest handler's JSON (§4.5): the workspace - * manifest as plain data — bindings with the honest handle status - * (`failed` for rejected handle calls — v1 showed rejected and - * fulfilled both as `settled`), the in-flight ids, the raised - * checkpoints, and the §6.2 diagnostics (reconcile summary, retained - * drain error, children-closed). */ - private workspaceJson(): string { - const manifest = this.workspaceManifest(); - const bindings = manifest.bindings.map((binding) => ({ - name: binding.name, - type: binding.type, - sizeBytes: binding.sizeBytes, - provenance: binding.provenance, - task: binding.task, - ...(binding.handleCallId !== null ? { callId: binding.handleCallId } : {}), - ...(binding.handleStatus !== null - ? { - status: - binding.handleStatus === 'settled' - ? this.isFailedCall(binding.handleCallId) - ? 'failed' - : 'settled' - : binding.handleStatus, - } - : {}), - })); - return JSON.stringify({ - bindings, - inFlight: this.inFlightIds(), - checkpoints: [...this.checkpoints.values()].map((c) => ({ - id: c.callId, - question: headTailDescription(c.question, 200), - })), - diagnostics: { - reconcile: this.lastReconcileReport, - reconcileNotes: this.reconcileNotes, - drainError: this.retainedDrainError, - childrenClosed: this.drained, - }, - }); - } - - /** The `agents()` guest handler's JSON (§4.5): the live-agent entries - * as plain data (v1's liveAgents entries — including the addressable - * queued turn turns). */ - private agentsJson(): string { - return JSON.stringify(this.liveAgents()); - } - - /** Did the store record this handle call as REJECTED (the honest - * `failed` handle status — §4.5)? */ - private isFailedCall(callId: string | null): boolean { - if (callId === null) return false; - const completion = this.callStore.lookup(callId)?.completion; - return completion !== null && completion !== undefined && completion.outcome === 'reject'; - } - - /** The call store (read access for diagnostics and the crash-window - * tests). */ - store(): CallStore { - return this.callStore; - } - - /** True once the client-presence drain released every child (see - * `drainForDisconnect`): the workspace stays live, and later - * queued turn/steer/cancel on a settled handle lazily re-attaches the - * recorded backend session. */ - get isDrained(): boolean { - return this.drained; - } - - /** True once the broker was disposed — the reset() guest function's - * teardown (which runs in the operation's post-hook, AFTER the eval - * result was rendered) or the daemon's shutdown/reset path. The - * daemon reads this after every tool operation: a reset eval leaves - * its own broker disposed, so the project state must clear its live - * workspace/broker references for the NEXT touch to create a fresh - * workspace. */ - get isDisposed(): boolean { - return this.disposed; - } - - /** The number of sessions with a turn running (the drain's progress - * probe). */ - busySessionCount(): number { - let count = 0; - for (const lane of this.lanes.values()) { - if (lane.activeTurnId !== null || lane.promptInFlight) count++; - } - return count; - } - - /** - * The fused-eval pump (the eval-plane redesign's §3.1): pump until the - * target call ids settle (or `timeoutMs` elapses — "still running" on - * timeout), then return the SAME tool-result shape as an eval — output - * lines included (the pumps drain console events and restored-call warn - * lines into the buffer, so an eval whose continuation completed in a - * previous drain reports its output here — phase-D review round 2). - * `ids` omitted waits for the calls pending at ENTRY (with other - * operations interleaving between pumps, the entry-time set is the - * only stable "every pending call" reading — a call a concurrent eval - * dispatches after entry is not waited on; see the phase-E review - * rejection round 2 note below). **The eval-token settlement is the - * authoritative "the code's own work settled" signal and SHORT-CIRCUITS - * the target set**: the moment THIS eval's continuation completes - * during a pump, the wait returns the finished shape — an unrelated - * long-running call elsewhere in the workspace (a start-and-don't-await - * from an earlier eval) can never hold the finished shape to the bound - * (§3.1; review finding).** `evalToken` (the suspended eval's - * continuation token from its `ReplEvalResult`) attributes the result: - * when THAT eval's continuation completes during the pumps, the result - * carries its completion (`kind` `value`/`error`, the §4.4 repr in - * `result` when it resolved); the token-keyed attribution means a - * concurrent client's eval can never steal the seam. Returns the - * rendered result plus whether the target set drained within the bound. - */ - async waitForCalls( - ids: string[] | undefined, - timeoutMs: number, - evalToken?: string, - ): Promise<{ result: ReplEvalResult; drained: boolean }> { - const deadline = Date.now() + Math.max(0, timeoutMs); - // The target set is captured at ENTRY (phase-E review rejection - // round 2: the wait used to run its whole bounded poll inside ONE - // serialized op, so a concurrent interrupt — `cancelCall` / - // `armEvalBreak` — queued behind it and could not cancel or break - // until the wait finished or timed out, up to 120 s, by which point - // the target could already have completed). The wait now runs each - // PUMP as its own serialized unit and RELEASES the chain between - // pumps, so other operations — interrupts, other waits, the - // client-presence drain — interleave mid-wait: an interrupt landing - // mid-wait arms the eval-break signal against the eval the wait is - // pumping, and the wait's very next pump breaks it mid-run. With - // other operations interleaving, "wait for every pending call" can - // only mean "the calls pending when the wait started" — a call a - // concurrent eval dispatches after entry is not waited on (the wait - // stays bounded and deterministic). - // - // The CHAIN ACQUISITION itself is bounded by the wait's absolute - // deadline (phase-E review round 4's carried defect: the entry - // capture, every pump, and the final re-check used to enqueue onto - // the serialization chain with NO deadline, so a wait queued behind - // a long eval — 20 ms behind a 250 ms eval — took the eval's whole - // remaining run (~253 ms) instead of returning at its bound; the - // absolute deadline must bound chain contention as well as polling - // sleep and guest drains). A pump that cannot acquire the chain - // within the remaining budget reports "still running" — it - // observed nothing settle — and never touches the VM while another - // operation is mid-flight. - let targets: Set; - // The last pending read taken UNDER the chain (rendered in the - // result). Initialized to the explicit ids when given — the wait's - // own target set, none of which was observed to settle before any - // read — and empty until the entry capture for the ids-omitted - // form (the pending surface is unreadable while another operation - // holds the chain). - let lastPending: string[] = ids ?? []; - if (ids !== undefined) { - // Explicit ids need no chain read — the target set is the input. - targets = new Set(ids); - } else { - const captured = await this.trySerialized(async () => { - this.assertAlive(); - const pending = this.pendingIds(); - lastPending = pending; - return new Set(pending); - }, deadline); - if (!captured.acquired) { - // The chain was busy for the whole budget: no pending read was - // possible, nothing was observed to settle — the honest - // "still running" with an empty (unreadable) pending surface. - return { result: this.renderWaitResult([], [], evalToken), drained: false }; - } - targets = captured.value!; - } - const completed: string[] = []; - let drained = false; - for (;;) { - const pumped = await this.trySerialized(async () => { - this.assertAlive(); - // Each pump runs under the REMAINING wait time: a settlement - // drain that resumes a runaway continuation near the deadline is - // interrupted at the wait's bound, never at the eval deadline — - // the wait's bound is absolute (the same posture as the - // disconnect drain). - const { settled } = await this.pumpUnlocked(deadline); - if (settled.length > 0) { - // The per-call settlement boundaries fired inside - // `pumpUnlocked` (one per settled call's continuation drain). - completed.push(...settled); - } - // A pump drain failure (the armed eval-break signal's target, or - // the wait-bound interrupting a continuation) is RETAINED under - // workspace().diagnostics (§6.2 — it leaves the eval result - // surface); the settled ids are still reported. - const pending = this.pendingIds(); - lastPending = pending; - const drainedNow = [...targets].every((id) => !pending.includes(id)); - // §3.1: THIS eval's own settlement is the authoritative - // "everything the code waits on settled" signal — the - // token-keyed seam is set when its continuation completes. - // When it settles, the wait reports the finished shape - // IMMEDIATELY: an unrelated long-running call elsewhere in the - // workspace (a start-and-don't-await from an earlier eval) - // must never hold the finished shape to the bound (review - // finding: the pump captured the WHOLE pending registry and - // waited for all of it). - return drainedNow || (evalToken !== undefined && this.sweptEvalSettlements.has(evalToken)); - }, deadline); - if (!pumped.acquired) { - // The chain was held past the deadline (a long eval): the wait - // reports "still running" at its bound instead of queueing - // behind the stuck operation (the round-4 carried defect). - drained = false; - break; - } - drained = pumped.value!; - if (drained) break; - // Sleep only for the REMAINING wait budget — never a fixed 50 ms - // past the deadline (phase-E review round 3's carried defect: the - // unconditional sleep made every bounded wait take ~51 ms, so a - // 5/10/20/30 ms timeout all reported ~51 ms, violating - // `timeoutMs`'s bounded-wait contract). The next pump still runs - // when the deadline is already passed (the deadline check above - // handles the terminal state; the pump itself is bounded by the - // remaining budget through `pumpUnlocked(deadline)`). - const remaining = deadline - Date.now(); - if (remaining <= 0) break; - await new Promise((resolve) => setTimeout(resolve, Math.min(50, remaining))); - } - if (!drained) { - // The deadline tripped between the last pump and the timeout - // check — an interleaved operation (an interrupt's cancel, or - // another pump) may have settled the targets in that window: - // re-check once so the result is not a stale "still running" - // (the re-check's acquisition is bounded by the REMAINING budget - // — it may itself report the chain still busy, which is the - // honest not-drained). - const recheck = await this.trySerialized(async () => { - this.assertAlive(); - const pending = this.pendingIds(); - lastPending = pending; - // The token short-circuit applies to the re-check too: the - // deadline may have tripped in the same window this eval's - // continuation settled — the finished shape wins over a stale - // "still running". - return ( - [...targets].every((id) => !pending.includes(id)) || - (evalToken !== undefined && this.sweptEvalSettlements.has(evalToken)) - ); - }, deadline); - if (recheck.acquired) drained = recheck.value!; - } - const result = this.renderWaitResult(completed, lastPending, evalToken); - return { result, drained }; - } - - /** Render the wait result in the eval-result shape (output lines from - * the console buffer — including a suspended eval's completion output - * and late-error rendering, pending ids, checkpoints, completed ids). - * `pending` is the last pending read taken UNDER the chain (the wait's - * pumps capture it; a wait that could never acquire the chain passes - * the empty unreadable surface) — the renderer never re-enters the VM - * outside the chain. `evalToken` keys the fused-eval seam: when the - * caller's suspended eval completed during the pumps, `kind` reports - * its outcome and `result` carries the completion's §4.4 repr (the - * entry is consumed on read). - */ - /** - * The §3.1 empty-eval poll seam: claim the OLDEST swept settlement - * whose owning eval's held call has ENDED — its continuation token - * is in `timedOutTokens`, the tool layer's record of the evals IT - * returned as still-running. Settlements of evals whose held calls - * are still pumping are NEVER claimable here: the token-keyed wait - * read (`renderWaitResult`) owns them, so a concurrent client's - * in-flight wait can never lose its eval's attribution. Consumed on - * claim — one settlement, one poll — so repeated polls drain the - * settled timed-out evals in settlement order (the §3.1 drain: - * "any later eval reports what settled in the meantime"). - */ - claimSweptEvalSettlement( - timedOutTokens: ReadonlySet, - ): { token: string; kind: 'value' | 'error'; result?: string } | undefined { - for (const [token, settlement] of this.sweptEvalSettlements) { - if (timedOutTokens.has(token)) { - this.sweptEvalSettlements.delete(token); - return { token, ...settlement }; - } - } - return undefined; - } - - /** - * §6.2: retain a client-presence drain failure observed OUTSIDE the - * broker under `workspace().diagnostics.drainError` — the demoted - * diagnostics home. The tool layer calls this when its own drain - * (`drainForDisconnect` + the snapshot flush) rethrew a failure the - * broker's internal drain paths did not classify (a store write - * error, for example): the loss notice leads the next eval's output - * (never silent), and this record keeps the failure visible where - * §4.5 says drain errors live. - */ - retainDrainError(name: string, message: string): void { - this.retainedDrainError = { name, message, atMs: now() }; - } - - private renderWaitResult(completed: string[], pending: string[], evalToken?: string): ReplEvalResult { - const lines: string[] = []; - for (const event of this.consoleBuffer.splice(0)) { - lines.push(this.renderConsoleEvent(event)); - } - // §7: no output caps on guest output. - const swept = evalToken !== undefined ? this.sweptEvalSettlements.get(evalToken) : undefined; - if (evalToken !== undefined) this.sweptEvalSettlements.delete(evalToken); - return { - output: lines, - kind: swept?.kind ?? 'pending', - ...(swept?.result !== undefined ? { result: swept.result } : {}), - pending, - checkpoints: this.checkpointSummaries(), - completed, - }; - } - - /** - * The doc's client-presence drain: on last-client disconnect the daemon - * calls this — in-flight subagent turns DRAIN TO COMPLETION (their - * results settle into the VM and each settlement boundary snapshots, so - * "close the laptop while two researchers run" ends with the findings - * durable in the workspace — never a cancel of running work), bounded - * by `boundMs` (the daemon reuses its session-eviction TTL — the - * spec-owed concrete drain bound; individual turns already run under - * the runner's runaway protections, so the bound is the outer ceiling), - * and then every idle child CLOSES (sessions released — `keepSession` - * keeps the backend sessions re-openable). The workspace and broker - * stay alive; on the next client connect `queued turn` re-attaches the - * subagent session lazily via the capability matrix (see - * `canLazyReattach`/`lazyReattach`). Queued-but-undelivered steers are - * re-queued durably against their founding session ids (the same - * rebuild reconcile uses — their payloads live in the store), so the - * next lazy re-attach delivers them exactly once. Returns `true` when - * every turn drained within the bound, `false` when the bound forced - * the remainder to cancel (the honest bounded teardown — the cancel - * settles the calls as the recoverable `AGENT_CANCELLED`, recorded and - * snapshotted like any settlement) or when `shouldAbort` reported a - * reconnecting client (nothing was cancelled or released — the next - * disconnect drains again). - * - * `shouldAbort` is the daemon's mid-drain presence probe (phase-D - * review round 6): the drain consults it every iteration and before - * every destructive phase, and aborts — children stay warm, the drain - * latch stays clear — the moment a client is connected again. - * - * **The drain covers OPENING calls too** (phase-D review round 3: a - * call blocked in `openSession` has no session entry yet, so a drain - * that considered only registered busy sessions returned `true` - * immediately, cleared its bookkeeping, and let the child open and run - * after the last client disconnected). The drain waits for opening - * calls and in-flight lazy re-attaches exactly like busy sessions; - * when the bound expires with an open still parked, the call is - * STOPPED — the child that eventually opens is closed immediately - * (never prompts), the call settles as the recoverable `AGENT_CANCELLED`, - * and queued steers are dropped durably. - * - * **The outer bound is absolute** (phase-D review round 3: the - * cancel/release phases used to await `cancelSession`/`release` with no - * remaining-time bound, so a hung backend could block disconnect/ - * shutdown indefinitely past the eviction TTL): every post-deadline - * await races the remaining bound, and once it expires the drain - * returns without waiting — the best-effort cancellations and releases - * already issued keep running in the background (all promises carry - * catch handlers, so nothing can become an unhandled rejection). The - * GUEST DRAINS the drain's pumps trigger are bounded by the same - * remaining bound (phase-D review round 6: a ready settlement resumes - * the guest continuation through `drainJobs`, which used to run under - * the per-eval deadline alone — a runaway continuation near the - * disconnect deadline could exceed the session-eviction TTL). - * - * **A client reconnecting mid-drain aborts the drain** (phase-D review - * round 6: the drain used to run to its release phase and close every - * child regardless of presence — the doc requires children to remain - * warm while any client is connected): the daemon passes - * `shouldAbort` (its presence check — `clients.size > 0`), which the - * drain consults every iteration and before every destructive phase. - * An abort leaves every child attached and running, clears the - * drain latch (`isDrained` stays false), and returns `false` — the - * NEXT disconnect drains again. - * - * **The bound's forced stop never orphans a pending call** (phase-D - * review round 6: a re-attached call whose seam rejected mid-drain - * used to resolve `hold`, then the release phase discarded its - * session — the call stayed pending forever, uncancelable except by - * reset, because reconcile never runs again on a live workspace): - * after the bound expires every call still pending on an attached - * session is settled with the recoverable `AGENT_CANCELLED` (recorded - * FIRST, settled into the guest, one bounded drain + settlement - * boundary) — the same forced-stop vocabulary as a stopped open. A - * still-observing task's later outcome is a first-wins no-op against - * the recorded completion. - * - * **The bound's forced stop settles a still-OPENING call DURABLY at - * the bound, not when its openSession eventually lands** (phase-D - * review round 7: a bound-expired openSession used to be only flagged - * in `stoppedOpens` — its call was not recorded, guest-settled, - * drained or snapshotted until `openSession` resolved, so a parked - * open that NEVER resolves left the broker reporting drained with - * the call pending and uncancelable). The opening call is settled - * with the recoverable `AGENT_CANCELLED` at the bound — recorded - * FIRST, settled into the guest, one bounded drain + settlement - * boundary — while the `stoppedOpens` fence is RETAINED: an eventual - * landing still closes the child immediately without prompting, and - * the late task's reject is a first-wins no-op against the recorded - * completion. The same pass settles in-flight STEER wire calls the - * bound cut off (a lazy re-attach whose load never lands, an - * injection/delivery the release phase is about to cut) with the - * honest `failed` — exactly what their fenced late landing would - * have settled — so the drain never reports drained with a pending - * call of any kind. - * - * **The forced stop also settles every restored call the serialized - * reconcile had NOT yet reached** (phase-D review rejection: the - * reconciliation registers calls in `openingCalls` only as its loop - * reaches them — parked on the FIRST pending call's never-resolving - * `loadSession`, it never processes the entries behind it, so a - * forced stop that covered only the tracked calls left those entries - * pending and uncancelable while `isDrained` reported true, and a - * load that later landed let the resumed loop initiate SUBSEQUENT - * loads after the drain/disposal generation bump — children opening - * and running after the last client disconnected). Every pending - * registry entry that is not tracked is settled at the bound - * (completed-while-down entries from the store — the store arm's - * semantics; agent entries with the recoverable `AGENT_CANCELLED`; - * steers with the honest `failed`), and `reconcileAgentCall` refuses - * to initiate any load or re-issue while the broker is - * draining/disposed — the resumed loop settles the recorded - * completions from the store, first-wins, and opens nothing. - * - * **The outer bound is measured from METHOD ENTRY, before the - * serialized-chain wait** (phase-D review round 7: the clock used to - * start inside the serialized closure, so a drain queued behind a - * long operation ran its whole window AFTER the queue wait — the - * total could exceed the session-eviction TTL by the queue wait — - * and the loop's yield was a fixed 50 ms sleep that could land past - * the deadline). A deadline already past at chain acquisition skips - * straight to the forced stop; the loop's yield races the remaining - * bound. Everything below races the remaining bound — INCLUDING the - * chain acquisition itself (phase-D review round 8: the chain wait - * used to have no deadline race, so a YIELDFUL queued operation — a - * long `wait` op polling a pending call, an async op on a stuck - * backend — could delay the drain indefinitely past its bound; when - * the deadline expires while queued, the drain body runs WITHOUT the - * chain, and its forced-stop settlement is first-wins and - * generation-fenced against the stuck op's eventual landing, so an - * unlocked forced stop settles exactly like a chained one). A hung - * cancel/release can never block disconnect past the eviction TTL. - */ - async drainForDisconnect(boundMs: number, shouldAbort?: () => boolean): Promise { - // The ABSOLUTE bound: measured at METHOD ENTRY — before the - // serialized-chain wait — so the queue wait counts against it and - // the drain can never run a fresh full window after the chain - // finally freed (review round 7). The chain wait itself races the - // remaining bound (review round 8 — see `serialized`). - const deadline = Date.now() + Math.max(0, boundMs); - return this.serialized(async () => { - this.assertAlive(); - if (this.drained) return true; - if (shouldAbort?.()) return false; - this.draining = true; - // Drain phase: pump until no session has a turn running, no session - // is still opening, and no lazy re-attach is in flight (or the - // bound expires). Each pump settles into the VM and fires the - // settlement boundary — the daemon's snapshot sink persists each - // drain boundary, so a kill mid-drain loses nothing. An empty pump - // yields briefly (the turns complete asynchronously at the - // backends; a busy spin would starve the event loop). - for (;;) { - if (shouldAbort?.()) { - // A client reconnected mid-drain: the children stay warm — - // nothing is cancelled, nothing is released, and the drain - // latch stays clear so the next disconnect drains again. - this.draining = false; - return false; - } - const outstandingOpens = this.openingCalls.size + this.pendingReattaches.size; - if (this.busySessionCount() === 0 && outstandingOpens === 0) break; - if (Date.now() >= deadline) break; - const { drainError } = await this.pumpUnlocked(deadline); - // The pump's per-call settlement boundaries already fired inside - // `pumpUnlocked` (one per settled call's continuation drain — - // the daemon's snapshot sink persists each drain boundary, so a - // kill mid-drain loses nothing). A continuation the bound - // interrupted is RETAINED under workspace().diagnostics (§6.2 — - // the drain error leaves the eval result surface). - if (drainError !== undefined) { - this.retainedDrainError = { name: drainError.info.name, message: drainError.info.message, atMs: now() }; - } - if (this.busySessionCount() > 0 || this.openingCalls.size > 0 || this.pendingReattaches.size > 0) { - // The yield is bounded by the REMAINING bound — never a fixed - // sleep past the deadline (review round 7: a deadline that - // expired during the pump used to add a full 50 ms overshoot). - const remaining = deadline - Date.now(); - if (remaining > 0) await sleep(Math.min(50, remaining)); - } - } - const drainedWithinBound = - this.busySessionCount() === 0 && this.openingCalls.size === 0 && this.pendingReattaches.size === 0; - if (!drainedWithinBound) { - if (shouldAbort?.()) { - this.draining = false; - return false; - } - // The bound is the outer ceiling: stop what it caught - // (best-effort; the cancellations settle the calls as the - // recoverable AGENT_CANCELLED, and a parked open's eventual - // landing is stopped the same way), then one final pump settles - // the cancellations into the VM (each settlement boundary - // snapshots). Every await below races the remaining bound — a - // hung cancel/release can never block past the eviction TTL. - // The GENERATION bump fences every in-flight open and lazy - // re-attach that started before this moment: when it lands, the - // child is released immediately (never registers, never prompts - // — phase-D review round 5: the drain used to clear - // `pendingReattaches` without fencing the unresolved load, so a - // late landing ran a child after the disconnect). - this.generation++; - for (const callId of [...this.openingCalls]) this.stoppedOpens.add(callId); - const cancels: Promise[] = []; - for (const [sessionId, entry] of this.sessions) { - const lane = this.lanes.get(sessionId); - if (lane?.activeTurnId !== null && lane?.activeTurnId !== undefined) { - cancels.push(this.requestTurnCancellation(sessionId, lane.activeTurnId)); - } else if (lane?.promptInFlight) { - cancels.push(Promise.resolve(entry.session.cancel()).catch(() => undefined)); - } - } - await boundedAll(cancels, deadline); - // One final pump settles the cancellations into the VM; its - // per-call settlement boundaries already fired inside - // `pumpUnlocked` (one per settled call's continuation drain) and - // its guest drain is bounded by the remaining bound (review - // round 6 — the outer bound is absolute). - const final = await this.pumpUnlocked(deadline); - if (final.drainError !== undefined) { - this.retainedDrainError = { name: final.drainError.info.name, message: final.drainError.info.message, atMs: now() }; - } - // The bound's forced stop settles EVERY call still pending at - // the bound — recorded FIRST, settled into the guest, one - // bounded drain + settlement boundary — so the drain never - // reports drained with a pending, uncancelable call (review - // round 7): (a) calls still OPENING (a parked openSession whose - // child never landed — the settlement is DURABLE at the bound, - // not deferred until the open resolves; the `stoppedOpens` - // fence stays so an eventual landing still closes the child - // immediately without prompting, and the late task's reject is - // a first-wins no-op against the recorded completion), (b) calls - // still pending on an attached session (a held re-attach (the - // seam rejected and the pump dropped its in-flight entry) or a - // seam the release phase is about to cut off — a call left - // pending here would be ORPHANED: the release phase discards its - // session, no task tracks it, and (the workspace stays live — - // reconcile never runs again) it would be uncancelable except - // by reset (phase-D review round 6); a still-observing task's - // later outcome is a first-wins no-op against the recorded - // completion), and (c) in-flight STEER wire calls the bound cut - // off (a lazy re-attach whose load never lands, an - // injection/delivery turn the release phase is about to cut) — - // settled with the honest `failed`, exactly what their fenced - // late landing would have settled. - const stopped: Array<[string, SessionEntry]> = []; - for (const [callId, entry] of this.sessions) { - if (!entry.callSettled && !this.openingCalls.has(callId)) stopped.push([callId, entry]); - } - const stoppedSteers = [...this.inFlight.values()].filter((task) => task.kind === 'steer' && !task.done); - if (stopped.length > 0 || this.openingCalls.size > 0 || stoppedSteers.length > 0) { - const settledIds: string[] = []; - for (const callId of [...this.openingCalls]) { - const value = toRejectionValue( - new WorkflowError( - `call ${callId} was cancelled by the client-presence drain: its bound expired while the session ` + - `was still opening — the call is settled, and a late child, if any, is closed without prompting`, - CODE.AGENT_CANCELLED, - { recoverable: true }, - ), - ); - this.recordCompletion(callId, { outcome: 'reject', value, completedAtMs: now() }); - if (this.settleIntoGuest(callId, 'reject', value)) settledIds.push(callId); - // The settled opening call's concurrency token is released - // at the bound (its parked task is never pumped). - this.agentSlots.delete(callId); - this.warnLine('warn', `call ${callId}: ${value.message}`); - } - for (const [callId, entry] of stopped) { - const value = toRejectionValue( - new WorkflowError( - `call ${callId} was cancelled by the client-presence drain: its bound expired before the call's ` + - `outcome became observable — the session is closing`, - CODE.AGENT_CANCELLED, - { recoverable: true }, - ), - ); - this.recordCompletion(callId, { outcome: 'reject', value, completedAtMs: now() }); - if (this.settleIntoGuest(callId, 'reject', value)) settledIds.push(callId); - entry.callSettled = true; - this.warnLine('warn', `call ${callId}: ${value.message}`); - } - for (const task of stoppedSteers) { - const value = toRejectionValue( - executionError( - `steer ${task.callId}: cut off by the client-presence drain`, - 'steering_interrupted', - true, - ), - ); - if (this.recordCompletion(task.callId, { outcome: 'reject', value, completedAtMs: now() })) { - if (this.settleIntoGuest(task.callId, 'reject', value)) settledIds.push(task.callId); - } else { - const record = this.callStore.lookup(task.callId); - const completion = record?.completion; - if (completion === null || completion === undefined) { - throw new Error(`Broker: store lost the recorded completion for ${task.callId}`); - } - if (this.settleIntoGuest(task.callId, completion.outcome, completion.value)) settledIds.push(task.callId); - } - this.warnLine( - 'warn', - `steer ${task.callId}: cut off by the client-presence drain — nothing was delivered`, - ); - } - // The bound's forced stop also settles EVERY restored call the - // serialized reconcile had NOT yet reached. The passes above - // own the tracked calls (openingCalls, registered sessions, - // in-flight tasks), but a registry entry behind a parked - // loadSession — the serialized reconcile parks on the FIRST - // pending call's never-resolving load and never processes the - // entries after it — is in NONE of them: without this pass it - // would stay pending and uncancelable while `isDrained` - // reports true, and a load that later landed would let the - // resumed loop initiate SUBSEQUENT loads after the generation - // bump (a fresh child opening and running after the last - // client disconnected — phase-D review rejection). A - // completed-while-down entry settles from the store (the - // reconcile store arm's semantics — a recorded completion is - // the authority, never overwritten by the forced stop); an - // unreached agent entry settles with the recoverable - // AGENT_CANCELLED; an unreached steer with the honest - // `failed` (exactly what its fenced late landing would have - // settled). All recorded FIRST, then settled, drained and - // snapshotted with the other bound settlements below. (The - // gate above is always entered when unreached entries exist: - // the reconcile loop's only await — the re-attach load — is - // covered by the opening-call registry before it parks.) - const pendingEntries = this.workspace.surface()?.pending() ?? []; - for (const registryEntry of pendingEntries) { - if (registryEntry.kind === 'checkpoint') { - // A checkpoint the parked reconcile never re-surfaced is - // re-registered in the broker's checkpoint table here — it - // stays pending (a checkpoint awaits the human's answer; - // the drain must never fabricate one) but must stay - // ANSWERABLE across the cut-off restore (the doc: - // "answering works across a restore" — without the - // re-surface, `checkpoint.answer` could not find it). - this.requeueCheckpoint(registryEntry, this.callStore.lookup(registryEntry.id)); - continue; - } - if (this.openingCalls.has(registryEntry.id)) continue; // owned by the pass above - if (this.isTracked(registryEntry.id)) continue; // owned by the passes above - const storedRecord = this.callStore.lookup(registryEntry.id); - const storedCompletion = storedRecord?.completion; - if (storedCompletion !== null && storedCompletion !== undefined) { - // Completed while down: settle from the store (the same - // settle the store arm would have performed). - if (this.settleIntoGuest(registryEntry.id, storedCompletion.outcome, storedCompletion.value)) { - settledIds.push(registryEntry.id); - } - continue; - } - if (registryEntry.kind === 'steer') { - // The deliver() discipline: the store write first; a - // store that already holds a first completion stays the - // authority. A QUEUED answer-mode queued turn (the store - // record carries the queued marker) also gains the - // durable DROPPED marker — the cut-off turn settled - // `failed` here, and a restore must never re-queue it - // for a spurious second delivery turn. - const value = toRejectionValue( - executionError( - `steer ${registryEntry.id}: cut off by the client-presence drain`, - 'steering_interrupted', - true, - ), - ); - if (this.recordCompletion(registryEntry.id, { outcome: 'reject', value, completedAtMs: now() })) { - if (this.settleIntoGuest(registryEntry.id, 'reject', value)) settledIds.push(registryEntry.id); - } else { - const completion = this.callStore.lookup(registryEntry.id)?.completion; - if (completion === null || completion === undefined) { - throw new Error(`Broker: store lost the recorded completion for ${registryEntry.id}`); - } - if (this.settleIntoGuest(registryEntry.id, completion.outcome, completion.value)) { - settledIds.push(registryEntry.id); - } - } - this.warnLine( - 'warn', - `steer ${registryEntry.id}: cut off by the client-presence drain before its wire call started — nothing was delivered`, - ); - continue; - } - const value = toRejectionValue( - new WorkflowError( - `call ${registryEntry.id} was cancelled by the client-presence drain: its restore reconciliation ` + - `was cut off at the bound — the call is settled`, // eslint-disable-line max-len - CODE.AGENT_CANCELLED, - { recoverable: true }, - ), - ); - this.recordCompletion(registryEntry.id, { outcome: 'reject', value, completedAtMs: now() }); - if (this.settleIntoGuest(registryEntry.id, 'reject', value)) settledIds.push(registryEntry.id); - this.warnLine('warn', `call ${registryEntry.id}: ${value.message}`); - } - if (settledIds.length > 0) { - try { - // The interrupted-drain release — when the drain ran a - // tracked eval's continuation — happens inside `drain` - // itself (the interrupted job's continuation lease). - this.drain(deadline); - } catch (drainError) { - if (drainError instanceof DrainJobError) { - // The forced-stop settlements resumed a continuation that - // the disconnect bound interrupted; the failure is - // RETAINED under workspace().diagnostics (§6.2 — the - // drain error leaves the eval result surface). - this.retainedDrainError = { - name: drainError.info.name, - message: drainError.info.message, - atMs: now(), - }; - } else { - throw drainError; - } - } - this.provenancePass('settlement', settledIds); - this.sink?.boundary('settlement'); - } - } - } - if (shouldAbort?.()) { - this.draining = false; - return false; - } - // Release phase: every child closes while broker-owned future - // turns remain in their lane FIFO for later lazy reattachment. - const sessions = [...this.sessions.values()]; - for (const entry of sessions) { - const lane = this.lanes.get(entry.callId); - if (lane !== undefined && lane.laneState !== 'fatal') { - lane.laneState = 'released'; - lane.activeTurnId = null; - lane.promptInFlight = false; - } - } - const releases = sessions.map((entry) => - Promise.resolve(entry.session.release()).catch(() => undefined), - ); - await boundedAll(releases, deadline); - this.sessions.clear(); - this.agentSlots.clear(); - this.queueSlots.clear(); - this.pendingReattaches.clear(); - this.drained = true; - return drainedWithinBound; - }, deadline); - } - - /** - * The broker-enriched workspace manifest — the `status` tool's bindings - * surface (the roadmap doc: "top-level bindings with name, type, size, - * provenance (which subagent produced the value, from what task, when), - * and live-handle status. Metadata, never content — ls for the data - * plane"). The engine enumerates the user bindings trap-free (fresh- - * realm baseline set difference) with structure-only tokens and - * provenance labels; this layer appends the LIVE-HANDLE STATUS from the - * call store (`pending` / `settled` — the call id maps to the task and - * timestamps in the store, so any claim audits back to the worker that - * made it), the in-flight call ids, and the pending checkpoints. - */ - workspaceManifest(): WorkspaceManifestReport { - this.assertAlive(); - const manifest = this.workspace.manifest(); - const bindings = manifest.bindings.map((binding) => { - let token = binding.token; - let handleStatus: 'pending' | 'settled' | null = null; - if (binding.handleCallId !== null) { - const record = this.callStore.lookup(binding.handleCallId); - handleStatus = record === undefined || record.completion === null ? 'pending' : 'settled'; - // The size travels with the handle token too (phase-E review - // rejection: the manifest's size surface used to stop at the - // handle marker). `formatByteSize` is the previewer's decimal - // formatter — the same one the structure tokens use. The - // status and call id are ALSO reported as their own structured - // fields (phase-E review round 4: they used to live only in - // this string). - token = `agent handle \u00b7 ${handleStatus} \u00b7 call ${binding.handleCallId} \u00b7 ${formatByteSize(binding.sizeBytes)}`; - } - return { - name: binding.name, - token, - type: binding.type, - sizeBytes: binding.sizeBytes, - handleCallId: binding.handleCallId, - handleStatus, - provenance: binding.provenance, - provenanceAtMs: binding.provenanceAtMs, - // The doc's "from what task" provenance half: the task text behind - // a worker provenance (`worker c1` → the founding agent() call's - // task from the call store) or an agent-handle binding's founding - // call. Capped so the manifest stays bounded metadata. - task: this.taskForBinding(binding.provenance, binding.handleCallId), - }; - }); - return { - bindings, - logs: manifest.logs, - evalSeq: manifest.evalSeq, - inFlight: this.inFlightIds(), - checkpoints: this.pendingCheckpoints(), - }; - } - - /** Resolve a binding's task text ("from what task"): the founding - * agent() call's detail for a `worker cN` provenance label, or the - * handle's own call record for an agent-handle binding. Null when - * neither applies or the record is missing; capped at 200 chars - * (head+tail elision) so the manifest stays bounded metadata. */ - private taskForBinding(provenance: string | null, handleCallId: string | null): string | null { - const ids: string[] = []; - if (provenance !== null && provenance.startsWith('worker ')) { - ids.push(...provenance.slice('worker '.length).split('+')); - } - if (handleCallId !== null && !ids.includes(handleCallId)) ids.push(handleCallId); - const tasks: string[] = []; - for (const id of ids) { - const record = this.callStore.lookup(id); - if (record !== undefined && record.kind === 'agent' && record.detail.length > 0) { - tasks.push(record.detail); - } - } - if (tasks.length === 0) return null; - const joined = tasks.join(' / '); - return joined.length > 200 ? headTail(joined, 200) : joined; - } - - /** The in-flight host-task call ids, in dispatch order (the manifest's - * in-flight seam; the harness's `in_flight_ids`). */ - inFlightIds(): string[] { - return [...this.inFlight.keys()]; - } - - /** - * Teardown: cancel in-flight turns (best-effort), release EVERY - * session the broker opened (whether or not it owns the runner — a - * host-injected runner keeps its own lifetime, but the broker's - * dedicated ACP processes are still released; review regression: an - * injected-runner disposal used to leak every session), then dispose - * the runner when this broker owns it, and drop the broker's state. - * The workspace (and its VM) is the caller's to dispose. - * - * The teardown is BOUNDED (phase-D review round 7: it used to await - * `cancelSession`, `session.release` and the owned runner's dispose - * with NO deadline — a hung backend could block daemon shutdown and - * the reset tool indefinitely, and the daemon's shutdown path entered - * this unbounded disposal right after a failed or deadline-expired - * drain, hanging on the exact hung backend the drain had already - * caught). `boundMs` defaults to `DEFAULT_DISPOSE_BOUND_MS` (5 s — - * the spec-owed decision: the engine's own default mirrors the - * daemon's shutdown deadline; callers with a stricter budget — the - * daemon's shutdown path, which shares ONE deadline across the drain - * and this teardown — pass the remaining time). The bound is - * ABSOLUTE, measured from method entry like the client-presence - * drain's: every await races the remaining bound — INCLUDING the - * serialized-chain acquisition (phase-D review round 8: the chain - * wait used to have no deadline race, so a yieldful queued operation - * could delay disposal indefinitely; when the deadline expires while - * queued, the disposal body runs WITHOUT the chain, and its - * bookkeeping clear is safe unlocked — the stuck op's eventual - * landing is absorbed by the same first-wins/fenced paths as a - * disposal that ran chained) — and once it expires the disposal - * returns without waiting — the best-effort - * cancellations, releases and the runner teardown already issued keep - * running in the background (every promise carries a catch handler, - * so nothing can become an unhandled rejection). - */ - async dispose(boundMs: number = DEFAULT_DISPOSE_BOUND_MS): Promise { - if (this.disposed) return; - this.disposed = true; - this.draining = true; - // The disposal GENERATION bump fences every in-flight open and lazy - // re-attach started before disposal: when it lands, the child is - // released immediately — it never registers and never prompts after - // disposal/reset (phase-D review round 5: `openingCalls`, - // `stoppedOpens` and `pendingReattaches` used to be cleared without - // fencing their unresolved promises, so a late landing could - // re-register or run a child on a disposed broker). - this.generation++; - // The ABSOLUTE bound: measured at method entry — before the - // serialized-chain wait — like the client-presence drain's (review - // round 7); the chain wait itself races it (review round 8). - const deadline = Date.now() + Math.max(0, boundMs); - await this.serialized(async () => { - const cancels: Promise[] = []; - const sessions = [...this.sessions.values()]; - for (const entry of sessions) { - const lane = this.lanes.get(entry.callId); - if (lane?.promptInFlight) { - cancels.push(Promise.resolve(entry.session.cancel()).catch(() => undefined)); - } - } - await boundedAll(cancels, deadline); - const releases: Promise[] = sessions.map((entry) => - Promise.resolve(entry.session.release()).catch(() => undefined), - ); - await boundedAll(releases, deadline); - this.sessions.clear(); - this.pendingReattaches.clear(); - this.checkpoints.clear(); - this.deferreds.clear(); - this.inFlight.clear(); - this.agentSlots.clear(); - this.queueSlots.clear(); - this.openingCalls.clear(); - this.stoppedOpens.clear(); - for (const lane of this.lanes.values()) { - if (lane.cancellationTimer !== null) clearTimeout(lane.cancellationTimer); - } - this.lanes.clear(); - // The eval-plane additions: queued dispatches, first-class future - // turns, steering controls, and live sleep timers are dropped with the - // broker (their guest calls are gone with the workspace). The - // owed-but-unconsumed reset() teardown dies with the broker (a - // disposed broker owns no children to tear down). - this.dispatchQueue.length = 0; - this.queuedTurns.clear(); - this.steeringControls.clear(); - this.sleepCalls.clear(); - this.resetDue = false; - this.resetRequested = false; - this.resetOwningCompletions.clear(); - // The retained suspended-eval completions and the eval-break - // signal die with the broker (the handles are released before the - // VM is disposed by the caller). - for (const completion of this.activeEvalCompletions) completion.dispose(); - this.activeEvalCompletions.clear(); - this.evalTokens.clear(); - this.evalBreakTargets = new Set(); - this.evalBreakTokens = new Set(); - this.evalBreakArmed = false; - this.lastEvalToken = undefined; - this.jobLease.cell.current = undefined; - if (this.ownsRunner) { - // The owned runner's disposal races the remaining bound too - // (review round 7: it used to be awaited without any deadline). - // Its rejection still propagates when it wins the race — a - // failing runner teardown is a host-side failure; a deadline - // that wins leaves the runner's own best-effort teardown (its - // internal allSettled releases) running in the background, and - // the tail catch below absorbs a later rejection. - const runnerDispose = Promise.resolve(this.runner.dispose()); - runnerDispose.catch(() => undefined); - await boundedOne(runnerDispose, deadline); - } - }, deadline); - // The workspace's eval-break slot is released with the broker (the - // channel's slots are per-project and reusable — phase-F review - // round 3: the old channel never released slots, and a fresh - // workspace for the same project re-registers on its broker's - // attach). Fire-and-forget: the channel is a best-effort relay. - this.evalBreakChannel?.unregister(this.workspace.projectDir); - } - - // ── Guest bridge handlers ───────────────────────────────────────────── - - /** - * `__host_agent`: ADMISSION VALIDATION (§4.1 — the backend segment - * against the registry, the option keys, the `configOptions` - * vocabulary where it is knowable; all refusals settle the call - * SYNCHRONOUSLY, recorded first — a refused call must never be - * re-issued after a restore — and never throw), then the concurrency - * gate: dispatches above the cap QUEUE in dispatch order for the - * next free slot — NEVER a rejection (the workflow engine's - * semantics; `parallel(items.map(...))` must not lose work). - */ - private onAgent(call: GuestCall, callId: string, modelSpec: string, task: string, optionsJson: string | null): void { - if (this.drained) { - this.draining = false; - this.drained = false; - } - const admissionSequence = ++this.admissionSequence; - let parsed: ParsedAgentOptions; - try { - parsed = this.parseAgentOptions(optionsJson); - } catch (error) { - this.refuseAdmitted(call, callId, task, optionsJson, modelSpec, error); - return; - } - const admissionError = this.validateAdmission(modelSpec, parsed); - if (admissionError !== undefined) { - this.refuseAdmitted(call, callId, task, optionsJson, modelSpec, admissionError); - return; - } - this.recordDispatch(callId, 'agent', task, optionsJson, null, modelSpec, admissionSequence); - this.deferreds.set(callId, call); - const lane = this.newLane('opening'); - lane.activeTurnId = callId; - this.lanes.set(callId, lane); - // Founding dispatches and queued-turn heads share one admission - // arbiter. Enrolling first preserves the workspace-wide sequence - // order even when capacity is currently free. - this.dispatchQueue.push({ kind: 'dispatch', call, callId, modelSpec, task, optionsJson, parsed, admissionSequence }); - this.scheduleAdmissions(); - } - - /** - * The admission validation (§4.1): the backend segment MUST resolve - * against the registry at call time (built-ins plus registered custom - * agents) — an unknown segment rejects synchronously, naming the - * segment and enumerating the known backends, never a silent route to - * the default backend; a spec with no known-backend prefix is an - * error. EVERY spec validates — there is no sentinel bypass (the v1 - * reserved 'default' sentinel is deleted; verify/judgePanel resolve - * their reviewers/graders through the runner's real - * `defaultBackendId()`, which IS a registered segment). - * `configOptions` keys validate at admission against the - * resolved backend's known vocabulary WHERE IT IS KNOWABLE (the - * runner's `knownConfigOptionIds` seam — a backend whose vocabulary - * is genuinely dynamic returns undefined and the [C]5 fallback in - * `runAgentTask` covers it). Returns the refusal error, or undefined - * when the call is admitted. - */ - private validateAdmission(modelSpec: string, parsed: ParsedAgentOptions): unknown { - const segment = backendSegment(modelSpec); - const known = this.knownBackends(); - if (!known.includes(segment)) { - return new WorkflowError( - `unknown backend "${segment}" in model spec "${modelSpec}" (known backends: ${known.join(', ')})`, - CODE.SCRIPT_VALIDATION_ERROR, - { recoverable: false }, - ); - } - if (parsed.configOptions !== undefined) { - if ('model' in parsed.configOptions) { - return new WorkflowError( - `configOptions option "model" with authored value ${JSON.stringify(parsed.configOptions.model)} is reserved; ` + - 'use the first modelSpec argument instead', - CODE.SCRIPT_VALIDATION_ERROR, - { recoverable: false }, - ); - } - const vocabulary = this.runner.knownConfigOptionIds?.(segment); - if (vocabulary !== undefined) { - for (const key of Object.keys(parsed.configOptions)) { - if (!vocabulary.includes(key)) { - return new WorkflowError( - `configOptions: unknown option "${key}" for backend "${segment}" ` + - `(known options: ${vocabulary.length > 0 ? vocabulary.join(', ') : 'none'})`, - CODE.SCRIPT_VALIDATION_ERROR, - { recoverable: false }, - ); - } - } - } - } - return undefined; - } - - /** The known backend ids (built-ins plus registered custom agents), - * lowercased and sorted — cached (the registry is fixed at runner - * construction). Every runner publishes its registry (the seam's - * `listBackends` is REQUIRED — admission validation runs on every - * dispatch path). */ - private knownBackends(): string[] { - if (this.knownBackendsCache !== undefined) return this.knownBackendsCache; - const set = new Set(); - for (const id of this.runner.listBackends()) set.add(id.toLowerCase()); - this.knownBackendsCache = [...set].sort(); - return this.knownBackendsCache; - } - - /** - * A dispatch-time admission refusal, with the §4.6 backend attribution: - * a call whose backend segment RESOLVED (an option-key/config-option - * failure — the call was admitted to that backend) stamps `replBackend` - * onto the rejection so the uncaught-error rendering names the backend - * alongside the call id; an unknown-backend refusal has no resolved - * backend to name (its message enumerates the vocabulary instead). - */ - private refuseAdmitted( - call: GuestCall, - callId: string, - task: string, - optionsJson: string | null, - modelSpec: string, - error: unknown, - ): void { - const segment = backendSegment(modelSpec); - if (this.knownBackends().includes(segment)) { - (error as { replBackend?: string }).replBackend = segment; - } - this.refuse(call, callId, 'agent', task, optionsJson, error); - } - - /** The dispatch body (shared by the live path and the queued path): - * record the dispatch, register the deferred and the concurrency - * token, and start the async task. */ - private startDispatch( - call: GuestCall, - callId: string, - modelSpec: string, - task: string, - optionsJson: string | null, - parsed: ParsedAgentOptions, - ): void { - this.recordDispatch(callId, 'agent', task, optionsJson, null, modelSpec); - this.deferreds.set(callId, call); - this.agentSlots.add(callId); - const lane = this.lanes.get(callId) ?? this.newLane('opening'); - lane.laneState = 'opening'; - lane.activeTurnId = callId; - this.lanes.set(callId, lane); - // The opening-call registry (the client-presence drain's in-flight - // probe): the session does not exist yet — the drain must wait for it - // exactly like a busy session (a call blocked in openSession is still - // in flight; draining past it would let the child open and run after - // the last client disconnected). - this.openingCalls.add(callId); - // A fresh dispatch makes the workspace warmable again: the drain - // latch is STALE the moment a child may open (phase-D review round - // 5: it used to stay set until the open RESOLVED, so a second - // disconnect while the open was still parked skipped the drain - // entirely and the child could prompt after the last client - // disconnected). The latch re-sets when the drain runs. - this.drained = false; - const taskPromise = this.runAgentTask(callId, modelSpec, task, parsed); - this.trackInFlight(callId, 'agent', taskPromise); - } - - /** - * `__host_checkpoint`: question mode parks the call and records the - * dispatch; answer mode delivers the user's answer for the matching - * pending checkpoint — recorded FIRST (an accepted answer must survive - * a kill between the eval and the next snapshot; a FAILING store write - * must leave the checkpoint pending so a later answer retry can - * succeed — review regression: the checkpoint used to be forgotten - * before its answer was durable, and a failed write left the guest - * promise pending forever), then settled within the same eval, - * first-wins. Returns whether a pending checkpoint with that id was - * answered; an answer never touches the agent/steer call table (the - * phase-B review regression's id-space separation). - */ - private onCheckpoint( - call: GuestCall | null, - callId: string, - question: string | null, - optionsJson: string | null, - answerJson: string | null, - ): boolean | void { - if (answerJson !== null) { - const pending = this.checkpoints.get(callId); - if (pending === undefined) return false; - let answer: unknown; - try { - answer = JSON.parse(answerJson); - } catch { - // A host-side contract violation (the guest only sends - // JSON.stringify output): reject rather than park the question - // forever. Recorded FIRST, then the checkpoint is consumed, then - // the call settles (a failing record leaves the checkpoint - // pending and the call unsettled — the next answer retry works). - this.recordCompletion(callId, { - outcome: 'reject', - value: toRejectionValue(new Error(`checkpoint ${callId}: answer was not valid JSON`)), - completedAtMs: now(), - }); - this.checkpoints.delete(callId); - this.settleCheckpoint(callId, pending.call, 'reject', new Error(`checkpoint ${callId}: answer was not valid JSON`)); - return true; - } - // Record FIRST (durable), THEN consume the pending checkpoint, - // THEN settle — a failing store write propagates as a guest error - // in the answering eval and the checkpoint stays pending. - this.recordCompletion(callId, { outcome: 'resolve', value: answer, completedAtMs: now() }); - this.checkpoints.delete(callId); - this.settleCheckpoint(callId, pending.call, 'resolve', answer); - return true; - } - this.recordDispatch(callId, 'checkpoint', question ?? '', optionsJson, null); - this.checkpoints.set(callId, { - callId, - call: call!, - question: question ?? '', - optionsJson, - raisedAtMs: now(), - }); - // The §4.3 output line: a raised checkpoint surfaces as - // `checkpoint c9: ` in the eval's output stream — the - // question rendered as PLAIN head+tail metadata text (the retained - // 200-char preview, §7; the §4.3 fix: never a double-JSON-quoted - // form). - this.consoleBuffer.push({ - level: 'log', - line: `checkpoint ${callId}: ${headTailDescription(question ?? '', 200)}`, - }); - return undefined; - } - - /** - * `__host_agent_steer`: a steering operation on a live agent handle. - * `callId` is the operation's own registry id (the settlement key), - * `sessionId` the founding call id (the session being steered). The - * dispatch is recorded, then the operation runs per the steering - * mechanism table (module docs); the outcome settles with what - * actually happened — never a hard error. The session's cancel path - * also settles the CANCELLED call itself through its own task. - */ - private onQueue(call: GuestCall, callId: string, sessionId: string, payloadJson: string | null): void { - if (this.drained) { - this.draining = false; - this.drained = false; - } - const admissionSequence = ++this.admissionSequence; - const admittedAtMs = now(); - try { - this.recordDispatch(callId, 'queue', rawPromptDetail(payloadJson), payloadJson, sessionId, null, admissionSequence, admittedAtMs); - this.callStore.recordQueued(callId, admittedAtMs); - } catch (cause) { - const failure = persistenceFailure(cause); - this.markPersistenceFatal(sessionId, failure); - call.reject(failure); - this.syncSettled.push(callId); - return; - } - let payload: { prompt: string; promptMeta?: Record }; - try { - payload = this.parseTurnPayload(payloadJson, 'queue'); - } catch (error) { - this.refuseRecorded(call, callId, error); - return; - } - const founding = this.callStore.lookup(sessionId); - if (founding?.kind !== 'agent') { - this.refuseRecorded(call, callId, executionError(`queue ${callId}: founding session ${sessionId} does not exist`, 'session_not_found', false)); - return; - } - const lane = this.lanes.get(sessionId) ?? this.newLane(founding.completion === null ? 'opening' : 'released'); - this.lanes.set(sessionId, lane); - if (lane.laneState === 'fatal') { - this.refuseRecorded( - call, - callId, - executionError(`queue ${callId}: founding session ${sessionId} is fatally unavailable`, 'session_unusable', false), - ); - return; - } - const turn: QueuedTurn = { - callId, - sessionId, - prompt: payload.prompt, - promptMeta: payload.promptMeta, - call, - admissionSequence, - state: 'pending', - cancelRequested: false, - }; - this.queuedTurns.set(callId, turn); - lane.queuedTurnIds.push(callId); - this.deferreds.set(callId, call); - if (lane.laneState === 'released') this.scheduleQueueReattach(sessionId); - this.scheduleAdmissions(); - } - - private onSteer(call: GuestCall, callId: string, sessionId: string, payloadJson: string | null): void { - this.recordDispatch(callId, 'steer', 'steer', payloadJson, sessionId, null); - let payload: { prompt: string; promptMeta?: Record }; - try { - payload = this.parseTurnPayload(payloadJson, 'steer'); - } catch (error) { - this.refuseRecorded(call, callId, error); - return; - } - const lane = this.lanes.get(sessionId); - const targetTurnId = lane?.promptInFlight === true ? lane.activeTurnId : null; - if (lane === undefined || targetTurnId === null) { - this.settleControlSync(call, callId, 'idle'); - return; - } - const entry = this.sessions.get(sessionId); - if (entry === undefined) { - this.settleControlSync(call, callId, 'idle'); - return; - } - if (!advertisesSteering(entry.initializeMeta)) { - this.settleControlSync(call, callId, 'unsupported'); - return; - } - this.deferreds.set(callId, call); - this.steeringControls.set(callId, { callId, sessionId, targetTurnId, ...payload, call }); - lane.steeringControlIds.push(callId); - this.processSteeringControls(sessionId); - } - - private onSessionCancel(call: GuestCall, callId: string, sessionId: string): void { - this.recordDispatch(callId, 'cancel', 'session', null, sessionId, null); - const lane = this.lanes.get(sessionId); - if (lane === undefined || lane.activeTurnId === null) { - this.settleControlSync(call, callId, 'idle'); - this.scheduleAdmissions(); - return; - } - if (lane.laneState === 'opening' && !lane.promptInFlight) { - const cancelled = this.cancelFoundingBeforeSession(sessionId, 'handle cancel', false); - this.settleControlSync(call, callId, cancelled ? 'cancelled' : 'idle'); - return; - } - this.deferreds.set(callId, call); - this.trackInFlight(callId, 'cancel', this.requestTurnCancellation(sessionId, lane.activeTurnId)); - } - - private onQueueCancel(call: GuestCall, callId: string, queueCallId: string): void { - const turn = this.queuedTurns.get(queueCallId); - this.recordDispatch(callId, 'cancel', 'queue', null, turn?.sessionId ?? queueCallId, null); - if (turn === undefined || turn.state === 'settled') { - this.settleControlSync(call, callId, 'idle'); - return; - } - this.deferreds.set(callId, call); - this.trackInFlight(callId, 'cancel', this.requestTurnCancellation(turn.sessionId, queueCallId)); - } - - /** `__host_console`: buffer the event; the next tool result renders it - * (the guest-rendered one line per call, non-log levels prefixed). */ - private onConsole(event: { level: string; line: string }): void { - this.consoleBuffer.push(event); - } - - /** `__host_sleep`: settle the call from a HOST-side timer (the VM - * itself stays timer-free — §4.7). The settlement is tracked under a - * host-minted key (not a guest call id — sleeps never enter the - * guest registry or the call store) so the pump's readiness probe - * sees it; the pump resolves the guest promise and drains the - * continuation. A timer firing after the broker was disposed is a - * harmless no-op (the call is gone with the workspace). */ - private onSleep(call: GuestCall, ms: number): void { - const key = `sleep${++this.sleepSeq}`; - const task = { call, done: false }; - this.sleepCalls.set(key, task); - const delay = Number.isFinite(ms) && ms > 0 ? Math.min(ms, 2 ** 31 - 1) : 0; - setTimeout(() => { - task.done = true; - }, delay); - } - - // ── Dispatch, tasks, settlement ─────────────────────────────────────── - - /** Record a dispatch (idempotent per id — a re-issue of a known id - * keeps the original record). Steer records carry the FOUNDING session - * id (`sessionId` — the restore path's queue rebuild keys on it); - * agent/checkpoint records pass null. Agent records ALSO persist the - * model spec verbatim (`modelSpec` — the re-attach routing source; a - * restore or lazy re-attach must not re-resolve the spec against the - * current default backend, phase-D review round 2). */ - private recordDispatch( - callId: string, - kind: 'agent' | 'checkpoint' | 'queue' | 'steer' | 'cancel', - detail: string, - optionsJson: string | null, - foundingCallId: string | null = null, - modelSpec: string | null = null, - admissionSequence: number = ++this.admissionSequence, - admittedAtMs: number = now(), - ): void { - this.callStore.recordDispatched({ - callId, - kind, - detail, - optionsJson, - modelSpec, - backendId: null, - foundingCallId, - admittedAtMs, - admissionSequence, - dispatchedAtMs: admittedAtMs, - reissues: 0, - completion: null, - sessionId: null, - queuedAtMs: null, - handoffAtMs: null, - cancelledAtMs: null, - }); - } - - /** Record a completion (first-wins — returns whether newly recorded). - * Every rejected AGENT call whose backend resolved is attributed at - * this single durable boundary, covering live, restored, cancelled, - * held, and disconnect-forced settlements alike. */ - private recordCompletion(callId: string, outcome: CallOutcome): boolean { - if (outcome.outcome === 'reject' && typeof outcome.value === 'object' && outcome.value !== null) { - const record = this.callStore.lookup(callId); - if (record?.kind === 'agent' || record?.kind === 'queue') { - const founding = record.kind === 'queue' && record.foundingCallId !== null - ? this.callStore.lookup(record.foundingCallId) - : record; - const modelSegment = - founding?.modelSpec !== null && founding?.modelSpec !== undefined && founding.modelSpec !== '' - ? backendSegment(founding.modelSpec) - : undefined; - const segment = - (record.kind === 'queue' && record.foundingCallId !== null - ? this.sessions.get(record.foundingCallId)?.backendId - : undefined) ?? - founding?.backendId ?? - (modelSegment !== undefined && this.knownBackends().includes(modelSegment) - ? modelSegment - : undefined); - if (segment !== undefined) { - const value = outcome.value as { replBackend?: unknown }; - if (typeof value.replBackend !== 'string') value.replBackend = segment; - } - } - } - return this.callStore.recordCompleted(callId, outcome); - } - - /** A dispatch-time refusal: record dispatched + rejected FIRST (a - * refused call must never be re-issued after a restore), then settle - * the guest call with the recoverable/non-recoverable error. */ - private refuse(call: GuestCall, callId: string, kind: 'agent' | 'queue' | 'steer' | 'cancel', detail: string, optionsJson: string | null, error: unknown): void { - this.recordDispatch(callId, kind, detail, optionsJson); - const value = toRejectionValue(error); - this.recordCompletion(callId, { outcome: 'reject', value, completedAtMs: now() }); - call.reject(value); - this.syncSettled.push(callId); - } - - private refuseRecorded(call: GuestCall, callId: string, error: unknown): void { - const value = toRejectionValue( - isWorkflowError(error) - ? error - : new WorkflowError(toRejectionValue(error).message, CODE.SCRIPT_VALIDATION_ERROR, { - recoverable: false, - }), - ); - this.recordCompletion(callId, { outcome: 'reject', value, completedAtMs: now() }); - call.reject(value); - this.syncSettled.push(callId); - } - - /** A synchronous steering settlement (no session, idle cancel, queued - * delivery, bad action): recorded then settled, like every other - * settlement. */ - private settleControlSync(call: GuestCall, callId: string, outcome: SteeringOutcomeValue | CancelOutcomeValue): void { - this.recordCompletion(callId, { outcome: 'resolve', value: outcome, completedAtMs: now() }); - call.resolve(outcome); - this.syncSettled.push(callId); - } - - /** Track an in-flight host task with the poll-shaped readiness flag. */ - private trackInFlight( - callId: string, - kind: 'agent' | 'queue' | 'steer' | 'cancel', - promise: Promise<{ outcome: 'resolve' | 'reject' | 'hold'; value: unknown }>, - ): void { - const entry: InFlightTask = { callId, kind, promise, done: false }; - promise.then( - () => { - entry.done = true; - }, - () => { - entry.done = true; - }, - ); - this.inFlight.set(callId, entry); - } - - /** The agent call task: open the session, run the prompt, shape the - * result (schema ladder or text), and report the outcome. The session - * stays open after the call settles — the live-handle contract. The - * model spec is admission-validated (its backend segment resolved - * against the registry — §4.1) and passed verbatim. The schema rides - * session creation (`openSession` - * folds it into the backend's native session/new channel — Claude); - * the per-turn channels (Codex's `outputSchema` forward, the in-band - * contract for pi/custom) are the runner's own, applied inside - * `InteractiveSession.prompt` from that same schema. - */ - private async runAgentTask( - callId: string, - modelSpec: string, - task: string, - parsed: ParsedAgentOptions, - backendIdOverride: string | null = null, - ): Promise<{ outcome: 'resolve' | 'reject' | 'hold'; value: unknown }> { - let openedSession: BrokerSession | undefined; - // The drain/disposal generation captured at START: an `openSession` - // that lands after the drain's bound expired (or after a - // dispose/reset) is STOPPED exactly like a drain-stopped open — the - // child is released immediately and never prompts (phase-D review - // round 5: the disposal cleared `openingCalls`/`stoppedOpens` - // without fencing the unresolved open, so a late landing could - // re-register and prompt after disposal/reset). - const generation = this.generation; - try { - const session = await this.runner.openSession({ - // The routing pin: a re-issue of a call that once had a backend - // re-routes to the ORIGINAL backend (a backend id doubles as a - // model routing spec), never to the current configured default - // (phase-D review round 2: routing by the current default across - // a restart could open the re-issued session on the wrong - // backend). - model: backendIdOverride ?? (modelSpec ?? undefined), - schema: parsed.schema as never, - cwd: parsed.cwd ?? this.workspace.projectDir, - configOptions: parsed.configOptions, - mode: parsed.mode, - label: `repl:${callId}`, - runId: callId, - keepSession: true, - retainSessionLog: true, - }); - openedSession = session; - if (this.stoppedOpens.has(callId) || this.disposed || this.generation !== generation) { - // The client-presence drain's bound expired (or the broker was - // disposed/reset) while this open was in flight: the call is - // STOPPED — the child is closed immediately (it never prompts — - // nothing runs after the last client disconnected, and nothing - // runs after disposal), queued steers are dropped durably, and - // the call settles as the recoverable AGENT_CANCELLED. The - // stopped marker is consumed here so a later reconcile re-issues - // normally. - this.stoppedOpens.delete(callId); - this.openingCalls.delete(callId); - this.failQueuedTurns( - callId, - executionError( - `session ${callId}: founding turn was stopped before a reusable session was established`, - 'founding_turn_cancelled_before_session', - false, - ), - ); - // Detached, never awaited: the drain already settled the call at - // its bound, and a hung release must not park the stopped task - // (the same boundless-release family as the restore fence). - void Promise.resolve(session.release()).catch(() => undefined); - return { - outcome: 'reject', - value: toRejectionValue( - new WorkflowError( - `call ${callId} was stopped by the client-presence drain while its session was still opening`, - CODE.AGENT_CANCELLED, - { recoverable: true }, - ), - ), - }; - } - this.openingCalls.delete(callId); - this.drained = false; // children are warm again - const entry: SessionEntry = { - session, - callId, - modelSpec, - task, - backendId: session.backendId ?? backendSegment(modelSpec), - initializeMeta: initializeMetaOf(session), - callSettled: false, - callCancelled: false, - cancelWaiters: new Set(), - }; - this.sessions.set(callId, entry); - const lane = this.lanes.get(callId) ?? this.newLane('usable'); - lane.laneState = 'usable'; - lane.activeTurnId = callId; - this.lanes.set(callId, lane); - this.watchSessionRelease(entry); - // Durable re-attach key (phase D): record the backend session id - // (and the RESOLVED backend id — the re-attach routing pin) the - // moment the session opens — BEFORE the prompt is sent — so a - // crash with a turn in flight leaves a restore able to re-attach - // this session on the RIGHT backend (without the record, the - // restore would re-issue a call whose turn may still be running at - // the backend — duplicated work — or route the load by the current - // default backend and miss the original session). A failing record - // is a host-side failure: the call rejects (the session stays open - // and tracked, so dispose releases it). - this.callStore.recordAttached(callId, session.sessionId, now(), session.backendId ?? null); - lane.promptInFlight = true; - let turn: BrokerTurn; - try { - turn = await session.prompt(task); - } finally { - lane.promptInFlight = false; - } - this.assertNormalStopReason(turn.stopReason, callId); - const value = - parsed.schema !== undefined - ? await this.resolveStructuredOutput(entry, parsed) - : this.finalText(entry); - return { outcome: 'resolve', value }; - } catch (error) { - if (openedSession === undefined) { - this.openingCalls.delete(callId); - this.failQueuedTurns( - callId, - executionError( - `founding session ${callId} failed to open: ${toRejectionValue(error).message}`, - 'session_open_failed', - false, - ), - ); - } - // The §4.6 attribution: the rejecting call names its resolved - // backend. The session's own backend id when it opened, else the - // admission-validated segment. - const value = toRejectionValue(error); - const backend = - openedSession !== undefined - ? (openedSession.backendId ?? backendSegment(modelSpec)) - : backendSegment(modelSpec); - (value as { replBackend?: string }).replBackend = backend; - // The [C]5 fallback: a backend whose config-option vocabulary is - // genuinely dynamic cannot be validated at admission. When the call - // carried configOptions and the failure does not already name one, one - // bounded diagnostic reopen WITHOUT the config options decides whether - // configuration caused the failure. Only a successful baseline reopen - // permits blaming and isolating a config key; an independently failing - // reopen preserves the backend's original error. - if (openedSession === undefined && parsed.configOptions !== undefined && Object.keys(parsed.configOptions).length > 0) { - return { - outcome: 'reject', - value: await this.configOptionLateError(callId, modelSpec, parsed, value, backendIdOverride), - }; - } - return { outcome: 'reject', value }; - } - } - - /** - * The [C]5 fallback body: decide whether the openSession failure was - * the config options' doing and, only when established, make the rejection - * name the offending key. The diagnostic reopen (configOptions - * omitted, session NOT kept open) establishes whether configuration - * caused the failure. For multiple keys, prompt-free prefix probes - * isolate the actual offending key. These run only on the failure path. - */ - private async configOptionLateError( - callId: string, - modelSpec: string, - parsed: ParsedAgentOptions, - original: { name: string; message: string; code?: string; recoverable?: boolean }, - backendIdOverride: string | null, - ): Promise<{ name: string; message: string; code?: string; recoverable?: boolean; replBackend?: string }> { - const keys = Object.keys(parsed.configOptions!); - // The backend may already have named the offending key in its own - // message. That shortcut is only decisive with a SINGLE key: with - // several keys the message can name an accepted sibling while - // omitting the actual rejected key (the round-6 review repro: - // { good: true, bad: true } + "accepted option good; another - // config option is invalid" emitted the vague message verbatim), - // so multi-key failures NEVER skip the diagnostic reopen and the - // prefix-probe isolation below on the strength of a message hit. - if (keys.length === 1 && original.message.includes(keys[0])) return original; - let diagnostic: BrokerSession | undefined; - try { - diagnostic = await this.runner.openSession({ - model: backendIdOverride ?? (modelSpec ?? undefined), - schema: parsed.schema as never, - cwd: parsed.cwd ?? this.workspace.projectDir, - mode: parsed.mode, - label: `repl:${callId}`, - runId: callId, - keepSession: false, - retainSessionLog: true, - }); - } catch { - // The same failure without configOptions proves that configuration was not - // observably causal. Preserve the backend's original error (for example an - // unsupported ACP mode) instead of falsely accusing one of the carried keys. - return original; - } - if (keys.length > 1) { - // Dynamic vocabularies publish no admission-time key list. Isolate - // the actual rejected key by adding options in caller order until - // the first prefix fails. Every preceding prefix was accepted, so - // the newly added key is the offending one; when all proper - // prefixes succeed, the final key is the one that turns the known - // failing full bag invalid. Successful probes never send a prompt - // and are released immediately. - const prefix: Record = {}; - let offendingKey = keys[keys.length - 1]; - let backend = diagnostic?.backendId ?? backendIdOverride ?? backendSegment(modelSpec); - for (let index = 0; index < keys.length - 1; index++) { - const key = keys[index]; - prefix[key] = parsed.configOptions![key]; - let probe: BrokerSession | undefined; - try { - probe = await this.runner.openSession({ - model: backendIdOverride ?? (modelSpec ?? undefined), - schema: parsed.schema as never, - cwd: parsed.cwd ?? this.workspace.projectDir, - configOptions: { ...prefix }, - mode: parsed.mode, - label: `repl:${callId}`, - runId: callId, - keepSession: false, - retainSessionLog: true, - }); - backend = probe.backendId ?? backend; - } catch { - offendingKey = key; - break; - } finally { - if (probe !== undefined) void Promise.resolve(probe.release()).catch(() => undefined); - } - } - if (diagnostic !== undefined) void Promise.resolve(diagnostic.release()).catch(() => undefined); - return { - name: 'ConfigOptionsError', - message: - `backend ${backend} rejected the call's configOptions — offending key "${offendingKey}" ` + - `(backend error: ${original.message})`, - recoverable: false, - replBackend: backend, - }; - } - try { - // The config options caused the failure: name the offending key - // (the multiple-key path above isolates by accepted prefixes). - const carried = keys.map((key) => `"${key}"`).join(', '); - const backend = diagnostic!.backendId ?? backendSegment(modelSpec); - return { - name: 'ConfigOptionsError', - message: - `backend ${backend} rejected the call's configOptions — offending key ${carried}` + - ` (backend error: ${original.message})`, - recoverable: false, - // The §4.6 attribution: the [C]5 fallback's replacement error - // names the resolved backend too (the call id is stamped by the - // guest library at settlement). - replBackend: backend, - }; - } finally { - void Promise.resolve(diagnostic!.release()).catch(() => undefined); - } - } - - /** The schema ladder, driven by acp-agents' own resolver over the - * session (the same convert/check + re-prompt machinery `run()` uses; - * the one divergence is documented in the module docs). */ - private async resolveStructuredOutput( - entry: SessionEntry, - parsed: ParsedAgentOptions, - queuedTurn?: QueuedTurn, - ): Promise { - const session = entry.session; - const structuredSession: StructuredSession = { - prompt: async (repromptText: string) => { - if (queuedTurn?.cancelRequested) { - throw new WorkflowError(`queued turn ${queuedTurn.callId} was cancelled`, CODE.AGENT_CANCELLED, { - recoverable: true, - agentLabel: `repl:${queuedTurn.callId}`, - }); - } - const lane = this.lanes.get(entry.callId); - if (lane !== undefined) lane.promptInFlight = true; - let turn: BrokerTurn; - try { - turn = await session.prompt( - repromptText, - queuedTurn === undefined - ? undefined - : { - promptMeta: this.queuePromptMeta(queuedTurn.promptMeta, queuedTurn.callId), - onHandoff: () => this.recordQueueHandoff(queuedTurn.sessionId, queuedTurn.callId), - }, - ); - } finally { - if (lane !== undefined) lane.promptInFlight = false; - } - // A repair turn that refuses / truncates / cancels must surface - // distinctly instead of silently continuing the ladder (the - // runner's own ladder does the same). - this.assertNormalStopReason(turn.stopReason, queuedTurn?.callId ?? entry.callId); - }, - // Final message only, matching the runner: prose extraction over - // the whole turn would resurrect the first-JSON-wins bug. - lastText: () => session.finalMessageText(), - tryNative: () => session.rawStructuredOutput() ?? parseFinalJson(session.finalMessageText()), - }; - return resolveStructuredOutput(structuredSession, parsed.schema as never, { - label: `repl:${queuedTurn?.callId ?? entry.callId}`, - }); - } - - /** The no-schema result fold (§5 [C]12): the latest turn's assistant - * text with the "\n\n" chunk joiner when the session exposes the - * folded surface; a third-party adapter without it degrades to - * `currentTurnText()` (its own fold). */ - private finalTurnText(session: BrokerSession): string { - return session.foldedTurnText !== undefined ? session.foldedTurnText() : session.currentTurnText(); - } - - /** The no-schema result: the latest turn's assistant text, mirroring - * the runner's `AGENT_EMPTY_OUTPUT` refusal. */ - private finalText(entry: SessionEntry): string { - return this.finalTextOf(this.finalTurnText(entry.session), entry.callId); - } - - /** The shared empty-output gate for a completed turn's text (used by - * the live path and the re-attached path alike). */ - private finalTextOf(text: string, callId: string): string { - const trimmed = text.trim(); - if (!trimmed) { - throw new WorkflowError('Subagent produced no assistant output', CODE.AGENT_EMPTY_OUTPUT, { - recoverable: true, - }); - } - return trimmed; - } - - /** The stop-reason gate, mirroring the runner's own (with the REPL's - * `AGENT_CANCELLED` code for the orchestrator-driven cancel). */ - private assertNormalStopReason(stopReason: string, callId: string): void { - switch (stopReason) { - case 'refusal': - throw new WorkflowError('model refused to respond', CODE.AGENT_EXECUTION_ERROR, { - recoverable: false, - agentLabel: `repl:${callId}`, - }); - case 'max_tokens': - case 'max_turn_requests': - throw new WorkflowError(`output truncated (stop reason: ${stopReason})`, CODE.AGENT_EXECUTION_ERROR, { - recoverable: false, - agentLabel: `repl:${callId}`, - }); - case 'cancelled': - // Orchestrator-driven cancellation is RECOVERABLE (review - // regression: this used to be recoverable: false, which the guest - // combinators treat as a halt signal — cancelling one worker then - // aborted the parallel()/pipeline() owning it). The module docs' - // cancel contract ("the cancelled call itself rejects with the - // recoverable CancelledError") and the monorepo convention agree: - // one call's cancellation must never abort the surrounding - // orchestration. The store's cancelled-founding-call check keys on - // the CODE, not the flag, so the queue-drop semantics are - // unchanged. - throw new WorkflowError(`call ${callId} was cancelled`, CODE.AGENT_CANCELLED, { - recoverable: true, - agentLabel: `repl:${callId}`, - }); - default: - return; // "end_turn" and any unrecognized future reason: normal - } - } - - private recordQueueHandoff(sessionId: string, callId: string): void { - try { - this.callStore.recordHandoff(callId, now()); - } catch (cause) { - const failure = persistenceFailure(cause); - this.markPersistenceFatal(sessionId, failure); - throw new WorkflowError(failure.message, CODE.PERSISTENCE_ERROR, { - recoverable: false, - details: { reason: 'queue_persistence_failed' }, - }); - } - } - - private queuePromptMeta( - promptMeta: Record | undefined, - callId: string, - ): Record { - const ownedNamespace = promptMeta?.['@automatalabs/agentprism']; - return { - ...promptMeta, - '@automatalabs/agentprism': { - ...(isPlainObject(ownedNamespace) ? ownedNamespace : {}), - replCallId: callId, - }, - }; - } - - private strictSteeringMeta(promptMeta: Record | undefined): Record { - const steering = promptMeta?.steering; - return { - ...promptMeta, - steering: { - ...(isPlainObject(steering) ? steering : {}), - idleBehavior: 'promptRequired', - }, - }; - } - - private foundingOptions(sessionId: string): ParsedAgentOptions { - const record = this.callStore.lookup(sessionId); - if (record?.optionsJson === null || record?.optionsJson === undefined) return {}; - return this.parseAgentOptions(record.optionsJson); - } - - private async runQueuedTurnTask( - turn: QueuedTurn, - entry: SessionEntry, - ): Promise<{ outcome: 'resolve' | 'reject'; value: unknown }> { - const lane = this.lanes.get(turn.sessionId); - try { - if (turn.cancelRequested || turn.state === 'settled') { - throw new WorkflowError(`queued turn ${turn.callId} was cancelled`, CODE.AGENT_CANCELLED, { - recoverable: true, - agentLabel: `repl:${turn.callId}`, - }); - } - if (lane === undefined || lane.laneState !== 'usable' || lane.activeTurnId !== turn.callId) { - throw executionError( - `queued turn ${turn.callId}: its session lane is not usable`, - 'session_unusable', - false, - ); - } - lane.promptInFlight = true; - let promptTurn: BrokerTurn; - try { - promptTurn = await entry.session.prompt(turn.prompt, { - promptMeta: this.queuePromptMeta(turn.promptMeta, turn.callId), - onHandoff: () => { - this.recordQueueHandoff(turn.sessionId, turn.callId); - }, - }); - } finally { - lane.promptInFlight = false; - } - this.assertNormalStopReason(promptTurn.stopReason, turn.callId); - if (turn.cancelRequested) { - throw new WorkflowError(`queued turn ${turn.callId} was cancelled`, CODE.AGENT_CANCELLED, { - recoverable: true, - agentLabel: `repl:${turn.callId}`, - }); - } - const parsed = this.foundingOptions(turn.sessionId); - const value = parsed.schema !== undefined - ? await this.resolveStructuredOutput(entry, parsed, turn) - : this.finalTextOf(this.finalTurnText(entry.session), turn.callId); - return { outcome: 'resolve', value }; - } catch (error) { - const value = toRejectionValue(error); - value.replBackend = entry.backendId; - return { outcome: 'reject', value }; - } - } - - private async runRestoredQueuedTurnTask( - turn: QueuedTurn, - entry: SessionEntry, - ): Promise<{ outcome: 'resolve' | 'reject'; value: unknown }> { - const awaitTurn = entry.session.awaitCurrentTurn; - if (awaitTurn === undefined) { - const error = executionError( - `queued turn ${turn.callId}: the handed-off turn cannot be classified after restore`, - 'queued_turn_indeterminate', - false, - ); - this.markLaneFatal(turn.sessionId, error); - return { outcome: 'reject', value: toRejectionValue(error) }; - } - try { - const promptTurn = await awaitTurn.call(entry.session); - this.assertNormalStopReason(promptTurn.stopReason, turn.callId); - const parsed = this.foundingOptions(turn.sessionId); - const value = parsed.schema !== undefined - ? await this.resolveStructuredOutput(entry, parsed, turn) - : this.finalTextOf(promptTurn.text || this.finalTurnText(entry.session), turn.callId); - return { outcome: 'resolve', value }; - } catch (cause) { - if (turn.cancelRequested || isCancellation(cause)) { - return { - outcome: 'reject', - value: toRejectionValue( - new WorkflowError(`queued turn ${turn.callId} was cancelled`, CODE.AGENT_CANCELLED, { - recoverable: true, - agentLabel: `repl:${turn.callId}`, - }), - ), - }; - } - const error = executionError( - `queued turn ${turn.callId}: handed-off turn recovery failed: ${toRejectionValue(cause).message}`, - 'queued_turn_indeterminate', - false, - ); - this.markLaneFatal(turn.sessionId, error); - return { outcome: 'reject', value: toRejectionValue(error) }; - } - } - - private async runSteeringTask( - control: SteeringControl, - entry: SessionEntry, - ): Promise<{ outcome: 'resolve' | 'reject'; value: unknown }> { - try { - const response = await entry.session.steer(control.prompt, { - promptMeta: this.strictSteeringMeta(control.promptMeta), - }); - if (!isPlainObject(response) || typeof response.outcome !== 'string') { - const error = executionError( - `steer ${control.callId}: backend returned an invalid steering response`, - 'invalid_steering_response', - false, - ); - void Promise.resolve(entry.session.cancel()).catch(() => undefined); - this.markLaneFatal(control.sessionId, error); - return { outcome: 'reject', value: toRejectionValue(error) }; - } - switch (response.outcome) { - case 'injected': - return { outcome: 'resolve', value: 'injected' }; - case 'promptRequired': - return { outcome: 'resolve', value: 'idle' }; - case 'failed': - return { - outcome: 'reject', - value: toRejectionValue( - executionError(`steer ${control.callId}: backend rejected steering`, 'steering_failed', true), - ), - }; - case 'startedNewTurn': { - const error = executionError( - `steer ${control.callId}: backend violated strict steering by starting a new turn`, - 'steering_started_new_turn', - false, - ); - void Promise.resolve(entry.session.cancel()).catch(() => undefined); - this.markLaneFatal(control.sessionId, error); - return { outcome: 'reject', value: toRejectionValue(error) }; - } - default: { - const error = executionError( - `steer ${control.callId}: backend returned unknown outcome ${JSON.stringify(response.outcome)}`, - 'invalid_steering_response', - false, - ); - void Promise.resolve(entry.session.cancel()).catch(() => undefined); - this.markLaneFatal(control.sessionId, error); - return { outcome: 'reject', value: toRejectionValue(error) }; - } - } - } catch (cause) { - const shaped = toRejectionValue(cause); - const methodMissing = - (cause as { code?: unknown } | null)?.code === -32601 || - /method\s+not\s+found/i.test(shaped.message); - if (methodMissing) { - return { - outcome: 'reject', - value: toRejectionValue( - executionError( - `steer ${control.callId}: ${shaped.message}`, - 'advertised_steering_missing', - false, - ), - ), - }; - } - return { - outcome: 'reject', - value: { - ...shaped, - code: shaped.code ?? CODE.AGENT_EXECUTION_ERROR, - recoverable: shaped.recoverable ?? true, - details: { reason: 'steering_request_failed' }, - }, - }; - } - } - - private processSteeringControls(sessionId: string): void { - const lane = this.lanes.get(sessionId); - if (lane === undefined || lane.steeringInFlight || lane.laneState === 'fatal') return; - for (;;) { - const callId = lane.steeringControlIds[0]; - if (callId === undefined) { - this.scheduleAdmissions(); - return; - } - const control = this.steeringControls.get(callId); - if (control === undefined) { - lane.steeringControlIds.shift(); - continue; - } - lane.steeringInFlight = true; - const entry = this.sessions.get(sessionId); - const task = - entry === undefined || - lane.activeTurnId !== control.targetTurnId || - !lane.promptInFlight - ? Promise.resolve({ outcome: 'resolve' as const, value: 'idle' }) - : this.runSteeringTask(control, entry); - this.trackInFlight(callId, 'steer', task); - return; - } - } - - private startQueuedTurn(turn: QueuedTurn, entry: SessionEntry, lane: SessionLaneState): void { - turn.state = 'active'; - lane.activeTurnId = turn.callId; - this.queueSlots.add(turn.callId); - this.trackInFlight(turn.callId, 'queue', this.runQueuedTurnTask(turn, entry)); - } - - /** Fill free workspace slots with the oldest eligible founding - * dispatch or per-session queue head. Ineligible older work does not - * block a newer eligible candidate. */ - private scheduleAdmissions(): void { - if (this.disposed || this.draining) return; - for (;;) { - if (this.agentSlots.size + this.queueSlots.size >= this.maxConcurrentAgents) return; - let bestSequence = Number.POSITIVE_INFINITY; - let dispatchIndex = -1; - for (let index = 0; index < this.dispatchQueue.length; index++) { - const item = this.dispatchQueue[index]; - const sequence = item.kind === 'dispatch' - ? item.admissionSequence - : (this.callStore.lookup(item.entry.id)?.admissionSequence ?? Number.POSITIVE_INFINITY); - if (sequence < bestSequence) { - bestSequence = sequence; - dispatchIndex = index; - } - } - let queueCandidate: { turn: QueuedTurn; entry: SessionEntry; lane: SessionLaneState } | undefined; - for (const [sessionId, lane] of this.lanes) { - if (lane.laneState === 'released' && lane.queuedTurnIds.length > 0) { - this.scheduleQueueReattach(sessionId); - continue; - } - if ( - lane.laneState !== 'usable' || - lane.activeTurnId !== null || - lane.promptInFlight || - lane.steeringInFlight || - lane.steeringControlIds.length > 0 || - lane.cancellingTurnId !== null - ) continue; - const headId = lane.queuedTurnIds[0]; - const turn = headId === undefined ? undefined : this.queuedTurns.get(headId); - const entry = this.sessions.get(sessionId); - if (turn === undefined || turn.state !== 'pending' || turn.cancelRequested || entry === undefined) continue; - if (turn.admissionSequence < bestSequence) { - bestSequence = turn.admissionSequence; - dispatchIndex = -1; - queueCandidate = { turn, entry, lane }; - } - } - if (queueCandidate !== undefined) { - this.startQueuedTurn(queueCandidate.turn, queueCandidate.entry, queueCandidate.lane); - continue; - } - if (dispatchIndex < 0) return; - const [dispatch] = this.dispatchQueue.splice(dispatchIndex, 1); - if (dispatch.kind === 'dispatch') { - this.startDispatch( - dispatch.call, - dispatch.callId, - dispatch.modelSpec, - dispatch.task, - dispatch.optionsJson, - dispatch.parsed, - ); - } else { - this.startReissue(dispatch.entry, dispatch.parsed, dispatch.reason, dispatch.report); - } - } - } - - private scheduleQueueReattach(sessionId: string): void { - const lane = this.lanes.get(sessionId); - if ( - lane === undefined || - lane.laneState !== 'released' || - lane.queuedTurnIds.length === 0 || - this.pendingReattaches.has(sessionId) || - this.disposed || - this.draining - ) return; - if (!this.canLazyReattach(sessionId)) { - this.markLaneFatal( - sessionId, - executionError(`session ${sessionId}: queued turn reattachment is unavailable`, 'session_reattach_failed', false), - ); - return; - } - void this.lazyReattach(sessionId).then((entry) => { - if (entry === undefined) { - this.markLaneFatal( - sessionId, - executionError(`session ${sessionId}: queued turn reattachment failed`, 'session_reattach_failed', false), - ); - return; - } - const current = this.lanes.get(sessionId); - if (current !== undefined && current.laneState !== 'fatal') current.laneState = 'usable'; - this.watchSessionRelease(entry); - this.scheduleAdmissions(); - }); - } - - private restoreHandedOffQueue(callId: string): void { - const turn = this.queuedTurns.get(callId); - if (turn === undefined || this.inFlight.has(callId) || this.queueSlots.has(callId)) return; - const lane = this.lanes.get(turn.sessionId); - if (lane === undefined) return; - lane.activeTurnId = callId; - turn.state = 'active'; - this.queueSlots.add(callId); - void this.lazyReattach(turn.sessionId).then((entry) => { - if (entry === undefined) { - this.markLaneFatal( - turn.sessionId, - executionError( - `queued turn ${callId}: its handed-off session could not be reattached`, - 'session_reattach_failed', - false, - ), - ); - return; - } - const current = this.lanes.get(turn.sessionId); - if (current !== undefined && current.laneState !== 'fatal') current.laneState = 'usable'; - this.watchSessionRelease(entry); - this.trackInFlight(callId, 'queue', this.runRestoredQueuedTurnTask(turn, entry)); - }); - } - - private cancelledTurnValue(callId: string): ReturnType { - return toRejectionValue( - new WorkflowError(`turn ${callId} was cancelled`, CODE.AGENT_CANCELLED, { - recoverable: true, - agentLabel: `repl:${callId}`, - }), - ); - } - - private settleStored(callId: string, outcome: 'resolve' | 'reject', value: unknown): void { - const newlyRecorded = this.recordCompletion(callId, { outcome, value, completedAtMs: now() }); - const completion = newlyRecorded - ? { outcome, value } - : this.callStore.lookup(callId)?.completion; - if (completion !== null && completion !== undefined) { - this.settleIntoGuest(callId, completion.outcome, completion.value); - } - } - - private async requestTurnCancellation( - sessionId: string, - turnId: string, - ): Promise<{ outcome: 'resolve' | 'reject' | 'hold'; value: unknown }> { - if (this.disposed) return { outcome: 'resolve', value: 'idle' }; - const lane = this.lanes.get(sessionId); - const record = this.callStore.lookup(turnId); - if (lane === undefined || record?.completion !== null || lane.laneState === 'fatal') { - return { outcome: 'resolve', value: 'idle' }; - } - const queued = this.queuedTurns.get(turnId); - if (queued !== undefined && queued.state === 'pending') { - queued.cancelRequested = true; - queued.state = 'settled'; - try { - this.callStore.recordCancelled(turnId, now()); - this.settleStored(turnId, 'reject', this.cancelledTurnValue(turnId)); - } catch (error) { - const failure = persistenceFailure(error); - this.markPersistenceFatal(sessionId, failure); - return { outcome: 'reject', value: failure }; - } - lane.queuedTurnIds = lane.queuedTurnIds.filter((id) => id !== turnId); - this.scheduleAdmissions(); - return { outcome: 'resolve', value: 'cancelled' }; - } - if (lane.activeTurnId !== turnId) return { outcome: 'resolve', value: 'idle' }; - if (lane.cancellingTurnId === turnId) return { outcome: 'resolve', value: 'cancelled' }; - - if (queued !== undefined) { - queued.cancelRequested = true; - queued.state = 'cancelling'; - } - const entry = this.sessions.get(sessionId); - if (turnId === sessionId && entry !== undefined) this.markCancelled(entry); - try { - this.callStore.recordCancelled(turnId, now()); - this.settleStored(turnId, 'reject', this.cancelledTurnValue(turnId)); - } catch (error) { - const failure = persistenceFailure(error); - this.markPersistenceFatal(sessionId, failure); - return { outcome: 'reject', value: failure }; - } - lane.cancellingTurnId = turnId; - - if (lane.promptInFlight && entry !== undefined) { - lane.cancellationTimer = setTimeout(() => { - const current = this.lanes.get(sessionId); - if ( - current?.cancellingTurnId === turnId && - current.activeTurnId === turnId && - current.promptInFlight - ) { - this.markLaneFatal( - sessionId, - executionError( - `turn ${turnId}: backend did not honor cancellation within ${CANCELLATION_SETTLEMENT_BOUND_MS}ms`, - 'cancellation_not_honored', - false, - ), - ); - } - }, CANCELLATION_SETTLEMENT_BOUND_MS); - try { - await entry.session.cancel(); - } catch (error) { - this.warnLine('warn', `turn ${turnId}: ACP cancellation attempt failed: ${toRejectionValue(error).message}`); - } - } - return { outcome: 'resolve', value: 'cancelled' }; - } - - private cancelFoundingBeforeSession(callId: string, source: string, drain: boolean): boolean { - const lane = this.lanes.get(callId); - const record = this.callStore.lookup(callId); - if ( - lane === undefined || - lane.laneState !== 'opening' || - lane.promptInFlight || - record?.kind !== 'agent' || - record.completion !== null - ) return false; - - const dispatchIndex = this.dispatchQueue.findIndex((item) => - item.kind === 'dispatch' ? item.callId === callId : item.entry.id === callId, - ); - if (dispatchIndex >= 0) this.dispatchQueue.splice(dispatchIndex, 1); - if (this.openingCalls.has(callId)) this.stoppedOpens.add(callId); - this.callStore.recordCancelled(callId, now()); - this.settleStored(callId, 'reject', this.cancelledTurnValue(callId)); - this.agentSlots.delete(callId); - lane.activeTurnId = null; - lane.laneState = 'fatal'; - this.failQueuedTurns( - callId, - executionError( - `session ${callId}: founding turn was cancelled by ${source} before a reusable session was established`, - 'founding_turn_cancelled_before_session', - false, - ), - ); - this.scheduleAdmissions(); - if (drain) { - try { - this.drain(); - this.provenancePass('settlement', [callId]); - this.sink?.boundary('settlement'); - } catch (error) { - if (!(error instanceof DrainJobError)) throw error; - this.retainedDrainError = { name: error.info.name, message: error.info.message, atMs: now() }; - this.sink?.boundary('settlement'); - } - } - return true; - } - - private watchSessionRelease(entry: SessionEntry): void { - const released = entry.session.released?.(); - if (released === undefined) return; - void released.then(() => { - const lane = this.lanes.get(entry.callId); - if ( - lane === undefined || - lane.laneState === 'released' || - lane.laneState === 'fatal' || - this.sessions.get(entry.callId) !== entry || - this.draining || - this.disposed - ) return; - this.markLaneFatal( - entry.callId, - executionError(`session ${entry.callId}: ACP session was released`, 'session_released', false), - ); - }); - } - - /** Set the entry's cancel flag and wake its waiters (the - * non-re-armable settlement wait's cancel signal — a held re-attach - * call is settled as the recoverable `AGENT_CANCELLED` the moment a - * cancel lands, never left pending until the drain). Idempotent; - * waiters are one-shot and removed on fire. */ - private markCancelled(entry: SessionEntry): void { - if (entry.callCancelled) return; - entry.callCancelled = true; - for (const wake of [...entry.cancelWaiters]) { - entry.cancelWaiters.delete(wake); - wake(); - } - } - - private failQueuedTurns(sessionId: string, error: unknown): void { - const lane = this.lanes.get(sessionId); - if (lane === undefined) return; - lane.laneState = 'fatal'; - const value = toRejectionValue(error); - for (const callId of [...lane.queuedTurnIds]) { - const turn = this.queuedTurns.get(callId); - if (turn === undefined || turn.state === 'settled') continue; - turn.state = 'settled'; - turn.cancelRequested = true; - this.queueSlots.delete(callId); - this.settleStored(callId, 'reject', value); - } - lane.queuedTurnIds = []; - if (lane.activeTurnId !== null && this.queuedTurns.has(lane.activeTurnId)) { - lane.activeTurnId = null; - lane.promptInFlight = false; - } - } - - /** Fatal containment when the mandatory store itself cannot record settlements. */ - private markPersistenceFatal(sessionId: string, failure: unknown): void { - const lane = this.lanes.get(sessionId); - if (lane === undefined) return; - lane.laneState = 'fatal'; - if (lane.cancellationTimer !== null) clearTimeout(lane.cancellationTimer); - lane.cancellationTimer = null; - const ids = [ - ...(lane.activeTurnId !== null ? [lane.activeTurnId] : []), - ...lane.queuedTurnIds, - ...lane.steeringControlIds, - ]; - for (const callId of ids) { - try { this.settleIntoGuest(callId, 'reject', failure); } catch { /* store is already unusable */ } - const queued = this.queuedTurns.get(callId); - if (queued !== undefined) queued.state = 'settled'; - this.agentSlots.delete(callId); - this.queueSlots.delete(callId); - } - lane.activeTurnId = null; - lane.promptInFlight = false; - lane.queuedTurnIds = []; - lane.steeringControlIds = []; - const entry = this.sessions.get(sessionId); - this.sessions.delete(sessionId); - if (entry !== undefined) void Promise.resolve(entry.session.release()).catch(() => undefined); - } - - private markLaneFatal(sessionId: string, error: unknown): void { - if (this.disposed) return; - const existing = this.lanes.get(sessionId); - if (existing?.laneState === 'fatal') return; - const lane = existing ?? this.newLane('usable'); - lane.laneState = 'fatal'; - if (lane.cancellationTimer !== null) clearTimeout(lane.cancellationTimer); - lane.cancellationTimer = null; - lane.cancellingTurnId = null; - this.lanes.set(sessionId, lane); - const value = toRejectionValue(error); - const activeTurnId = lane.activeTurnId; - if (activeTurnId !== null && this.callStore.lookup(activeTurnId)?.completion === null) { - this.settleStored(activeTurnId, 'reject', value); - } - if (activeTurnId !== null) { - this.agentSlots.delete(activeTurnId); - this.queueSlots.delete(activeTurnId); - const activeQueue = this.queuedTurns.get(activeTurnId); - if (activeQueue !== undefined) activeQueue.state = 'settled'; - } - for (const controlId of lane.steeringControlIds) { - if (this.callStore.lookup(controlId)?.completion === null) { - this.settleStored(controlId, 'reject', value); - } - } - lane.steeringControlIds = []; - lane.steeringInFlight = false; - this.failQueuedTurns(sessionId, error); - lane.activeTurnId = null; - lane.promptInFlight = false; - const entry = this.sessions.get(sessionId); - if (entry !== undefined) { - this.sessions.delete(sessionId); - void Promise.resolve(entry.session.release()).catch(() => undefined); - } - this.scheduleAdmissions(); - } - - // ── The settlement pump ─────────────────────────────────────────────── - - private async pumpUnlocked(boundDeadlineMs?: number): Promise<{ settled: string[]; drainError?: DrainJobError }> { - this.assertAlive(); - const settled: string[] = []; - for (;;) { - const ready = [...this.inFlight.values()].filter((t) => t.done); - const readySleeps = [...this.sleepCalls.entries()].filter(([, task]) => task.done); - if (ready.length === 0 && readySleeps.length === 0) break; - for (const [sleepKey, sleepTask] of readySleeps) { - // A `sleep(ms)` settlement (the host timer fired): settle the - // guest call directly (no store record — sleeps are never - // registry calls), then one drain + provenance pass like every - // settled call. A drain failure keeps the already-settled ids - // and reports the error like the agent/steer arm. - this.sleepCalls.delete(sleepKey); - try { - sleepTask.call.resolve(undefined); - } catch { - // The call was already settled (first-wins) — nothing to do. - } - try { - this.drain(boundDeadlineMs); - this.provenancePass('settlement'); - this.sink?.boundary('settlement'); - } catch (error) { - if (error instanceof DrainJobError) { - this.sink?.boundary('settlement'); - this.retainedDrainError = { name: error.info.name, message: error.info.message, atMs: now() }; - this.sweepActiveEvals(); - return { settled, drainError: error }; - } - throw error; - } - } - for (const entry of ready) { - const outcome: { outcome: 'resolve' | 'reject' | 'hold'; value: unknown } = await entry.promise; - if (outcome.outcome === 'hold') { - // The drain/disposal fences only (phase-F review: the re-attach - // arm's unobservable-turn degradation was deleted — a - // non-re-armable seam rejection and a missing seam now re-issue - // under the same call id, the doc's honest fallback): the - // in-flight entry is dropped WITHOUT recording or settling — - // either the client-presence drain's forced stop already settled - // the call DURABLY (recorded + guest-settled at the bound), or - // the broker was disposed and the state owning the call is being - // torn down. The condition was surfaced guest-visibly by the - // task. ALSO the queue-scheduler shape (a restored queued - // queued turn's delivery scheduler and the lazy re-attach arm's - // cap-queue): the hold entry's call id is REUSED by the - // delivery task that starts once a slot frees — the drop must - // only remove the map entry when it still holds THIS task (the - // review probe: the stale hold arm deleted the freshly tracked - // delivery task by id, leaving a completed delivery turn with - // no recorded completion and a pending guest promise). - if (this.inFlight.get(entry.callId) === entry) { - this.inFlight.delete(entry.callId); - } - continue; - } - try { - // Narrowed past the `hold` branch above: the pump only ever - // delivers resolve/reject outcomes (hold drops the entry). - this.deliver(entry.callId, outcome as { outcome: 'resolve' | 'reject'; value: unknown }); - } catch (error) { - // The outcome stays IN FLIGHT (its readiness flag is already - // set) — the next pump retries. Both the store write and the - // guest settlement are first-wins idempotent, so the retry - // settles exactly once. The failure propagates to the caller: - // a store IO failure is a host-side failure, not a guest - // outcome. - throw error; - } - this.inFlight.delete(entry.callId); - if (entry.kind === 'agent' || entry.kind === 'queue') { - this.onPublicTurnSettled(entry.callId, entry.kind); - } else if (entry.kind === 'steer') { - this.onSteeringSettled(entry.callId); - } - settled.push(entry.callId); - // ONE drain + ONE provenance pass PER SETTLED CALL (phase-D - // review round 5: the pump used to deliver every simultaneously - // ready call and then run one drain + one provenance pass - // labelled with all their ids — two independent continuations - // producing separate bindings were both attributed to both - // workers/tasks, violating the doc's per-value producer/task - // provenance). Each call's continuation drain is attributed to - // THAT call alone (`worker c1`, not `worker c1+c2`), and each - // drain that changed VM state fires the state-changing boundary - // (the sink's burst debounce coalesces the batch into one write - // at the operation's flush). A drain failure stops the pump with - // the already-settled ids still reported (review regression: an - // interrupted continuation used to erase every id the pump had - // settled). - try { - // The drain itself performs the interrupted-drain release - // when it ran a tracked eval's continuation (the interrupted - // job's continuation lease — see `releaseInterruptedEval`). - this.drain(boundDeadlineMs); - this.provenancePass('settlement', [entry.callId]); - this.sink?.boundary('settlement'); - } catch (error) { - if (error instanceof DrainJobError) { - // The delivery happened; the continuation drain failed. The - // settled call ids must NOT be lost with the error (review - // regression: an interrupted continuation used to erase - // every id the pump had settled) — the caller reports them - // alongside the drain-failure line. The state-changing - // boundary still fires: the settlement landed (the VM - // changed) and the operation-end flush must have a dirty - // boundary to persist it. - this.sink?.boundary('settlement'); - // §6.2: the drain error DEMOTES to workspace().diagnostics - // (retained; the eval surface reports it only as a line in - // the next result). - this.retainedDrainError = { name: error.info.name, message: error.info.message, atMs: now() }; - // A suspended eval's continuation may still have completed - // before the drain failure — the sweep reads any settled - // completion into `_` and releases it. - this.sweepActiveEvals(); - return { settled, drainError: error }; - } - throw error; - } - } - } - // The pump's drains may have completed suspended evals (their - // continuations resumed by the deliveries): the sweep reads the - // settled values into `_` right here, so a `pump()` returns with - // the result history already updated (the §4.4 seam — `await - // sleep(10); 42`, pump, then `_` reads `42`). - this.sweepActiveEvals(); - return { settled }; - } - - private onPublicTurnSettled(callId: string, kind: 'agent' | 'queue'): void { - const queued = kind === 'queue' ? this.queuedTurns.get(callId) : undefined; - const sessionId = queued?.sessionId ?? callId; - const lane = this.lanes.get(sessionId); - const entry = this.sessions.get(sessionId); - if (kind === 'agent') { - this.agentSlots.delete(callId); - if (entry !== undefined) entry.callSettled = true; - } else { - this.queueSlots.delete(callId); - if (queued !== undefined) queued.state = 'settled'; - if (lane !== undefined) lane.queuedTurnIds = lane.queuedTurnIds.filter((id) => id !== callId); - } - if (lane !== undefined) { - lane.promptInFlight = false; - if (lane.activeTurnId === callId) lane.activeTurnId = null; - if (lane.cancellingTurnId === callId) lane.cancellingTurnId = null; - if (lane.cancellationTimer !== null) clearTimeout(lane.cancellationTimer); - lane.cancellationTimer = null; - } - if (entry !== undefined) entry.callCancelled = false; - this.processSteeringControls(sessionId); - this.scheduleAdmissions(); - } - - private onSteeringSettled(callId: string): void { - const control = this.steeringControls.get(callId); - if (control === undefined) return; - const lane = this.lanes.get(control.sessionId); - if (lane !== undefined) { - lane.steeringControlIds = lane.steeringControlIds.filter((id) => id !== callId); - lane.steeringInFlight = false; - } - this.steeringControls.delete(callId); - this.processSteeringControls(control.sessionId); - this.scheduleAdmissions(); - } - - /** Record → settle → consume for one ready outcome (the exactly-once - * discipline). The store write happens BEFORE the guest settlement; - * on a crash between the two, the next delivery (or the restore's - * reconcile) settles from the store — never twice. When the store - * ALREADY holds a first completion (a re-delivery after a crash, or a - * second pump attempt), the guest settles with the STORE's completion, - * never the newer host outcome — the store's first completion is the - * authority and the guest must never see a different value than the - * store records (review regression: the newer live outcome used to - * settle the guest while the store kept the first, leaving them - * disagreeing). */ - private deliver( - callId: string, - outcome: { outcome: 'resolve' | 'reject'; value: unknown }, - ): void { - const newlyRecorded = this.recordCompletion(callId, { - outcome: outcome.outcome, - value: outcome.value, - completedAtMs: now(), - }); - if (!newlyRecorded) { - const record = this.callStore.lookup(callId); - const completion = record?.completion; - if (completion === null || completion === undefined) { - throw new Error(`Broker: store lost the recorded completion for ${callId}`); - } - this.settleIntoGuest(callId, completion.outcome, completion.value); - return; - } - this.settleIntoGuest(callId, outcome.outcome, outcome.value); - } - - /** Settle one call into the guest: through its live deferred when this - * broker issued it, through the reconciliation surface otherwise - * (the restored-broker route). Both converge on the guest's - * idempotent first-wins settle. Returns whether the guest entry was - * newly settled (a no-op replay of an already-settled id reports - * false — the changed-VM bookkeeping's source of truth). */ - private settleIntoGuest(callId: string, outcome: 'resolve' | 'reject', value: unknown): boolean { - const call = this.deferreds.get(callId); - this.deferreds.delete(callId); - if (call !== undefined) { - if (outcome === 'resolve') call.resolve(value); - else call.reject(value); - return true; - } - const surface = this.workspace.surface(); - if (surface === undefined) { - throw new Error(`Broker: cannot settle ${callId} — the guest surface is not installed`); - } - return surface.settle(callId, outcome, value); - } - - /** One settlement drain with the broker's interrupt handler armed (a - * continuation resumed by settlement cannot run away unguarded). The - * drain also runs under the per-eval wall-clock deadline (a - * settlement drain can itself resume a runaway continuation — it - * gets the same bound) and, when the drain is the client-presence - * teardown's pump, under the REMAINING disconnect bound - * (`boundDeadlineMs` — phase-D review round 6: the outer drain bound - * is absolute, so a settlement that resumed a runaway continuation - * near the disconnect deadline can never exceed the session-eviction - * TTL). - * - * The drain carries the per-job CONTINUATION-LEASE plumbing (see - * `jobLease`): the drain loop mirrors the guest lease per job, and - * the eval-break signal's handler fires only while the executing job - * holds an armed token — the executing job IS the armed eval's - * continuation segment (phase-E review rounds 3/5: the carried - * defect's drainInterruptHandler fired on every later drain - * regardless of which continuation it was actually executing). A - * drain interrupted while running a TRACKED eval's continuation - * releases exactly the tracked eval holding the interrupted job's - * token here (the `releaseInterruptedEval` gate); an unrelated - * interrupted drain releases nothing and leaves the eval-break armed - * state intact. */ - private drain(boundDeadlineMs?: number): void { - // The OUT-OF-BAND probe's execution marker: this drain began now (a - // break armed mid-drain breaks the running job — an interrupt lands - // promptly mid-wait). - this.currentExecutionStartSeq = this.evalBreakChannel?.executionStartMarker() ?? 0; - const boundHandler = - boundDeadlineMs === undefined ? undefined : () => Date.now() >= boundDeadlineMs; - try { - this.workspace.drainJobs({ - // The eval-break signal rides ONLY the settlement drains (see - // `evalBreakHandler`): a fresh eval's own code and its own job - // drain never consult it (phase-E review rejection — the armed - // signal used to be the broker's DEFAULT eval handler, so an - // unrelated eval consumed it before the intended continuation). - // The OUT-OF-BAND probe rides BOTH (see `evalBreakProbe`): a - // synchronously running eval OR drain is exactly the blocked- - // main-thread case the worker channel exists for. - interruptHandler: this.composedInterrupt( - this.evalBreakProbe(), - this.interruptHandler, - this.evalBreakHandler(), - boundHandler, - ), - // The per-job continuation-lease plumbing (see `jobLease`): the - // drain loop reads the guest lease before each job into the - // mirror and clears it after a lease-carrying job — the - // eval-break signal's firing identity and the interrupted-drain - // release decision. - jobLease: this.jobLease, - }); - } catch (error) { - if (error instanceof DrainJobError) { - // The interrupted drain RAN a tracked suspended eval's - // continuation (the interrupted job's continuation lease — see - // `releaseInterruptedEval`): the continuation is broken and its - // wrapper never settles — release the intersecting tracked - // eval NOW. An unrelated interrupted drain releases nothing: - // the armed state and the tracked evals stay intact (phase-E - // review round 3's carried defect). - this.releaseInterruptedEval(); - } - throw error; - } - } - - /** The per-eval wall-clock deadline interrupt (the harness's eval - * guard; see `BrokerOptions.evalTimeoutMs`): a fresh handler per - * operation, interrupting once the operation exceeds the budget. - * Returns `undefined` when the deadline is disabled (`0`/`null`), so - * the caller falls back to no handler. */ - private deadlineHandler(): (() => boolean) | undefined { - if (!(this.evalTimeoutMs > 0)) return undefined; - const deadline = Date.now() + this.evalTimeoutMs; - return () => Date.now() >= deadline; - } - - /** One maintenance pass of the per-binding provenance registry after a - * guest-entering operation (the workspace manifest's provenance seam; - * see `provenance.ts`). Orientation metadata only — never throws. */ - private provenancePass(origin: 'eval' | 'settlement', callIds: string[] = []): void { - try { - this.workspace.provenanceRecord( - origin === 'eval' ? { kind: 'eval' } : { kind: 'settlement', callIds }, - ); - } catch { - // Orientation metadata: a failing pass must never break the - // operation that triggered it. - } - } - - // ── Eval + rendering ────────────────────────────────────────────────── - - /** Run the eval (with the rejection bridge armed) and read the - * completion. The broker-level interrupt handler is the DEFAULT for - * evals too: a direct eval that runs away must be bounded even when - * the caller supplies no per-eval handler (review regression: the - * configured handler used to apply only to settlement drains, so a - * runaway eval could hang the workspace indefinitely). A caller's - * per-eval handler still overrides it. The per-eval wall-clock - * deadline ALWAYS applies on top (phase-D review round 2: the - * interrupt tool's armed signal alone could only break the NEXT - * execution, because a synchronous runaway eval blocks the event loop - * — the deadline makes the CURRENTLY running eval always breakable - * through the quickjs interrupt handler, even when an explicit signal - * handler is armed and unset). */ - private runEval(code: string, options: ReplEvalOptions): { outcome: ReplEvalOutcome; completion?: unknown; interruptedInDrain?: boolean } { - // The OUT-OF-BAND probe's execution marker: this eval's code phase - // began now — a break armed after this instant breaks THIS eval; a - // stale flag (armed before) is dropped on first observation. - this.currentExecutionStartSeq = this.evalBreakChannel?.executionStartMarker() ?? 0; - // The continuation-lease mirror starts clean: the code phase must - // never read a token a PREVIOUS drain's last lease-carrying job - // left in the mirror (the reset handler's continuation attribution - // reads it — a code-phase reset() must read undefined and be - // attributed by the eval op's own snapshot, never by a stale - // token). The drain phases re-set the mirror per job. - this.jobLease.cell.current = undefined; - this.inRunEval = true; - try { - return this.runEvalInner(code, options); - } finally { - this.inRunEval = false; - } - } - - /** The `runEval` body (see above): the `inRunEval` marker frames the - * whole execution — code phase and own drain alike. */ - private runEvalInner(code: string, options: ReplEvalOptions): { outcome: ReplEvalOutcome; completion?: unknown; interruptedInDrain?: boolean } { - // The eval's CONTINUATION TOKEN (phase-E review round 5): minted - // per eval, embedded in the instrumented code's `__replAwait(value, - // token)` calls (see `await-instrument.ts`), and attributed to the - // completion wrapper when the eval suspends (`eval()`). The guest - // library's wrap-settling reaction sets the continuation lease to - // this token immediately before the eval's continuation segment — - // the eval-break signal's genuine continuation identity. The token - // is minted even when the library lacks the lease surface (the arm - // refuses then); the attribution is harmless. - const token = `e${++this.evalTokenSeq}`; - this.lastEvalToken = token; - // The top-level-await instrumenter (see `await-instrument.ts`): - // rewrites the eval's top-level `await x` into - // `await (x, TOKEN)` so the guest library can wrap - // the awaited value — the continuation-lease seam (phase-E review - // round 5). Gated on the workspace's library carrying the 0.3.1+ - // lease surface (version-gated — the 0.3.0 copy's lease-set - // ordering carries the sibling-reaction defect, phase-E review - // rejection round 7): a restored snapshot with the 0.1.0/0.2.0/0.3.0 - // library is served as-is and simply gets no instrumentation (the - // interrupt degrades to the honest refusal — the 0.2.0 log-only - // targeting is the rejected settled-call-ids identity). - const instrumented = this.continuationLeaseAvailable() - ? instrumentTopLevelAwaits(code, token, { wrapIterables: this.iterableLeaseAvailable() }) - : code; - const result = this.workspace.evalWithCompletion(instrumented, { - ...options, - // The per-eval handler overrides the broker-level default (the - // documented contract); the per-eval wall-clock deadline ALWAYS - // composes on top (phase-D review round 2: a currently-running - // runaway eval is always breakable). - interruptHandler: this.composedInterrupt( - this.evalBreakProbe(), - options.interruptHandler ?? this.interruptHandler, - ), - // The eval-break signal rides the eval's OWN DRAIN as well - // (phase-E review rejection round 2: the signal used to be - // consulted only by settlement drains, but a suspended eval's - // continuation can be resumed by a SYNCHRONOUS host-callback - // settlement — `checkpoint.answer` in a later eval — and that - // execution runs inside the answering eval's own drain, where the - // old signal was blind: the runaway continuation burned the eval - // deadline instead of being broken by the interrupt). The eval's - // own CODE still never consults the signal — an unrelated eval's - // code is never broken by it (the phase-E review rejection's - // targeting discipline). - drainInterruptHandler: this.evalBreakHandler(), - // The per-job continuation-lease plumbing: the drain loop mirrors - // the guest lease per job, and the handler above fires only while - // the mirror holds an armed token (the executing job IS the armed - // eval's continuation segment). - jobLease: this.jobLease, - rejectionBridge: true, - }); - return result; - } - - /** Whether the workspace's guest library carries the 0.3.1+ - * CONTINUATION-LEASE surface — the corrected lease ordering (the - * eval-break targeting seam). VERSION-GATED (phase-E review - * rejection round 7): the 0.3.0 copy reports 'supportsContinuationLease: - * true' but its lease-setting reaction still runs on the awaited - * VALUE's settlement — the carried sibling-reaction interrupt- - * targeting defect (a sibling 'q.then' registered after the target's - * await runs between the lease set and the continuation, consumes the - * armed signal, and the target runs later unprotected). Accepting the - * flag alone would re-arm the original defect on a restored 0.3.0 - * snapshot; the version gate refuses it: a restored 0.3.0 workspace is - * served as-is, its awaits are left UNINSTRUMENTED (native semantics), - * and the eval-break interrupt degrades to the honest refusal. 0.3.1 - * is the first copy with the corrected ordering (its wrapper reaction - * rides the wrapper promise itself, immediately before the await - * machinery's own — the lease is associated with the actual - * continuation job), so it passes the gate. A restored snapshot with - * the 0.1.0/0.2.0 library is refused by the flag itself (the 0.2.0 - * log-only targeting is the rejected settled-call-ids identity). - * Cached per check: the library never changes within a broker's - * lifetime (restore keeps the snapshot's copy). */ - private continuationLeaseAvailable(): boolean { - if (this.leaseCapabilityCached !== undefined) return this.leaseCapabilityCached; - try { - const surface = this.workspace.surface(); - this.leaseCapabilityCached = - surface?.supportsContinuationLease === true && - guestVersionAtLeast(surface.version, '0.3.1'); - } catch { - this.leaseCapabilityCached = false; - } - return this.leaseCapabilityCached; - } - - /** Whether the workspace's guest library carries the 0.3.1 - * ITERABLE-LEASE surface (`__replAwaitIterable` — the for-await - * iterable wrap that preserves the iterable protocol while setting - * the continuation lease per iteration). The instrumenter's - * for-await sites are gated on this: a restored snapshot carrying - * the 0.3.0 library (whose for-await wrap returned a promise and - * broke every `for await` loop — phase-E review rejection round 6) - * is served as-is, its for-await sites are left UNWRAPPED, and the - * loops run natively (no mid-loop eval-break targeting — the honest - * degradation). Cached per check like `continuationLeaseAvailable`. - */ - private iterableLeaseAvailable(): boolean { - if (this.iterableLeaseCapabilityCached !== undefined) return this.iterableLeaseCapabilityCached; - try { - const surface = this.workspace.surface(); - this.iterableLeaseCapabilityCached = surface?.supportsIterableLease === true; - } catch { - this.iterableLeaseCapabilityCached = false; - } - return this.iterableLeaseCapabilityCached; - } - - /** Compose the per-operation interrupt handlers with the per-eval - * wall-clock deadline: any handler returning true interrupts. The - * deadline is the last resort — a runaway operation is ALWAYS - * breakable (see `BrokerOptions.evalTimeoutMs`; `0`/`null` disables - * it). */ - private composedInterrupt(...handlers: Array<(() => boolean) | undefined>): () => boolean { - const live = handlers.filter((handler): handler is () => boolean => handler !== undefined); - const deadline = this.deadlineHandler(); - return () => { - for (const handler of live) { - if (handler()) return true; - } - return deadline !== undefined && deadline(); - }; - } - - /** The OUT-OF-BAND eval-break probe (phase-F review round 2; see - * `BrokerOptions.evalBreakChannel` and `eval-break-channel.ts`): - * consumes the channel's break flag for this workspace and breaks - * the executing eval when the flag was armed after the execution - * began (the arm-after-start rule — a stale flag never breaks a - * later eval; it is consumed-and-dropped on first observation). - * Composed into every execution: a fresh eval's own code (the - * `while (true)` case — the daemon's main thread is blocked, the - * worker armed the flag, and the quickjs interrupt handler breaks - * the eval mid-run) and the settlement drains (an interrupt lands - * promptly mid-wait). `undefined` when no channel is wired. */ - private evalBreakProbe(): (() => boolean) | undefined { - const channel = this.evalBreakChannel; - if (channel === undefined) return undefined; - const projectDir = this.workspace.projectDir; - return () => { - if (channel.consumeBreak(projectDir, this.currentExecutionStartSeq)) { - this.outOfBandBreakCount++; - this.lastOutOfBandBreakAtMs = Date.now(); - return true; - } - return false; - }; - } - - /** The interrupt tool's honest-outcome record: the moment an - * out-of-band break was CONSUMED by a running eval, consumed on - * read (one request → one report — a later interrupt can never - * inherit an earlier break's delivery record). Null when no - * out-of-band break was delivered since the last read. The daemon - * reads it AFTER `armEvalBreak` (the eval's chain hold releases - * before the interrupt's processing — the break already happened - * by then, which is exactly what the record reports). */ - consumeOutOfBandBreakReport(): number | null { - const at = this.lastOutOfBandBreakAtMs; - this.lastOutOfBandBreakAtMs = null; - return at; - } - - /** Render the tool-result shape: output lines (console events drained - * from the buffer, then the pump-drain error line when one occurred, - * then the eval's own error line when it threw), the previewed - * result, pending ids, checkpoints, completed ids. */ - private render( - outcome: ReplEvalOutcome, - completion: unknown, - completed: string[], - ): ReplEvalResult { - const lines: string[] = []; - for (const event of this.consoleBuffer.splice(0)) { - lines.push(this.renderConsoleEvent(event)); - } - // §6.2: a retained settlement-drain failure is demoted to - // workspace().diagnostics — it no longer renders as an output line - // (losses are surfaced by the tool's one-line notice instead). - if (outcome.kind === 'error') { - lines.push(errorLine(outcome.error)); - } - // §7: the engine applies NO output caps to guest output — the lines - // ship verbatim (the Python posture). - const result: ReplEvalResult = { - output: lines, - kind: outcome.kind, - evalToken: this.lastEvalToken, - pending: this.pendingIds(), - checkpoints: this.checkpointSummaries(), - completed, - }; - if (outcome.kind === 'value' && completion !== undefined) { - try { - result.result = renderCompletionLine(completion); - } finally { - (completion as JSValueHandle).dispose(); - } - } - return result; - } - - /** One console event → its guest-rendered line (non-log levels - * prefixed `warn:`/`error:`/…). */ - private renderConsoleEvent(event: { level: string; line: string }): string { - const prefix = event.level === 'log' ? '' : `${event.level}: `; - return `${prefix}${event.line}`; - } - - /** The pending call ids, in registry order. */ - private pendingIds(): string[] { - return this.workspace.surface()?.pending().map((entry) => entry.id) ?? []; - } - - /** The raised-checkpoint summaries, previewed — the `status` seam's - * checkpoint surface (the same bounded form the eval/wait result's - * `checkpoints` field carries: questions previewed through the - * top-level string rule, never unbounded guest text). */ - checkpointSummaries(): CheckpointSummary[] { - return [...this.checkpoints.values()].map((c) => ({ - id: c.callId, - // The §4.3 double-JSON-quote fix: the question renders as PLAIN - // head+tail metadata text (the retained 200-char metadata preview, - // §7) — never a JSON-stringified quoted form. - question: headTailDescription(c.question, 200), - })); - } - - // ── Options validation ──────────────────────────────────────────────── - - /** Parse + validate the agent options bag (the exact surface is the - * module docs' table). Throws a `WorkflowError` for every invalid - * shape — the caller refuses the call with it. */ - private parseAgentOptions(optionsJson: string | null): ParsedAgentOptions { - if (optionsJson === null) return {}; - let raw: unknown; - try { - raw = JSON.parse(optionsJson); - } catch (error) { - throw new WorkflowError(`agent options are not valid JSON: ${(error as Error).message}`, CODE.SCRIPT_VALIDATION_ERROR, { - recoverable: false, - }); - } - if (typeof raw !== 'object' || raw === null || Array.isArray(raw)) { - throw new WorkflowError('agent options must be an object', CODE.SCRIPT_VALIDATION_ERROR, { - recoverable: false, - }); - } - const opts = raw as Record; - for (const key of Object.keys(opts)) { - if (!AGENT_OPTION_KEYS.has(key)) { - // §4.1: an unknown option key rejects synchronously, and the - // error lists the valid keys (the enumerated teaching error). - throw new WorkflowError( - `agent options: unknown option "${key}" (valid options: ${AGENT_OPTION_KEYS_TEXT})`, - CODE.SCRIPT_VALIDATION_ERROR, - { recoverable: false }, - ); - } - } - const parsed: ParsedAgentOptions = {}; - if (opts.schema !== undefined) { - if (typeof opts.schema !== 'object' || opts.schema === null || Array.isArray(opts.schema)) { - throw new WorkflowError('agent options: "schema" must be a JSON Schema object', CODE.SCRIPT_VALIDATION_ERROR, { - recoverable: false, - }); - } - parsed.schema = opts.schema as Record; - } - if (opts.cwd !== undefined) { - if (typeof opts.cwd !== 'string' || opts.cwd.length === 0 || !isAbsolute(opts.cwd)) { - throw new WorkflowError('agent options: "cwd" must be an absolute path string', CODE.SCRIPT_VALIDATION_ERROR, { - recoverable: false, - }); - } - parsed.cwd = opts.cwd; - } - if (opts.configOptions !== undefined) { - parsed.configOptions = requireStringBoolRecord(opts.configOptions, 'configOptions'); - } - if (opts.mode !== undefined) parsed.mode = requireString(opts.mode, 'mode'); - return parsed; - } - - /** Parse + validate a steering payload's options bag (exactly - * `{ promptMeta }`). */ - private parseSteerOptions(options: unknown): Record { - if (typeof options !== 'object' || options === null || Array.isArray(options)) { - throw new WorkflowError('steer options must be an object', CODE.SCRIPT_VALIDATION_ERROR, { - recoverable: false, - }); - } - const opts = options as Record; - for (const key of Object.keys(opts)) { - if (!STEER_OPTION_KEYS.has(key)) { - throw new WorkflowError(`steer options: unknown option "${key}"`, CODE.SCRIPT_VALIDATION_ERROR, { - recoverable: false, - }); - } - } - if (opts.promptMeta !== undefined) opts.promptMeta = requireRecord(opts.promptMeta, 'promptMeta'); - return opts; - } - - private parseTurnPayload( - payloadJson: string | null, - kind: 'queue' | 'steer', - ): { prompt: string; promptMeta?: Record } { - if (payloadJson === null) { - throw new WorkflowError(`${kind} payload is missing`, CODE.SCRIPT_VALIDATION_ERROR, { - recoverable: false, - }); - } - let payload: unknown; - try { - payload = JSON.parse(payloadJson); - } catch (error) { - throw new WorkflowError(`${kind} payload is not valid JSON: ${(error as Error).message}`, CODE.SCRIPT_VALIDATION_ERROR, { - recoverable: false, - }); - } - if (!isPlainObject(payload)) { - throw new WorkflowError(`${kind} payload must be an object`, CODE.SCRIPT_VALIDATION_ERROR, { - recoverable: false, - }); - } - if (typeof payload.prompt !== 'string') { - throw new WorkflowError(`${kind}(prompt, options?) requires a string prompt`, CODE.SCRIPT_VALIDATION_ERROR, { - recoverable: false, - }); - } - const options = payload.options; - if (options === undefined || options === null) return { prompt: payload.prompt }; - const parsedOptions = this.parseSteerOptions(options); - return { - prompt: payload.prompt, - ...(parsedOptions.promptMeta !== undefined - ? { promptMeta: parsedOptions.promptMeta as Record } - : {}), - }; - } - - private newLane(laneState: SessionLaneState['laneState']): SessionLaneState { - return { - activeTurnId: null, - promptInFlight: false, - queuedTurnIds: [], - steeringControlIds: [], - steeringInFlight: false, - laneState, - cancellingTurnId: null, - cancellationTimer: null, - }; - } - - // ── Misc ────────────────────────────────────────────────────────────── - - /** Serialize async broker operations (eval/pump/reconcile/dispose) — - * two overlapping tool calls must never interleave settlement - * bookkeeping or the eval's pump-before-eval ordering. - * - * `deadline` (the client-presence drain's and the disposal's ABSOLUTE - * bound) races the CHAIN WAIT itself (phase-D review round 8: the - * chain acquisition used to be awaited with no deadline, so a - * YIELDFUL queued operation — a long `wait` op polling a pending - * call, anything async that never resolves — could delay the drain - * and the teardown indefinitely past their bounds, exactly the - * unbounded-wait family the outer bound exists to kill). When the - * deadline expires while queued, `fn` runs WITHOUT the chain: the - * chain is, by definition, stuck past the caller's absolute ceiling, - * and the caller's body is safe unlocked because it is first-wins - * (the store's first-completion authority) and generation-fenced - * against the stuck op's eventual landing — the same protections the - * drain's late-landing paths already rely on. A deadline already - * past at call time skips straight to the unlocked run. When the - * chain frees within the bound, `fn` runs INSIDE it exactly as - * before (subsequent ops queue behind it). - * - * The enqueue is ATOMIC with a changed-chain re-check: the race - * resolves when the chain promise captured AT RACE TIME settles, but - * other microtasks run between that resolution and this continuation - * — an op enqueued in that window chains onto the just-released - * chain and REPLACES `this.opChain` (the no-deadline path enqueues - * synchronously, so it can land exactly there). Re-reading the - * mutable field after the await would enqueue behind the NEW chain - * with no deadline race on it (the review-rejected round-8 code did - * exactly this: a 20 ms drain took 307 ms behind an op queued as the - * prior chain released). So the post-race path re-checks the field - * and, when it changed, re-races the new chain against the REMAINING - * time — each loop pass only consumes remaining budget, so the total - * wait can never exceed the deadline plus timer slop no matter how - * many ops enqueue around a release; and the check-and-assign when - * the field is unchanged run in ONE synchronous block, so no op can - * interleave between them. */ - private async serialized(fn: () => Promise, deadline?: number): Promise { - if (deadline === undefined) { - const run = this.opChain.then( - () => this.runSerialized(fn), - () => this.runSerialized(fn), - ); - this.opChain = run.then( - () => undefined, - () => undefined, - ); - return this.afterOp(run); - } - for (;;) { - const remaining = deadline - Date.now(); - if (remaining <= 0) { - // The deadline is already past: run WITHOUT the chain — waiting - // for it would exceed the absolute bound. - return this.afterOp(this.runSerialized(fn)); - } - const raced = this.opChain; - let timer: NodeJS.Timeout | undefined; - const bound = new Promise<'bound'>((resolve) => { - timer = setTimeout(() => resolve('bound'), remaining); - }); - const winner = await Promise.race([ - raced.then( - () => 'chain' as const, - () => 'chain' as const, - ), - bound, - ]); - if (timer !== undefined) clearTimeout(timer); - if (winner === 'bound') { - // The bound won the race: the chain is stuck past the deadline - // — run WITHOUT it and return at the bound (the caller's body - // is first-wins/generation-fenced, so the stuck op's eventual - // landing cannot interleave into a double settlement). - return this.afterOp(this.runSerialized(fn)); - } - if (this.opChain === raced) { - // The chain we raced is still the current one — enqueue onto it - // atomically (the check and the replacement share one - // synchronous block, so no operation can interleave between - // them). - const run = raced.then( - () => this.runSerialized(fn), - () => this.runSerialized(fn), - ); - this.opChain = run.then( - () => undefined, - () => undefined, - ); - return this.afterOp(run); - } - // The chain CHANGED while we awaited the race (an op enqueued in - // the microtasks between the chain's release and this continuation - // replaced it): re-race the new chain with the remaining time. - } - } - - /** - * After a serialized operation settles: run the owed reset() teardown - * (§4.5 — the guest's reset() tears the workspace down once the eval - * that called it completed). OUTSIDE the chain slot: the disposal - * acquires the chain itself, and the op's own promise has settled - * (its result shipped) before the teardown starts. A failing op still - * tears down (the eval completed with an error — the teardown is - * owed all the same). - */ - private async afterOp(p: Promise): Promise { - try { - return await p; - } finally { - await this.resetIfDue(); - } - } - - /** - * The reset() teardown owed by a completed eval (`resetDue`, flipped - * by the sweep or by `eval` for an in-call completion): dispose the - * broker's children (bounded, generation-fenced), then the workspace - * VM — the host-side effect the deleted `reset` action performed; the - * daemon's next touch creates a fresh workspace. Idempotent: dispose - * is first-wins and the flags are consumed once. `boundMs` is - * forwarded to the disposal — the eval path passes an already-expired - * bound so the disposal body runs WITHOUT the chain (the eval op - * holds it); the post-op path uses the disposal's own default. - */ - private async resetIfDue(boundMs?: number): Promise { - if (!this.resetDue || this.disposed) return; - this.resetDue = false; - this.resetRequested = false; - this.resetOwningCompletions.clear(); - await this.dispose(boundMs); - this.workspace.dispose(); - } - - /** One serialized operation with the sink's end-of-burst flush: the - * boundaries fired inside the op are written (debounced) before the - * op's promise resolves — a kill after the op returns loses nothing. - * The active-eval sweep runs first: a suspended eval's completion - * can only settle inside an operation's drain, so the operation - * boundaries are exactly the moments the tracking can advance. */ - private async runSerialized(fn: () => Promise): Promise { - try { - // Every serialized operation waits for the workspace's eval-break - // slot ACK (phase-F review round 4): guest-executing operations - // must not start before the relay worker knows the workspace's - // key — an interrupt fired during the operation would 404 against - // an unapplied mapping. The ack resolves once (milliseconds after - // attach) and is a settled-promise microtask from then on; a dead - // channel rejects it and the swallow degrades to the documented - // per-eval deadline bound — never a hang. - await this.evalBreakReady.catch(() => undefined); - this.sweepActiveEvals(); - return await fn(); - } finally { - this.sink?.flush(); - } - } - - /** - * Race the serialization chain against an ABSOLUTE deadline: acquire - * the chain when it frees within the remaining budget, otherwise - * report `{ acquired: false }` — WITHOUT running the body (a - * VM-touching body must never execute while another operation is - * mid-flight: the chain is held by the operation itself, and running - * unlocked would re-enter the single-threaded engine). This is the - * `wait` tool's chain-contention bound (phase-E review round 4's - * carried defect: the wait enqueued onto the chain with no deadline, - * so a bounded wait queued behind a long eval took the eval's whole - * remaining run instead of returning at its bound). Unlike - * `serialized(fn, deadline)` — which runs the body WITHOUT the chain - * when the bound trips (the disconnect drain's choice, whose body is - * settlement-safe) — the caller here treats a failed acquisition as - * "the operation did not run": the wait reports still-running with - * the last observation it actually made. - */ - private async trySerialized( - fn: () => Promise, - deadline: number, - ): Promise<{ acquired: boolean; value?: T }> { - for (;;) { - const remaining = deadline - Date.now(); - if (remaining <= 0) { - // The deadline is already past: ONE IMMEDIATE acquisition - // attempt (phase-E review round 5's carried defect: the wait - // used to return unacquired right here, so a zero-timeout wait - // could not perform even an immediately available state read — - // an idle workspace reported "still running" and a pending - // call's surface read as empty). The chain is acquirable - // WITHOUT any wait when it is currently free: its settle - // continuation is a microtask, which runs before a zero timer - // (macrotask), so the race resolves 'chain' for a free chain - // and 'bound' for a busy one — nothing was waited for either - // way, the body just ran when the read was immediately - // available. A busy chain loses to the timer: unacquired, - // nothing ran (a VM-touching body must never execute while - // another operation is mid-flight). - const raced = this.opChain; - let timer: NodeJS.Timeout | undefined; - const zero = new Promise<'bound'>((resolve) => { - timer = setTimeout(() => resolve('bound'), 0); - }); - const winner = await Promise.race([ - raced.then( - () => 'chain' as const, - () => 'chain' as const, - ), - zero, - ]); - if (timer !== undefined) clearTimeout(timer); - if (winner === 'bound') { - // The chain is busy: do NOT run the body (it would touch the - // VM while the stuck operation is mid-flight). - return { acquired: false }; - } - if (this.opChain === raced) { - // The chain we raced is still the current one — enqueue onto it - // atomically (the check and the replacement share one - // synchronous block, so no operation can interleave between - // them). - const run = raced.then( - () => this.runSerialized(fn), - () => this.runSerialized(fn), - ); - this.opChain = run.then( - () => undefined, - () => undefined, - ); - return { acquired: true, value: await run }; - } - // The chain CHANGED while we awaited the race: re-race the new - // chain (still zero budget — the same one-attempt semantics). - continue; - } - const raced = this.opChain; - let timer: NodeJS.Timeout | undefined; - const bound = new Promise<'bound'>((resolve) => { - timer = setTimeout(() => resolve('bound'), remaining); - }); - const winner = await Promise.race([ - raced.then( - () => 'chain' as const, - () => 'chain' as const, - ), - bound, - ]); - if (timer !== undefined) clearTimeout(timer); - if (winner === 'bound') { - // The chain is stuck past the deadline: do NOT run the body (it - // would touch the VM while the stuck operation is mid-flight). - return { acquired: false }; - } - if (this.opChain === raced) { - // The chain we raced is still the current one — enqueue onto it - // atomically (the check and the replacement share one - // synchronous block, so no operation can interleave between - // them). - const run = raced.then( - () => this.runSerialized(fn), - () => this.runSerialized(fn), - ); - this.opChain = run.then( - () => undefined, - () => undefined, - ); - return { acquired: true, value: await run }; - } - // The chain CHANGED while we awaited the race (an op enqueued in - // the microtasks between the chain's release and this continuation - // replaced it): re-race the new chain with the remaining time. - } - } - - private assertAlive(): void { - if (this.disposed) { - throw new Error(`Broker for ${this.workspace.projectDir}: operation on a disposed broker`); - } - } -} - -// ──────────────────────────────────────────────────────────────────────── -// Module helpers -// ──────────────────────────────────────────────────────────────────────── - -function now(): number { - return Date.now(); -} - -function isPlainObject(value: unknown): value is Record { - if (typeof value !== 'object' || value === null || Array.isArray(value)) return false; - const prototype = Object.getPrototypeOf(value); - return prototype === Object.prototype || prototype === null; -} - -/** Read only the raw initialize metadata surface. The capabilities - * fallback is the current InteractiveSession carrier for that same raw - * initialize response; no derived capability boolean is consulted. */ -function initializeMetaOf(session: BrokerSession): Readonly> | undefined { - if (isPlainObject(session.initializeMeta)) return session.initializeMeta; - const capabilities = (session as BrokerSession & { capabilities?: unknown }).capabilities; - if (!isPlainObject(capabilities)) return undefined; - return isPlainObject(capabilities.initializeMeta) ? capabilities.initializeMeta : undefined; -} - -/** Exact raw steering advertisement parser. */ -function advertisesSteering(initializeMeta: unknown): boolean { - return ( - isPlainObject(initializeMeta) && - isPlainObject(initializeMeta.steering) && - initializeMeta.steering.supported === true - ); -} - -function persistenceFailure(cause: unknown): ReturnType { - return toRejectionValue( - new WorkflowError( - `mandatory queue persistence failed: ${toRejectionValue(cause).message}`, - CODE.PERSISTENCE_ERROR, - { recoverable: false, details: { reason: 'queue_persistence_failed' } }, - ), - ); -} - -function executionError( - message: string, - reason: string, - recoverable: boolean, -): WorkflowError { - return new WorkflowError(message, CODE.AGENT_EXECUTION_ERROR, { - recoverable, - details: { reason }, - }); -} - -function rawPromptDetail(payloadJson: string | null): string { - if (payloadJson === null) return ''; - try { - const payload: unknown = JSON.parse(payloadJson); - return isPlainObject(payload) && typeof payload.prompt === 'string' ? payload.prompt : ''; - } catch { - return ''; - } -} - -/** Restore-only parser. Live admissions use the throwing class method; - * corrupt durable rows are ignored rather than resurrected. */ -function parseTurnPayload( - payloadJson: string | null, -): { prompt: string; promptMeta?: Record } | null { - if (payloadJson === null) return null; - let payload: unknown; - try { - payload = JSON.parse(payloadJson); - } catch { - return null; - } - if (!isPlainObject(payload) || typeof payload.prompt !== 'string') return null; - const options = payload.options; - if (options === undefined || options === null) return { prompt: payload.prompt }; - if (!isPlainObject(options)) return null; - if (Object.keys(options).some((key) => !STEER_OPTION_KEYS.has(key))) return null; - if (options.promptMeta === undefined) return { prompt: payload.prompt }; - if (!isPlainObject(options.promptMeta)) return null; - return { prompt: payload.prompt, promptMeta: options.promptMeta }; -} - -/** Await a batch of best-effort teardown promises (cancels, releases) - * ONLY until the drain deadline — the doc's outer drain bound (phase-D - * review round 3: a hung cancel/release used to block disconnect/ - * shutdown indefinitely past the session-eviction TTL). After the - * deadline the drain returns without waiting; every promise in the batch - * already carries its own catch handler, so the fire-and-forget tail can - * never become an unhandled rejection. The bound timer is CLEARED when - * the batch wins the race — a satisfied drain must not leave a timer - * pending for the whole remaining bound (it would keep the process - * alive and fire needlessly later). */ -async function boundedAll(promises: Promise[], deadline: number): Promise { - const remaining = deadline - Date.now(); - if (remaining <= 0) return; - let timer: NodeJS.Timeout | undefined; - const bound = new Promise((resolve) => { - timer = setTimeout(resolve, remaining); - }); - await Promise.race([Promise.allSettled(promises), bound]); - if (timer !== undefined) clearTimeout(timer); -} - -/** Await ONE teardown promise (the owned runner's dispose) only until - * `deadline` — the same absolute-bound discipline as `boundedAll`. A - * rejection still propagates when the promise wins the race; a deadline - * that wins returns `undefined` and the promise keeps running in the - * background (callers must attach their own tail catch — the broker's - * dispose does — so the abandoned tail can never become an unhandled - * rejection). The bound timer is CLEARED when the promise wins, so a - * satisfied teardown must not leave a timer pending for the whole - * remaining bound. */ -async function boundedOne(promise: Promise, deadline: number): Promise { - const remaining = deadline - Date.now(); - if (remaining <= 0) return undefined; - let timer: NodeJS.Timeout | undefined; - const bound = new Promise((resolve) => { - timer = setTimeout(() => resolve(undefined), remaining); - }); - try { - return await Promise.race([promise, bound]); - } finally { - if (timer !== undefined) clearTimeout(timer); - } -} - -/** Timer sleep (the drain's yield and bounded-wait primitive). */ -function sleep(ms: number): Promise { - return new Promise((resolve) => setTimeout(resolve, ms)); -} - -/** The guest-library version gate (phase-E review rejection round 7): - * parse a surface version string ('0.3.1') into its numeric triple and - * compare it against a minimum. Returns false for unparseable or - * missing versions — an unknown library version can never pass a - * capability gate. The gate exists for the corrected continuation-lease - * ordering (0.3.1 is the first copy whose lease-setting reaction rides - * the wrapper promise itself; see `continuationLeaseAvailable`). */ -function guestVersionAtLeast(version: string, atLeast: string): boolean { - const parsed = /^(\d+)\.(\d+)\.(\d+)/.exec(version); - const minimum = /^(\d+)\.(\d+)\.(\d+)/.exec(atLeast); - if (parsed === null || minimum === null) return false; - for (let i = 1; i <= 3; i++) { - const a = Number(parsed[i]); - const b = Number(minimum[i]); - if (a > b) return true; - if (a < b) return false; - } - return true; -} - -/** Head+tail elision at `max` chars (the manifest task cap). */ -function headTail(value: string, max: number): string { - if (value.length <= max) return value; - const keep = Math.max(0, max - 1); - const half = Math.floor(keep / 2); - return `${value.slice(0, half)}…${value.slice(value.length - (keep - half))}`; -} - -function requireString(value: unknown, what: string): string { - if (typeof value !== 'string') { - throw new WorkflowError(`agent options: "${what}" must be a string`, CODE.SCRIPT_VALIDATION_ERROR, { - recoverable: false, - }); - } - return value; -} - -function requireRecord(value: unknown, what: string): Record { - if (typeof value !== 'object' || value === null || Array.isArray(value)) { - throw new WorkflowError(`agent options: "${what}" must be an object`, CODE.SCRIPT_VALIDATION_ERROR, { - recoverable: false, - }); - } - return value as Record; -} - -function requireStringBoolRecord(value: unknown, what: string): Record { - const record = requireRecord(value, what); - for (const [key, entry] of Object.entries(record)) { - if (typeof entry !== 'string' && typeof entry !== 'boolean') { - throw new WorkflowError(`agent options: "${what}.${key}" must be a string or boolean`, CODE.SCRIPT_VALIDATION_ERROR, { - recoverable: false, - }); - } - } - return record as Record; -} - -function requireStringArray(value: unknown, what: string): string[] { - if (!Array.isArray(value) || !value.every((entry) => typeof entry === 'string')) { - throw new WorkflowError(`agent options: "${what}" must be an array of strings`, CODE.SCRIPT_VALIDATION_ERROR, { - recoverable: false, - }); - } - return value; -} - -/** Normalize any thrown value into the guest rejection shape: `{ name, - * message, code?, recoverable? }` (the guest's `toError` turns it into - * an Error carrying code/recoverable). WorkflowErrors keep their code - * and recoverable flag; plain errors are recoverable by default; a - * plain rejection-shaped object (the broker's own refusals) passes - * through with its name/recoverable. */ -function toRejectionValue(error: unknown): { name: string; message: string; code?: string; recoverable?: boolean; details?: unknown; replBackend?: string } { - // The §4.6 backend stamp passes THROUGH the conversion (an error - // pre-stamped with `replBackend` — an admission refusal whose segment - // resolved — keeps it on the rejection value the guest sees). - const backend = (error as { replBackend?: unknown } | null | undefined)?.replBackend; - const stamped = (value: { name: string; message: string; code?: string; recoverable?: boolean; details?: unknown; replBackend?: string }): typeof value => { - if (typeof backend === 'string') value.replBackend = backend; - return value; - }; - if (isWorkflowError(error)) { - return stamped({ - name: 'WorkflowError', - message: error.message, - code: error.code, - recoverable: error.recoverable, - ...(error.details !== undefined ? { details: error.details } : {}), - }); - } - if (error instanceof Error) { - return stamped({ name: error.name || 'Error', message: error.message }); - } - if (typeof error === 'object' && error !== null && typeof (error as { message?: unknown }).message === 'string') { - const value = error as { name?: unknown; message: string; code?: unknown; recoverable?: unknown; details?: unknown }; - return stamped({ - name: typeof value.name === 'string' ? value.name : 'Error', - message: value.message, - ...(typeof value.code === 'string' ? { code: value.code } : {}), - ...(typeof value.recoverable === 'boolean' ? { recoverable: value.recoverable } : {}), - ...(value.details !== undefined ? { details: value.details } : {}), - }); - } - return { name: 'Error', message: String(error) }; -} - -/** Is this error the session-cancel signal (the prompt's stop reason or - * the released-session error after a cancel)? */ -function isCancellation(error: unknown): boolean { - if (isWorkflowError(error)) return error.code === CODE.AGENT_CANCELLED || error.code === CODE.WORKFLOW_ABORTED; - return false; -} - -/** The §4.6 uncaught-eval-error rendering: the error name and message, - * the guest stack's top frames with LINE NUMBERS in the submitted code - * (the eval's filename — the broker evals under the VM's default - * `''`), and — when the error came from a subagent call — the - * call id and the resolved backend (the guest library stamps - * `replCallId`, the broker stamps `replBackend` — see - * `EvalErrorInfo`). */ -function errorLine(info: EvalErrorInfo): string { - let line = `${info.name}: ${info.message}`; - if (info.replCallId !== undefined) { - line += ` (call ${info.replCallId}${info.replBackend !== undefined ? ` on backend ${info.replBackend}` : ''})`; - } - const frames = replStackFrames(info.stack); - if (frames.length > 0) line += '\n' + frames.join('\n'); - return line; -} - -/** The guest stack's frames with line numbers in the submitted code: - * quickjs stacks are newest-first, so the FIRST matching frames are the - * top of the guest stack; frames from the guest library (filename - * `''`) and the host are skipped. At most 8 frames — the - * render is attribution, not a transcript (the §4.6 rule: name, message, - * top frames with line numbers). */ -function replStackFrames(stack: string | undefined): string[] { - if (stack === undefined) return []; - const frames: string[] = []; - for (const raw of stack.split('\n')) { - if (frames.length >= 8) break; - const match = /^\s*at\s+(?:(.+?)\s+\()?:(\d+):(\d+)\)?\s*$/.exec(raw); - if (match === null) continue; - frames.push( - match[1] !== undefined - ? ` at ${match[1]} (:${match[2]}:${match[3]})` - : ` at :${match[2]}:${match[3]}`, - ); - } - return frames; -} - -/** The backend segment of a model spec (the §4.1 grammar: `"backend/ - * model"` — the first `/`-delimited segment, ASCII-lowercased; a bare - * `"backend"` spec is the whole string). */ -function backendSegment(modelSpec: string): string { - const slash = modelSpec.indexOf('/'); - return (slash >= 0 ? modelSpec.slice(0, slash) : modelSpec).toLowerCase(); -} diff --git a/packages/repl-engine/src/errors.ts b/packages/repl-engine/src/errors.ts deleted file mode 100644 index 12bdabe2..00000000 --- a/packages/repl-engine/src/errors.ts +++ /dev/null @@ -1,96 +0,0 @@ -/** - * Error reporting for failed evals. - * - * Error info is plain data, not an `Error` subclass, because it crosses the - * MCP tool boundary: the `repl` tool must return eval failures as part of - * its result payload, never by throwing across the transport. - */ - -/** - * A coded refusal: the global lexical binding enumeration could not be - * established for this VM (the running binary's `JSContext` layout or - * value encoding does not match the adjacency invariant the scan in - * `global-lexical.ts` calibrates against). Thrown from - * `globalVarObjectHandle` — and thereby from every manifest/provenance - * read that needs lexical bindings — so a layout regression surfaces - * loudly instead of silently dropping the workspace's - * `let`/`const`/`class` state. Defined here (not in `global-lexical.ts`) - * so the published type graph stays free of quickjs-wasi imports: the - * package index re-exports this class, and the consumer-facing - * declaration of `global-lexical.ts` must never be pulled into a - * non-DOM program. - */ -export class LexicalEnumerationError extends Error { - constructor(detail: string) { - super(`global lexical binding enumeration is unavailable: ${detail}`); - this.name = 'LexicalEnumerationError'; - } -} - -/** - * Structured information about a failed eval, safe to ship to an MCP client. - * - * `name`/`message`/`stack` are read from the guest error **trap-free** - * (own-data-property descriptor reads only — guest getters are never - * invoked while rendering error state; see the roadmap doc's transfer - * lesson R69). When the guest threw a primitive, `name` is `"Error"` and - * `message` is the native string conversion. - */ -export interface EvalErrorInfo { - /** Error constructor name as seen in the guest (e.g. `SyntaxError`, `TypeError`, `InternalError`). */ - name: string; - /** Human-readable message. */ - message: string; - /** Guest stack trace when it is available as an own data property. */ - stack?: string; - /** - * The repl call id of the subagent call this error came from, when the - * guest library attached it (the §4.6 error attribution: an uncaught - * error from a rejected agent/steer call names the call). Own data - * string only — guest-forgeable like the message itself. - */ - replCallId?: string; - /** - * The RESOLVED backend of the subagent call this error came from, - * when the host stamped it onto the rejection value (the §4.6 - * attribution's second half). - */ - replBackend?: string; - /** - * True when the per-eval `interruptHandler` fired, aborting execution - * with quickjs's `InternalError: interrupted` (or the drain threw the - * same as a job error). This is how `interrupt` breaks runaway evals. - */ - interrupted: boolean; - /** - * True when the per-VM `memoryLimit` was exceeded (quickjs's - * `InternalError: out of memory`). The limit is a malloc cap on live - * allocations; after the failed eval the VM stays usable. - */ - outOfMemory: boolean; -} - -/** - * Classify a (name, message) pair captured from the guest into the - * engine-level flags the tool layer needs. The quickjs interrupt and - * malloc-limit failures are the only engine-injected errors; everything - * else is guest-authored, so classification matches the exact built-in - * message strings and nothing fuzzier. - */ -export function classifyError( - name: string, - message: string, - stack?: string, - replCallId?: string, - replBackend?: string, -): EvalErrorInfo { - return { - name, - message, - stack, - ...(replCallId !== undefined ? { replCallId } : {}), - ...(replBackend !== undefined ? { replBackend } : {}), - interrupted: name === 'InternalError' && message === 'interrupted', - outOfMemory: name === 'InternalError' && message === 'out of memory', - }; -} diff --git a/packages/repl-engine/src/eval-break-channel.ts b/packages/repl-engine/src/eval-break-channel.ts deleted file mode 100644 index 9c487965..00000000 --- a/packages/repl-engine/src/eval-break-channel.ts +++ /dev/null @@ -1,524 +0,0 @@ -/** - * The out-of-band eval-break channel — the `interrupt` tool's no-id path - * made deliverable to a SYNCHRONOUSLY RUNNING eval (the REPL orchestrator - * roadmap: "break a runaway eval (the quickjs interrupt handler)"). - * - * ## Why a channel at all - * - * The broker runs the QuickJS VM on the daemon's single thread. A - * fully synchronous (never-yielding) eval — `while (true) {}` — blocks - * that thread, so the interrupt tool call itself cannot be PROCESSED by - * the daemon (its HTTP request sits in the kernel backlog until the eval - * ends or the per-eval wall-clock deadline breaks it). The quickjs - * interrupt handler is the only thing that runs DURING the eval — it is - * invoked periodically by the VM, synchronously, inside the execution. - * The interrupt therefore needs a side channel that stays writable while - * the main thread is blocked: a WORKER THREAD. - * - * ## The mechanism - * - * The channel owns one worker thread per daemon. The worker hosts a tiny - * loopback HTTP endpoint (`POST /break` with the workspace key). The - * worker's event loop never blocks (it is a separate thread), so the - * MCP server's shim can reach it while the daemon's main thread is - * wedged in the eval. The break flag lives in a `SharedArrayBuffer` - * (flags + arm sequence numbers + slot generations), written by the - * worker with `Atomics.store` and read by the main thread inside the - * eval's interrupt handler with `Atomics.compareExchange` (consume-on- - * observation). - * - * ## The arm-after-start rule (no stale breaks, no lost breaks) - * - * A bare flag would let an interrupt armed while the workspace was idle - * break a LATER, unrelated eval — the phase-E review's targeting - * discipline. The channel therefore orders every arm against every - * execution start on a SHARED MONOTONIC ARM-SEQUENCE COUNTER (one - * `Atomics.add` per arm, the first word of the buffer), giving a TOTAL - * order across the worker and the main thread — no clock comparison at - * all (phase-F review round 3: the old scheme compared millisecond - * `Date.now()` stamps and required `armedAt > executionStartMs`, so a - * break arriving in the SAME millisecond as the execution start was - * consumed as stale and permanently lost; a sequence number has no - * resolution window — an arm strictly after an execution's start marker - * ALWAYS carries a greater sequence, an arm before it ALWAYS carries a - * lesser one). The probe (`consumeBreak(key, sinceSeq)`) consumes the - * flag ONLY when its arm sequence exceeds the sequence observed at the - * consuming execution's start: an eval that was already running when - * the interrupt arrived is broken; a fresh eval that starts after the - * interrupt was armed is not (the stale flag is consumed-and-dropped by - * its first execution — the daemon's own interrupt handling clears it - * too, see the repl tool). - * - * ## Scoping and lifecycle (no project-count ceiling) - * - * One channel per daemon, keyed by the workspace's projectDir; slots - * are assigned on registration and RELEASED on `unregister` (the broker - * disposes a workspace's slot when its broker is torn down; a released - * slot is reused by the next registration). The shared buffer GROWS on - * demand (a resizable `SharedArrayBuffer`; the worker's length-tracking - * view follows the growth automatically), so there is no fixed workspace - * ceiling. `dispose()` closes the HTTP server and terminates the - * worker. A crashed daemon takes the channel with it — a fresh daemon - * starts a fresh channel, so no stale flags survive a restart. - * - * ## Acknowledged, generation-safe registration (phase-F review round 4) - * - * The carried defect: registration was fire-and-forget while the worker - * applied it asynchronously, and `unregister` immediately cleared and - * reused the slot. Two concrete failures followed. (a) A FIRST interrupt - * could 404: the shim fired `/break` before the worker had applied the - * key→slot mapping (the main thread's map was updated synchronously at - * `register`), so the out-of-band break was lost and the eval ran to - * the per-eval deadline. (b) A STALE interrupt could break a later - * workspace: the worker could still hold the RELEASED key's mapping when - * a `/break` for it landed (the `unregister`/re-`register` messages were - * still in flight), arm the REUSED slot's flag with the old key's - * sequence — and the NEW key's `consumeBreak` read that flag as its own. - * - * The fix has two halves: - * - * - REGISTRATION IS ACKNOWLEDGED: `register(key)` returns a promise the - * worker resolves (`{ type: "ack", key, slot, gen }`) only after it - * applied the mapping. The broker awaits the ack before ANY - * guest-executing operation runs (`runSerialized`), so by the time an - * interrupt can meaningfully arrive (during a running eval) the relay - * can never 404 for this workspace. A dead worker rejects the pending - * acks, so an awaiting broker degrades to the per-eval deadline bound - * instead of hanging. - * - SLOT ASSIGNMENTS CARRY GENERATIONS: each registration bumps the - * slot's generation, the worker writes the ARMING key's generation - * into the shared slot (before the flag, release order), and the - * main-thread probe drops any consumed flag whose generation does not - * match the consuming key's CURRENT generation. A stale arm for a - * released incarnation therefore can never break the workspace that - * later reuses the slot, even while the worker still held the old - * mapping — the old-generation flag is consumed-and-dropped. The - * worker additionally clears the flag when a mapping takes the slot - * over, and `unregister` clears the flag AND invalidates the - * generation word, so the fence holds in every interleaving. - */ - -import { existsSync } from "node:fs"; -import { fileURLToPath } from "node:url"; -import { Worker } from "node:worker_threads"; - -/** The channel's initial slot capacity (workspace keys); the shared - * buffer grows by doubling when it is exhausted, so this is a starting - * size, never a ceiling. */ -export const EVAL_BREAK_CHANNEL_INITIAL_SLOTS = 16; - -/** The shared buffer's absolute ceiling in bytes (the growth bound — - * a memory bound for the resizable `SharedArrayBuffer`, not a - * project-count cap: ~2 M slots at 12 bytes each). */ -export const EVAL_BREAK_CHANNEL_MAX_BYTES = 4 + 2_097_152 * 12; - -/** The worker's readiness message (the bound loopback port). */ -interface BreakWorkerReady { - type: "ready"; - port: number; -} - -/** The worker's registration acknowledgement (the mapping is APPLIED — - * the relay will no longer 404 for this key). */ -interface BreakWorkerAck { - type: "ack"; - key: string; - slot: number; - gen: number; -} - -/** The channel's published surface (the broker's probe + the daemon's - * wiring). */ -export interface EvalBreakChannel { - /** The worker's loopback break endpoint (`POST /break` with a JSON - * `{ key }` body) — the address the MCP shim fires while the daemon's - * main thread is blocked in a synchronous eval. Resolves once the - * worker is listening. */ - breakUrl(): Promise; - /** Assign the workspace's slot (idempotent; the worker learns the - * key→slot mapping for its HTTP endpoint). Grows the shared buffer - * when the current capacity is exhausted — no project-count ceiling. - * A slot freed by `unregister` is reused under a NEW GENERATION. - * RESOLVES when the worker ACKNOWLEDGES the mapping — the broker - * gates every guest-executing operation on this ack, so a no-id - * interrupt can never 404 against an unapplied mapping (phase-F - * review round 4). Rejects when the worker dies before - * acknowledging (callers degrade to the per-eval deadline bound — - * the relay is best-effort by design). */ - register(key: string): Promise; - /** Release the workspace's slot (the broker's teardown side): the - * worker drops the key's mapping and the slot returns to the free - * pool for the next registration. The slot's armed flag is cleared - * and its generation word INVALIDATED — an arm still in flight for - * the released key writes the old generation, which no consume for - * the next key can satisfy. Idempotent; unknown keys are a no-op. */ - unregister(key: string): void; - /** The execution-start marker: the arm-sequence counter's current - * value, read at the moment an execution begins and passed to - * `consumeBreak` as `sinceSeq`. The sequence gives a total order - * across the worker and the main thread (one shared counter), so an - * arm that strictly followed the execution's start ALWAYS breaks it - * — down to the same instant, with no clock-resolution window. */ - executionStartMarker(): number; - /** The interrupt-handler probe: consume the break flag when it was - * armed after `sinceSeq` (the consuming execution's start marker) - * AND under the consuming key's CURRENT slot generation (a stale arm - * for a released incarnation of the slot is dropped — it can never - * break the workspace that reused the slot). Returns whether THIS - * execution must break. Consumes the flag either way — a stale flag - * (armed before the execution began, or under an old generation) is - * dropped so it can never break a later execution. */ - consumeBreak(key: string, sinceSeq: number): boolean; - /** Clear the workspace's flag (the daemon's own interrupt handling — - * once the request is processed, the continuation-targeted signal - * owns the break). */ - clearBreak(key: string): void; - dispose(): Promise; -} - -/** The worker entry: the compiled `eval-break-worker.js` in dist, or - * the TypeScript source when running from src (tsx dev/tests — the - * worker inherits the parent's loader, so the .ts runs directly). */ -function workerEntryUrl(): URL { - const tsEntry = new URL("./eval-break-worker.ts", import.meta.url); - if (existsSync(fileURLToPath(tsEntry))) return tsEntry; - return new URL("./eval-break-worker.js", import.meta.url); -} - -/** The shared-buffer layout (all Int32 words): word 0 is the arm - * sequence counter; slot i occupies words `1 + 3*i` (the flag), `2 + - * 3*i` (the arm's sequence) and `3 + 3*i` (the ARMING key's slot - * generation). The stride is fixed, so a growth only appends — every - * side's view of the old range stays valid, and the length-tracking - * views (`new Int32Array(sab)` without a length) follow the growth - * automatically. */ -const SLOT_STRIDE = 3; -const FLAG_WORD = 1; - -function slotFlagWord(slot: number): number { - return FLAG_WORD + SLOT_STRIDE * slot; -} - -function slotSeqWord(slot: number): number { - return FLAG_WORD + SLOT_STRIDE * slot + 1; -} - -function slotGenWord(slot: number): number { - return FLAG_WORD + SLOT_STRIDE * slot + 2; -} - -/** The resizable-`SharedArrayBuffer` surface (the ES2024 `SharedArrayBuffer`; - * the repo's `lib` target is ES2022, so the growth surface is declared - * locally — Node ≥ 22 implements it at runtime, and the package's - * consumer fixtures pin the shipped behavior). */ -type ResizableSharedArrayBuffer = SharedArrayBuffer & { - readonly growable: boolean; - readonly maxByteLength: number; - grow(newByteLength: number): void; -}; - -/** The ES2024 constructor surface (the `{ maxByteLength }` option). */ -interface ResizableSharedArrayBufferCtor { - new (byteLength: number, options: { maxByteLength: number }): ResizableSharedArrayBuffer; -} - -/** One key's slot assignment (the main thread's authoritative map). */ -interface SlotAssignment { - slot: number; - /** The assignment's generation: bumped on every (re)registration of - * the slot — the fence that makes a stale arm for a released - * incarnation unreadable by the slot's next key (see `consumeBreak` - * and the module docs). */ - gen: number; -} - -/** A registration awaiting the worker's acknowledgement. */ -interface PendingAck { - gen: number; - promise: Promise; - resolve: () => void; - reject: (error: Error) => void; -} - -/** The channel implementation (see the module docs). */ -export class EvalBreakChannelImpl implements EvalBreakChannel { - /** The resizable shared buffer: [armSeq][slot 0 flag+seq+gen][slot 1 …]. */ - private readonly sab: SharedArrayBuffer; - /** The length-tracking view (follows `grow()` automatically — the - * worker's identical view follows it too, being the same buffer). */ - private readonly view: Int32Array; - private readonly slots = new Map(); - private readonly freeSlots: number[] = []; - /** The per-slot generation counter (the NEXT assignment's generation - * minus one — see `register`). */ - private readonly slotGenerations: number[] = []; - /** Registrations awaiting the worker's acknowledgement, by key. */ - private readonly acks = new Map(); - /** Slots ever allocated (the next fresh slot index). */ - private allocated = 0; - private readonly worker: Worker; - private readonly ready: Promise; - /** The worker's message listener is attached ONLY while it is needed - * (boot + pending registration acks): Node re-refs the worker's - * port when a `message` listener is attached, so a permanently - * attached listener would keep the parent process alive after the - * channel's work is done (phase-F review round 4 — the ack listener - * used to stay attached forever and every test suite that created a - * server hung on exit). See `attachWorkerListener` / - * `detachWorkerListenerIfIdle`. */ - private workerListenerAttached = false; - /** The ready promise's resolve (captured for the shared listener). */ - private readyResolve: ((port: number) => void) | undefined; - private disposed = false; - - constructor() { - this.sab = new (SharedArrayBuffer as unknown as ResizableSharedArrayBufferCtor)( - (FLAG_WORD + SLOT_STRIDE * EVAL_BREAK_CHANNEL_INITIAL_SLOTS) * 4, - { maxByteLength: EVAL_BREAK_CHANNEL_MAX_BYTES }, - ); - this.view = new Int32Array(this.sab); - this.worker = new Worker(workerEntryUrl(), { - workerData: { sab: this.sab }, - }); - this.worker.unref(); - this.ready = new Promise((resolve, reject) => { - this.readyResolve = resolve; - const onError = (error: Error): void => { - this.worker.off("message", this.onWorkerMessage); - this.workerListenerAttached = false; - reject(error); - this.rejectPendingAcks(error); - }; - const onExit = (code: number): void => { - this.worker.off("message", this.onWorkerMessage); - this.workerListenerAttached = false; - if (code !== 0) { - reject(new Error(`eval-break worker exited with code ${code}`)); - } - // Either way the worker is gone: no pending registration can - // ever be acknowledged — awaiting callers degrade to the - // per-eval deadline bound instead of hanging. - this.rejectPendingAcks(new Error(`eval-break worker exited with code ${code}`)); - }; - this.worker.once("error", onError); - this.worker.once("exit", onExit); - }); - this.attachWorkerListener(); - } - - /** The shared message listener: resolves `ready` (the bound loopback - * port) and applies registration acknowledgements. Detached when - * neither is pending — see `detachWorkerListenerIfIdle`. */ - private readonly onWorkerMessage = (message: unknown): void => { - const msg = message as Partial & Partial; - if (msg.type === "ready" && typeof msg.port === "number") { - const resolve = this.readyResolve; - this.readyResolve = undefined; - resolve?.(msg.port); - this.detachWorkerListenerIfIdle(); - return; - } - if (msg.type === "ack" && typeof msg.key === "string" && typeof msg.gen === "number") { - this.onAck(msg.key, msg.gen); - this.detachWorkerListenerIfIdle(); - } - }; - - /** Attach the worker's message listener (idempotent). Needed while - * the worker boots (the ready message) and while any registration - * awaits its ack. */ - private attachWorkerListener(): void { - if (this.workerListenerAttached) return; - this.workerListenerAttached = true; - this.worker.on("message", this.onWorkerMessage); - } - - /** Detach the message listener once it has nothing left to hear - * (boot done AND no pending acks): a permanently attached listener - * re-refs the worker's port and keeps the parent process alive after - * the channel's work is done (phase-F review round 4). */ - private detachWorkerListenerIfIdle(): void { - if (this.acks.size === 0 && this.readyResolve === undefined) { - this.worker.off("message", this.onWorkerMessage); - this.workerListenerAttached = false; - } - } - - async breakUrl(): Promise { - const port = await this.ready; - return `http://127.0.0.1:${port}/break`; - } - - register(key: string): Promise { - if (this.disposed) return Promise.resolve(); - const existing = this.slots.get(key); - if (existing !== undefined) { - // Idempotent re-registration: the live mapping's ack — already - // resolved once the worker applied it, or still pending when the - // first registration has not been acknowledged yet (the caller's - // readiness gate must cover that first application, not a - // spuriously-resolved duplicate). - const pending = this.acks.get(key); - return pending?.promise ?? Promise.resolve(); - } - const slot = this.freeSlots.pop() ?? this.allocateFreshSlot(); - // The slot's NEXT generation: an arm written for a previous - // incarnation of this slot (the worker still held the released - // key's mapping when the arm landed) carries the old generation, - // which no consume for this key can satisfy (see `consumeBreak`). - const gen = (this.slotGenerations[slot] ?? 0) + 1; - this.slotGenerations[slot] = gen; - this.slots.set(key, { slot, gen }); - let resolve!: () => void; - let reject!: (error: Error) => void; - const promise = new Promise((res, rej) => { - resolve = res; - reject = rej; - }); - this.acks.set(key, { gen, promise, resolve, reject }); - // The ack travels as a worker message: make sure the listener is - // attached (it may have been detached once boot completed and no - // acks were pending). - this.attachWorkerListener(); - try { - this.worker.postMessage({ type: "register", key, slot, gen }); - } catch (error) { - // A dead worker: roll the assignment back and reject — the - // awaiting broker degrades to the per-eval deadline bound (the - // relay is best-effort by design). - this.acks.delete(key); - this.slots.delete(key); - this.freeSlots.push(slot); - reject(error instanceof Error ? error : new Error(String(error))); - } - return promise; - } - - /** The worker's registration acknowledgement: resolve the pending - * ack ONLY when its generation matches — an ack for a released - * incarnation (the key was unregistered and possibly re-registered - * under a new generation while the ack was in flight) resolves - * nothing. */ - private onAck(key: string, gen: number): void { - const pending = this.acks.get(key); - if (pending === undefined || pending.gen !== gen) return; - this.acks.delete(key); - pending.resolve(); - } - - private allocateFreshSlot(): number { - if (this.allocated >= this.slotsPerCapacity()) { - // No fixed ceiling: grow the shared buffer (doubling). The - // worker's length-tracking view follows the growth automatically, - // and existing slots' layout is unchanged (the stride is fixed, - // growth only appends), so no re-view handshake is needed. - const capacity = this.slotsPerCapacity(); - const next = Math.min(capacity * 2, Math.floor((EVAL_BREAK_CHANNEL_MAX_BYTES / 4 - FLAG_WORD) / SLOT_STRIDE)); - if (next <= capacity) { - throw new Error( - `eval-break channel shared-buffer ceiling (${EVAL_BREAK_CHANNEL_MAX_BYTES} bytes) exhausted — ` + - `too many repl workspaces in one daemon`, - ); - } - (this.sab as ResizableSharedArrayBuffer).grow((FLAG_WORD + SLOT_STRIDE * next) * 4); - } - const slot = this.allocated; - this.allocated++; - return slot; - } - - /** The current capacity in slots (the view is length-tracking, so it - * reflects the latest growth). */ - private slotsPerCapacity(): number { - return Math.floor((this.view.length - FLAG_WORD) / SLOT_STRIDE); - } - - unregister(key: string): void { - if (this.disposed) return; - const entry = this.slots.get(key); - if (entry === undefined) return; - this.slots.delete(key); - // Drop any armed flag with the slot (a stale flag must never fire - // for a later workspace that reuses the slot) and INVALIDATE the - // generation word: an arm still in flight for the released key — - // the worker can still hold the released mapping until the - // unregister message lands — writes the OLD generation, which no - // consume for the next key can satisfy (the generation fence; see - // the module docs and `consumeBreak`). - Atomics.store(this.view, slotFlagWord(entry.slot), 0); - Atomics.store(this.view, slotGenWord(entry.slot), 0); - this.freeSlots.push(entry.slot); - this.worker.postMessage({ type: "unregister", key }); - // A registration still awaiting its ack is released with the - // mapping (the worker drops it when the unregister lands; a - // re-registered key starts a fresh ack under its new generation, - // and the old ack's generation never matches it). - const pending = this.acks.get(key); - if (pending !== undefined) { - this.acks.delete(key); - pending.resolve(); - this.detachWorkerListenerIfIdle(); - } - } - - executionStartMarker(): number { - return Atomics.load(this.view, 0); - } - - consumeBreak(key: string, sinceSeq: number): boolean { - if (this.disposed) return false; - const entry = this.slots.get(key); - if (entry === undefined) return false; - // Consume the flag first (a stale flag must not survive into a - // later execution), then decide by the arm's sequence AND - // generation. - if (Atomics.compareExchange(this.view, slotFlagWord(entry.slot), 1, 0) !== 1) return false; - // The worker writes the arm's sequence AND the arming key's - // generation BEFORE the flag (release order), so a consumed flag - // always carries both. - const armedSeq = Atomics.load(this.view, slotSeqWord(entry.slot)); - const armedGen = Atomics.load(this.view, slotGenWord(entry.slot)); - // The GENERATION fence (phase-F review round 4): an arm written for - // a PREVIOUS incarnation of this slot — the worker still held the - // released key's mapping when its `/break` landed — carries the old - // generation and can never break THIS key's execution. The stale - // flag is consumed-and-dropped, exactly like an arm-before-start. - if (armedGen !== entry.gen) return false; - return armedSeq > sinceSeq; - } - - clearBreak(key: string): void { - if (this.disposed) return; - const entry = this.slots.get(key); - if (entry === undefined) return; - Atomics.store(this.view, slotFlagWord(entry.slot), 0); - } - - /** Reject every pending registration ack (the worker is gone — no - * pending registration can ever be applied). */ - private rejectPendingAcks(error: Error): void { - for (const pending of this.acks.values()) pending.reject(error); - this.acks.clear(); - this.detachWorkerListenerIfIdle(); - } - - async dispose(): Promise { - if (this.disposed) return; - this.disposed = true; - this.rejectPendingAcks(new Error("eval-break channel disposed")); - try { - this.worker.postMessage({ type: "dispose" }); - } catch { - // The worker already died — nothing to signal. - } - await this.worker.terminate().catch(() => undefined); - } -} - -/** Create the channel (the daemon's composition root). */ -export function createEvalBreakChannel(): EvalBreakChannel { - return new EvalBreakChannelImpl(); -} - -// The worker's HTTP contract is implemented in `eval-break-worker.js` -// (the shim's fire side posts `{ key }` to `POST /break`). diff --git a/packages/repl-engine/src/eval-break-worker.ts b/packages/repl-engine/src/eval-break-worker.ts deleted file mode 100644 index 96b9438c..00000000 --- a/packages/repl-engine/src/eval-break-worker.ts +++ /dev/null @@ -1,138 +0,0 @@ -/** - * The eval-break channel's worker thread (see `eval-break-channel.ts`): - * owns the break flags' write side — a loopback HTTP endpoint the MCP - * shim can reach while the daemon's main thread is blocked in a - * synchronous eval. The worker's event loop is a separate thread, so it - * never blocks with the daemon. - * - * Wire contract: `POST /break` with a JSON body `{ "key": "" }` arms the key's flag (arm sequence first, generation second, - * flag last — release order, so a consumed flag always carries its arm - * sequence AND the arming key's generation); 204 when the key is - * registered, 404 otherwise. `{ type: "register", key, slot, gen }` - * messages from the main thread teach the key→slot mapping and are - * ACKNOWLEDGED with `{ type: "ack", key, slot, gen }` once applied (the - * channel's registration gate — phase-F review round 4); the slot's - * flag is cleared as the mapping takes it over (an arm still in flight - * for the slot's previous key must never break the new key), and the - * slot's generation word is set to the new key's generation. - * `{ type: "unregister", key }` drops the mapping (the slot returns to - * the main thread's free pool); `dispose` closes the server and exits. - * - * The shared buffer is RESIZABLE: the length-tracking `Int32Array` view - * below follows the main thread's growth automatically, and the slot - * stride is fixed, so no re-view handshake is ever needed. - */ - -import { createServer } from "node:http"; -import { parentPort, workerData } from "node:worker_threads"; - -interface WorkerData { - sab: SharedArrayBuffer; -} - -const { sab } = workerData as WorkerData; -const view = new Int32Array(sab); -/** The applied key→slot mapping: slot + the generation the mapping was - * assigned under (written into the shared slot on every arm — the - * main thread's consume drops an arm whose generation does not match - * the consuming key's current one). */ -const slotsByKey = new Map(); - -function slotFlagWord(slot: number): number { - return 1 + 3 * slot; -} - -function slotSeqWord(slot: number): number { - return 1 + 3 * slot + 1; -} - -function slotGenWord(slot: number): number { - return 1 + 3 * slot + 2; -} - -const server = createServer((req, res) => { - if (req.method !== "POST" || req.url !== "/break") { - res.writeHead(404).end(); - return; - } - let body = ""; - req.setEncoding("utf8"); - req.on("data", (chunk: string) => { - body += chunk; - }); - req.on("end", () => { - let key: unknown; - try { - key = (JSON.parse(body) as { key?: unknown }).key; - } catch { - res.writeHead(400).end(); - return; - } - if (typeof key !== "string") { - res.writeHead(400).end(); - return; - } - const entry = slotsByKey.get(key); - if (entry === undefined) { - res.writeHead(404).end(); - return; - } - // Release order: the arm's sequence and the ARMING key's generation - // are visible before the flag. The sequence is the SHARED monotonic - // arm counter (word 0) — a total order across this thread and the - // main thread, so a break armed after an execution began always - // carries a greater sequence than the execution's start marker (no - // clock-resolution window — the phase-F review round 3 - // same-millisecond loss is impossible). The generation is the - // fence against slot reuse: a consume under a LATER generation of - // this slot drops the arm (phase-F review round 4 — see - // `consumeBreak` in the channel). - const seq = Atomics.add(view, 0, 1) + 1; - Atomics.store(view, slotSeqWord(entry.slot), seq); - Atomics.store(view, slotGenWord(entry.slot), entry.gen); - Atomics.store(view, slotFlagWord(entry.slot), 1); - res.writeHead(204).end(); - }); -}); - -parentPort?.on("message", (message: { type?: string; key?: string; slot?: number; gen?: number }) => { - if ( - message.type === "register" && - typeof message.key === "string" && - typeof message.slot === "number" && - typeof message.gen === "number" - ) { - // The mapping takes the slot over: clear any flag an in-flight arm - // for the slot's PREVIOUS key left behind (the worker can still - // hold the released mapping until the unregister message lands) and - // stamp the slot with the new key's generation — a stale arm can - // never break the new key (the main thread's consume also drops it - // on the generation mismatch; the clear makes it vanish entirely). - slotsByKey.set(message.key, { slot: message.slot, gen: message.gen }); - Atomics.store(view, slotFlagWord(message.slot), 0); - Atomics.store(view, slotGenWord(message.slot), message.gen); - // The ACKNOWLEDGEMENT (phase-F review round 4): the mapping is - // APPLIED — the channel's registration promise resolves only now, - // so the broker never runs guest code against an unapplied - // mapping. - parentPort?.postMessage({ type: "ack", key: message.key, slot: message.slot, gen: message.gen }); - return; - } - if (message.type === "unregister" && typeof message.key === "string") { - slotsByKey.delete(message.key); - return; - } - if (message.type === "dispose") { - server.close(() => process.exit(0)); - // A hanging keep-alive connection must not block exit. - setTimeout(() => process.exit(0), 100).unref(); - return; - } -}); - -server.listen(0, "127.0.0.1", () => { - const address = server.address(); - const port = typeof address === "object" && address !== null ? address.port : 0; - parentPort?.postMessage({ type: "ready", port }); -}); diff --git a/packages/repl-engine/src/global-lexical.ts b/packages/repl-engine/src/global-lexical.ts deleted file mode 100644 index f8665c62..00000000 --- a/packages/repl-engine/src/global-lexical.ts +++ /dev/null @@ -1,189 +0,0 @@ -/** - * The host's door into the realm's GLOBAL LEXICAL bindings — top-level - * `let`/`const`/`class` declarations, the roadmap doc's canonical - * workspace state (`const research = agent(...)`). The workspace - * manifest's binding enumeration (see `Workspace.manifest`) and the - * provenance registry's attribution pass both need these names; this - * module is the seam that reaches them. - * - * ## Why a seam at all - * - * ECMAScript's global declarative record is deliberately - * non-reflectable: no guest API can enumerate global lexical bindings - * (`Object.getOwnPropertyNames(globalThis)` lists only the object-record - * side — `var`/function declarations and host globals), and no - * JS-visible surface exposes them. quickjs-ng (the engine the shipped - * `quickjs.wasm` binary runs) stores them in an INTERNAL object, - * `ctx->global_var_obj` — a null-prototype object holding the lexical - * declarations as own properties (`JS_DefineGlobalVar`), reachable from - * C but from nothing the guest can see. - * - * ## The mechanism - * - * The shipped binary (quickjs-ng 0.15.1) exposes `qjs_get_context_ptr()` - * and the global object's own JSValue through the shim, and — verified - * against the v0.15.1 source — the `JSContext` struct holds - * `global_obj` and `global_var_obj` as ADJACENT JSValue fields. The - * binary is NaN-boxed, so a JSValue is 8 bytes - * (`[payload u32][tag i32]`, `JS_TAG_OBJECT = -1`) and the object - * pointer is the payload. This module therefore LOCATES the global-var - * object with a self-calibrating scan: - * - * 1. read the global object's pointer from the shim's cached global - * handle (the payload of its JSValue); - * 2. scan the `JSContext` for the 8-byte slot holding exactly that - * pointer with the object tag — that is `global_obj`'s slot; - * 3. the adjacent slot (offset +8) is `global_var_obj` (the adjacency - * invariant); - * 4. fabricate a handle over that slot: 8 bytes of scratch wasm memory - * holding `[global_var_obj pointer][object tag]`, wrapped as a - * VM-lifetime handle whose `dispose` is a no-op (the reference it - * exposes belongs to the context, never to the scratch). - * - * The fabricated handle then exposes the whole trap-free introspection - * machinery (own-key enumeration, own-property-descriptor reads) over - * the lexical bindings — the same machinery the manifest uses for - * global-object bindings. - * - * The scan is verified end-to-end by the test suite: a workspace whose - * evals created lexical bindings must list them in the manifest. A - * layout the scan cannot find (a quickjs build where the fields moved or - * the value encoding changed) REFUSES with `LexicalEnumerationError` — - * a loud, coded error, never silent omission of the workspace's - * bindings (the package's "never silently discards data" posture; the - * pinned quickjs-wasi version's layout is stable, and the snapshot - * envelope already refuses loudly on a binary change). - */ - -import { JSValueHandle, type QuickJS } from 'quickjs-wasi'; - -import { LexicalEnumerationError } from './errors.js'; -import { getVmShim, type ReplVm } from './vm.js'; -import { getPropRaw, hasOwnRaw, rawOwnKeys } from './trapfree.js'; - -/** The object tag in the NaN-boxed JSValue encoding (`JS_TAG_OBJECT`). */ -const JS_TAG_OBJECT = -1; - -/** The JSValue slot size in the NaN-boxed encoding (8 bytes). */ -const JSVALUE_BYTES = 8; - -/** How far into the `JSContext` struct to scan for the `global_obj` - * slot (the field sits ~300 bytes in on the shipped build; the window - * is generous). */ -const CONTEXT_SCAN_WINDOW = 4096; - -/** The per-VM fabricated handle (see the module docs; a VM-lifetime - * scratch value, never disposed). */ -const lexicalHandles = new WeakMap(); - -/** - * The fabricated handle over the realm's global-var object (see the - * module docs): the VM-lifetime handle the trap-free introspection - * machinery reads the lexical bindings through. The handle is created - * once per VM (the scan is self-calibrating per wasm instance) and its - * `dispose` is a no-op — the object reference it exposes is owned by - * the context, not by this handle. - * - * **Internal**: returns a quickjs-wasi handle type, so it is NOT - * re-exported from the package index (the published type graph stays - * free of quickjs-wasi types). - */ -export function globalVarObjectHandle(vm: ReplVm): JSValueHandle { - const cached = lexicalHandles.get(vm); - if (cached !== undefined) return cached; - const shim = getVmShim(vm) as QuickJS; - const e = shim._getExports(); - const memory = e.memory; - const view = new DataView(memory.buffer); - const ctxPtr = e.qjs_get_context_ptr(); - const limit = Math.min(ctxPtr + CONTEXT_SCAN_WINDOW + JSVALUE_BYTES, memory.buffer.byteLength); - // The global object's pointer: the payload of the shim's cached - // global handle (an object JSValue — payload + object tag). - const globalPayload = view.getUint32(shim.global.ptr, true); - let varSlot = -1; - for (let off = ctxPtr; off + 2 * JSVALUE_BYTES <= limit; off += JSVALUE_BYTES) { - if (view.getUint32(off, true) !== globalPayload) continue; - if (view.getInt32(off + 4, true) !== JS_TAG_OBJECT) continue; - // The adjacency invariant: the next slot is `global_var_obj` — a - // plausible object JSValue (a nonzero pointer into wasm memory with - // the object tag). Without the adjacency check a stray slot holding - // the global pointer would be misread as the pair. - const varPayload = view.getUint32(off + JSVALUE_BYTES, true); - if (varPayload === 0 || varPayload >= memory.buffer.byteLength) continue; - if (view.getInt32(off + JSVALUE_BYTES + 4, true) !== JS_TAG_OBJECT) continue; - varSlot = off + JSVALUE_BYTES; - break; - } - if (varSlot < 0) { - throw new LexicalEnumerationError( - `no global-var-object slot found in the JSContext (the running binary's layout does not match ` + - `the adjacency invariant; the manifest cannot enumerate top-level let/const/class bindings)`, - ); - } - // Fabricate the handle: an 8-byte scratch JSValue in wasm memory, - // owned by the VM for its lifetime (never freed — the reference it - // holds belongs to the context, not to this scratch slot). The - // singleton flag makes `dispose` a no-op, so no later path can - // decrement the context's reference. - const scratch = e.wasm_malloc(JSVALUE_BYTES); - const write = new DataView(memory.buffer); - write.setUint32(scratch, view.getUint32(varSlot, true), true); - write.setInt32(scratch + 4, JS_TAG_OBJECT, true); - const handle = new JSValueHandle(shim, scratch, true); - lexicalHandles.set(vm, handle); - return handle; -} - -/** - * The global lexical binding names of a realm, trap-free: ALL own - * string keys of the internal global-var object (the semantic - * equivalent of `Object.getOwnPropertyNames` over the lexical - * declarations — the same discipline `rawOwnKeys` applies to the global - * object). **Internal** (see `globalVarObjectHandle`). - */ -export function rawLexicalKeys(vm: ReplVm): string[] { - return rawOwnKeys(globalVarObjectHandle(vm)); -} - -/** - * Resolve a global lexical binding trap-free and return its VALUE - * handle, or `undefined` when the name is not a lexical binding (or the - * read failed). The returned handle is owned by the caller. Mirrors the - * global-object reader (`preview.ts`'s `readSlotValue`) over the - * global-var object's descriptor machinery. **Internal** (see - * `globalVarObjectHandle`). - */ -export function readLexicalSlotValue(vm: ReplVm, name: string): JSValueHandle | undefined { - const shim = getVmShim(vm) as QuickJS; - const e = shim._getExports(); - const base = globalVarObjectHandle(vm); - const keyHandle = shim.newString(name); - let descPtr: number; - try { - descPtr = e.qjs_get_own_property_descriptor(base.ptr, keyHandle.ptr); - } finally { - keyHandle.dispose(); - } - if (descPtr === 0) return undefined; - const desc = new JSValueHandle(shim, descPtr); - try { - if (e.qjs_is_exception(desc.ptr) !== 0) { - const excPtr = e.qjs_get_exception(); - if (excPtr !== 0) new JSValueHandle(shim, excPtr).dispose(); - return undefined; - } - // Data vs accessor via `hasOwnProperty` on the descriptor object - // (raw): lexical bindings are always data (let/const/class), but the - // discipline mirrors the global reader exactly. - if (hasOwnRaw(e, shim, desc.ptr, 'value')) { - const valueProp = getPropRaw(e, shim, desc.ptr, 'value'); - if (valueProp !== undefined) return valueProp; - return undefined; // allocation failure edge — reads as absent - } - getPropRaw(e, shim, desc.ptr, 'get')?.dispose(); - getPropRaw(e, shim, desc.ptr, 'set')?.dispose(); - return undefined; - } finally { - desc.dispose(); - } -} diff --git a/packages/repl-engine/src/guest/guest-library.ts b/packages/repl-engine/src/guest/guest-library.ts deleted file mode 100644 index 5009498f..00000000 --- a/packages/repl-engine/src/guest/guest-library.ts +++ /dev/null @@ -1,2256 +0,0 @@ -/** - * The REPL orchestrator's guest-side library — a fresh TypeScript-authored - * implementation of the sandbox vocabulary (NOT a vendor of the harness's - * `guest/dsl.js`; the harness's evolution disciplines are the model). - * - * `GUEST_LIBRARY_SOURCE` is a plain script evaluated exactly ONCE at VM - * creation, inside the capability-free QuickJS realm — no modules, no - * imports, no host assumptions beyond the documented `__host_*` - * functions (see the package README's "Guest library ⇄ host contract"). - * After the first snapshot this library travels INSIDE the snapshot: it is - * versioned with the workspace, not with the host, and a host must accept - * snapshots carrying an older copy than the one it ships. - * - * What it defines in the realm (the roadmap doc's DSL split — only the - * sliver that needs host effects calls out; everything else is pure JS): - * - * - `agent(modelSpec, task, options?)` → Promise (host effect). `modelSpec` - * is the backend-routing spec (`"pi/deepseek-v4-flash-max"`, per the - * roadmap doc's own example); `task` the worker's prompt; `options` - * (structured-output schema, cwd, backend config) cross the bridge as - * JSON. The returned promise IS the live handle: started-not-awaited - * handles come free with top-level await, and the doc's handle methods - * ride it — `queue(prompt, opts?)` / `steer(prompt, opts?)` / - * `cancel()` — each resolving with what actually happened (the host - * settles with the steering outcome, mirroring the outcome values - * acp-agents surfaces in its steering events). `id` carries the stable - * call id (`"c1"`, …) used by `status`/`interrupt`. - * - `checkpoint(question, options?)` → Promise (host effect), and - * `checkpoint.answer(callId, value)` → boolean — answer delivery - * through the same host function's trailing-argument mode. - * - `console.{log,info,warn,error,debug}` — the bridge: every call - * renders ONE joined line (the arguments' §4.4 reprs joined with a - * single space — direct strings whole, objects/arrays to depth 2, - * 20 entries per level, nested strings head-limited at 200 chars) - * and forwards it to `__host_console`. - * - `sleep(ms)` → Promise (host effect — a host-side timer; the VM - * itself stays timer-free). - * - `workspace()` / `agents()` → plain JSON-round-tripped values - * served by the host (`__host_workspace` / `__host_agents`); - * `reset()` → void, asking the host to tear the workspace down - * after the current eval completes (`__host_reset`). The - * verify/judgePanel reviewers/graders resolve their model spec - * through `__host_default_backend` (the host's configured default - * backend id — a real registered segment; the v1 reserved - * 'default' sentinel is deleted). - * - `parallel` / `pipeline` / `verify` / `judgePanel` / `gate` / - * `retry` / `loopUntilDry` — pure JavaScript layered on `agent()`, - * following `packages/workflows/src/dsl.d.ts` semantics. - * - `_` — the previous eval's completion value (IPython-style result - * history; set by the host after every eval that resolved with a - * value). The per-argument `$N` capture globals are deleted. - * - `__REPL_GUEST_VERSION` — the version marker global. - * - `globalThis[Symbol.for("repl.guest")]` — the frozen reconciliation - * surface (version / pending / settle / stats) the host uses after a - * snapshot restore. - * - * Deleted vocabulary, per the roadmap doc: `phase()` (it presupposes "a - * run" that no longer exists) and the whole budget surface — no `budget()` - * global, no ledger, no caps vocabulary. Resource limits are server - * configuration, invisible to the guest; the host signals non-recoverable - * failures exclusively through `recoverable: false` on rejections (the - * harness's reserved `BUDGET_EXHAUSTED`/`AGENT_LIMIT_EXCEEDED` codes have - * no counterpart here). - * - * The pending-call registry (callId → { resolve, reject, kind, detail, - * optionsJson, createdAt, sessionId, modelSpec }) lives in the library's - * closure, so the table of in-flight host calls travels inside the snapshot - * itself. Every entry records the id the host addresses the call by - * (`sessionId` — the founding session id for queue/control calls, the call's own - * id otherwise), so each pending operation survives with full correlation. - * Queue work may restore; steering is settled as interrupted and never replayed. - * The host settles calls by callId — through the - * Deferred it returned from a `__host_*` function in a live session, or - * through the reconciliation surface after a restore. Both routes converge - * on the same idempotent settlement function; the first settlement wins. - */ - -/** The guest library's version (the `__REPL_GUEST_VERSION` marker value). - * 0.2.0 adds the eval-await tracking surface: '__replAwait' (the global - * the host's top-level-await instrumenter inserts), the registry - * entries' 'promise' field (which promise each pending call's - * settlement resolves — the await-attribution look-up), the 'awaitLog' - * (the chronological record of awaited call ids), and the surface's - * 'supportsAwaitTracking'/'awaitLogTake' members. - * 0.3.0 replaces the LOG-based targeting with a genuine per-eval - * CONTINUATION IDENTITY: '__replAwait(value, token)' now WRAPS the - * awaited value in a fresh promise whose settling reaction — the job - * that runs IMMEDIATELY BEFORE the eval's continuation segment — sets - * the CONTINUATION LEASE to the eval's token (the writable - * '__replLease' accessor global). The host's drain loop reads the - * lease between jobs: a job that starts with a lease set IS the armed - * eval's continuation, so the interrupt fires only while THAT - * execution runs (an unawaited sibling `.then` job — which runs - * before the lease-setting reaction — can neither fire nor consume - * the signal), and the lease is cleared after the segment ends. The - * wrap also makes INDIRECT awaits targetable: `await - * Promise.all([q])` wraps the combinator promise, whose settlement - * queues the eval's continuation exactly like a direct call's — the - * eval's identity is the promise graph, not a logged call-id list. - * The surface gains 'supportsContinuationLease'; the 0.2.0 log - * surface stays (an older host may still drive it). A snapshot - * carrying 0.1.0 is served as-is (the doc's rule: the host serves - * snapshots carrying older library versions) — the host degrades by - * not instrumenting awaits on it (no eval-break targeting, honest - * refusal). - * 0.3.1 fixes two continuation-lease defects (phase-E review rejection - * round 6): the lease-setting reaction moved from the awaited VALUE's - * settlement onto the WRAPPER itself (registered before the await - * machinery's own reaction), so a sibling `q.then(...)` registered - * after the eval started awaiting `q` can no longer run with the - * lease set — the lease is associated with the actual continuation - * job; and the for-await ITERABLE wrap became a real async-iterable - * (`__replAwaitIterable` — the 0.3.0 instrumenter wrapped `for await` - * iterables in `__replAwait`, whose promise result made every `for - * await` loop throw `TypeError: not a function`). The surface gains - * 'supportsIterableLease'; a 0.3.0 snapshot is served as-is — its - * for-await sites are left unwrapped by the instrumenter (native - * semantics, no mid-loop targeting) while its awaits stay instrumented - * (the 0.3.0 lease-set defect is the older copy's own, never - * re-injected). - * - * 0.4.0 is the eval-plane redesign surface (docs/roadmap/repl-eval-redesign.md): - * the `$N` capture system is deleted (console renders one joined line per call - * with the §4.4 depth-limited repr; `_` is the sole result-history global), - * `sleep(ms)` / `workspace()` / `agents()` / `reset()` join the guest library, - * rejections of registry calls carry `replCallId` (and `replBackend` when the - * host stamps it) for the §4.6 error attribution, the agent options bag is - * narrowed to exactly `{ schema, cwd, configOptions, mode }`, and the - * verify/judgePanel combinators resolve their reviewer/grader spec through - * `__host_default_backend` (the reserved 'default' sentinel that bypassed - * registry validation is deleted) while rejected calls augment their Error - * with the CALL-SITE stack (the §4.6 submitted-code line numbers). The guest - * environment changed, so this version invalidates older stored snapshots - * (they take the §6.1 auto-reset path on first touch). - * - * The 0.3.1 copy also hardens the instrumentation surface (phase-E - * review rejection round 7, same version — nothing shipped between): - * the await/iterable helpers run on the CAPTURED pristine Promise - * intrinsics (a guest that replaces 'Promise.prototype.then', overwrites - * 'Promise.resolve' or shadows 'Promise' lexically cannot change - * instrumentation semantics — the instrumented 'await 40' stays '40' - * and the continuation lease is still set); the for-await iterable wrap - * propagates ACQUISITION errors exactly once (an observable/throwing - * 'Symbol.asyncIterator' getter runs a single time and reports its - * original error — the old degrade-to-unwrapped made the machinery - * acquire the iterable a second time and could surface 'boom2' instead - * of native 'boom1'); and a SYNC iterator's results pass through - * AsyncFromSyncIteratorContinuation semantics (the result VALUE is - * awaited and unwrapped — 'for await (const x of - * [Promise.resolve(1)])' yields '1', never the promise object). - * The provenance registry reads descriptors off the CAPTURED global - * object too (a top-level lexical 'const globalThis = 7' no longer - * blanks every binding's provenance). - * - * 0.5.0 removes `followUp`, makes `steer` strict active-prompt control, - * and adds first-class durable `queue` handles plus distinct host callbacks - * for queue creation, steering, session cancellation, and queue cancellation. - * Snapshot format 3 refuses older guest state before execution. - * - * HOST GATE: the broker's continuation-lease availability check is - * VERSION-GATED on >= 0.3.1 — a restored snapshot carrying the 0.3.0 - * library (whose lease-setting reaction still runs on the awaited - * VALUE's settlement — the sibling-reaction interrupt-targeting defect) - * reports 'supportsContinuationLease: true' but is served WITHOUT - * instrumentation: no eval-break targeting, honest refusal (phase-E - * review rejection round 7: the flag alone re-armed the original - * defect on a supported older snapshot). */ -export const GUEST_LIBRARY_VERSION = '0.5.0'; - -/** `Symbol.for` key of the reconciliation surface on `globalThis`. */ -export const GUEST_SURFACE_KEY = 'repl.guest'; - -/** `Symbol.for` key of the per-binding provenance registry on - * `globalThis` (the workspace manifest's provenance seam — which eval - * created/rebound a binding, or which worker call's settlement produced - * it; metadata only, travels inside snapshots). The registry is HOST - * policy, not guest injection: the workspace layer's bootstrap installs - * it (with the fresh-realm baseline as its `known` set) on fresh and - * restored workspaces alike — the harness manifest's own placement — so - * the library source itself never grows the realm's baseline. - */ -export const GUEST_PROVENANCE_KEY = 'repl.provenance'; - -/** - * The provenance registry factory (see `provenance.ts`): evaluated by - * the host bootstrap on every workspace start (fresh installs and - * pre-provenance restores), so all installers produce a byte-identical - * registry. Captures its own intrinsics at evaluation time (install-time - * captures are pristine — the bootstrap runs before any guest code on a - * fresh workspace; a bootstrap over a hostile pre-provenance snapshot - * degrades to no provenance, never to content, via the record/read - * try/catch). `names` is the fresh-realm baseline key set (the 'known' - * skip set for GLOBAL-OBJECT properties), plus two newer optional - * arguments the bootstrap passes: the baseline TYPE TOKENS (name → - * fresh-realm typeof token — a known name whose token changes has been - * REBOUND by user code and is attributed like any other rebinding) and - * the LEXICAL baseline key set (the fresh realm's own top-level - * let/const/class bindings — the lexical pass skips THESE instead of - * the known set, because a lexical declaration shadows a same-named - * baseline global and is always the user's). A factory invoked without - * them (an older host) keeps the pure known-set skip. - */ -export const PROVENANCE_FACTORY = `(function (names, typeToks, lexKnownArr) { - var gOPN = Object.getOwnPropertyNames; - var gOPD = Object.getOwnPropertyDescriptor; - var hasOwnProp = Object.prototype.hasOwnProperty; - var jparse = JSON.parse; - // Captured at CREATION (the bootstrap runs before any user eval, so - // the realm is pristine): the pass's own code must keep working when - // user code SHADOWS a baseline global — 'const Math = 42' is a - // legitimate user program, and the lexical binding shadows the - // factory's free-variable Math/Object references at call time - // (phase-E review round 5: the pass threw on Math.max under a - // lexical Math shadow, swallowing every attribution). - var jmax = Math.max; - var jcreate = Object.create; - // The realm's global object, captured when the factory runs (before - // any user code on a fresh workspace): the pass's own code must keep - // working when user code SHADOWS a baseline global — 'const - // globalThis = …' is a legitimate user program, and the lexical - // binding would shadow the factory's free-variable globalThis at - // call time (the same discipline as the captured Math/Object - // intrinsics above). - var g = globalThis; - var reg = { - evalSeq: 0, - origins: Object.create(null), - prev: Object.create(null), - // The lexical pass's value tracker (see record): the CURRENT value - // of each global lexical binding (top-level let/const/class), by - // name — the SameValue comparison base for re-attribution. The - // guest cannot read lexical bindings (the global declarative record - // is non-reflectable), so the HOST passes the values in; this map - // is where the registry keeps them (a strong reference, exactly - // like the property pass's prev values). - lexPrev: Object.create(null), - known: Object.create(null), - // The ORIGINAL baseline VALUES of the known names (see the - // typeToks loop below): NEVER updated on attribution — the - // manifest's "changed from the baseline" comparison needs the - // pristine value forever. - baseVal: Object.create(null), - // The LAST-ATTRIBUTED value of each known name (initialized to the - // baseline value): the record pass's SameValue rebind detector — a - // SECOND rebind re-attributes to its own eval, and a restored - // registry's last-attributed values survive the snapshot (a - // pre-snapshot rebind is not re-attributed by the first - // post-restore pass). - knownPrev: Object.create(null), - }; - // The fresh-realm BASELINE TYPE TOKENS (name -> typeof token of the - // pristine value, captured by the host in a throwaway realm): a KNOWN - // (baseline) name whose current token differs has been REBOUND by - // user code — 'Math = 42' overwrites the built-in — and is - // attributed like any other rebinding (phase-E review rejection: the - // known-set skip made overwritten built-ins invisible to the - // manifest). SameValue against the throwaway realm's baseline VALUE - // is impossible (different realm), so the tokens are the host-side - // change detector, while the registry's OWN baseline values - // (reg.baseVal, captured below) extend it to SAME-TYPE replacements - // ('Math = { userOwned: true }' — the token cannot see those; the - // value identity can, within this realm). The token observed at each - // pass is remembered (reg.baseTok), so an untouched builtin is a - // no-op forever and a SECOND rebind re-attributes to its own eval. - // The tokens are CONSTANT per registry lifetime (the baseline never - // changes), so they arrive at factory creation. - reg.baseTok = Object.create(null); - if (typeof typeToks === 'string') { - try { typeToks = jparse(typeToks); } catch (e) { typeToks = null; } - } - if (typeToks !== null && typeToks !== undefined && typeof typeToks === 'object') { - for (var tk in typeToks) { - if (typeof typeToks[tk] === 'string') { - reg.baseTok[tk] = typeToks[tk]; - // The ORIGINAL baseline VALUE of the known name (descriptor - // read, never a [[Get]] — the pass's discipline): captured when - // the factory runs. On a fresh workspace the realm is pristine - // (the bootstrap runs before any user code), so this is the - // true baseline; a restored registry carries the references - // inside the snapshot. The values are NEVER updated on - // attribution: the manifest's same-type-replacement detector - // ('Math = { userOwned: true }' keeps the 'object' type token) - // compares the CURRENT value against the ORIGINAL baseline — - // value identity is the only detector a type token cannot - // provide (phase-E review rejection round 6). A pre-provenance - // restore runs the factory over the restored (dirty) realm — - // the captured values may be user-rebound; the type-token - // detector still catches token-changing overwrites there (the - // same corner the bootstrap accepts). - var bd = gOPD(g, tk); - var bv; - if (bd !== undefined && hasOwnProp.call(bd, 'value')) bv = bd.value; - else if (bd !== undefined && hasOwnProp.call(bd, 'get')) bv = bd.get; - reg.baseVal[tk] = bv; - reg.knownPrev[tk] = bv; - } - } - } - // The LEXICAL baseline key set (the fresh realm's own top-level - // let/const/class bindings — empty on the shipped library; a future - // library that declared lexically would otherwise be attributed as - // user bindings). The lexical pass skips THESE names instead of the - // global baseline's: a lexical declaration SHADOWS a same-named - // baseline global, and the shadowing binding is the user's — it must - // be attributed (phase-E review rejection: the known-set skip made - // 'const Math = 42' invisible to the manifest). Also constant per - // registry lifetime. - reg.lexKnown = Object.create(null); - if (typeof lexKnownArr === 'string') { - try { lexKnownArr = jparse(lexKnownArr); } catch (e) { lexKnownArr = null; } - } - if (lexKnownArr !== null && lexKnownArr !== undefined && typeof lexKnownArr.length === 'number') { - for (var lk0 = 0; lk0 < lexKnownArr.length; lk0++) { - if (typeof lexKnownArr[lk0] === 'string') reg.lexKnown[lexKnownArr[lk0]] = true; - } - } - function record(label, atMs) { - try { - if (label === null || label === undefined) { - reg.evalSeq = (reg.evalSeq | 0) + 1; - label = 'eval ' + reg.evalSeq; - } - // The global LEXICAL bindings first (top-level let/const/class — - // the roadmap's canonical 'const research = agent(...)' state): - // they are not global-object properties and cannot be enumerated - // guest-side (ECMAScript's global declarative record is - // non-reflectable), so the HOST enumerates them through the - // engine's internal global-var object (see global-lexical.ts) and - // passes the names as this pass's THIRD argument (a JSON array - // string). A lexical binding SHADOWS a same-named global-object - // property for identifier resolution, and the manifest displays - // the binding code sees — the lexical one — so names in the - // lexical set are SKIPPED by the property pass below (one binding - // per name, the lexical view authoritative). A pass without the - // argument (an older host, or a registry snapshot whose record - // closure predates the feature) skips the merge. - // - // The pass's FOURTH+ arguments carry the CURRENT lexical VALUES, - // one realm value per name in the names array's order (the host - // reads them through the internal global-var object — the same - // host-driven channel as the names; a guest can never forge the - // values). With the values the registry can detect a CHANGE - // (SameValue) and RE-ATTRIBUTE: a 'let' binding assigned a worker - // result, or a suspended 'const finding = await research' whose - // continuation assigned the settled value, re-attributes to the - // settlement's 'worker cN' label — the manifest then reports - // WHICH subagent produced the current value, from what task, when - // (phase-E review rejection: the lexical entry was recorded on - // first sight only, so a value the worker settlement produced - // kept the declaring eval's label with no task). Without the - // values (an older host) the pass degrades to first-sight-only - // attribution, the pre-feature behavior. - var lexNames = null; - try { - if (arguments.length >= 3 && typeof arguments[2] === 'string' && arguments[2].length > 0) { - lexNames = jparse(arguments[2]); - } - } catch (e) { lexNames = null; } - var lexValueCount = jmax(0, arguments.length - 3); - var lexSet = jcreate(null); - if (lexNames !== null && typeof lexNames.length === 'number') { - for (var li = 0; li < lexNames.length; li++) { - var lk = lexNames[li]; - if (typeof lk === 'string') lexSet[lk] = true; - } - } - var names_ = gOPN(g); - var seen = jcreate(null); - for (var i = 0; i < names_.length; i++) { - var k = names_[i]; - // Descriptor read, never a [[Get]]: a binding rebound to an - // accessor must not have its getter fired by host bookkeeping. - // The getter FUNCTION serves as the rebind-detection identity for - // accessor bindings. Read from the CAPTURED global object 'g' — - // never the free variable globalThis: a top-level lexical 'const - // globalThis = 7' is a legitimate user program, and the lexical - // binding shadows the factory's free-variable globalThis at call - // time, so gOPD(globalThis, k) read descriptors off the NUMBER - // (throwing in QuickJS) and the pass's catch swallowed the whole - // attribution — every subsequent binding reached the manifest - // with null provenance (phase-E review rejection round 7: 'var - // userValue = 42' appeared without producer/task/time metadata - // after a 'const globalThis' shadow). - var d = gOPD(g, k); - var v = undefined; - if (d !== undefined) { - if (hasOwnProp.call(d, 'value')) v = d.value; - else if (hasOwnProp.call(d, 'get')) v = d.get; - } - if (reg.known[k]) { - // A KNOWN baseline name (a builtin or a library global): - // attribute only when user code REBOUND it — the current type - // token differs from the fresh-realm baseline token ('Math = - // 42', 'globalThis.JSON = "x"' — the phase-E review - // rejection: the baseline filter hid overwritten built-ins - // from the manifest), OR the current value is no longer - // SameValue to the last-attributed value — a SAME-TYPE - // replacement ('Math = { userOwned: true }' keeps the - // 'object' token; phase-E review rejection round 6: the - // token-only detector missed same-type overwrites entirely, - // leaving them absent from the manifest with no provenance). - // The token and the value are remembered per pass - // (reg.baseTok / reg.knownPrev), so an untouched builtin is a - // no-op forever and a SECOND rebind re-attributes to its own - // eval. The name is present (it is in the property list), so - // it is marked seen — an attributed rebinding must never be - // swept by the gone pass in the same sweep. - seen[k] = true; - if (reg.baseTok && typeof reg.baseTok[k] === 'string') { - var tok = hasOwnProp.call(d, 'value') ? typeof d.value : 'accessor'; - var rebound = tok !== reg.baseTok[k]; - if (!rebound && reg.knownPrev && hasOwnProp.call(reg.knownPrev, k)) { - var prevV = reg.knownPrev[k]; - // SameValue semantics (NaN-stable), like the pass's other - // value comparisons. - rebound = prevV !== v && !(prevV !== prevV && v !== v); - } - if (rebound) { - reg.origins[k] = { via: label, at: atMs }; - reg.prev[k] = v; - reg.baseTok[k] = tok; - reg.knownPrev[k] = v; - } - } - continue; - } - if (lexSet[k]) { seen[k] = true; continue; } - seen[k] = true; - var tracked = reg.origins[k] !== undefined; - var same = tracked && (reg.prev[k] === v || (reg.prev[k] !== reg.prev[k] && v !== v)); - if (!same) { - reg.origins[k] = { via: label, at: atMs }; - reg.prev[k] = v; - } - } - // The lexical pass: attribute on first sight; with the host's - // VALUE arguments, RE-ATTRIBUTE on a value change (SameValue — - // the current label produced the current value: a 'let' assigned - // a worker result, or a suspended 'const finding = await - // research' whose continuation assigned the settled value, - // re-attributes to the settlement's 'worker cN' label). A name - // first attributed as a global PROPERTY and later shadowed by a - // lexical declaration is re-attributed when the lexical binding - // appears — the property path stored the property VALUE in - // prev[k], and a stored value is the pass's marker that the name - // predates the lexical binding (after re-attribution prev[k] is - // undefined and stays; the corner where the property value itself - // was literally undefined is accepted — orientation metadata). - if (lexNames !== null && typeof lexNames.length === 'number') { - for (var li2 = 0; li2 < lexNames.length; li2++) { - var lk2 = lexNames[li2]; - if (typeof lk2 !== 'string') continue; - // Only the LEXICAL baseline is skipped: a lexical declaration - // always comes from user code, and it SHADOWS a same-named - // baseline global — 'const Math = 42' is a user binding even - // though the name is in the known set (phase-E review - // rejection: the known-set skip hid it from the manifest). - if (reg.lexKnown[lk2]) continue; - seen[lk2] = true; - if (lexValueCount > 0) { - var cur = arguments[3 + li2]; - if (!hasOwnProp.call(reg.lexPrev, lk2)) { - reg.origins[lk2] = { via: label, at: atMs }; - reg.lexPrev[lk2] = cur; - } else if (reg.lexPrev[lk2] !== cur && !(reg.lexPrev[lk2] !== reg.lexPrev[lk2] && cur !== cur)) { - reg.origins[lk2] = { via: label, at: atMs }; - reg.lexPrev[lk2] = cur; - } - } else { - if (reg.origins[lk2] === undefined || reg.prev[lk2] !== undefined) { - reg.origins[lk2] = { via: label, at: atMs }; - reg.prev[lk2] = undefined; - } - } - } - } - for (var gone in reg.origins) { - if (!seen[gone]) { delete reg.origins[gone]; delete reg.prev[gone]; delete reg.lexPrev[gone]; } - } - } catch (e) {} - } - function read() { - try { - var out = jcreate(null); - for (var k in reg.origins) { - var o = reg.origins[k]; - out[k] = { via: o.via, at: o.at }; - } - // The KNOWN names whose CURRENT value is no longer the baseline: - // the type token differs from the pristine token, OR the value is - // no longer SameValue to the ORIGINAL baseline value (reg.baseVal - // — never updated on attribution) — the manifest's - // changed-binding detector for overwritten built-ins. A SAME-TYPE - // overwrite ('Math = { userOwned: true }') is caught by the value - // identity, which the type token alone cannot see (phase-E review - // rejection round 6). Computed at READ time (the manifest is - // rendered under the operation chain), descriptor-based and - // trap-free like the record pass: an accessor-rebound name is - // detected through its getter function identity, never invoked. - var changed = []; - for (var ck in reg.baseTok) { - if (!reg.known[ck]) continue; - var cd; - try { - cd = gOPD(g, ck); - } catch (e) { - continue; - } - if (cd === undefined) continue; - var cv = hasOwnProp.call(cd, 'value') - ? cd.value - : hasOwnProp.call(cd, 'get') - ? cd.get - : undefined; - var ctok = hasOwnProp.call(cd, 'value') ? typeof cd.value : 'accessor'; - var cchanged = ctok !== reg.baseTok[ck]; - if (!cchanged && reg.baseVal && hasOwnProp.call(reg.baseVal, ck)) { - var cbv = reg.baseVal[ck]; - // SameValue semantics (NaN-stable). - cchanged = cbv !== cv && !(cbv !== cbv && cv !== cv); - } - if (cchanged) changed.push(ck); - } - return { evalSeq: reg.evalSeq, origins: out, changed: changed }; - } catch (e) { - return { evalSeq: 0, origins: Object.create(null), changed: [] }; - } - } - reg.record = record; - reg.read = read; - if (names !== undefined && names !== null) { - for (var n = 0; n < names.length; n++) reg.known[names[n]] = true; - } - return reg; -})`; - -/** Name of the version-marker global the library installs. */ -export const GUEST_VERSION_GLOBAL = '__REPL_GUEST_VERSION'; - -/** Host-callback names the guest library calls (the whole effect surface). */ -export const HOST_AGENT = '__host_agent'; -export const HOST_CHECKPOINT = '__host_checkpoint'; -export const HOST_CONSOLE = '__host_console'; -export const HOST_STEER = '__host_agent_steer'; -export const HOST_QUEUE = '__host_agent_queue'; -export const HOST_SESSION_CANCEL = '__host_agent_cancel'; -export const HOST_QUEUE_CANCEL = '__host_queue_cancel'; -export const HOST_SLEEP = '__host_sleep'; -export const HOST_WORKSPACE = '__host_workspace'; -export const HOST_AGENTS = '__host_agents'; -export const HOST_RESET = '__host_reset'; -export const HOST_DEFAULT_BACKEND = '__host_default_backend'; - -/** - * Build the injectable library script. `version` is substituted into the - * source so the version marker and the exported constant can never drift. - */ -export function buildGuestLibrarySource(version: string = GUEST_LIBRARY_VERSION): string { - return GUEST_LIBRARY_SOURCE.replaceAll('__REPL_GUEST_VERSION__', JSON.stringify(version)); -} - -/** - * The library as a plain script (ES2017-level JavaScript, evaluated in the - * realm with no module system). Written as a single template literal with - * no interpolation: the guest code is deliberately plain JS (string - * concatenation, no backticks) so the source is exactly what the VM - * evaluates. All `\\` escapes below are doubled so the guest code receives - * the literal escape sequences (`\\n` in the guest source → the guest's - * `\n`). - */ -const GUEST_LIBRARY_SOURCE = `/* - * REPL orchestrator guest-side library, version __REPL_GUEST_VERSION__. - * Evaluated exactly ONCE at VM creation; travels inside snapshots. The - * four __host_* functions below are the realm's entire effect surface. - */ -(function () { - 'use strict'; - - // The realm's REAL global object, captured when the library is - // evaluated (before any user code runs): the library's own code must - // keep working when user code SHADOWS a baseline global — a top-level - // lexical 'const globalThis = 7' is a legitimate user program, and the - // lexical binding shadows the library's free-variable globalThis at - // call time, breaking every internal reference (the provenance - // registry's descriptor reads, the host-function lookups, the global - // installs). Everything below that means "the realm's - // global object" reads 'g' — the same discipline as the provenance - // factory's capture (phase-E review rejection round 7). - var g = globalThis; - - // ──────────────────────────────────────────────────────────────────────── - // Identity and idempotence - // ──────────────────────────────────────────────────────────────────────── - - var VERSION = __REPL_GUEST_VERSION__; - var SURFACE_KEY = 'repl.guest'; - var VERSION_GLOBAL = '__REPL_GUEST_VERSION'; - - // Evaluating this script twice in one realm (e.g. a host bug that - // re-injects it into a restored snapshot) must never wipe the live - // registry. If the surface is already installed, this evaluation is a - // no-op. - if (g[Symbol.for(SURFACE_KEY)]) return; - - // ──────────────────────────────────────────────────────────────────────── - // Internal state — all of it lives in this closure, so all of it travels - // inside the snapshot. - // ──────────────────────────────────────────────────────────────────────── - - var state = { - callSeq: 0, // monotonic call-id counter ("c1", "c2", ...) - registry: new Map(), // callId -> { id, kind, detail, optionsJson, createdAt, resolve, reject } - // The eval-await tracking surface (version 0.2.0): the registry - // entries' 'promise' field maps every registry promise - // (agent/checkpoint/steer) to its call id — the look-up table - // '__replAwait' resolves an awaited value against; 'awaitLog' is - // the chronological record of awaited call - // ids (the host's top-level-await instrumenter rewrites 'await x' - // into 'await __replAwait(x)', and the library logs every awaited - // value that IS one of its registry promises). The log is the - // 0.2.0-era targeting seam; the 0.3.0 library keeps it for older - // hosts (surface.awaitLogTake) while the broker's targeting rides - // the CONTINUATION LEASE (see '__replLease' below). - awaitLog: [], - // The CONTINUATION LEASE (version 0.3.0): the token of the eval - // whose continuation is about to run (set by '__replAwait''s - // wrap-settling reaction — the job immediately before the eval's - // continuation segment — and cleared by the host's drain loop - // after the segment ends). The host reads it between jobs; a job - // that starts with a lease set IS the armed eval's continuation — - // the eval-break interrupt's genuine per-eval identity (phase-E - // review rejection round 5: the signal used to be keyed to settled - // call ids, so an unawaited sibling '.then' job running before the - // target's continuation consumed it). Exposed as the writable - // '__replLease' accessor global (its getter/setter are this - // closure's — trusted host-installed code, never guest-authored). - continuationLease: undefined, - }; - - // Captured intrinsics: the registry is the host's settlement table, so - // its operations must stay immune to guest Map.prototype pollution (a - // guest that replaces Map.prototype.set must not be able to break - // settlement or the host's post-restore reconciliation reads). - var registryGet = Map.prototype.get; - var registrySet = Map.prototype.set; - var registryDelete = Map.prototype.delete; - var registryForEach = Map.prototype.forEach; - var registrySize = Object.getOwnPropertyDescriptor(Map.prototype, 'size').get; - - // Captured intrinsics for the argument gatherers. - // This library is evaluated exactly once, at VM creation, BEFORE any - // guest code runs — so everything captured here is pristine, and a guest - // that later pollutes a realm global or prototype cannot change what the - // captured functions do: - // - // - arraySlice is a BOUND copy of Array.prototype.slice (created via - // Function.prototype.call.bind at installation — both pristine). - // console.* and pipeline() gather their arguments through it; a guest - // that replaces Array.prototype.slice or Function.prototype.call with - // a throwing function must not make console.log (or pipeline) throw — - // console.* NEVER throws by contract (review regression, pinned by - // test). A bound function performs no property lookups at call time, - // so neither replacement can reach it. - var arraySlice = Function.prototype.call.bind(Array.prototype.slice); - var arrayFrom = Array.from.bind(Array); - - // The continuation-lease instrumentation's pristine PROMISE intrinsics - // (phase-E review rejection round 7): \`__replAwait\` / - // \`__replAwaitIterable\` mirror the awaited value through the realm's - // ORIGINAL Promise machinery — the captured constructor, the captured - // bound statics, and the captured \`then\` function value — never the - // guest-resolvable \`Promise\` global / \`Promise.resolve\` / public - // \`.then\`. A guest that REPLACES \`Promise.prototype.then\` (the - // reviewer's repro: the instrumented \`await 40\` returned \`99\` where - // the native evaluation returned \`40\`) or overwrites \`Promise.resolve\` - // itself, or SHADOWS \`Promise\` with a top-level lexical, must not - // change the instrumentation's semantics: the wrapped value mirrors - // natively and the continuation lease is still set. \`P\` is the SAME - // object as the realm's \`globalThis.Promise\`, so the statics are bound - // and the instance method captured as the bare function value at - // installation — later property replacement cannot reach them. - var P = Promise; - var PResolve = P.resolve.bind(P); - var PReject = P.reject.bind(P); - var pThen = P.prototype.then; - - // ──────────────────────────────────────────────────────────────────────── - // Small utilities - // ──────────────────────────────────────────────────────────────────────── - - function safeString(value) { - try { - return String(value); - } catch (_err) { - // String() throws for e.g. objects with a throwing toString/Symbol.toPrimitive. - } - try { - return Object.prototype.toString.call(value); - } catch (_err) { - // Even the brand fallback can throw (revoked proxies, all-trap proxies). - } - return '[unstringifiable]'; - } - - /** - * Normalize an arbitrary rejection value into an Error. Hosts may reject - * with a realm Error, a host Error marshalled into the realm, or a plain - * { name?, message, code?, recoverable? } object; all of them come out as - * an Error carrying code/recoverable when present. - */ - function toError(value) { - if (value instanceof Error) { - copyErrorAttribution(value, value); - return value; - } - if (value && typeof value === 'object') { - var err = new Error(typeof value.message === 'string' ? value.message : safeString(value)); - if (typeof value.name === 'string') err.name = value.name; - if (typeof value.stack === 'string') err.stack = value.stack; - if (value.code !== undefined) err.code = value.code; - if (value.recoverable !== undefined) err.recoverable = !!value.recoverable; - if (value.details !== undefined) err.details = value.details; - copyErrorAttribution(err, value); - return err; - } - return new Error(safeString(value)); - } - - /** - * Copy the §4.6 error-attribution fields onto a rejected call's Error: - * 'replBackend' is stamped by the host onto the rejection value (the - * resolved backend the subagent call failed on); 'replCallId' is the - * registry entry's own id, attached by settleCall (see there). Both - * render in the host's uncaught-error line so a failure that came from - * a subagent call names the call and its backend. - */ - function copyErrorAttribution(err, source) { - try { - if (typeof source.replBackend === 'string') err.replBackend = source.replBackend; - } catch (_e) {} - try { - if (typeof source.replCallId === 'string') err.replCallId = source.replCallId; - } catch (_e) {} - } - - /** - * A failure is recoverable unless the host said otherwise. Recoverable - * failures become null slots in parallel()/pipeline(); non-recoverable - * ones (recoverable: false) propagate and halt the surrounding - * orchestration. There is NO budget vocabulary in this guest (the - * roadmap doc deletes it): the recoverable flag is the only signal. - */ - function isRecoverable(err) { - return !(err && err.recoverable === false); - } - - // ──────────────────────────────────────────────────────────────────────── - // The pending-call registry and settlement - // ──────────────────────────────────────────────────────────────────────── - - /** - * Settle a pending call by id. Idempotent: the first settlement wins; a - * second settlement of the same id (e.g. the live deferred resolving - * after the reconciliation surface already settled it, or vice versa) - * returns false and does nothing. Returns true iff a pending entry was - * settled. - */ - function settleCall(callId, outcome, value) { - var entry = registryGet.call(state.registry, callId); - if (!entry) return false; - registryDelete.call(state.registry, callId); - if (outcome === 'resolve') entry.resolve(value); - else { - var err = toError(value); - // The §4.6 attribution: the rejecting call's id rides the error - // into the eval's uncaught-error rendering (the host stamps the - // backend). A guest-visible own property; a hostile realm that - // forges it is forging only its own error attribution. - if (typeof err.replCallId !== 'string') { - try { - err.replCallId = callId; - } catch (_e) {} - } - // The §4.6 submitted-code frames: the error was created HERE (its - // own stack names only library frames) — augment it with the - // CALL-SITE stack captured at issue time (see 'issueCall'), which - // carries the user's '' frames. The host's renderer filters - // to exactly those frames; the library frames in both halves are - // skipped. - if (entry && typeof entry.siteStack === 'string' && entry.siteStack.length > 0) { - try { - err.stack = - typeof err.stack === 'string' && err.stack.length > 0 - ? err.stack + '\\n' + entry.siteStack - : entry.siteStack; - } catch (_e) {} - } - entry.reject(err); - } - return true; - } - - /** - * Issue a host call: mint a call id, park {resolve, reject} in the - * registry, and invoke the host function. The host function may return a - * thenable (the quickjs-wasi Deferred idiom) — if it does, its settlement - * is forwarded into the registry. It may also return undefined and settle - * later purely through the reconciliation surface; both routes are always - * valid and converge on settleCall. - * - * 'detail' is kept VERBATIM in the registry entry (the prompt for agent - * calls, the question for checkpoints, the action for steering): after a - * restore the host may need it to re-issue work it lost track of, so it - * must not be truncated. 'sessionId' is the id the HOST addresses the - * call by — the founding call id for steering calls (the session being - * steered), the call's own id otherwise — and 'modelSpec' the agent - * call's backend-routing spec (null otherwise); both are recorded in the - * entry so a pending call survives a snapshot/restore with full - * correlation. - * - * 'hostArgs' builds the actual host invocation (each __host_* function - * has its own argument layout): it receives the host function and the - * freshly minted id and returns the host result. - */ - function issueCall(kind, hostFnName, detail, optionsJson, sessionId, modelSpec, hostArgs) { - var hostFn = g[hostFnName]; - if (typeof hostFn !== 'function') { - throw new Error( - hostFnName + ' is not installed — the host must register it before evaluating ' + - 'guest code (and re-register it by name after every snapshot restore)', - ); - } - // The §4.6 CALL-SITE STACK: captured HERE, synchronously inside the - // library function the user code invoked, so the stack carries the - // SUBMITTED-CODE frames (the agent()/steer()/checkpoint() call site - // at '') below the library's own. A rejection's Error is - // created at settlement time in this library (its own stack names - // only library frames) — settleCall AUGMENTS it with these frames so - // the host's uncaught-error rendering can show line numbers in the - // submitted code. Best-effort: a failure to capture leaves the - // rejection with its library-only stack. - var siteStack; - try { - siteStack = new Error('repl call site').stack; - } catch (_err) { - siteStack = undefined; - } - var id = 'c' + ++state.callSeq; - var resolveFn; - var rejectFn; - var promise = new Promise(function (resolve, reject) { - resolveFn = resolve; - rejectFn = reject; - }); - registrySet.call(state.registry, id, { - id: id, - kind: kind, - detail: detail, - optionsJson: optionsJson === undefined ? null : optionsJson, - siteStack: siteStack, - createdAt: Date.now(), - // The id the host addresses this call by: the founding session id for - // steering calls, this call's own id for everything else. Recorded so - // the pending-call manifest never omits the correlation the host - // needs to settle or restore each pending operation after a restart. - sessionId: sessionId === undefined ? id : sessionId, - modelSpec: modelSpec === undefined ? null : modelSpec, - resolve: resolveFn, - reject: rejectFn, - // Track the promise for the eval-await attribution ('__replAwait'): - // the registry entry carries the exact promise the settlement - // resolves, so awaiting THIS promise (or awaiting it again from a - // later eval — the "running eval awaiting an earlier binding" - // case) is attributable by identity. The 'promise' field is never - // exposed by the pending() manifest (it builds explicit fields) — - // it is closure-internal bookkeeping. - promise: promise, - }); - var returned; - try { - returned = hostArgs(hostFn, id); - } catch (err) { - // Synchronous host refusal (e.g. a per-call cap enforced at dispatch). - settleCall(id, 'reject', err); - return { id: id, promise: promise }; - } - if (returned && typeof returned.then === 'function') { - // Adopt the host-returned thenable (the quickjs-wasi Deferred - // idiom) through the CAPTURED pristine Promise surface - // ('PResolve'/'pThen' — phase-E review rejection round 7): the - // guest-visible 'Promise.resolve(returned).then(...)' broke under - // a replaced 'Promise.prototype.then' — the reactions were never - // registered, so a settled host call NEVER reached the registry - // and the awaiting eval stayed pending forever (the reviewer's - // repro: replacing 'Promise.prototype.then' made the - // instrumented 'await 40' return '99'; the SAME mutation also - // silently killed every settlement through this forwarding). - pThen.call( - PResolve(returned), - function (value) { settleCall(id, 'resolve', value); }, - function (err) { settleCall(id, 'reject', err); }, - ); - } - return { id: id, promise: promise }; - } - - // ──────────────────────────────────────────────────────────────────────── - // agent() — the delegation primitive, and the live handle - // ──────────────────────────────────────────────────────────────────────── - - /** Shallow-copy options (plain data by construction — they cross the - * bridge as JSON). */ - function normalizeAgentOptions(options, knownKey1, knownKey2, knownKey3, knownKey4) { - if (options === undefined || options === null) return undefined; - if (typeof options !== 'object') { - throw new TypeError('agent options must be an object'); - } - var out = {}; - var keys = Object.keys(options); - for (var i = 0; i < keys.length; i++) { - var key = keys[i]; - var optionValue = options[key]; - var knownKey = key === knownKey1 || key === knownKey2 || key === knownKey3 || key === knownKey4; - // Classify the top-level key BEFORE JSON serialization can erase it. - // Known undefined/function/symbol values keep ordinary JSON omission - // semantics (the host treats them as absent). An unknown key must - // survive long enough for host admission validation to reject it and - // enumerate the valid vocabulary, even when its value is otherwise - // not JSON-representable. - out[key] = !knownKey && - (optionValue === undefined || - typeof optionValue === 'function' || - typeof optionValue === 'symbol' || - typeof optionValue === 'bigint') - ? null - : optionValue; - } - return out; - } - - function issueAgentCall(modelSpec, task, options) { - if (typeof modelSpec !== 'string') { - throw new TypeError( - 'agent(modelSpec, task, options?) needs a model spec string (e.g. "pi/deepseek-v4-flash-max")', - ); - } - if (typeof task !== 'string') { - throw new TypeError('agent(modelSpec, task, options?) needs a task string'); - } - var normalized = normalizeAgentOptions(options, 'schema', 'cwd', 'configOptions', 'mode'); - // Options cross the bridge as JSON: plain data by construction, one - // flat, unambiguous decoding host-side (functions or cycles in options - // would be meaningless host-side). normalizeAgentOptions has already - // preserved non-representable UNKNOWN top-level keys for host admission; - // ordinary JSON semantics intentionally erase undefined KNOWN values, so - // the { cwd: maybeCwd } idiom continues to mean "omit cwd" when maybeCwd is - // undefined. - var optionsJson = normalized === undefined ? undefined : JSON.stringify(normalized); - // The registry entry records the model spec verbatim so a restore can - // re-issue the call against the same backend routing. The host - // receives (callId, modelSpec, task, optionsJson). - return issueCall('agent', '__host_agent', task, optionsJson, undefined, modelSpec, function (hostFn, id) { - return hostFn(id, modelSpec, task, optionsJson); - }); - } - - /** Preserve malformed option bags for host admission. Queue validation - * is host-owned because even a refused queue call must mint an id and - * receive a durable call-store record. */ - function normalizeTurnOptions(options) { - if (options === undefined || options === null) return options; - if (typeof options !== 'object') return options; - return normalizeAgentOptions(options, 'promptMeta'); - } - - /** Serialize without preventing ID minting. Even cyclic/BigInt malformed input reaches host - * admission as a durable validation refusal rather than throwing before issueCall(). */ - function turnPayloadJson(prompt, options) { - try { - var normalized = normalizeTurnOptions(options); - return JSON.stringify({ prompt: prompt, options: normalized }); - } catch (_err) { - return JSON.stringify({ prompt: null, options: null, serializationError: true }); - } - } - - /** Mint a queue/steer control call. No semantic validation happens here: - * the host records the dispatch before accepting or refusing it. */ - function turnCall(kind, hostFnName, foundingCallId, prompt, options) { - try { - var payloadJson = turnPayloadJson(prompt, options); - return issueCall(kind, hostFnName, typeof prompt === 'string' ? prompt : safeString(prompt), payloadJson, foundingCallId, null, function (hostFn, id) { - return hostFn(id, foundingCallId, payloadJson); - }).promise; - } catch (err) { - return Promise.reject(err); - } - } - - function cancelSessionCall(foundingCallId) { - return issueCall('cancel', '__host_agent_cancel', 'session', null, foundingCallId, null, function (hostFn, id) { - return hostFn(id, foundingCallId); - }).promise; - } - - function cancelQueueCall(queueCallId) { - return issueCall('cancel', '__host_queue_cancel', 'queue', null, queueCallId, null, function (hostFn, id) { - return hostFn(id, queueCallId); - }).promise; - } - - function queuedTurnHandle(foundingCallId, prompt, options) { - var call = (function () { - try { - var payloadJson = turnPayloadJson(prompt, options); - return issueCall('queue', '__host_agent_queue', typeof prompt === 'string' ? prompt : safeString(prompt), payloadJson, foundingCallId, null, function (hostFn, id) { - return hostFn(id, foundingCallId, payloadJson); - }); - } catch (err) { - return { id: undefined, promise: PReject(err) }; - } - })(); - var handle = call.promise; - if (call.id !== undefined) { - Object.defineProperties(handle, { - id: { - value: call.id, - writable: false, - enumerable: false, - configurable: false, - }, - cancel: { - value: function () { return cancelQueueCall(call.id); }, - writable: false, - enumerable: false, - configurable: false, - }, - }); - } - return handle; - } - - /** - * Run one worker agent to completion and resolve with its result (final - * text, or the schema-validated object when options.schema is given — - * result shaping is host policy). 'modelSpec' is the backend-routing - * spec ("pi/deepseek-v4-flash-max"); 'task' is the worker's prompt; - * 'options' (structured-output schema, cwd, backend config) cross the - * bridge as JSON. Recoverable worker failures reject with an Error - * whose recoverable is not false. - * - * The returned promise IS the live handle: it may sit in a REPL variable - * across turns (and across snapshot/restore) and be awaited, or driven - * with the handle methods queue/steer/cancel (own, non-enumerable - * properties of the promise; 'id' carries the stable call id). - */ - function agent(modelSpec, task, options) { - try { - var call = issueAgentCall(modelSpec, task, options); - var handle = call.promise; - Object.defineProperties(handle, { - id: { - value: call.id, - writable: false, - enumerable: false, - configurable: false, - }, - queue: { - value: function (nextPrompt, nextOptions) { - return queuedTurnHandle(call.id, nextPrompt, nextOptions); - }, - writable: false, - enumerable: false, - configurable: false, - }, - steer: { - value: function (nextPrompt, nextOptions) { - return turnCall('steer', '__host_agent_steer', call.id, nextPrompt, nextOptions); - }, - writable: false, - enumerable: false, - configurable: false, - }, - cancel: { - value: function () { - return cancelSessionCall(call.id); - }, - writable: false, - enumerable: false, - configurable: false, - }, - }); - return handle; - } catch (err) { - return Promise.reject(err); - } - } - - // ──────────────────────────────────────────────────────────────────────── - // checkpoint() — the data plane interrupting the intent plane - // ──────────────────────────────────────────────────────────────────────── - - /** - * Raise a question from a running orchestration up into the - * conversation. The returned promise resolves with the user's answer, - * delivered by the host whenever it arrives — possibly turns later, - * possibly after a snapshot/restore cycle (the pending entry travels in - * the registry like any agent call). 'options' (e.g. { choices, default }) - * are host policy, passed through as JSON. - */ - function checkpoint(question, options) { - try { - if (typeof question !== 'string') { - throw new TypeError('checkpoint(question, options?) needs a question string'); - } - var optionsJson = - options === undefined || options === null ? undefined : JSON.stringify(options); - return issueCall('checkpoint', '__host_checkpoint', question, optionsJson, undefined, null, function (hostFn, id) { - return hostFn(id, question, optionsJson); - }).promise; - } catch (err) { - return Promise.reject(err); - } - } - - /** - * Deliver the user's answer for a pending checkpoint, by call id — the - * orchestrator calls this from an eval after the user replies in the - * conversation. Answering is a root-mediated act: the host never captures - * user text as an answer; the answer enters the data plane only through - * this call (the __host_checkpoint answer mode: the same host function - * the question left through, with the JSON-encoded answer as a fourth - * argument). - * - * Returns true iff a checkpoint with that id was pending when the call - * was made; false for unknown or already-answered ids. Delivery is - * first-wins idempotent (the settlement rule every call follows), and - * the checkpoint's promise resolves with 'value' during the same - * evaluation's settlement flush. No registry entry is minted: nothing - * new pends, and a snapshot can never capture an answer in flight. - */ - checkpoint.answer = function answer(callId, value) { - if (typeof callId !== 'string' || callId.length === 0) { - throw new TypeError('checkpoint.answer(callId, value) needs a call id string (e.g. "c3")'); - } - if (typeof g.__host_checkpoint !== 'function') { - throw new Error( - '__host_checkpoint is not installed — the host must register it before evaluating ' + - 'guest code (and re-register it by name after every snapshot restore)', - ); - } - // The answer crosses the bridge as JSON (plain data by construction); - // undefined normalizes to null so the mode marker — a PRESENT fourth - // argument — is unambiguous. - var answerJson; - try { - answerJson = JSON.stringify(value === undefined ? null : value); - } catch (_err) { - throw new TypeError( - 'checkpoint.answer(callId, value): value must be JSON-serializable', - ); - } - return !!g.__host_checkpoint(callId, undefined, undefined, answerJson); - }; - - // ──────────────────────────────────────────────────────────────────────── - // The eval-plane helpers: sleep (host-side timer — the VM itself stays - // timer-free), the introspection pair (workspace()/agents() — plain - // values served by the host as JSON), and reset (teardown after the - // current eval completes). - // ──────────────────────────────────────────────────────────────────────── - - /** - * Sleep for 'ms' milliseconds: the universal idiom agents reach for, - * implemented HOST-side (the VM itself stays timer-free — the promise - * is settled by a host timer through '__host_sleep'). Returns a - * promise resolving undefined after the host timer fires; the eval's - * continuation resumes at the next settlement drain, exactly like a - * subagent call's. - */ - function sleep(ms) { - try { - if (typeof ms !== 'number' || !Number.isFinite(ms) || ms < 0) { - throw new TypeError('sleep(ms) needs a non-negative number of milliseconds'); - } - if (typeof g.__host_sleep !== 'function') { - throw new Error( - '__host_sleep is not installed — the host must register it before evaluating ' + - 'guest code (and re-register it by name after every snapshot restore)', - ); - } - return g.__host_sleep(ms); - } catch (err) { - // Like agent(): the validation failure is a REJECTED promise, never - // a synchronous throw from the DSL surface. - return Promise.reject(err); - } - } - - /** - * The introspection host round-trip: call the host function, parse the - * returned JSON string, hand back the plain value. The value is an - * ORDINARY object/array in the realm — sliceable in the same eval - * (the 'dir()' / '%who' idiom). - */ - function introspect(hostName, apiName) { - var hostFn = g[hostName]; - if (typeof hostFn !== 'function') { - throw new Error( - hostName + ' is not installed — the host must register it before evaluating ' + - 'guest code (and re-register it by name after every snapshot restore)', - ); - } - var raw = hostFn(); - if (typeof raw !== 'string') { - throw new TypeError(apiName + ': the host returned a non-string (host contract violation)'); - } - return JSON.parse(raw); - } - - /** - * The workspace manifest as a plain value (the roadmap doc's 'status' - * replacement): { bindings, inFlight, checkpoints, diagnostics } — see - * the doc for the exact shape. Bindings are name/type/size/provenance/ - * task/callId/status records (the status is the honest one — 'failed' - * for rejected handle calls). - */ - function workspace() { - return introspect('__host_workspace', 'workspace()'); - } - - /** - * The live subagents as a plain value: one { callId, modelSpec, task, - * state, supportsSteering, queuedTurns } entry per live agent, - * including every unsettled queued turn (each with its own - * addressable call id). - */ - function agents() { - return introspect('__host_agents', 'agents()'); - } - - /** - * Ask the host to tear the workspace down AFTER the current eval - * completes (the host-side effect the roadmap doc's deleted 'reset' - * action performed). Returns nothing meaningful; the eval that called - * this still completes normally first. - */ - function reset() { - if (typeof g.__host_reset !== 'function') { - throw new Error( - '__host_reset is not installed — the host must register it before evaluating ' + - 'guest code (and re-register it by name after every snapshot restore)', - ); - } - g.__host_reset(); - return undefined; - } - - // ──────────────────────────────────────────────────────────────────────── - // Combinators — pure JavaScript over agent(). No host effects of their - // own; every one of them bottoms out in agent() (or in caller-supplied - // thunks). Semantics follow packages/workflows/src/dsl.d.ts, adapted for - // the persistent REPL (no run, no journal, no phases). - // ──────────────────────────────────────────────────────────────────────── - - /** - * Run an array of THUNKS concurrently; resolve to their results in input - * order. Pass functions, not promises: parallel([() => agent("a"), ...]). - * A recoverable failure becomes null in its slot (reported via - * console.warn); a non-recoverable one rejects the whole parallel(). - */ - async function parallel(thunks) { - if (!Array.isArray(thunks)) { - throw new TypeError('parallel() expects an array of functions'); - } - if (thunks.some(function (t) { return typeof t !== 'function'; })) { - throw new TypeError( - 'parallel() expects an array of functions, not promises. Wrap each call: () => agent(...)', - ); - } - return Promise.all( - thunks.map(async function (thunk, index) { - try { - return await thunk(); - } catch (error) { - var err = toError(error); - if (!isRecoverable(err)) throw err; - // Closure-internal reference (not the global): sabotaging - // console.warn must not be able to alter parallel()'s semantics. - consoleObject.warn('parallel[' + index + '] failed: ' + err.message); - return null; - } - }), - ); - } - - /** - * Map 'items' through one or more sequential async 'stages', concurrently - * across items. Each stage receives (prev, original, index). Resolves to - * the final value per item; a recoverable per-item failure yields null - * for that item, a non-recoverable one rejects the whole pipeline(). - */ - async function pipeline(items) { - // Captured intrinsic (see the captured-intrinsics note): guest - // pollution of Array.prototype.slice / Function.prototype.call must - // not be able to break the combinators. - var stages = arraySlice(arguments, 1); - if (!Array.isArray(items)) { - throw new TypeError('pipeline() expects an array as the first argument'); - } - if (stages.some(function (s) { return typeof s !== 'function'; })) { - throw new TypeError( - 'pipeline() stages must be functions: pipeline(items, item => ..., result => ...)', - ); - } - return Promise.all( - items.map(async function (item, index) { - var value = item; - for (var i = 0; i < stages.length; i++) { - try { - value = await stages[i](value, item, index); - } catch (error) { - var err = toError(error); - if (!isRecoverable(err)) throw err; - consoleObject.warn('pipeline[' + index + '] failed: ' + err.message); - return null; - } - } - return value; - }), - ); - } - - var VERIFY_SCHEMA = { - type: 'object', - properties: { real: { type: 'boolean' }, reason: { type: 'string' } }, - required: ['real'], - }; - - /** - * The reviewer/grader model spec for verify/judgePanel: the HOST's - * configured default backend id, served by '__host_default_backend' - * (§4.7 — the DSL options carry no per-call model, so the spawned - * workers inherit the run's default model; §4.1 — the spec is a REAL - * registered backend segment, validated at admission like any agent() - * call; the v1 reserved 'default' sentinel that bypassed registry - * validation is deleted). A host with no backend registry (the parking - * bridge) returns undefined and the combinators reject NON-recoverably - * (they cannot work without a backend). - */ - function defaultBackendSpec() { - if (typeof g.__host_default_backend !== 'function') { - var err = new Error( - '__host_default_backend is not installed — the host must register it before evaluating ' + - 'guest code (and re-register it by name after every snapshot restore)', - ); - err.recoverable = false; - throw err; - } - var id = g.__host_default_backend(); - if (typeof id !== 'string' || id.length === 0) { - var err2 = new Error( - 'verify/judgePanel need a default backend, but no backend registry is attached to this workspace', - ); - err2.recoverable = false; - throw err2; - } - return id; - } - - /** - * Adversarial verification panel: 'reviewers' workers vote on whether - * 'item' is real/correct; passes when the share voting real meets - * 'threshold'. Reviewers that fail recoverably are dropped from the vote - * (they are neither yes nor no). - * - * The DSL options are EXACTLY { reviewers, threshold, lens } - * (packages/workflows/src/dsl.d.ts) — there is no per-call model option - * (an invented opts.model was removed in review; the dsl.d.ts verify - * lets reviewers inherit the run's default model, so the spawned - * reviewers route through the host's configured default backend id — - * a real registered segment, never the deleted 'default' sentinel). - */ - async function verify(item, opts) { - opts = opts || {}; - var reviewers = Math.max(1, opts.reviewers !== undefined ? opts.reviewers : 2); - var threshold = opts.threshold !== undefined ? opts.threshold : 0.5; - var lenses = opts.lens ? (Array.isArray(opts.lens) ? opts.lens : [opts.lens]) : []; - var modelSpec = defaultBackendSpec(); - var claim; - if (typeof item === 'string') { - claim = item; - } else { - try { - claim = JSON.stringify(item); - } catch (_err) { - // Non-serializable item (circular, hostile) — degrade to a safe - // string rather than failing the whole panel. - claim = safeString(item); - } - } - var votes = ( - await parallel( - Array.from({ length: reviewers }, function (_v, i) { - return function () { - return agent( - modelSpec, - 'Adversarially review whether the following is REAL/correct. Try to refute it; default to real=false if unsure.' + - (lenses.length ? ' Focus lens: ' + lenses[i % lenses.length] + '.' : '') + - '\\n\\n' + claim, - { schema: VERIFY_SCHEMA }, - ); - }; - }), - ) - ).filter(Boolean); - var realCount = votes.filter(function (v) { return v && v.real; }).length; - return { - real: votes.length > 0 && realCount / votes.length >= threshold, - realCount: realCount, - total: votes.length, - votes: votes, - }; - } - - var JUDGE_SCHEMA = { - type: 'object', - properties: { score: { type: 'number' }, reason: { type: 'string' } }, - required: ['score'], - }; - - /** - * LLM-judge panel: score each candidate in 'attempts' with 'judges' - * graders against 'rubric' and return the highest mean-scoring candidate - * as { index, attempt, score, judgments } (stable tie-break by index). - * The DSL options are EXACTLY { judges, rubric } - * (packages/workflows/src/dsl.d.ts) — no per-call model option; the - * spawned graders route through the host's configured default backend id - * (same decision as verify). - */ - async function judgePanel(attempts, opts) { - opts = opts || {}; - var judges = Math.max(1, opts.judges !== undefined ? opts.judges : 3); - var rubric = opts.rubric !== undefined ? opts.rubric : 'overall quality and correctness'; - var modelSpec = defaultBackendSpec(); - var scored = ( - await parallel( - (Array.isArray(attempts) ? attempts : []).map(function (att, idx) { - return async function () { - var text = typeof att === 'string' ? att : JSON.stringify(att); - var js = ( - await parallel( - Array.from({ length: judges }, function (_v, j) { - return function () { - return agent( - modelSpec, - 'Score this candidate from 0 to 1 on: ' + rubric + - '. Reply with the score.\\n\\nCandidate:\\n' + text, - { schema: JUDGE_SCHEMA }, - ); - }; - }), - ) - ).filter(Boolean); - var score = js.length - ? js.reduce(function (s, v) { return s + (Number(v && v.score) || 0); }, 0) / js.length - : 0; - return { index: idx, attempt: att, score: score, judgments: js }; - }; - }), - ) - ).filter(Boolean); - // Highest mean score; stable tie-break by input index. - var best = scored[0]; - for (var i = 0; i < scored.length; i++) { - var s = scored[i]; - if (s.score > best.score || (s.score === best.score && s.index < best.index)) best = s; - } - return best; - } - - /** Dedupe key default: JSON identity, degrading to a safe string for - * non-serializable items (a circular item must not kill the loop). */ - function defaultKey(x) { - try { - return JSON.stringify(x); - } catch (_err) { - return safeString(x); - } - } - - /** - * Repeatedly invoke round(i), collecting fresh (deduped by 'key') items - * until it yields nothing 'consecutiveEmpty' rounds in a row (or - * 'maxRounds' is hit). Returns every unique item gathered. Round - * failures propagate (recoverable ones are not nulled — a round is the - * loop's contract, not a slot). - */ - async function loopUntilDry(opts) { - if (!opts || typeof opts.round !== 'function') { - throw new TypeError('loopUntilDry requires { round: (i) => items[] }'); - } - var key = opts.key || defaultKey; - var consecutiveEmpty = Math.max(1, opts.consecutiveEmpty !== undefined ? opts.consecutiveEmpty : 2); - var maxRounds = opts.maxRounds !== undefined ? opts.maxRounds : 50; - var seen = new Set(); - var all = []; - var dry = 0; - for (var r = 0; r < maxRounds && dry < consecutiveEmpty; r++) { - var items = (await opts.round(r)) || []; - var fresh = (Array.isArray(items) ? items : []).filter(function (x) { - return x !== null && x !== undefined && !seen.has(key(x)); - }); - if (!fresh.length) { - dry++; - continue; - } - dry = 0; - for (var i = 0; i < fresh.length; i++) { - var k = key(fresh[i]); - if (seen.has(k)) continue; // within-round duplicates stay deduped - seen.add(k); - all.push(fresh[i]); - } - } - return all; - } - - /** - * Bounded retry: call thunk(attempt) up to 'attempts' times, stopping - * early once until(result) holds. Without 'until' the FIRST result is - * accepted — exactly the repository DSL behavior - * (workflow-engine/src/workflow.ts: if (!opts.until || opts.until(last)) - * return last); the final return last only runs when an 'until' - * predicate was given and never held (attempts exhausted — the caller - * inspects the last result). No backoff: there is no timer in the realm - * and delegation retries gain nothing from delay. - */ - async function retry(thunk, opts) { - opts = opts || {}; - var attempts = Math.max(1, opts.attempts !== undefined ? opts.attempts : 3); - var last; - for (var i = 0; i < attempts; i++) { - last = await thunk(i); - if (!opts.until || opts.until(last)) return last; - } - return last; - } - - /** - * Validation gate: call thunk(feedback, attempt), validate the result, - * and feed the validator's feedback into the next attempt until it - * passes or 'attempts' run out. The verdict may be a boolean or - * { ok, feedback? }. Returns { ok, value, verdict, attempts }. - */ - async function gate(thunk, validator, opts) { - opts = opts || {}; - var attempts = Math.max(1, opts.attempts !== undefined ? opts.attempts : 3); - var feedback; - var last; - var lastVerdict = null; - for (var i = 0; i < attempts; i++) { - last = await thunk(feedback, i); - lastVerdict = await validator(last); - var accepted = - typeof lastVerdict === 'boolean' ? lastVerdict : Boolean(lastVerdict && lastVerdict.ok); - if (accepted) { - return { - ok: true, - value: last, - verdict: lastVerdict === undefined ? null : lastVerdict, - attempts: i + 1, - }; - } - feedback = - typeof lastVerdict === 'boolean' - ? undefined - : lastVerdict - ? lastVerdict.feedback - : undefined; // fed into the next attempt - } - return { - ok: false, - value: last, - verdict: lastVerdict === undefined ? null : lastVerdict, - attempts: attempts, - }; - } - - // ──────────────────────────────────────────────────────────────────────── - // ──────────────────────────────────────────────────────────────────────── - // The §4.4 depth-limited repr (docs/roadmap/repl-eval-redesign.md): - // printing conventions, not budgets — there is NO byte ceiling anywhere - // on this path. The rules, chosen for familiarity with Python's defaults: - // - // - strings passed DIRECTLY to console.* print WHOLE, unquoted (they - // are the output the orchestrator asked for); - // - objects/arrays render to DEPTH 2; deeper levels render as - // '{…}' / '[…]'; - // - collections render their first 20 entries per level, then - // '… +N more'; - // - NESTED strings (inside a collection) render head-limited at - // 200 chars, quoted, with a trailing '…' marker when clipped; - // - everything deeper/longer is reached by evaluating a narrower - // expression — the values are alive in the VM; slicing is the API. - // ──────────────────────────────────────────────────────────────────────── - - var REPR_DEPTH_LIMIT = 2; // expand levels 0..1, collapse level 2+ - var REPR_ENTRY_LIMIT = 20; // entries rendered per level - var REPR_NESTED_STRING_CHARS = 200; // nested-string head bound - - /** The depth-limited repr of one value ('depth' is the distance from - * the top-level console argument). NEVER throws — a hostile value - * degrades to '[unstringifiable]' (console.* never throws by - * contract). */ - function reprValue(value, depth, seen) { - try { - var t = typeof value; - if (value === null) return 'null'; - if (t === 'string') { - if (depth === 0) return value; // direct strings print whole - var stringChars = arrayFrom(value); - if (stringChars.length <= REPR_NESTED_STRING_CHARS) return "'" + value + "'"; - return "'" + stringChars.slice(0, REPR_NESTED_STRING_CHARS).join('') + "…'"; - } - if (t === 'undefined') return 'undefined'; - if (t === 'number') return value === 0 && 1 / value === -Infinity ? '-0' : String(value); - if (t === 'boolean') return value ? 'true' : 'false'; - if (t === 'bigint') return String(value) + 'n'; - if (t === 'symbol') return 'Symbol'; - if (t === 'function') { - var fnName = '(anonymous)'; - try { - fnName = value.name || '(anonymous)'; - } catch (_err) { - // A proxy-of-function with a throwing get trap. - } - return 'ƒ ' + fnName + '()'; - } - if (t !== 'object') return safeString(value); - // Objects/arrays below. - if (depth >= REPR_DEPTH_LIMIT) { - return Array.isArray(value) ? '[…]' : '{…}'; - } - if (seen.has(value)) { - // A cycle (or a shared reference already rendered at a - // shallower depth): collapse instead of recursing forever. - return Array.isArray(value) ? '[…]' : '{…}'; - } - seen.add(value); - try { - // Branded objects render as their brand word (the predictable - // leaf, like Python's '' — their content is reached - // by slicing a narrower expression). - if (value instanceof Error) { - var eName = typeof value.name === 'string' ? value.name : 'Error'; - var eMessage = typeof value.message === 'string' ? value.message : ''; - var errorBody = eMessage === '' ? eName : eName + ': ' + eMessage; - // The §4.6 attribution: an error that came from a subagent - // call names the call (the library stamps replCallId on every - // rejected registry call) and the resolved backend (the host - // stamps replBackend) — visible wherever the error renders. - if (typeof value.replCallId === 'string') { - errorBody += ' (call ' + value.replCallId; - if (typeof value.replBackend === 'string') errorBody += ' on backend ' + value.replBackend; - errorBody += ')'; - } - var errorChars = arrayFrom(errorBody); - return errorChars.length <= REPR_NESTED_STRING_CHARS - ? errorBody - : errorChars.slice(0, REPR_NESTED_STRING_CHARS).join('') + '…'; - } - if (value instanceof Promise) return 'Promise'; - if (value instanceof Date) return 'Date'; - if (value instanceof RegExp) return 'RegExp'; - if (value instanceof Map) return 'Map'; - if (value instanceof Set) return 'Set'; - if (value instanceof WeakMap) return 'WeakMap'; - if (value instanceof WeakSet) return 'WeakSet'; - if (value instanceof ArrayBuffer) return 'ArrayBuffer'; - if (ArrayBuffer.isView(value)) return 'TypedArray'; - if (Array.isArray(value)) { - var parts = []; - var n = Math.min(value.length, REPR_ENTRY_LIMIT); - for (var i = 0; i < n; i++) parts.push(reprValue(value[i], depth + 1, seen)); - if (value.length > n) parts.push('… +' + (value.length - n) + ' more'); - return '[' + parts.join(', ') + ']'; - } - var keys; - try { - keys = Object.keys(value); - } catch (_err) { - return '{…}'; // a proxy's ownKeys trap threw — collapse - } - var objParts = []; - var kn = Math.min(keys.length, REPR_ENTRY_LIMIT); - for (var j = 0; j < kn; j++) { - objParts.push(keys[j] + ': ' + reprValue(value[keys[j]], depth + 1, seen)); - } - if (keys.length > kn) objParts.push('… +' + (keys.length - kn) + ' more'); - return '{' + objParts.join(', ') + '}'; - } finally { - seen.delete(value); - } - } catch (_err) { - return '[unstringifiable]'; - } - } - - /** - * The guest half of the console bridge: ONE joined line per call — - * the arguments' reprs joined with a single space (the doc deletes the - * per-argument '$N' capture system). The line is forwarded to - * __host_console as the JSON payload { line }. console.* NEVER throws - * — a broken value or a missing/failing host sink must not take down - * guest code; every argument renders under its own guard. - */ - function emitLog(level, args) { - try { - var line = ''; - var seen = new Set(); - for (var i = 0; i < args.length; i++) { - if (i > 0) line += ' '; - line += reprValue(args[i], 0, seen); - } - if (typeof g.__host_console === 'function') { - g.__host_console(level, JSON.stringify({ line: line })); - } - } catch (_err) { - // Deliberately swallowed: the bridge is best-effort by contract. - } - } - - var consoleObject = {}; - ['log', 'info', 'warn', 'error', 'debug'].forEach(function (level) { - consoleObject[level] = function () { - // Captured intrinsic (see the captured-intrinsics note): the - // bridge contract is console.* NEVER throws, and a guest that - // replaces Array.prototype.slice (or Function.prototype.call) with - // a throwing function must not be able to break it (review - // regression, pinned by test). ONE joined line per call — the - // arguments' reprs joined with a single space (§4.4). - emitLog(level, arraySlice(arguments)); - }; - }); - // Method-level sabotage protection: the console global is non-writable - // (installGlobal), and the OBJECT is frozen so its methods cannot be - // reassigned or deleted either — combinator diagnostics (parallel/ - // pipeline warn on swallowed failures) always reach the bridge. - Object.freeze(consoleObject); - - // ──────────────────────────────────────────────────────────────────────── - // The eval-await tracking: '__replAwait' — the global the host's - // top-level-await instrumenter inserts around every top-level 'await' - // ('await x' → 'await __replAwait(x, TOKEN)' — the 0.3.0 form; the - // 0.2.0 host inserted no token). With a TOKEN the awaited value is - // WRAPPED in a fresh promise: the wrap's settling reaction — the job - // that runs IMMEDIATELY BEFORE the eval's continuation segment (the - // reaction is registered at the await, so earlier-registered - // reactions — an unawaited sibling's '.then' — run first) — sets the - // CONTINUATION LEASE to the eval's token. The host's drain loop reads - // the lease between jobs: the segment starts with the lease set (the - // eval-break interrupt's genuine continuation identity — it can only - // fire while THIS eval's continuation executes) and the host clears - // it after the segment ends. The wrap also makes INDIRECT awaits - // targetable — 'await Promise.all([q])' wraps the combinator's - // promise, and its settlement queues the eval's continuation exactly - // like a direct call's: the identity is the promise graph, not a - // logged call-id list (phase-E review rejection round 5: the 0.2.0 - // log refused indirect waits). The 0.2.0 no-token form passes the - // value through and logs registry promises (an older host still - // drives the log). NEVER throws: a wrap failure (guest promise - // sabotage) degrades to the unwrapped value, so guest semantics are - // preserved either way. - // ──────────────────────────────────────────────────────────────────────── - - function setContinuationLease(token) { - state.continuationLease = token; - } - - function replAwait(value, token) { - try { - if (typeof token === 'string' && token.length > 0) { - // The 0.3.0 continuation-lease form: wrap the awaited value in - // a fresh promise. The CONTINUATION LEASE is set by a reaction - // registered on the WRAPPER itself — BEFORE the await machinery - // registers its own reaction on the wrapper (the machinery's - // registration happens at the await site, after this function - // returns). The wrapper's settlement therefore queues the - // lease-setting job IMMEDIATELY BEFORE the machinery job that - // runs the eval's continuation segment: the job after the - // lease-setting reaction IS the segment, and NO job queued - // between the awaited value's settlement and the wrapper's - // settlement can run with the lease set (phase-E review - // rejection round 6: the 0.3.0 reaction set the lease inside - // the job that resolved the wrapper — the job right after the - // awaited value's settlement — so a sibling \`q.then(...)\` - // registered AFTER the eval started awaiting \`q\` ran between - // the lease set and the continuation, consumed the armed - // signal, and the target's continuation ran later unprotected; - // the lease is now associated with the actual continuation - // job, not with whichever job runs next). The wrapper mirrors - // the value (identity for the resolution value, same rejection - // value) — the async machinery sees exactly what it would have - // seen. The mirroring machinery is the CAPTURED pristine - // Promise surface ('P'/'PResolve'/'pThen' — see the captures at - // the top of the library): a guest that replaces - // 'Promise.prototype.then' or overwrites 'Promise.resolve' (or - // shadows 'Promise' with a top-level lexical) must not change - // the instrumentation's semantics — the reviewer's repro: - // replacing 'Promise.prototype.then' made the instrumented - // 'await 40' return '99' while the native evaluation returned - // '40' (phase-E review rejection round 7). The wrapper is - // adopted by the guest's own await machinery through its - // INTERNAL promise reactions, so the guest-visible prototype - // cannot intercept the continuation either way; the lease- - // setting reaction rides the pristine 'then' function value, so - // a replaced prototype cannot skip it. - var wrapper = new P(function (resolve, reject) { - try { - pThen.call(PResolve(value), resolve, reject); - } catch (e) { - reject(e); - } - }); - pThen.call( - wrapper, - function () { - try { - setContinuationLease(token); - } catch (_e) {} - }, - function () { - try { - setContinuationLease(token); - } catch (_e) {} - }, - ); - return wrapper; - } - // The 0.2.0 form (no token): record the awaited call id when the - // awaited value is one of this library's registry promises and - // otherwise pass the value through untouched. - if (value !== null && (typeof value === 'object' || typeof value === 'function')) { - // Identity scan of the registry (see the state note): the - // awaited value is one of this library's promises iff it is - // some pending entry's 'promise'. A guest cannot forge an - // entry (the registry is closure-private), so attribution is - // precise: only values the LIBRARY minted are logged. - var found = null; - registryForEach.call(state.registry, function (entry) { - if (found === null && entry.promise === value) found = entry.id; - }); - if (found !== null) state.awaitLog.push(found); - } - } catch (_err) { - // Never throws by contract: a broken value must not take down - // guest code (the bridge's stance, mirrored here); a wrap - // failure degrades to the unwrapped value below. - } - return value; - } - - /** - * The for-await ITERABLE wrap (version 0.3.1): the instrumenter - * rewrites every top-level \`for await (... of )\` iterable - * into \`this["__replAwaitIterable"](, TOKEN)\` — the same - * continuation-lease discipline as \`__replAwait\`, WITHOUT breaking the - * iterable protocol (phase-E review rejection round 6: the 0.3.0 - * instrumenter wrapped for-await iterables in \`__replAwait\`, whose - * promise result made \`for await (const x of [1, 2])\` throw - * \`TypeError: not a function\` instead of iterating — the eval's - * continuation is queued by the loop's \`next()\`-result awaits, so the - * wrap must RIDE those promises, not replace the iterable with one). - * - * The returned object is an ASYNC-ITERABLE wrapper over the - * underlying iterable, resolved exactly like \`for await\` resolves one: - * \`@@asyncIterator\` first, then \`@@iterator\` (a plain array iterates - * synchronously; a promise is not iterable and throws the same - * TypeError the un-instrumented loop throws). Every \`next()\` (and - * \`return()\`/\`throw()\` — an abrupt completion must never leak the - * underlying iterator without its cleanup) returns a FRESH promise - * whose settling reaction — registered BEFORE the for-await machinery - * registers its own on the same promise (the machinery awaits the - * \`next()\` result — its registration happens after this function - * returns) — sets the continuation lease to the eval's token: the job - * after the lease-setting reaction IS the loop segment's continuation, - * so the eval-break interrupt can break a runaway for-await loop - * mid-iteration, exactly like any other awaited segment. - * - * ACQUISITION errors PROPAGATE (phase-E review rejection round 7): - * resolving the underlying iterator — the \`@@asyncIterator\`/ - * \`@@iterator\` property reads (guest accessors can throw) and the - * method calls — must run exactly ONCE and report exactly what the - * un-instrumented loop reports. The old implementation caught - * acquisition failures and returned the UNWRAPPED iterable, so the - * for-await machinery acquired it a SECOND time: an observable or - * throwing \`Symbol.asyncIterator\` getter ran twice and could produce - * a different error (\`boom2\` instead of native \`boom1\`). The wrap - * also follows GetMethod semantics: a present-but-not-callable - * \`@@asyncIterator\` is a TypeError, never a silent fallback to - * \`@@iterator\`. - * - * A SYNC iterator's results pass through AsyncFromSyncIterator- - * CONTINUATION semantics (phase-E review rejection round 7): the raw - * result's VALUE is awaited and unwrapped, so \`for await (const x of - * [Promise.resolve(1)])\` yields \`1\`, never the promise object — the - * old wrapper resolved with the RAW iterator result, and because the - * wrapper is an ASYNC iterable, the machinery used the value as-is - * (the promise object leaked through). An ASYNC iterator's results - * pass through untouched (its \`value\` is used as-is — native async - * iteration semantics). - * - * Never throws AFTER acquisition: a broken iterator's per-step - * failures (a throwing \`next()\`, a non-object result, a hostile - * result value) surface as REJECTED result promises — the loop - * observes the same rejection it would have observed unwrapped. The - * mirroring machinery is the CAPTURED pristine Promise surface - * ('P'/'PResolve'/'PReject'/'pThen') like \`__replAwait\` — a guest - * that replaces 'Promise.prototype.then' or shadows 'Promise' must - * not change the wrap's semantics. - */ - function replAwaitIterable(iterable, token) { - if (typeof token !== 'string' || token.length === 0) return iterable; - // ACQUISITION (GetIterator/GetMethod semantics): \`@@asyncIterator\` - // first — present-but-not-callable is a TypeError, never a fallback - // to \`@@iterator\` — then \`@@iterator\`; both absent is the same - // TypeError the un-instrumented loop throws. Property reads and - // method calls can throw (guest accessors); they PROPAGATE — the - // machinery must observe the exact acquisition error native \`for - // await\` reports, and the iterable's methods must be touched - // EXACTLY ONCE (phase-E review rejection round 7: the old catch - // degraded to the unwrapped iterable, so an observable/throwing - // \`@@asyncIterator\` getter ran twice and could report a different - // error than native). - var asyncIterMethod = - iterable === null || iterable === undefined ? undefined : iterable[Symbol.asyncIterator]; - var isAsync = typeof asyncIterMethod === 'function'; - var syncIterMethod; - if (!isAsync) { - if (asyncIterMethod !== undefined) { - throw new TypeError('Symbol.asyncIterator is not callable'); - } - syncIterMethod = - iterable === null || iterable === undefined ? undefined : iterable[Symbol.iterator]; - if (typeof syncIterMethod !== 'function') { - throw new TypeError('not async iterable'); - } - } - var underlying = isAsync ? asyncIterMethod.call(iterable) : syncIterMethod.call(iterable); - // One lease-wrapped iterator-result promise: the settle reaction - // is registered on the FRESH promise before the for-await - // machinery registers its own (the machinery awaits \`next()\`'s - // result), so the lease-setting job runs immediately before the - // loop segment's continuation job — the same ordering discipline - // as \`__replAwait\` (no job in between can run with the lease - // set). A synchronous throw from the underlying iterator is - // converted to a rejected promise — the loop observes the same - // rejection either way. - var wrapLease = function () { - try { - setContinuationLease(token); - } catch (_e) {} - }; - // The result wrapper (phase-E review rejection round 7): for an - // ASYNC underlying, the result (a promise of the iterator result) - // is adopted and the result object passes through untouched. For a - // SYNC underlying, the raw result object goes through the - // AsyncFromSyncIteratorContinuation transformation: a non-object - // result is a TypeError, and the result's VALUE is awaited and - // unwrapped into a fresh \`{ value, done }\` object — native \`for - // await\` over a sync iterable yields the RESOLVED value, never a - // promise object. The lease-setting reaction rides the pristine - // 'then' function value, so a replaced 'Promise.prototype.then' - // cannot skip it. - var wrapResult = isAsync - ? function (result) { - var p = new P(function (resolve, reject) { - try { - pThen.call(PResolve(result), resolve, reject); - } catch (e) { - reject(e); - } - }); - pThen.call(p, wrapLease, wrapLease); - return p; - } - : function (result) { - var p = new P(function (resolve, reject) { - try { - if (result === null || typeof result !== 'object') { - throw new TypeError('iterator result is not an object'); - } - pThen.call( - PResolve(result.value), - function (value) { - resolve({ value: value, done: result.done }); - }, - reject, - ); - } catch (e) { - reject(e); - } - }); - pThen.call(p, wrapLease, wrapLease); - return p; - }; - var wrapped = {}; - wrapped[Symbol.asyncIterator] = function () { - return wrapped; - }; - // Forward with the EXACT argument count the for-await machinery - // uses: \`next()\` is called with NO arguments (the loop's value - // travels through the iterator, never into \`next()\`), while - // \`return()\`/\`throw()\` receive the completion value even when it - // is undefined — an \`arguments.length\`-sensitive underlying - // iterator must observe the same calls it would have observed - // unwrapped. - var forward = function (method, hasArg, arg) { - var result; - try { - result = hasArg ? method.call(underlying, arg) : method.call(underlying); - } catch (e) { - return wrapResult(PReject(e)); - } - return wrapResult(result); - }; - wrapped.next = function () { - return forward(underlying.next, false); - }; - if (typeof underlying.return === 'function') { - wrapped.return = function (value) { - return forward(underlying.return, true, value); - }; - } - if (typeof underlying.throw === 'function') { - wrapped.throw = function (value) { - return forward(underlying.throw, true, value); - }; - } - return wrapped; - } - - // ──────────────────────────────────────────────────────────────────────── - // The reconciliation surface — the host's post-restore door back into the - // registry. Keyed by Symbol.for so it stays out of the workspace manifest - // and out of the DSL vocabulary the orchestrator is conditioned on, while - // remaining reachable from any host (global symbols survive snapshots and - // round-trip through every host binding). See the package README's host - // contract. - // ──────────────────────────────────────────────────────────────────────── - - var surface = { - /** Guest library version (same value as __REPL_GUEST_VERSION). */ - version: VERSION, - /** True when this library copy carries the 0.2.0 eval-await tracking - * surface ('__replAwait' + 'awaitLog' + the entries' 'promise' - * field). The host - * gates its top-level-await instrumenter on this: a restored - * snapshot carrying the 0.1.0 library is served as-is (the doc's - * older-library rule) and simply gets no await attribution — the - * eval-break interrupt degrades to the honest refusal. */ - supportsAwaitTracking: true, - /** True when this library copy carries the 0.3.0 continuation-lease - * surface ('__replAwait(value, token)' + the '__replLease' - * accessor global): the host's drain loop reads the lease between - * jobs and the eval-break interrupt keys to the lease token — the - * armed eval's genuine continuation identity. A snapshot carrying - * the 0.2.0 library reports false and the host degrades: no - * instrumentation (the 0.2.0 log-only targeting is the rejected - * settled-call-ids identity), no eval-break targeting — the - * interrupt refuses honestly. */ - supportsContinuationLease: true, - /** True when this library copy carries the 0.3.1 iterable-leash - * surface ('__replAwaitIterable' — the for-await iterable wrap - * that preserves the iterable protocol while setting the - * continuation lease per iteration). The host gates its - * instrumenter's for-await sites on this: a snapshot carrying the - * 0.3.0 library (whose for-await wrap returned a promise and - * broke every \`for await\` loop) reports false, its for-await - * sites are left unwrapped, and the loops run natively (no - * mid-loop eval-break targeting — the honest degradation). */ - supportsIterableLease: true, - /** The awaits logged since the host last took them, oldest first - * (call-id strings only — the library's own registry ids; a - * pathologically large log is bounded by one operation's awaits - * because the host takes it at every operation boundary). The - * returned array is a fresh copy; the log is cleared in the same - * call (take semantics — the host is the only consumer). Kept for - * older hosts; the 0.3.0 broker's targeting rides the - * continuation lease instead. */ - awaitLogTake: function () { - var out = state.awaitLog; - state.awaitLog = []; - return out; - }, - /** - * JSON-safe manifest of every pending host call, oldest first: - * [{ id, kind: "agent" | "checkpoint" | "queue" | "steer" | "cancel", detail, optionsJson, - * createdAt, sessionId, modelSpec }]. 'detail' is the verbatim - * prompt/question/action, 'optionsJson' the verbatim options string - * (or null), 'sessionId' the id the host addresses the call by (the - * founding session id for steering calls — a pending steer is fully - * reconcilable after a restore), 'modelSpec' the agent call's backend - * routing spec (null otherwise) — enough for the host to re-issue - * lost work. - */ - pending: function () { - var out = []; - registryForEach.call(state.registry, function (entry) { - out.push({ - id: entry.id, - kind: entry.kind, - detail: entry.detail, - optionsJson: entry.optionsJson, - createdAt: entry.createdAt, - sessionId: entry.sessionId, - modelSpec: entry.modelSpec, - }); - }); - return out; - }, - /** - * Settle a pending call by id: outcome is "resolve" or "reject", value - * is the result (or the error / { message, code?, recoverable? } - * object). Returns true iff a pending entry was settled; false for - * unknown or already-settled ids (idempotent — safe to call on both - * the live path and the reconciliation path). - */ - settle: function (callId, outcome, value) { - if (outcome !== 'resolve' && outcome !== 'reject') { - throw new TypeError('settle(callId, outcome, value): outcome must be "resolve" or "reject"'); - } - return settleCall(callId, outcome, value); - }, - /** Counters for diagnostics and the workspace manifest. */ - stats: function () { - return { - version: VERSION, - callSeq: state.callSeq, - pendingCalls: registrySize.call(state.registry), - }; - }, - }; - Object.freeze(surface); - - // ──────────────────────────────────────────────────────────────────────── - // Install the globals - // ──────────────────────────────────────────────────────────────────────── - - function installGlobal(name, value) { - try { - Object.defineProperty(g, name, { - value: value, - writable: false, - enumerable: true, - configurable: false, - }); - } catch (_err) { - // The realm predefined the name non-configurably — fall back to - // assignment so the DSL still works. - g[name] = value; - } - } - - // Freeze every installed function object so its methods and properties - // cannot be reassigned (agent carries the handle-method factory surface; - // checkpoint carries 'answer'; the combinators are pure functions). - Object.freeze(agent); - Object.freeze(checkpoint); - Object.freeze(parallel); - Object.freeze(pipeline); - Object.freeze(verify); - Object.freeze(judgePanel); - Object.freeze(gate); - Object.freeze(retry); - Object.freeze(loopUntilDry); - Object.freeze(sleep); - Object.freeze(workspace); - Object.freeze(agents); - Object.freeze(reset); - Object.freeze(replAwait); - Object.freeze(replAwaitIterable); - - installGlobal('agent', agent); - installGlobal('checkpoint', checkpoint); - installGlobal('parallel', parallel); - installGlobal('pipeline', pipeline); - installGlobal('verify', verify); - installGlobal('judgePanel', judgePanel); - installGlobal('gate', gate); - installGlobal('retry', retry); - installGlobal('loopUntilDry', loopUntilDry); - installGlobal('sleep', sleep); - installGlobal('workspace', workspace); - installGlobal('agents', agents); - installGlobal('reset', reset); - installGlobal('console', consoleObject); - // The result-history global (§4.4): '_' holds the previous eval's - // completion value (IPython-style) — the HOST sets it after every eval - // that resolved with a value. Installed HERE (as an ordinary writable - // global initialized to undefined) so it sits in the fresh-realm - // baseline and never pollutes the workspace manifest as a user binding. - Object.defineProperty(g, '_', { - value: undefined, - writable: true, - enumerable: true, - configurable: true, - }); - // The host's top-level-await instrumenter inserts calls to this global - // ('await x' → 'await __replAwait(x)'); a bare VM without the library - // never has the instrumenter applied (the broker gates on - // 'supportsAwaitTracking'). - installGlobal('__replAwait', replAwait); - // The for-await iterable wrap (version 0.3.1): the instrumenter - // inserts calls to this global at every top-level \`for await\` - // iterable ('for await (const x of y)' → 'for await (const x of - // __replAwaitIterable(y, TOKEN))'); a bare VM without the library - // never has the instrumenter applied (the broker gates on - // 'supportsIterableLease'). - installGlobal('__replAwaitIterable', replAwaitIterable); - - // The continuation lease (version 0.3.0): a WRITABLE accessor whose - // getter/setter are this closure's own (the host's drain loop reads it - // between jobs and clears it after a lease-carrying job; the eval-break - // targeting identity). Non-configurable and non-enumerable: guest code - // can neither redefine the accessor nor observe it through the - // manifest's baseline difference (it IS part of the fresh-realm - // baseline). A guest that WRITES the lease (through the setter) is - // sabotaging only its own interrupt targeting — the same self- - // sabotage stance as the rest of the tracking surface. - Object.defineProperty(g, '__replLease', { - get: function () { return state.continuationLease; }, - set: function (v) { state.continuationLease = v; }, - enumerable: false, - configurable: false, - }); - - // Version marker (snapshot versioning: hosts read this — or - // surface.version — to know which guest library a restored workspace - // carries; see the README's version-compatibility rules). - Object.defineProperty(g, VERSION_GLOBAL, { - value: VERSION, - writable: false, - enumerable: false, - configurable: false, - }); - - Object.defineProperty(g, Symbol.for(SURFACE_KEY), { - value: surface, - writable: false, - enumerable: false, - configurable: false, - }); -})(); -`; diff --git a/packages/repl-engine/src/index.ts b/packages/repl-engine/src/index.ts deleted file mode 100644 index 86740bf5..00000000 --- a/packages/repl-engine/src/index.ts +++ /dev/null @@ -1,186 +0,0 @@ -/** - * @automatalabs/repl-engine — the REPL orchestrator's engine package. - * - * A persistent JavaScript REPL in a capability-free QuickJS-in-WASM VM. - * One VM per workspace; the workspace object owns the VM lifecycle - * (create, eval, drain, dispose). This package is the engine tier of the - * REPL orchestrator roadmap doc (docs/roadmap/repl-orchestrator.md); the - * `repl` MCP tool in `mcp-server` registers over it (the roadmap's - * phase E — shipped), and the broker drives subagents as ACP sessions - * through `@automatalabs/acp-agents` (the same backends the SDK's - * workflow engine drives). - * - * Engine posture (all quickjs-wasi built-ins, used as-is): - * `memoryLimit` per VM, `interruptHandler` per eval. - */ - -export { - ReplVm, - DrainJobError, - loadShippedWasm, - type ReplVmOptions, - type ReplEvalOptions, - type ReplDrainOptions, - type ReplEvalOutcome, -} from './vm.js'; -export { - Workspace, - WorkspaceRegistry, - type WorkspaceOptions, - type WorkspaceRegistryOptions, - type WorkspaceManifest, - type WorkspaceBinding, -} from './workspace.js'; -export { - baselineGlobalKeys, - baselineLexicalKeys, - provenanceBootstrap, - provenanceRecord, - provenanceView, - isValidOriginLabel, - type ProvenanceOrigin, - type ProvenanceView, - type OriginRecord, - type BaselineKeys, -} from './provenance.js'; -export { LexicalEnumerationError } from './errors.js'; -export type { EvalErrorInfo } from './errors.js'; -export type { WasmInput, WasmModule, ReplSnapshot, ReplSnapshotExtension } from './types.js'; - -// Phase B: the guest-side library bridge, the previewer, and the output -// caps. See the package README's "Guest library ⇄ host contract". -export { - GUEST_LIBRARY_VERSION, - GUEST_SURFACE_KEY, - GUEST_VERSION_GLOBAL, - GUEST_PROVENANCE_KEY, - HOST_AGENT, - HOST_CHECKPOINT, - HOST_CONSOLE, - HOST_QUEUE, - HOST_QUEUE_CANCEL, - HOST_SESSION_CANCEL, - HOST_STEER, - HOST_SLEEP, - HOST_WORKSPACE, - HOST_AGENTS, - HOST_RESET, -} from './guest/guest-library.js'; -export { - installGuestBridge, - registerGuestHostCallbacks, - GuestLibraryInstallError, - GuestCall, - readGuestSurface, - readRealmSlot, - type GuestBridgeHandlers, - type GuestSurface, - type GuestSurfaceEntry, - type ConsoleEvent, - type ConsoleLevel, - type RealmSlot, -} from './bridge.js'; -export { - renderCollapsed, - inspectGlobal, - manifestBinding, - formatByteSize, - formatNumber, - escapeString, - stringDescription, - shortString, - headTailDescription, - isCanonicalIndex, - MAX_PREVIEW_PROPERTIES, - MAX_PREVIEW_ARRAY_ITEMS, - MAX_PROPERTY_VALUE_CHARS, - PROPERTY_STRING_HEAD_CHARS, - PROPERTY_STRING_TAIL_CHARS, - MAX_ERROR_MESSAGE_CHARS, - MAX_STRING_PREVIEW_CHARS, - STRING_HEAD_CHARS, - STRING_TAIL_CHARS, - MAX_COLLAPSED_CHARS, - REPR_MAX_DEPTH, - REPR_MAX_ENTRIES, - REPR_NESTED_STRING_CHARS, - type PreviewType, - type PreviewSubtype, - type PropertyPreviewKind, - type PropertyPreview, - type ObjectPreview, -} from './preview.js'; -// The output-cap apparatus (applyOutputCaps / capFinalText / the caps -// constants) was deleted with the eval-plane redesign's §7 budget sweep — -// the engine applies NO caps to guest output. - -// Phase F review round 2: the out-of-band eval-break channel (the -// interrupt tool's no-id path deliverable to a synchronously running -// eval — see the module docs). -export { - EvalBreakChannelImpl, - createEvalBreakChannel, - EVAL_BREAK_CHANNEL_INITIAL_SLOTS, - EVAL_BREAK_CHANNEL_MAX_BYTES, - type EvalBreakChannel, -} from './eval-break-channel.js'; - -// Phase C: the broker, the append-only call store, and the eval -// tool-result semantics. See the package README's "The broker" section. -export { - Broker, - DEFAULT_MAX_CONCURRENT_AGENTS, - DEFAULT_EVAL_TIMEOUT_MS, - DEFAULT_DISPOSE_BOUND_MS, - type BrokerOptions, - type BrokerRunner, - type BrokerSession, - type BrokerTurn, - type BrokerOpenSessionOptions, - type BrokerPromptOptions, - type BrokerLoadSessionOptions, - type SteeringOutcomeValue, - type ReplEvalResult, - type CheckpointSummary, - type CheckpointInfo, - type LiveAgentInfo, - type ReconcileReport, - type SnapshotSink, - type SnapshotBoundaryKind, - type WorkspaceManifestReport, - type WorkspaceManifestBinding, -} from './broker.js'; -export { - InMemoryCallStore, - JsonlCallStore, - type CallStore, - type CallRecord, - type CallOutcome, - type CallKind, - type CallOutcomeKind, -} from './store.js'; - -// Phase D: enveloped snapshots, the per-project store, and the restore -// path. See the package README's "Snapshots and durability" section. -export { - SNAPSHOT_FORMAT, - SNAPSHOT_FORMAT_VERSION, - serializeSnapshot, - deserializeSnapshot, - wasmSha256Of, - SnapshotEnvelopeError, - SnapshotRestoreError, - type SnapshotEnvelopeMeta, - type SnapshotEnvelope, - type SnapshotEnvelopeErrorCode, -} from './snapshot-envelope.js'; -export { - ReplWorkspaceStore, - REPL_STORE_SUBDIR, - SNAPSHOT_FILENAME, - CALL_STORE_FILENAME, - type ReplStoreOptions, - type SnapshotWriteOptions, - type RestoredReplSnapshot, - type ReplStoreStats, -} from './repl-store.js'; diff --git a/packages/repl-engine/src/preview.ts b/packages/repl-engine/src/preview.ts deleted file mode 100644 index 9fd3e751..00000000 --- a/packages/repl-engine/src/preview.ts +++ /dev/null @@ -1,1414 +0,0 @@ -/** - * The previewer — the CDP-style collapsed preview of guest values that - * reaches the client agent's context. The Chrome DevTools Protocol's - * `RemoteObject`/`ObjectPreview` model, adopted as a spec; the exact rules - * (field order, caps, truncation, per-brand renderings) follow the - * harness's normative `previewer/FORMAT.md` (the roadmap doc names it as - * the normative reference — imitated, not copied). - * - * **The absolute rule: preview generation is side-effect-free.** Only the - * trap-free introspection surface is used (`trapfree.ts`): engine brand - * checks (never `instanceof`, never prototype inspection, never - * `Symbol.toStringTag`), `Reflect.ownKeys`-style key listing, and - * own-property-descriptor reads that never invoke getters. Proxies are - * detected FIRST and previewed as proxies (never enumerated); strings/ - * numbers/booleans/bigints are extracted only after brand checks; the - * only indexed `[[Get]]`s are on brand-checked typed arrays, where - * integer-indexed element access is guest-code-free by the language. - * Observing a value never executes guest code and never mutates guest - * state — a hostile getter, a polluted `Object.prototype`, or a - * trap-counting proxy cannot influence anything rendered here. - * - * `estimateByteSize` is the bounded, trap-free byte-size estimate for the - * header (FORMAT.md leaves the estimate to the caller); `inspectGlobal` - * is the workspace-manifest seam (content-free name/type/size metadata - * for one realm global slot, read through its own property descriptor — - * an accessor-rebound slot is marked, never invoked). The v1 `$N` - * capture-unit previewer surface (renderRefLine/renderGlobalLine/ - * renderPreviewLine/previewGlobal) is DELETED with the redesign's §7 - * sweep — the retained surface is the §4.4 completion repr - * (`renderCompletionLine`), the retained metadata-formatting tokens - * (stringDescription/shortString/headTailDescription/formatNumber/ - * formatByteSize), and the manifest seams (inspectGlobal/ - * manifestBinding). - */ - -import { JSValueHandle, type QuickJS } from 'quickjs-wasi'; - -import { getVmShim, type ReplVm } from './vm.js'; -import { readLexicalSlotValue } from './global-lexical.js'; -import { - arrayBufferByteLength, - getPropRaw, - hasOwnRaw, - readOwnDataProperty, - readOwnDescriptor, - rawOwnKeysAll, - readProxyTarget, - typedArrayInfo, - type OwnDescriptor, - type TypedArrayInfo, -} from './trapfree.js'; - -// ---- Caps (normative — FORMAT.md §3). All character counts are Unicode -// scalar values (code points), not bytes and not UTF-16 units. ---- - -/** Named properties listed per preview. */ -export const MAX_PREVIEW_PROPERTIES = 8; -/** Leading array / typed-array entries. */ -export const MAX_PREVIEW_ARRAY_ITEMS = 8; -/** String content chars in a property value. */ -export const MAX_PROPERTY_VALUE_CHARS = 40; -/** Head kept when a property string is elided. */ -export const PROPERTY_STRING_HEAD_CHARS = 24; -/** Tail kept when a property string is elided. */ -export const PROPERTY_STRING_TAIL_CHARS = 8; -/** Error-description chars (top level). */ -export const MAX_ERROR_MESSAGE_CHARS = 120; -/** Longest top-level string rendered whole in a PREVIEW (a string shown to - * convey shape — a nested/inspected value, a checkpoint question). This is - * the FORMAT.md §5.5 constant. */ -export const MAX_STRING_PREVIEW_CHARS = 200; -/** Head kept when a preview string is elided (= MAX_STRING_PREVIEW_CHARS × 3/5). */ -export const STRING_HEAD_CHARS = 120; -/** Tail kept when a preview string is elided (= MAX_STRING_PREVIEW_CHARS × 1/5). */ -export const STRING_TAIL_CHARS = 40; -/** Hard backstop on the rendered collapsed body. */ -export const MAX_COLLAPSED_CHARS = 400; - -// ---- The preview model (CDP shape — FORMAT.md §4). Field order is -// normative for serialized forms. ---- - -export type PreviewType = - | 'object' - | 'function' - | 'undefined' - | 'string' - | 'number' - | 'boolean' - | 'symbol' - | 'bigint'; - -export type PreviewSubtype = - | 'array' - | 'null' - | 'regexp' - | 'date' - | 'map' - | 'set' - | 'weakmap' - | 'weakset' - | 'weakref' - | 'error' - | 'proxy' - | 'promise' - | 'typedarray' - | 'arraybuffer' - | 'dataview'; - -export type PropertyPreviewKind = PreviewType | 'accessor'; - -export interface PropertyPreview { - name: string; - type: PropertyPreviewKind; - /** Abbreviated token; absent for accessors. */ - value?: string; - subtype?: PreviewSubtype; -} - -export interface ObjectPreview { - type: PreviewType; - subtype?: PreviewSubtype; - description: string; - overflow: boolean; - properties: PropertyPreview[]; -} - -// ---- Character helpers (code points, not UTF-16 units) ---- - -function toChars(s: string): string[] { - return Array.from(s); -} - -/** FORMAT.md §5.4: `\` `"` and C0 controls, everything else verbatim. */ -export function escapeString(s: string): string { - let out = ''; - for (const c of s) { - switch (c) { - case '"': - out += '\\"'; - break; - case '\\': - out += '\\\\'; - break; - case '\n': - out += '\\n'; - break; - case '\t': - out += '\\t'; - break; - case '\r': - out += '\\r'; - break; - default: { - const code = c.codePointAt(0)!; - if (code < 0x20) { - out += `\\u${code.toString(16).padStart(4, '0')}`; - } else { - out += c; - } - } - } - } - return out; -} - -/** FORMAT.md §5.2: ECMAScript Number::toString(10), plus -0/NaN/Infinity. */ -export function formatNumber(n: number): string { - if (Number.isNaN(n)) return 'NaN'; - if (n === Infinity) return 'Infinity'; - if (n === -Infinity) return '-Infinity'; - if (n === 0) return Object.is(n, -0) ? '-0' : '0'; - // Host Number::toString implements the spec's shortest round-trip - // decimal with exponent notation outside [1e-6, 1e21) and an explicit - // '+' for positive exponents — exactly FORMAT.md §5.2. - return n.toString(); -} - -/** Head+tail truncation for UNQUOTED description text (FORMAT.md §5.10). */ -export function headTailDescription(s: string, cap: number): string { - const chars = toChars(s); - if (chars.length <= cap) return s; - const headChars = Math.floor((cap * 3) / 5); - const tailChars = Math.floor(cap / 5); - const head = chars.slice(0, headChars).join(''); - const tail = chars.slice(chars.length - tailChars).join(''); - const elided = chars.length - headChars - tailChars; - return `${head}…[${elided} chars elided]…${tail}`; -} - -/** Head-only truncation with a trailing marker (function names). */ -function truncateChars(s: string, cap: number): string { - const chars = toChars(s); - if (chars.length <= cap) return s; - return chars.slice(0, cap).join('') + '…'; -} - -/** FORMAT.md §5.5: top-level strings — whole ≤ `wholeCap`, else head AND - * tail (proportional 3/5 head, 1/5 tail — the FORMAT.md 120/40 split at the - * default 200 cap). The §4.4 completion/console reprs bypass this - * (direct strings print whole there); `stringDescription` is the - * metadata/preview form (checkpoint questions, manifest tokens). */ -export function stringDescription(s: string, wholeCap: number = MAX_STRING_PREVIEW_CHARS): string { - const chars = toChars(s); - if (chars.length <= wholeCap) { - return `"${escapeString(s)}"`; - } - const headCount = Math.floor((wholeCap * 3) / 5); - const tailCount = Math.floor(wholeCap / 5); - const head = chars.slice(0, headCount).join(''); - const tail = chars.slice(chars.length - tailCount).join(''); - const elided = chars.length - headCount - tailCount; - return `"${escapeString(head)}" …[${elided} chars elided]… "${escapeString(tail)}"`; -} - -/** FORMAT.md §5.6: property-level strings — whole ≤ 40, else head AND tail - * inside one quoted token. */ -export function shortString(s: string): string { - const chars = toChars(s); - if (chars.length <= MAX_PROPERTY_VALUE_CHARS) { - return `"${escapeString(s)}"`; - } - const head = chars.slice(0, PROPERTY_STRING_HEAD_CHARS).join(''); - const tail = chars.slice(chars.length - PROPERTY_STRING_TAIL_CHARS).join(''); - const elided = chars.length - PROPERTY_STRING_HEAD_CHARS - PROPERTY_STRING_TAIL_CHARS; - return `"${escapeString(head)}…[${elided} chars elided]…${escapeString(tail)}"`; -} - -// ---- Preview generation ---- - -function leaf( - type: PreviewType, - subtype: PreviewSubtype | undefined, - description: string, -): ObjectPreview { - return { type, subtype, description, overflow: false, properties: [] }; -} - -/** - * Generate a collapsed first-level preview of any guest value. - * Side-effect-free by construction (module docs). Internal: takes a raw - * shim handle, so it is not exported from this module's public surface - * (the published declarations must stay free of quickjs-wasi types — a - * signature naming `JSValueHandle` would drag the shim's DOM-dependent - * declarations into the consumer's program); the public seam is - * `inspectGlobal`, and the broker - * layer reaches it through `renderCompletionLine` below. - */ -function previewHandle(handle: JSValueHandle): ObjectPreview { - // ---- Primitives (brand-checked before any conversion) ---- - if (handle.isUndefined) return leaf('undefined', undefined, 'undefined'); - if (handle.isNull) return leaf('object', 'null', 'null'); - if (handle.isBool) return leaf('boolean', undefined, handle.toBoolean() ? 'true' : 'false'); - if (handle.isNumber) return leaf('number', undefined, formatNumber(handle.toNumber())); - // A top-level string is rendered at EMISSION width: `previewHandle` feeds - // the emission line renderers (a console.log line, the eval result), and - // the manifest — which ignores this description (metadata only, see - // `describeManifest`) — so widening it carries directly-emitted output up - // to the byte budget without leaking string content into the manifest. - if (handle.isString) return leaf('string', undefined, stringDescription(handle.toString())); - if (handle.isBigInt) return leaf('bigint', undefined, handle.toBigInt().toString() + 'n'); - if (handle.isSymbol) { - // The description is not readable trap-free (it sits behind - // Symbol.prototype.description / Symbol.keyFor, both guest-replaceable) - // — render the bare brand. FORMAT.md §5.7. - return leaf('symbol', undefined, 'Symbol'); - } - - // ---- Proxy: detected BEFORE any other object handling (CDP convention) - // — a proxy is previewed as a proxy, never enumerated. ---- - if (handle.isProxy) { - return proxyPreview(handle); - } - - if (handle.isFunction) { - return leaf('function', undefined, functionDescription(handle)); - } - - // ---- Branded objects ---- - if (handle.isPromise) return promisePreview(handle); - if (handle.isError) return errorPreview(handle, MAX_ERROR_MESSAGE_CHARS); - if (handle.isMap) return brandedPreview(handle, 'map', 'Map(?)', []); - if (handle.isSet) return brandedPreview(handle, 'set', 'Set(?)', []); - if (handle.isWeakMap) return brandedPreview(handle, 'weakmap', 'WeakMap', []); - if (handle.isWeakSet) return brandedPreview(handle, 'weakset', 'WeakSet', []); - if (handle.isWeakRef) return brandedPreview(handle, 'weakref', 'WeakRef', []); - if (handle.isDate) return brandedPreview(handle, 'date', 'Date', []); - if (handle.isRegExp) { - return brandedPreview(handle, 'regexp', 'RegExp', ['lastIndex']); - } - if (handle.isArrayBuffer) { - const len = arrayBufferByteLength(handle) ?? 0; - return brandedPreview(handle, 'arraybuffer', `ArrayBuffer(${len})`, []); - } - if (handle.isDataView) return brandedPreview(handle, 'dataview', 'DataView(?)', []); - const tinfo = typedArrayInfo(handle); - if (tinfo !== undefined) return typedArrayPreview(handle, tinfo); - if (handle.isArray) return arrayPreview(handle); - if (handle.isObject) return plainObjectPreview(handle); - - // Unreachable for well-formed values; be total anyway. - return leaf('object', undefined, 'unknown'); -} - -/** FORMAT.md §5.8: `Proxy()` — the target's engine brand, read - * without traps; `Proxy(revoked)` for revoked proxies. */ -function proxyPreview(handle: JSValueHandle): ObjectPreview { - const target = readProxyTarget(handle); - let description = 'Proxy(revoked)'; - if (target !== undefined) { - try { - description = `Proxy(${brandWord(target)})`; - } finally { - target.dispose(); - } - } - return { type: 'object', subtype: 'proxy', description, overflow: false, properties: [] }; -} - -/** The engine brand of a value as a constructor-style word. */ -function brandWord(handle: JSValueHandle): string { - if (handle.isProxy) return 'Proxy'; - if (handle.isFunction) return 'Function'; - if (handle.isPromise) return 'Promise'; - if (handle.isError) return 'Error'; - if (handle.isMap) return 'Map'; - if (handle.isSet) return 'Set'; - if (handle.isWeakMap) return 'WeakMap'; - if (handle.isWeakSet) return 'WeakSet'; - if (handle.isWeakRef) return 'WeakRef'; - if (handle.isDate) return 'Date'; - if (handle.isRegExp) return 'RegExp'; - if (handle.isArrayBuffer) return 'ArrayBuffer'; - if (handle.isDataView) return 'DataView'; - const tinfo = typedArrayInfo(handle); - if (tinfo !== undefined) return typedArrayKind(handle, tinfo); - if (handle.isArray) return 'Array'; - return 'Object'; -} - -/** FORMAT.md §5.9: `ƒ ()` — own data string `name` only (an accessor - * `name` renders anonymous), capped at 40 chars. */ -function functionDescription(handle: JSValueHandle): string { - const nameHandle = readOwnDataProperty(handle, 'name'); - let name = ''; - if (nameHandle !== undefined) { - try { - if (nameHandle.isString) name = truncateChars(nameHandle.toString(), MAX_PROPERTY_VALUE_CHARS); - } finally { - nameHandle.dispose(); - } - } - return name === '' ? 'ƒ ()' : `ƒ ${name}()`; -} - -/** FORMAT.md §5.11: `Promise {}` / `Promise {: result}`. */ -function promisePreview(handle: JSValueHandle): ObjectPreview { - const state = handle.promiseState; // 0 pending, 1 fulfilled, 2 rejected - const stateStr = state === 0 ? 'pending' : state === 1 ? 'fulfilled' : 'rejected'; - const properties: PropertyPreview[] = [ - { name: '[[PromiseState]]', type: 'string', value: stateStr }, - ]; - if (state !== 0) { - const shim = handle.vm; - const resultPtr = shim._getExports().qjs_promise_result(handle.ptr); - const result = new JSValueHandle(shim, resultPtr); - try { - properties.push(propertyPreviewOf('[[PromiseResult]]', result)); - } finally { - result.dispose(); - } - } - return { - type: 'object', - subtype: 'promise', - description: 'Promise', - overflow: false, - properties, - }; -} - -/** FORMAT.md §5.10: `: ` — own data strings only, head+tail - * truncation with the given budget. */ -function errorDescription(handle: JSValueHandle, cap: number): string { - const name = ownDataString(handle, 'name') ?? 'Error'; - const message = ownDataString(handle, 'message') ?? ''; - if (message === '') return headTailDescription(name, cap); - return headTailDescription(`${name}: ${message}`, cap); -} - -function errorPreview(handle: JSValueHandle, cap: number): ObjectPreview { - const description = errorDescription(handle, cap); - // `name`/`message` are folded into the description; `stack` is a - // structural own property present on every engine error — none of the - // three counts as "more to see" (FORMAT.md §5.10). - return brandedPreview(handle, 'error', description, ['name', 'message', 'stack']); -} - -/** Branded object (Map, Date, ArrayBuffer, ...): fixed description plus - * any own enumerable string-keyed expando properties. */ -function brandedPreview( - handle: JSValueHandle, - subtype: PreviewSubtype, - description: string, - exempt: string[], -): ObjectPreview { - const { properties, overflow } = ownStringProperties(handle, MAX_PREVIEW_PROPERTIES, exempt, false); - return { type: 'object', subtype, description, overflow, properties }; -} - -/** FORMAT.md §5.12: `()` with leading elements via - * integer-indexed [[Get]] (guest-code-free on brand-checked typed - * arrays); overflow when len exceeds the cap or when expando keys exist - * (own-key count > element count — read engine-side, no descriptors). */ -function typedArrayPreview(handle: JSValueHandle, info: TypedArrayInfo): ObjectPreview { - const kindName = typedArrayKind(handle, info); - const len = info.length; - const show = Math.min(len, MAX_PREVIEW_ARRAY_ITEMS); - const properties: PropertyPreview[] = []; - for (let i = 0; i < show; i++) { - const elem = handle.vm._getExports().qjs_get_prop_uint32(handle.ptr, i); - const elemHandle = new JSValueHandle(handle.vm, elem); - try { - properties.push(propertyPreviewOf(String(i), elemHandle)); - } finally { - elemHandle.dispose(); - } - } - const ownKeys = ownKeyCount(handle); - const hasExpandos = ownKeys.count > len; - return { - type: 'object', - subtype: 'typedarray', - description: `${kindName}(${len})`, - // FORMAT.md §6: a corrupted key materialization degrades with - // overflow:true — omitted or unverifiable expandos must never be - // concealed behind a fabricated "no expandos" count (review: the - // corrupted count read as zero, hiding the expando signal). - overflow: len > show || hasExpandos || ownKeys.corrupted, - properties, - }; -} - -/** Resolve the precise typed-array kind from the engine class id, anchored - * to a host-created Uint8Array (no guest code) and cross-checked against - * the element width. Falls back to "TypedArray" if the layout of the - * binary's class table ever changes. FORMAT.md §5.12. */ -function typedArrayKind(handle: JSValueHandle, info: TypedArrayInfo): string { - // quickjs-ng registers the typed-array classes contiguously in this - // order (verified against the shipped binary: Uint8ClampedArray=-2 … - // Float64Array=+9 relative to the Uint8Array anchor); the anchor probe - // plus the bytes-per-element cross-check makes relying on that order - // safe (mismatch -> generic fallback). - const KINDS: Array<[number, string, number]> = [ - [-2, 'Uint8ClampedArray', 1], - [-1, 'Int8Array', 1], - [0, 'Uint8Array', 1], - [1, 'Int16Array', 2], - [2, 'Uint16Array', 2], - [3, 'Int32Array', 4], - [4, 'Uint32Array', 4], - [5, 'BigInt64Array', 8], - [6, 'BigUint64Array', 8], - [7, 'Float16Array', 2], - [8, 'Float32Array', 4], - [9, 'Float64Array', 8], - ]; - const classId = handle.classId; - const anchorPtr = handle.vm._getExports().qjs_new_uint8_array(0, 0); - const anchor = new JSValueHandle(handle.vm, anchorPtr); - let anchorId = 0; - try { - anchorId = anchor.classId; - } finally { - anchor.dispose(); - } - const delta = classId - anchorId; - for (const [d, name, bpe] of KINDS) { - if (d === delta && bpe === info.bytesPerElement) return name; - } - return 'TypedArray'; -} - -/** FORMAT.md §5.14: `Array()` — length from the own descriptor, - * leading entries via own descriptors (holes render `empty`, accessor - * elements render `(...)`), then named own enumerable properties. - * Overflow: len > 8, named props cut, or hidden properties. */ -function arrayPreview(handle: JSValueHandle): ObjectPreview { - const lengthHandle = readOwnDataProperty(handle, 'length'); - const len = lengthHandle === undefined ? 0 : lengthHandle.toNumber(); - lengthHandle?.dispose(); - - const show = Math.min(len, MAX_PREVIEW_ARRAY_ITEMS); - const properties: PropertyPreview[] = []; - for (let i = 0; i < show; i++) { - const desc = readOwnDescriptor(handle, String(i)); - if (desc === undefined) { - // A hole. Reading it with [[Get]] would consult the prototype chain - // (which can carry guest accessors), so it renders as a hole. - properties.push({ name: String(i), type: 'undefined', value: 'empty' }); - continue; - } - properties.push(descriptorPreview(String(i), desc)); - } - let overflow = len > show; - - // Named own properties on the array (`arr.foo = 1`), after the index - // entries — "length" is carried in the description and exempt. - const named = ownStringProperties(handle, MAX_PREVIEW_PROPERTIES, ['length'], true); - overflow = overflow || named.overflow; - properties.push(...named.properties); - - return { - type: 'object', - subtype: 'array', - description: `Array(${len})`, - overflow, - properties, - }; -} - -function plainObjectPreview(handle: JSValueHandle): ObjectPreview { - const { properties, overflow } = ownStringProperties(handle, MAX_PREVIEW_PROPERTIES, [], false); - return { type: 'object', description: 'Object', overflow, properties }; -} - -/** - * Own ENUMERABLE string-keyed properties (in `Reflect.ownKeys` order, - * which puts canonical indices first), abbreviated, up to `cap`. - * - * With `skipIndices` (the array path, which renders index entries - * positionally), canonical array indices are skipped; names in `exempt` - * always are. Overflow is set when enumerable properties were cut by the - * cap, or when symbol-keyed / non-enumerable properties exist at all (they - * are never listed — the preview says "there is more here" — FORMAT.md - * §5.16). A corrupted key enumeration degrades to "list nothing, flag - * overflow" (FORMAT.md §6). - */ -function ownStringProperties( - handle: JSValueHandle, - cap: number, - exempt: string[], - skipIndices: boolean, -): { properties: PropertyPreview[]; overflow: boolean } { - const { keys, corrupted } = rawOwnKeysAll(handle); - if (corrupted) return { properties: [], overflow: true }; - const properties: PropertyPreview[] = []; - let overflow = false; - for (const key of keys) { - if (key.symbol) { - overflow = true; - continue; - } - const name = key.name!; - if (exempt.includes(name) || (skipIndices && isCanonicalIndex(name))) continue; - const desc = readOwnDescriptor(handle, name); - if (desc === undefined) { - overflow = true; // vanished between calls; count it - continue; - } - if (!desc.enumerable || properties.length >= cap) { - // Omitted from the preview, but the descriptor's owned value handle - // must still be disposed — an omitted property is not listed, yet - // its handle is just as owned as a listed one's (review regression: - // every omitted data-property handle leaked, pinning one JSValue - // box per preview call; a 20,000-call inspectGlobal() probe on a - // 100-property object grew WASM memory from 1.31 MB to 30.74 MB). - if (desc.kind === 'data') desc.value.dispose(); - overflow = true; - continue; - } - properties.push(descriptorPreview(name, desc)); - } - return { properties, overflow }; -} - -/** Abbreviate one property from its descriptor: data properties preview - * their value; accessor properties render `(...)` with the getter left - * unfired (CDP `accessor`). */ -function descriptorPreview(name: string, desc: OwnDescriptor): PropertyPreview { - if (desc.kind === 'accessor') { - return { name, type: 'accessor' }; - } - try { - return propertyPreviewOf(name, desc.value); - } finally { - desc.value.dispose(); - } -} - -/** First-level abbreviation of a property VALUE (CDP `PropertyPreview`): - * primitives render inline (with string truncation); objects render as - * shorthand brand tokens, never expanded. Trap-free on every path. - * - * The constructed object's FIELD ORDER is normative (FORMAT.md §4: - * `name`, `type`, `value`, `subtype` — the serialized form must put - * `subtype` AFTER `value`; review regression: it was inserted before). - * `value` is the abbreviated token, absent for accessors; `subtype` is - * absent for primitives (and omitted by JSON serialization). */ -function propertyPreviewOf(name: string, value: JSValueHandle): PropertyPreview { - const { type, subtype, token } = shortForm(value); - return { name, type, value: token, subtype }; -} - -/** FORMAT.md §5.17: the property-level shorthand tokens. */ -function shortForm(value: JSValueHandle): { - type: PropertyPreviewKind; - subtype?: PreviewSubtype; - token: string; -} { - if (value.isUndefined) return { type: 'undefined', token: 'undefined' }; - if (value.isNull) return { type: 'object', subtype: 'null', token: 'null' }; - if (value.isBool) return { type: 'boolean', token: value.toBoolean() ? 'true' : 'false' }; - if (value.isNumber) return { type: 'number', token: formatNumber(value.toNumber()) }; - if (value.isString) return { type: 'string', token: shortString(value.toString()) }; - if (value.isBigInt) return { type: 'bigint', token: value.toBigInt().toString() + 'n' }; - if (value.isSymbol) return { type: 'symbol', token: 'Symbol' }; - if (value.isProxy) return { type: 'object', subtype: 'proxy', token: 'Proxy' }; - if (value.isFunction) return { type: 'function', token: 'ƒ' }; - if (value.isPromise) return { type: 'object', subtype: 'promise', token: 'Promise' }; - if (value.isError) return { type: 'object', subtype: 'error', token: errorDescription(value, MAX_PROPERTY_VALUE_CHARS) }; - if (value.isMap) return { type: 'object', subtype: 'map', token: 'Map(?)' }; - if (value.isSet) return { type: 'object', subtype: 'set', token: 'Set(?)' }; - if (value.isWeakMap) return { type: 'object', subtype: 'weakmap', token: 'WeakMap' }; - if (value.isWeakSet) return { type: 'object', subtype: 'weakset', token: 'WeakSet' }; - if (value.isWeakRef) return { type: 'object', subtype: 'weakref', token: 'WeakRef' }; - if (value.isDate) return { type: 'object', subtype: 'date', token: 'Date' }; - if (value.isRegExp) return { type: 'object', subtype: 'regexp', token: 'RegExp' }; - if (value.isArrayBuffer) { - const len = arrayBufferByteLength(value) ?? 0; - return { type: 'object', subtype: 'arraybuffer', token: `ArrayBuffer(${len})` }; - } - if (value.isDataView) return { type: 'object', subtype: 'dataview', token: 'DataView(?)' }; - const tinfo = typedArrayInfo(value); - if (tinfo !== undefined) { - return { - type: 'object', - subtype: 'typedarray', - token: `${typedArrayKind(value, tinfo)}(${tinfo.length})`, - }; - } - if (value.isArray) { - const lengthHandle = readOwnDataProperty(value, 'length'); - const len = lengthHandle === undefined ? 0 : lengthHandle.toNumber(); - lengthHandle?.dispose(); - return { type: 'object', subtype: 'array', token: `Array(${len})` }; - } - return { type: 'object', token: '{…}' }; -} - -function ownDataString(handle: JSValueHandle, name: string): string | undefined { - const valueHandle = readOwnDataProperty(handle, name); - if (valueHandle === undefined) return undefined; - try { - return valueHandle.isString ? valueHandle.toString() : undefined; - } finally { - valueHandle.dispose(); - } -} - -/** The own-key COUNT (Reflect.ownKeys length), materialized engine-side — - * no descriptor reads, no getters. Used for the typed-array expando - * signal (FORMAT.md §5.12). A corrupted materialization reports - * `corrupted: true` (count 0) so callers degrade with `overflow: true` - * instead of concealing the expando question (FORMAT.md §6). */ -function ownKeyCount(handle: JSValueHandle): { count: number; corrupted: boolean } { - const { keys, corrupted } = rawOwnKeysAll(handle); - return { count: corrupted ? 0 : keys.length, corrupted }; -} - -// ---- Rendering ---- - -/** FORMAT.md §5.18: canonical array indices render positionally. */ -export function isCanonicalIndex(name: string): boolean { - if (!/^\d+$/.test(name)) return false; - const n = Number(name); - return Number.isInteger(n) && n >= 0 && n < 2 ** 32 - 1 && String(n) === name; -} - -/** - * The broker's completion-preview seam: preview the eval completion - * value (an opaque quickjs-wasi handle, `unknown` here so the published - * declaration graph stays self-contained — see the module docs) and - * render the §4.4 depth-limited repr (direct strings whole, objects/ - * arrays to depth 2, 20 entries per level, nested strings head-limited - * at 200 chars — printing conventions, NOT budgets: there is no byte - * ceiling on this path, the Python posture). Trap-free by construction - * (module docs); the handle is NOT consumed or disposed here — the - * caller owns it. - */ -export function renderCompletionLine(completion: unknown): string { - return reprHandle(completion as JSValueHandle, 0, new Set()); -} - -// ── The §4.4 depth-limited repr (docs/roadmap/repl-eval-redesign.md) ── -// -// Printing conventions, not budgets — the same rules the guest library's -// console repr follows, kept in parity by tests: -// -// - strings rendered DIRECTLY (the top-level completion value) print -// WHOLE, unquoted — they are the output the orchestrator asked for, -// with no upper bound; -// - objects/arrays render to DEPTH 2; deeper levels render as -// `{…}` / `[…]`; -// - collections render their first 20 entries per level, then -// `… +N more`; -// - NESTED strings (inside a collection) render head-limited at -// 200 chars, quoted, with a trailing `…` marker when clipped; -// - branded objects (Date/Map/Set/…) render as their brand word; -// - everything deeper/longer is reached by evaluating a narrower -// expression — the values are alive in the VM; slicing is the API. -// -// Trap-free throughout: descriptor reads only (never `[[Get]]`), engine -// brand checks, no proxy enumeration, no recursion beyond depth 2. -// ────────────────────────────────────────────────────────────────────────── - -/** Expand levels 0..1, collapse level 2+ (the doc's depth-2 rule). */ -export const REPR_MAX_DEPTH = 2; -/** Entries rendered per collection level (the doc's 20-entry rule). */ -export const REPR_MAX_ENTRIES = 20; -/** Nested-string head bound in chars (the doc's 200-char rule). */ -export const REPR_NESTED_STRING_CHARS = 200; - -/** The guest-parity head-limited nested-string form: `'…'`. */ -function reprNestedString(s: string): string { - const chars = toChars(s); - if (chars.length <= REPR_NESTED_STRING_CHARS) return `'${s}'`; - return `'${chars.slice(0, REPR_NESTED_STRING_CHARS).join('')}…'`; -} - -/** The brand word for a branded object (guest parity — see the guest - * library's `reprValue`). */ -function reprBrand(handle: JSValueHandle): string | undefined { - if (handle.isPromise) return 'Promise'; - if (handle.isMap) return 'Map'; - if (handle.isSet) return 'Set'; - if (handle.isWeakMap) return 'WeakMap'; - if (handle.isWeakSet) return 'WeakSet'; - if (handle.isWeakRef) return 'WeakRef'; - if (handle.isDate) return 'Date'; - if (handle.isRegExp) return 'RegExp'; - if (handle.isDataView) return 'DataView'; - if (handle.isArrayBuffer) { - const len = arrayBufferByteLength(handle) ?? 0; - return `ArrayBuffer(${len})`; - } - const tinfo = typedArrayInfo(handle); - if (tinfo !== undefined) return typedArrayKind(handle, tinfo); - return undefined; -} - -/** The depth-limited repr of one guest value handle. Trap-free; the - * recursion is bounded by `REPR_MAX_DEPTH` (depth 2 collapses). `seen` - * tracks the current render path's object pointers (cycles and shared - * refs collapse to `{…}`/`[…]` instead of recursing). */ -function reprHandle(handle: JSValueHandle, depth: number, seen: Set): string { - // ---- Primitives (brand-checked before any conversion) ---- - if (handle.isUndefined) return 'undefined'; - if (handle.isNull) return 'null'; - if (handle.isBool) return handle.toBoolean() ? 'true' : 'false'; - if (handle.isNumber) return formatNumber(handle.toNumber()); - if (handle.isString) { - // The §4.4 rule: a DIRECT string prints whole, unbounded; a NESTED - // string (inside a collection) is head-limited at 200 chars. - return depth === 0 ? handle.toString() : reprNestedString(handle.toString()); - } - if (handle.isBigInt) return handle.toBigInt().toString() + 'n'; - if (handle.isSymbol) return 'Symbol'; - - // ---- Proxy: detected BEFORE any other object handling — never - // enumerated (descriptor reads would fire traps). ---- - if (handle.isProxy) { - const preview = proxyPreview(handle); - return preview.description === 'Proxy(revoked)' ? 'Proxy' : preview.description; - } - - if (handle.isFunction) return functionDescription(handle); - - // ---- Error: `name: message` (head-limited like a nested string), - // with the §4.6 attribution when the error came from a subagent call - // (replCallId stamped by the guest library, replBackend by the host). - if (handle.isError) { - const body = errorDescription(handle, REPR_NESTED_STRING_CHARS); - const callId = ownDataString(handle, 'replCallId'); - const backend = ownDataString(handle, 'replBackend'); - const attributed = - callId === undefined - ? body - : `${body} (call ${callId}${backend !== undefined ? ` on backend ${backend}` : ''})`; - return headTailDescription(attributed, REPR_NESTED_STRING_CHARS); - } - - // ---- Branded objects: the brand word (a predictable leaf — their - // content is reached by slicing a narrower expression) ---- - const brand = reprBrand(handle); - if (brand !== undefined) return brand; - - // ---- Objects/arrays: depth-2 entry rendering ---- - if (depth >= REPR_MAX_DEPTH) { - return handle.isArray ? '[…]' : '{…}'; - } - if (seen.has(handle.identity)) { - return handle.isArray ? '[…]' : '{…}'; - } - seen.add(handle.identity); - try { - if (handle.isArray) { - const lengthHandle = readOwnDataProperty(handle, 'length'); - const len = lengthHandle === undefined ? 0 : lengthHandle.toNumber(); - lengthHandle?.dispose(); - const show = Math.min(len, REPR_MAX_ENTRIES); - const parts: string[] = []; - for (let i = 0; i < show; i++) { - const desc = readOwnDescriptor(handle, String(i)); - if (desc === undefined) { - // A hole — the guest's `value[i]` read renders undefined; the - // host mirrors it without ever reading through the prototype - // chain. - parts.push('undefined'); - continue; - } - if (desc.kind === 'accessor') { - parts.push('(…)'); // never invoke the getter - continue; - } - try { - parts.push(reprHandle(desc.value, depth + 1, seen)); - } finally { - desc.value.dispose(); - } - } - if (len > show) parts.push(`… +${len - show} more`); - return `[${parts.join(', ')}]`; - } - const { keys, corrupted } = rawOwnKeysAll(handle); - if (corrupted) return '{…}'; - const stringKeys = keys.filter((key) => !key.symbol) as Array<{ name: string }>; - const show = Math.min(stringKeys.length, REPR_MAX_ENTRIES); - const parts: string[] = []; - for (let i = 0; i < show; i++) { - const name = stringKeys[i].name; - const desc = readOwnDescriptor(handle, name); - if (desc === undefined) continue; // vanished between reads - if (desc.kind === 'accessor') { - parts.push(`${name}: (…)`); - continue; - } - try { - parts.push(`${name}: ${reprHandle(desc.value, depth + 1, seen)}`); - } finally { - desc.value.dispose(); - } - } - if (stringKeys.length > show) parts.push(`… +${stringKeys.length - show} more`); - return `{${parts.join(', ')}}`; - } finally { - seen.delete(handle.identity); - } -} - -/** Render the collapsed preview body (everything after the header), capped - * at MAX_COLLAPSED_CHARS (FORMAT.md §3). */ -export function renderCollapsed(preview: ObjectPreview): string { - let body: string; - if (preview.type === 'string') { - // A top-level string that IS the emitted value is OUTPUT, not a - // preview of shape: its description is returned whole rather than - // re-clamped to the collapsed-preview backstop (the §4.4 rule — - // direct strings print whole; there is no byte ceiling). Non-string - // primitives and composites keep the backstop below. - return preview.description; - } - if (preview.type !== 'object') { - // Primitives, functions, symbols: the description IS the preview. - body = preview.description; - } else if (preview.subtype === 'null') { - body = 'null'; - } else if (preview.subtype === 'array' || preview.subtype === 'typedarray') { - body = `${preview.description} [${entries(preview)}]`; - } else if (preview.subtype === 'promise') { - body = renderPromise(preview); - } else if (preview.subtype === undefined && preview.description === 'Object') { - // Plain objects: bare braces, no "Object" prefix. - body = `{${entries(preview)}}`; - } else { - // Everything else (branded objects, proxies): description, plus braces - // only when there is something to put in them. - body = - preview.properties.length === 0 && !preview.overflow - ? preview.description - : `${preview.description} {${entries(preview)}}`; - } - return hardCap(body, MAX_COLLAPSED_CHARS); -} - -function hardCap(s: string, cap: number): string { - const chars = toChars(s); - if (chars.length <= cap) return s; - return chars.slice(0, cap - 1).join('') + '…'; -} - -/** Comma-joined property list: canonical-index names render positionally - * (value only); named properties render `name: value`; accessors render - * `(...)`; the overflow flag appends a final `…` entry. */ -function entries(preview: ObjectPreview): string { - const parts: string[] = []; - for (const p of preview.properties) { - parts.push(renderProperty(p)); - } - if (preview.overflow) parts.push('…'); - return parts.join(', '); -} - -function renderProperty(p: PropertyPreview): string { - const value = p.type === 'accessor' ? '(...)' : (p.value ?? ''); - if (isCanonicalIndex(p.name)) return value; - return `${p.name}: ${value}`; -} - -function renderPromise(preview: ObjectPreview): string { - const state = - preview.properties.find((p) => p.name === '[[PromiseState]]')?.value ?? 'pending'; - const result = preview.properties.find((p) => p.name === '[[PromiseResult]]'); - if (result === undefined) return `Promise {<${state}>}`; - const token = result.type === 'accessor' ? '(...)' : (result.value ?? ''); - return `Promise {<${state}>: ${token}}`; -} - -/** - * Decimal byte-size formatting (FORMAT.md §2.2): `B`, then `kB`/`MB`/`GB`/ - * `TB` at multiples of 1000, one decimal with a trailing `.0` stripped. - * The DISPLAYED value is kept below 1000: a value that one-decimal - * rounding would render as `1000.0` (anything ≥ 999.95) moves to the next - * unit instead — `999_999` is `1MB`, never `1000kB`. */ -export function formatByteSize(n: number): string { - if (n < 1000) return `${n}B`; - const units = ['kB', 'MB', 'GB', 'TB']; - let value = n; - let unit = 0; - value /= 1000; - while (value >= 999.95 && unit + 1 < units.length) { - value /= 1000; - unit += 1; - } - const s = value.toFixed(1); - const stripped = s.endsWith('.0') ? s.slice(0, -2) : s; - return `${stripped}${units[unit]}`; -} - -// ---- Byte-size estimation ---- - -/** Traversal bounds for the byte-size estimate: honest up to these - * budgets, degrading to an undercount beyond them (documented; the - * preview line marks sizes, it does not meter them). */ -const SIZE_MAX_NODES = 2048; -const SIZE_MAX_DEPTH = 32; - -/** - * Bounded, trap-free byte-size estimate of a guest value (internal; the - * public seams are `inspectGlobal`/`manifestBinding`). Uses only the - * introspection surface (brand checks, own-key listing, descriptor access - * that never fires getters, engine object identity), an explicit work - * stack (no recursion), a real VISITED SET (each object counts once, - * however many paths reach it — cycles terminate, shared subgraphs are - * never recounted) and node/depth budgets — so a hostile or cyclic graph - * costs bounded work and can never execute guest code. Sizes are estimates - * by design and the failure mode is an UNDERcount: strings count UTF-8 - * bytes, primitives use fixed costs, and container INTERNALS that are not - * readable trap-free (Map/Set entries, the Date time value, RegExp source) - * use a flat token — their own string-keyed expando properties, which ARE - * readable trap-free, count normally on top of it. - */ -function estimateSize(handle: JSValueHandle): number { - let total = 0; - let expanded = 0; - const visited = new Set(); - const stack: Array<{ handle: JSValueHandle; depth: number }> = [{ handle, depth: 0 }]; - while (stack.length > 0) { - const frame = stack.pop()!; - if (expanded >= SIZE_MAX_NODES) { - // Budget exhausted: dispose the remainder and stop sizing. - frame.handle.dispose(); - for (const rest of stack) rest.handle.dispose(); - break; - } - // Identity dedup (engine object pointer): a value reached twice — a - // cycle or a shared subgraph — is counted exactly once. Identity 0 is - // non-heap (primitives), which never recurse anyway. - const id = frame.handle.identity; - if (id !== 0) { - if (visited.has(id)) { - frame.handle.dispose(); - continue; - } - visited.add(id); - } - expanded += 1; - total += nodeSize(frame.handle, frame.depth, stack); - frame.handle.dispose(); - } - return total; -} - -function nodeSize( - node: JSValueHandle, - depth: number, - stack: Array<{ handle: JSValueHandle; depth: number }>, -): number { - // Primitives first (brand-checked before any conversion). - if (node.isUndefined || node.isNull) return 4; - if (node.isBool) return 4; - if (node.isNumber) return 8; - if (node.isBigInt) return 16; - if (node.isString) return byteLength(node.toString()); - if (node.isSymbol || node.isFunction) return 32; - if (!node.isObject) return 8; - - // Objects: proxies are never traversed; buffer-backed values report - // their real byte length; brands with unobservable internals get a - // flat token PLUS their trap-free-readable own expando properties. - if (node.isProxy) return 32; - const tinfo = typedArrayInfo(node); - if (tinfo !== undefined) return 16 + tinfo.byteLength; - const abLen = arrayBufferByteLength(node); - if (abLen !== undefined) return 16 + abLen; - const opaqueBrand = - node.isMap || - node.isSet || - node.isDate || - node.isRegExp || - node.isWeakRef || - node.isWeakMap || - node.isWeakSet || - node.isDataView || - node.isPromise; - - // Base cost: a flat token for opaque internals (entries/time/source are - // not readable trap-free — an undercount by design), 16 bytes overhead - // for ordinary objects. Both then count own string keys and - // (data-property) values within the depth budget. - let size = opaqueBrand ? 32 : 16; - if (depth >= SIZE_MAX_DEPTH) return size; - const { keys, corrupted } = rawOwnKeysAll(node); - if (corrupted) return size; - for (const key of keys) { - if (key.symbol) { - size += 16; - continue; - } - const name = key.name!; - size += byteLength(name); - const desc = readOwnDescriptor(node, name); - if (desc === undefined) continue; - if (desc.kind === 'data') { - stack.push({ handle: desc.value, depth: depth + 1 }); - } else { - size += 16; - } - } - return size; -} - -/** UTF-8 byte length of a host string. */ -function byteLength(s: string): number { - return Buffer.byteLength(s, 'utf8'); -} - -// ---- Trap-free global-slot reads (the manifest seam's resolver) ---- - -/** - * Resolve a realm global slot trap-free and return the VALUE handle, or - * `undefined` when the slot is absent, rebound to an accessor (never - * invoked), or the read failed. The returned handle is owned by the - * caller. - */ -function readSlotValue(vm: ReplVm, name: string): JSValueHandle | undefined { - const shim = getVmShim(vm) as QuickJS; - const global = shim.global; // cached singleton — do not dispose - const e = shim._getExports(); - const keyHandle = shim.newString(name); - let descPtr: number; - try { - descPtr = e.qjs_get_own_property_descriptor(global.ptr, keyHandle.ptr); - } finally { - keyHandle.dispose(); - } - if (descPtr === 0) return undefined; - const desc = new JSValueHandle(shim, descPtr); - try { - if (e.qjs_is_exception(desc.ptr) !== 0) { - const excPtr = e.qjs_get_exception(); - if (excPtr !== 0) new JSValueHandle(shim, excPtr).dispose(); - return undefined; - } - // Data vs accessor via `hasOwnProperty` on the descriptor object (raw): - // `getPropRaw` alone cannot distinguish an absent property (it returns - // an undefined-valued handle for a plain miss — a prototype walk on the - // engine-created descriptor, which is guest-code-free but ambiguous). - // A data descriptor whose VALUE is undefined is still data. - if (hasOwnRaw(e, shim, desc.ptr, 'value')) { - const valueProp = getPropRaw(e, shim, desc.ptr, 'value'); - if (valueProp !== undefined) return valueProp; - return undefined; // allocation failure edge — reads as absent - } - // Accessor: never invoke; free the owned get/set handles. - getPropRaw(e, shim, desc.ptr, 'get')?.dispose(); - getPropRaw(e, shim, desc.ptr, 'set')?.dispose(); - return undefined; - } finally { - desc.dispose(); - } -} - -/** - * Metadata for one realm global slot, content-free: the CDP label (type - * or subtype), the byte-size estimate, and the resolution kind. This is - * the workspace-manifest seam — `ls` for the data plane (the doc: top- - * level bindings with name, type, size; metadata, never content). - */ -export function inspectGlobal(vm: ReplVm, name: string): { - kind: 'data' | 'accessor' | 'absent'; - label: string; - sizeBytes: number; -} { - // The lexical view first: a global lexical binding (top-level - // let/const/class) shadows a same-named global-object property for - // identifier resolution, so the binding metadata is the lexical - // binding's. - const lexicalValue = readLexicalSlotValue(vm, name); - if (lexicalValue !== undefined) { - try { - const preview = previewHandle(lexicalValue); - return { - kind: 'data', - label: preview.subtype ?? preview.type, - sizeBytes: estimateSize(lexicalValue), - }; - } finally { - lexicalValue.dispose(); - } - } - const value = readSlotValue(vm, name); - if (value === undefined) { - const kind = readRealmSlotKind(vm, name); - return { kind, label: 'undefined', sizeBytes: 0 }; - } - try { - const preview = previewHandle(value); - return { - kind: 'data', - label: preview.subtype ?? preview.type, - sizeBytes: estimateSize(value), - }; - } finally { - value.dispose(); - } -} - -/** One binding's manifest metadata (the workspace-manifest seam): the - * structure-only token, the byte-size estimate, and the live-handle - * call id when the binding is an agent handle. Resolution is trap-free: - * the binding is read through its own property descriptor on - * globalThis, never a `[[Get]]` — an accessor-rebound binding renders - * the explicit sabotage marker and its getter is NEVER invoked - * (rendering the manifest must not execute guest code). - * - * The token NEVER embeds value content (the harness manifest's hygiene - * rule): type label plus shape count plus byte size — EVERY binding - * reports name, type, and size (the doc's manifest contract; phase-E - * review rejection: undefined/null/numbers/booleans/bigints/symbols/ - * functions and plain promises used to render without any size). For - * strings the bare `string` label plus size, for arrays/buffers the - * structural `()` description, for plain objects `{N keys}`, - * for every other brand the brand label plus size. A live agent handle - * (an own non-enumerable marker property under the registered handle - * symbol, read through the descriptor machinery — a guest-rebound - * handle can never forge the read) reports `agent handle` plus its call - * id; the caller (the broker) appends the live-handle status from the - * call store. - * - * `sizeBytes` is the trap-free byte-size estimate (`estimateSize` — an - * undercount by design for opaque internals; 0 for the accessor and - * unreadable cases, where no size exists without invoking the getter). - * - * Returns null for an absent/unreadable slot. - */ -export function manifestBinding( - vm: ReplVm, - name: string, -): { token: string; type: string; handleCallId: string | null; sizeBytes: number } | null { - // The lexical view first (see `inspectGlobal`): a global lexical - // binding shadows a same-named global-object property, so the - // manifest's binding is the lexical one. Lexical bindings are always - // data (let/const/class — never accessors), so no sabotage marker - // applies on this path. - const lexicalValue = readLexicalSlotValue(vm, name); - if (lexicalValue !== undefined) { - try { - return describeBindingValue(vm, lexicalValue); - } finally { - lexicalValue.dispose(); - } - } - const value = readSlotValue(vm, name); - if (value === undefined) { - const kind = readRealmSlotKind(vm, name); - if (kind === 'accessor') { - return { token: 'accessor (getter not invoked)', type: 'accessor', handleCallId: null, sizeBytes: 0 }; - } - return null; - } - try { - return describeBindingValue(vm, value); - } finally { - value.dispose(); - } -} - -/** The shared manifest-token logic over one resolved binding value (see - * `manifestBinding`): the live-handle detector first, then the - * structure-only token — EVERY binding carries its byte-size estimate - * (the doc's name/type/size contract; the size is computed for the - * agent-handle case too, before the handle marker short-circuit). - * Trap-free throughout (see `manifestBinding`); a preview failure - * degrades to the `?` token with size 0, never a throw. */ -function describeBindingValue( - vm: ReplVm, - value: JSValueHandle, -): { token: string; type: string; handleCallId: string | null; sizeBytes: number } { - try { - // The live-handle detector first: an own data property under the - // registered handle symbol, read through the raw descriptor export - // (never a [[Get]]; proxies are guarded before any descriptor read). - const handleCallId = agentHandleCallId(vm, value); - // The preview and the plain-object key count must be read BEFORE - // `estimateSize` (which consumes the handle — its established - // dispose contract; the preview is plain data, the key count is - // not). The size itself is computed for EVERY binding — the - // agent-handle case included (phase-E review rejection: the - // manifest used to omit size for handle bindings). - const preview = previewHandle(value); - const keyCount = - preview.type === 'object' && preview.subtype === undefined ? ownKeyCount(value) : null; - const size = estimateSize(value); - if (handleCallId !== null) { - return { token: 'agent handle', type: 'agent handle', handleCallId, sizeBytes: size }; - } - const token = describeManifest(preview, size, keyCount); - return { token, type: manifestTypeLabel(preview), handleCallId: null, sizeBytes: size }; - } catch { - return { token: '?', type: '?', handleCallId: null, sizeBytes: 0 }; - } -} - -/** The machine-readable type label of a previewed value — the - * structure-only vocabulary the structured manifest carries alongside - * the human token (phase-E review round 4: the manifest exposed only - * the formatted token, with the live-handle status and call id - * embedded in the string and the type implicit in it; a structured - * consumer must not have to parse the token). Mirrors the token's - * label vocabulary exactly (see `describeManifest`): `undefined`, - * `number`, `boolean`, `bigint`, `symbol`, `function`, `string`, the - * object subtypes (`null`, `array`, `typedarray`, `arraybuffer`, - * `map`, `set`, `weakmap`, `weakset`, `weakref`, `date`, `regexp`, - * `error`, `proxy`, `promise`, `dataview`), `object` for plain - * objects, `agent handle` for live agent handles (the caller appends - * the live-handle status), and `accessor`/`?` for the unreadable - * cases — metadata, never content. */ -function manifestTypeLabel(preview: ObjectPreview): string { - if (preview.type !== 'object') return preview.type; - return preview.subtype === undefined ? 'object' : preview.subtype; -} - -/** The structure-only token for a previewed value (see `manifestBinding`; - * mirrors the harness manifest's `describe`). `keyCount` is the plain- - * object own string-key count read before the size estimate consumed the - * handle (`null` for non-plain-objects). EVERY branch carries the byte - * size — the doc's manifest contract is name, type, AND size for every - * top-level binding (phase-E review rejection: primitives, null, - * functions and plain promises used to render without size). */ -function describeManifest( - preview: ObjectPreview, - size: number, - keyCount: { count: number; corrupted: boolean } | null, -): string { - switch (preview.type) { - case 'undefined': - return `undefined \u00b7 ${formatByteSize(size)}`; - case 'number': - return `number \u00b7 ${formatByteSize(size)}`; - case 'boolean': - return `boolean \u00b7 ${formatByteSize(size)}`; - case 'bigint': - return `bigint \u00b7 ${formatByteSize(size)}`; - case 'symbol': - return `symbol \u00b7 ${formatByteSize(size)}`; - case 'function': - return `function \u00b7 ${formatByteSize(size)}`; - case 'string': - return `string \u00b7 ${formatByteSize(size)}`; - case 'object': - switch (preview.subtype) { - case 'null': - return `null \u00b7 ${formatByteSize(size)}`; - case 'array': - case 'typedarray': - case 'arraybuffer': - // Structural descriptions only (FORMAT.md §5.12–5.14): the - // description here is `()` — never content. - return `${preview.description} \u00b7 ${formatByteSize(size)}`; - case 'map': - return `Map(?) \u00b7 ${formatByteSize(size)}`; - case 'set': - return `Set(?) \u00b7 ${formatByteSize(size)}`; - case 'promise': - // A promise WITHOUT the handle marker is not an agent handle - // (a plain guest promise): the brand label plus size (phase-E - // review rejection: the size used to be omitted). - return `Promise \u00b7 ${formatByteSize(size)}`; - case 'proxy': - return `Proxy \u00b7 ${formatByteSize(size)}`; - default: - if (preview.subtype === undefined) { - // A plain object: the own string-key count (structure only; - // a corrupted key materialization degrades to `{? keys}`). - const count = keyCount === null || keyCount.corrupted ? '?' : String(keyCount.count); - const plural = keyCount === null || keyCount.count === 1 ? 'key' : 'keys'; - return `{${count} ${plural}} \u00b7 ${formatByteSize(size)}`; - } - return `${preview.subtype} \u00b7 ${formatByteSize(size)}`; - } - default: - return `${preview.type} \u00b7 ${formatByteSize(size)}`; - } -} - -/** The live-handle call id of a realm value, trap-free: an agent handle - * is a promise carrying the library's own non-enumerable `id` (string), - * `queue` (function), and `steer` (function) data properties — the shape the harness's - * manifest detector uses. Own-property-descriptor reads only, never a - * [[Get]] (an accessor-rebound property reads as absent — never - * invoked); proxies are guarded before any descriptor read. A guest - * could forge the shape, but the manifest is orientation metadata about - * the orchestrator's own workspace — forging it is self-sabotage, the - * same stance the harness takes (and the honest alternative — a per- - * handle marker property — measurably grew every parked call's realm - * footprint, tipping the bounded-memory suite's knife-edge). */ -function agentHandleCallId(vm: ReplVm, handle: JSValueHandle): string | null { - if (handle.isProxy || !handle.isPromise) return null; - const shim = getVmShim(vm) as QuickJS; - const idHandle = readOwnDataProperty(handle, 'id'); - if (idHandle === undefined || !idHandle.isString) { - idHandle?.dispose(); - return null; - } - const queueHandle = readOwnDataProperty(handle, 'queue'); - const steerHandle = readOwnDataProperty(handle, 'steer'); - if ( - queueHandle === undefined || - !queueHandle.isFunction || - steerHandle === undefined || - !steerHandle.isFunction - ) { - idHandle.dispose(); - queueHandle?.dispose(); - steerHandle?.dispose(); - return null; - } - queueHandle.dispose(); - steerHandle.dispose(); - try { - return idHandle.toString(); - } finally { - idHandle.dispose(); - } -} - -/** The slot kind for the sabotage distinction (reuses bridge's reader — - * kept local to avoid a cross-module dependency on bridge internals). - * Own-descriptor read on globalThis, same discipline as bridge.ts's - * readRealmSlot (which callers may use instead; this one only - * distinguishes the accessor case for the marker line). */ -function readRealmSlotKind(vm: ReplVm, name: string): 'data' | 'accessor' | 'absent' { - // Own-descriptor read on globalThis, same discipline as bridge.ts's - // readRealmSlot (which callers may use instead; this one only - // distinguishes the accessor case for the marker line). - const shim = getVmShim(vm) as QuickJS; - const global = shim.global; - const e = shim._getExports(); - const keyHandle = shim.newString(name); - let descPtr: number; - try { - descPtr = e.qjs_get_own_property_descriptor(global.ptr, keyHandle.ptr); - } finally { - keyHandle.dispose(); - } - if (descPtr === 0) return 'absent'; - const desc = new JSValueHandle(shim, descPtr); - try { - if (e.qjs_is_exception(desc.ptr) !== 0) { - const excPtr = e.qjs_get_exception(); - if (excPtr !== 0) new JSValueHandle(shim, excPtr).dispose(); - return 'absent'; - } - if (hasOwnRaw(e, shim, desc.ptr, 'value')) { - getPropRaw(e, shim, desc.ptr, 'value')?.dispose(); - return 'data'; - } - getPropRaw(e, shim, desc.ptr, 'get')?.dispose(); - getPropRaw(e, shim, desc.ptr, 'set')?.dispose(); - return 'accessor'; - } finally { - desc.dispose(); - } -} diff --git a/packages/repl-engine/src/provenance.ts b/packages/repl-engine/src/provenance.ts deleted file mode 100644 index 1e9128d7..00000000 --- a/packages/repl-engine/src/provenance.ts +++ /dev/null @@ -1,586 +0,0 @@ -/** - * The host side of the workspace manifest's per-binding provenance (the - * roadmap doc's `status` manifest: "provenance (which subagent produced - * the value, from what task, when)" — metadata only, never content). - * - * The registry itself lives INSIDE the realm under - * `Symbol.for("repl.provenance")` (see `src/guest/guest-library.ts`): the - * guest library installs it at VM creation, so it travels inside - * snapshots, survives restore, and rolls back coherently with a workspace - * snapshot. This module drives it from the host: - * - * - `baselineGlobalKeys` — the fresh-realm key set (builtins + the four - * host functions + the guest library's globals + the structured-clone - * extension's `structuredClone`), captured once per process from a - * throwaway VM provisioned exactly like a real workspace. User bindings - * are recognized by set difference against this baseline (the harness - * manifest's own discipline), so the manifest lists exactly what the - * orchestrator (or its workers' results) put in the global scope. - * - `provenanceBootstrap` — fills the registry's `known` set from the - * baseline and CREATES the registry when a snapshot predates the - * feature (a pre-provenance restore: the library is never re-evaluated - * over a restored workspace, so the host installs a byte-identical - * registry and the restore sweep attributes pre-existing bindings to - * `session restore` — "first seen at restore", never a guessed origin). - * - `provenanceRecord` — one maintenance pass after a guest-entering - * operation (an eval, a settlement drain, the restore sweep): diffs the - * global scope against the registry's `prev`/`known` sets and - * attributes NEW or REBOUND bindings (SameValue — NaN is stable) to the - * operation's host-shaped origin label (`eval N` with the registry's - * own monotonic, snapshot-durable counter; `worker c1+c2`; `session - * restore`). GLOBAL LEXICAL bindings (top-level let/const/class) are - * tracked by their VALUES: the host reads each binding's current value - * through the internal global-var object and hands it to the registry, - * which RE-ATTRIBUTES a changed value to the operation that produced - * it — a `let` binding assigned a worker result, or a suspended `const - * finding = await research` whose continuation assigned the settled - * value, re-attributes to the settlement's `worker cN` label, so the - * manifest reports which subagent produced the current value, from - * what task, when. In-place mutation of a binding's VALUE (a property - * of the bound object, say) deliberately does NOT re-attribute (the - * binding still refers to the value its recorded origin produced). - * The pass is trap-free by construction: own-property-descriptor reads - * only (an accessor-rebound binding is detected through its getter - * FUNCTION identity, never invoked), and the whole pass is the guest - * library's frozen closure — no guest-authored code runs. - * - `provenanceView` — the registry read for rendering, SANITIZED: only - * host-shaped labels survive (`eval N` / `worker ` / `session - * restore`); a vandalized registry degrades to missing provenance, - * never to content in the manifest. - * - * The maintenance pass is a guest-side function (the library's frozen - * closure) driven over the raw call machinery — the same discipline as - * the reconciliation surface's member calls. - */ - -import { JSValueHandle, type QuickJS } from 'quickjs-wasi'; - -import { installGuestBridge, readRealmSlotTypeToken, type GuestBridgeHandlers } from './bridge.js'; -import { GUEST_PROVENANCE_KEY, PROVENANCE_FACTORY } from './guest/guest-library.js'; -import { rawLexicalKeys, readLexicalSlotValue } from './global-lexical.js'; -import { wasmSha256Of } from './snapshot-envelope.js';import { - getPropRaw, - hasOwnRaw, - rawOwnKeys, - readValueComplete, - takeAndFreeException, - type QuickJSExports, -} from './trapfree.js'; -import type { WasmInput } from './types.js'; -import { ReplVm, getVmShim } from './vm.js'; - -/** The origin of one maintenance pass (see the module docs). */ -export type ProvenanceOrigin = - /** A root eval: the label is `eval N` with the registry's own counter. */ - | { kind: 'eval' } - /** A settlement batch: the calls settled into the guest (`worker c3`, or - * `worker c3+c4` for a multi-call batch — batch granularity is honest - * ambiguity). */ - | { kind: 'settlement'; callIds: string[] } - /** The restore sweep for PRE-PROVENANCE snapshots (`session restore`). */ - | { kind: 'restore' }; - -/** One binding's provenance as read back for rendering (unvalidated until - * `provenanceView` sanitizes it). */ -export interface OriginRecord { - via: string; - at: number; -} - -/** The sanitized registry contents used for rendering. */ -export interface ProvenanceView { - evalSeq: number; - origins: Map; - /** The KNOWN (baseline) names whose CURRENT value is no longer the - * pristine baseline — the type token changed OR the value is no - * longer SameValue to the registry's ORIGINAL baseline value - * (same-type replacement: `Math = { userOwned: true }` keeps the - * `object` token; the value identity is the detector the token - * cannot provide — phase-E review rejection round 6). Computed at - * read time by the registry's own trap-free descriptor pass. The - * manifest's changed-binding filter for overwritten built-ins. */ - changed: Set; -} - -/** A fresh-realm baseline key set (see the module docs). */ -export type BaselineKeys = string[]; - -/** The fresh-realm baseline TYPE TOKENS (name → trap-free `typeof` - * token of the pristine value — see `readRealmSlotTypeToken`): the - * manifest's changed-binding detector and the provenance registry's - * rebinding detector for KNOWN (baseline) names. */ -export type BaselineTypeTokens = Map; - -/** One wasm binary's cached baseline computation. */ -interface BaselineCapture { - keys: BaselineKeys; - types: BaselineTypeTokens; -} - -// The baseline is computed once per process per wasm binary (a throwaway -// VM instantiation); the shipped binary dominates, and a custom wasm -// input hashes directly. -const baselineCache = new Map>(); -const baselineCacheByModule = new WeakMap>(); - -/** The fresh-realm baseline capture for a wasm binary (see - * `computeBaseline`). Cached per process per binary. */ -function baselineCapture(wasm: WasmInput): Promise { - let key: string | undefined; - try { - key = wasmSha256Of(wasm); - } catch { - key = undefined; - } - if (key !== undefined) { - const cached = baselineCache.get(key); - if (cached !== undefined) return cached; - } else if (typeof wasm === 'object' && wasm !== null) { - const cached = baselineCacheByModule.get(wasm as object); - if (cached !== undefined) return cached; - } - const promise = computeBaseline(wasm); - if (key !== undefined) baselineCache.set(key, promise); - else if (typeof wasm === 'object' && wasm !== null) baselineCacheByModule.set(wasm as object, promise); - return promise; -} - -/** - * The fresh-realm global key set for a wasm binary: a throwaway VM is - * provisioned exactly like a real workspace (same binary, same - * structured-clone extension, same host functions, same guest library) - * and its string-key set is captured trap-free. Cached per process per - * binary. Used both as the manifest's user-binding baseline and as the - * provenance registry's `known` set. - */ -export function baselineGlobalKeys(wasm: WasmInput): Promise { - return baselineCapture(wasm).then((capture) => capture.keys); -} - -/** - * The fresh-realm baseline TYPE TOKENS for a wasm binary (see - * `BaselineTypeTokens`): the pristine value of every baseline name is - * typed trap-free in the throwaway VM. Cached with the key set. The - * manifest's changed-binding detector and the provenance registry's - * known-name rebinding detector read these. - */ -export function baselineGlobalTypeTokens(wasm: WasmInput): Promise { - return baselineCapture(wasm).then((capture) => capture.types); -} - -/** The throwaway-VM baseline computation (see `baselineGlobalKeys`): - * the key set AND the per-name type token (the pristine value's - * trap-free `typeof` — `readRealmSlotTypeToken`, the same helper the - * manifest's changed-binding detector uses, so the two computations - * can never drift). */ -async function computeBaseline(wasm: WasmInput): Promise { - const vm = await ReplVm.create({ wasm }); - try { - await installGuestBridge(vm, NOOP_HANDLERS); - const shim = getVmShim(vm) as QuickJS; - const keys = rawOwnKeys(shim.global); - const types = new Map(); - for (const name of keys) { - types.set(name, readRealmSlotTypeToken(vm, name)); - } - return { keys, types }; - } finally { - vm.dispose(); - } -} - -// The fresh-realm LEXICAL baseline: top-level `let`/`const`/`class` -// bindings a provisioned realm carries before any user eval. The guest -// library deliberately declares only `var`s inside its IIFE, so the set -// is EMPTY on the shipped library — but a future library that used -// lexical declarations would otherwise leak its internals into the -// manifest's user bindings, so the baseline is computed, cached and -// subtracted exactly like the global one. -const lexicalBaselineCache = new Map>(); -const lexicalBaselineCacheByModule = new WeakMap>(); - -/** - * The fresh-realm GLOBAL LEXICAL key set for a wasm binary (top-level - * `let`/`const`/`class` bindings — see `global-lexical.ts`): a throwaway - * VM is provisioned exactly like a real workspace and its lexical key - * set is captured through the internal global-var object. Cached per - * process per binary, mirroring `baselineGlobalKeys`. The workspace - * manifest subtracts this set (alongside the global baseline) from its - * user-binding enumeration. - */ -export function baselineLexicalKeys(wasm: WasmInput): Promise { - let key: string | undefined; - try { - key = wasmSha256Of(wasm); - } catch { - key = undefined; - } - if (key !== undefined) { - const cached = lexicalBaselineCache.get(key); - if (cached !== undefined) return cached; - } else if (typeof wasm === 'object' && wasm !== null) { - const cached = lexicalBaselineCacheByModule.get(wasm as object); - if (cached !== undefined) return cached; - } - const promise = computeLexicalBaseline(wasm); - if (key !== undefined) lexicalBaselineCache.set(key, promise); - else if (typeof wasm === 'object' && wasm !== null) lexicalBaselineCacheByModule.set(wasm as object, promise); - return promise; -} - -/** The throwaway-VM lexical baseline computation (see - * `baselineLexicalKeys`). */ -async function computeLexicalBaseline(wasm: WasmInput): Promise { - const vm = await ReplVm.create({ wasm }); - try { - await installGuestBridge(vm, NOOP_HANDLERS); - return rawLexicalKeys(vm); - } finally { - vm.dispose(); - } -} - -/** The parking-bridge stand-in for the throwaway baseline VM: the host - * functions exist (the library only needs the names). */ -const NOOP_HANDLERS: GuestBridgeHandlers = { - agent: () => undefined, - checkpoint: () => undefined, - queue: () => undefined, - steer: () => undefined, - cancelSession: () => undefined, - cancelQueue: () => undefined, - console: () => undefined, - sleep: () => undefined, - workspace: () => '{}', - agents: () => '[]', - reset: () => undefined, - defaultBackend: () => undefined, -}; - -/** - * Install (or complete) the provenance registry on a workspace's VM: - * fills the `known` baseline set from `baselineGlobalKeys(wasm)` and - * CREATES the registry when the snapshot predates the feature (a - * pre-provenance restore). Returns whether the registry was created by - * this call (the caller then runs the `session restore` sweep so - * pre-existing bindings are attributed as "first seen at restore", never - * guessed) plus the baseline key set and the baseline TYPE TOKENS (the - * manifest's changed-binding detector). Never errors upward: provenance - * is orientation metadata; a realm hostile enough to break the bootstrap - * simply has none. - */ -export async function provenanceBootstrap( - vm: ReplVm, - wasm: WasmInput, -): Promise<{ created: boolean; baseline: BaselineKeys; baselineTypes: BaselineTypeTokens }> { - const [baseline, baselineTypes, lexBaseline] = await Promise.all([ - baselineGlobalKeys(wasm), - baselineGlobalTypeTokens(wasm), - baselineLexicalKeys(wasm), - ]); - const shim = getVmShim(vm) as QuickJS; - const symbol = shim.newSymbolFor(GUEST_PROVENANCE_KEY); - let existing: JSValueHandle | undefined; - try { - existing = shim.getProp(shim.global, symbol); - // The factory arguments are embedded as JSON literals (a JSON object - // is a valid JS object literal): the baseline key array (the known - // set), the baseline TYPE TOKENS object (the known-name rebinding - // detector), and the LEXICAL baseline array (the lexical pass's own - // skip set — a lexical declaration shadows a same-named baseline - // global and is always the user's). - const namesJson = JSON.stringify(baseline); - const typeTokensJson = JSON.stringify(Object.fromEntries(baselineTypes)); - const lexKnownJson = JSON.stringify(lexBaseline); - if (existing.isUndefined || !existing.isObject) { - // A pre-provenance snapshot: install the byte-identical registry - // (the same factory the library evaluates at install time) with the - // baseline as its `known` set, the baseline type tokens, and the - // lexical baseline as the lexical pass's own skip set. - const source = - `(function () { try { var KEY = Symbol.for(${JSON.stringify(GUEST_PROVENANCE_KEY)}); ` + - `var reg = (${PROVENANCE_FACTORY})(${namesJson}, ${typeTokensJson}, ${lexKnownJson}); ` + - `Object.defineProperty(globalThis, KEY, { value: reg, writable: false, enumerable: false, ` + - `configurable: false }); return 1; } catch (e) { return 0; } })()`; - const outcome = await vm.evalCode(source, { filename: '' }); - return { created: outcome.kind !== 'error', baseline, baselineTypes }; - } - // The registry already exists (a fresh install, or a post-feature - // snapshot whose registry travels with the workspace): fill the - // `known` set and the baseline type tokens from the current - // baseline (a same-version snapshot's sets are already identical — - // this is a no-op; an older-library snapshot's known set - // legitimately reflects the older library and is left alone except - // for names the current baseline adds). - const source = - `(function () { try { var KEY = Symbol.for(${JSON.stringify(GUEST_PROVENANCE_KEY)}); ` + - `var reg = globalThis[KEY]; if (!reg || typeof reg !== 'object') return 0; ` + - `var names = ${namesJson}; for (var i = 0; i < names.length; i++) reg.known[names[i]] = true; ` + - `if (!reg.baseTok) reg.baseTok = {}; var toks = ${typeTokensJson}; ` + - `for (var t in toks) reg.baseTok[t] = toks[t]; ` + - `if (!reg.lexKnown) reg.lexKnown = {}; var lk = ${lexKnownJson}; ` + - `for (var li = 0; li < lk.length; li++) reg.lexKnown[lk[li]] = true; ` + - // The baseline-VALUE fill (phase-E review rejection round 6): a - // registry that predates the same-type-replacement detector (a - // pre-0.3.1 snapshot) lacks reg.baseVal/reg.knownPrev — fill them - // from the CURRENT realm for every known name (the same - // descriptor-based capture the factory performs). Guarded in its - // own try/catch: a realm hostile enough to shadow Object/ - // globalThis degrades to the token-only detector, never failing - // the fill (a pre-snapshot same-type overwrite is the same - // undetectable corner the bootstrap accepts for pre-provenance - // restores). - `try { if (!reg.baseVal) reg.baseVal = {}; if (!reg.knownPrev) reg.knownPrev = {}; ` + - `var hOP = Object.prototype.hasOwnProperty; var gOPD = Object.getOwnPropertyDescriptor; var g = globalThis; ` + - `for (var n = 0; n < names.length; n++) { var kn = names[n]; if (hOP.call(reg.baseVal, kn)) continue; ` + - `var kd = gOPD(g, kn); var kv; if (kd !== undefined && hOP.call(kd, 'value')) kv = kd.value; ` + - `else if (kd !== undefined && hOP.call(kd, 'get')) kv = kd.get; ` + - `reg.baseVal[kn] = kv; reg.knownPrev[kn] = kv; } } catch (e) {} ` + - `return 1; } catch (e) { return 0; } })()`; - const outcome = await vm.evalCode(source, { filename: '' }); - void outcome; - return { created: false, baseline, baselineTypes }; - } finally { - existing?.dispose(); - symbol.dispose(); - } -} - -/** - * One maintenance pass (see the module docs): attribute new/rebound user - * bindings (including `$N` globals) to the operation's origin. Errors are - * swallowed by design — provenance is orientation metadata. - */ -export function provenanceRecord(vm: ReplVm, origin: ProvenanceOrigin): void { - const shim = getVmShim(vm) as QuickJS; - const e = shim._getExports(); - const symbol = shim.newSymbolFor(GUEST_PROVENANCE_KEY); - let registryHandle: JSValueHandle | undefined; - let recordFn: JSValueHandle | undefined; - let labelHandle: JSValueHandle | undefined; - let ownsLabel = false; - let atHandle: JSValueHandle | undefined; - let lexHandle: JSValueHandle | undefined; - try { - registryHandle = shim.getProp(shim.global, symbol); - if (registryHandle.isUndefined || !registryHandle.isObject) return; - recordFn = readOwnDataPropertyShim(e, shim, registryHandle, 'record'); - if (recordFn === undefined || !recordFn.isFunction) return; - if (origin.kind === 'eval') { - // The registry computes `eval N` from its own snapshot-durable - // counter; the null/undefined label is the "eval pass" marker. The - // singleton undefined handle is NOT owned by this function. - labelHandle = shim.undefined; - } else { - const label = - origin.kind === 'settlement' ? `worker ${origin.callIds.join('+')}` : 'session restore'; - labelHandle = shim.newString(label); - ownsLabel = true; - } - atHandle = shim.newNumber(Date.now()); - // The pass's THIRD argument: the realm's global LEXICAL binding - // names as a JSON array string (see the factory in - // `guest-library.ts`). Lexical bindings cannot be enumerated - // guest-side; the host reaches them through the engine's internal - // global-var object (see `global-lexical.ts`) and hands the names - // over here — the same host-driven channel as every other aspect of - // the pass (no guest-visible surface grows for it). A registry whose - // record closure predates the feature (an older snapshot) simply - // ignores the argument. - const lexNames = rawLexicalKeys(vm); - lexHandle = shim.newString(JSON.stringify(lexNames)); - // The pass's FOURTH+ arguments: the realm's CURRENT LEXICAL VALUES, - // one realm value per name in the names array's order (the factory - // reads them at `arguments[3 + i]`). The host reads each binding - // through the internal global-var object's descriptor machinery and - // hands the VALUES over so the registry can detect a CHANGE - // (SameValue) and RE-ATTRIBUTE: a `let` binding assigned a worker - // result, or a suspended `const finding = await research` whose - // continuation assigned the settled value, re-attributes to the - // settlement's `worker cN` label — the manifest then reports WHICH - // subagent produced the current value, from what task, when - // (phase-E review rejection: the lexical entry was recorded on - // first sight only, so a value the worker settlement produced kept - // the declaring eval's label with no task). A registry whose record - // closure predates the feature (an older snapshot) ignores the - // extra arguments and degrades to first-sight-only attribution. - // The handles are BORROWED by the call (callRaw never frees its - // arguments) and disposed afterwards; `shim.undefined` (a cached - // singleton — dispose is a no-op) stands in for an unreadable - // binding (a TDZ cell reads as its raw uninitialized marker, which - // is passed through unchanged — SameValue against the settled value - // differs, so the re-attribution still fires). - const lexValues: JSValueHandle[] = []; - for (const name of lexNames) { - const value = readLexicalSlotValue(vm, name); - lexValues.push(value ?? shim.undefined); - } - try { - const result = new JSValueHandle( - shim, - callRaw(e, shim, recordFn.ptr, [labelHandle, atHandle, lexHandle, ...lexValues]), - ); - try { - if (e.qjs_is_exception(result.ptr) !== 0) takeAndFreeException(e, shim); - } finally { - result.dispose(); - } - } finally { - for (const value of lexValues) value.dispose(); - } - } finally { - if (ownsLabel) labelHandle?.dispose(); - atHandle?.dispose(); - lexHandle?.dispose(); - recordFn?.dispose(); - registryHandle?.dispose(); - symbol.dispose(); - } -} - -/** - * Read the registry for rendering, SANITIZED: only host-shaped origin - * labels survive (`eval N` / `worker ` / `session restore`); a - * vandalized registry degrades to missing provenance, never to content in - * the manifest. The read itself is trap-free and COMPLETE (the registry's - * own `read` closure returns plain data; `readValueComplete` reads own - * data properties with no property-count cap — phase-E review round 4: - * the generic 256-property cap silently dropped bindings 256+ from the - * manifest's provenance, reporting null provenance for bindings the eval - * did create; the registry is the host's own metadata, bounded by the - * VM's memory like the bindings it describes). - */ -export function provenanceView(vm: ReplVm): ProvenanceView { - const shim = getVmShim(vm) as QuickJS; - const e = shim._getExports(); - const symbol = shim.newSymbolFor(GUEST_PROVENANCE_KEY); - let registryHandle: JSValueHandle | undefined; - let readFn: JSValueHandle | undefined; - try { - registryHandle = shim.getProp(shim.global, symbol); - if (registryHandle.isUndefined || !registryHandle.isObject) return emptyView(); - readFn = readOwnDataPropertyShim(e, shim, registryHandle, 'read'); - if (readFn === undefined || !readFn.isFunction) return emptyView(); - const result = new JSValueHandle(shim, callRaw(e, shim, readFn.ptr, [])); - try { - if (e.qjs_is_exception(result.ptr) !== 0) { - takeAndFreeException(e, shim); - return emptyView(); - } - if (result.isUndefined) return emptyView(); - const data = readValueComplete(result) as { - evalSeq?: unknown; - origins?: unknown; - changed?: unknown; - } | null; - if (typeof data !== 'object' || data === null) return emptyView(); - const origins = new Map(); - if (typeof data.origins === 'object' && data.origins !== null) { - for (const [name, record] of Object.entries(data.origins as Record)) { - const r = record as { via?: unknown; at?: unknown } | null; - if (typeof r !== 'object' || r === null) continue; - if (typeof r.via !== 'string' || !isValidOriginLabel(r.via)) continue; - origins.set(name, { via: r.via, at: typeof r.at === 'number' ? r.at : 0 }); - } - } - // The changed-known-names list, SANITIZED the same way (strings - // only — a vandalized registry degrades to no changed bindings, - // never to content in the manifest). - const changed = new Set(); - if (Array.isArray(data.changed)) { - for (const name of data.changed) { - if (typeof name === 'string' && name.length > 0) changed.add(name); - } - } - return { - evalSeq: typeof data.evalSeq === 'number' ? data.evalSeq : 0, - origins, - changed, - }; - } finally { - result.dispose(); - } - } finally { - readFn?.dispose(); - registryHandle?.dispose(); - symbol.dispose(); - } -} - -function emptyView(): ProvenanceView { - return { evalSeq: 0, origins: new Map(), changed: new Set() }; -} - -/** Whether an origin label matches one of the shapes this host writes - * (see the module docs; the harness manifest's validation shapes). */ -export function isValidOriginLabel(label: string): boolean { - if (label === 'session restore') return true; - if (label.startsWith('eval ')) { - const digits = label.slice('eval '.length); - return digits.length > 0 && digits.length <= 10 && /^[0-9]+$/.test(digits); - } - if (label.startsWith('worker ')) { - const ids = label.slice('worker '.length); - return ids.length > 0 && ids.split('+').every((id) => isValidCallId(id)); - } - return false; -} - -/** A plausible call id: short, no whitespace, the guest id charset. */ -function isValidCallId(id: string): boolean { - return id.length > 0 && id.length <= 32 && /^[A-Za-z0-9_-]+$/.test(id); -} - -/** Raw own-data-property read over the exports (no JSException ever). */ -function readOwnDataPropertyShim( - e: QuickJSExports, - shim: QuickJS, - handle: JSValueHandle, - key: string, -): JSValueHandle | undefined { - if (handle.isProxy) return undefined; - const keyHandle = shim.newString(key); - let descPtr: number; - try { - descPtr = e.qjs_get_own_property_descriptor(handle.ptr, keyHandle.ptr); - } finally { - keyHandle.dispose(); - } - if (descPtr === 0) return undefined; - const desc = new JSValueHandle(shim, descPtr); - try { - if (e.qjs_is_exception(desc.ptr) !== 0) { - const excPtr = e.qjs_get_exception(); - if (excPtr !== 0) new JSValueHandle(shim, excPtr).dispose(); - return undefined; - } - // Has an own "value" (data descriptor)? Read it raw. - const has = hasOwnRaw(e, shim, desc.ptr, 'value'); - if (!has) { - getPropRaw(e, shim, desc.ptr, 'get')?.dispose(); - getPropRaw(e, shim, desc.ptr, 'set')?.dispose(); - return undefined; - } - return getPropRaw(e, shim, desc.ptr, 'value'); - } finally { - desc.dispose(); - } -} - -/** Raw `qjs_call` with borrowed arguments (mirrors bridge.ts's callRaw). */ -function callRaw(e: QuickJSExports, shim: QuickJS, fnPtr: number, args: JSValueHandle[]): number { - const argc = args.length; - let argvPtr = 0; - if (argc > 0) { - argvPtr = e.wasm_malloc(argc * 4); - const view = new DataView(e.memory.buffer); - for (let i = 0; i < argc; i++) { - view.setUint32(argvPtr + i * 4, args[i].ptr, true); - } - } - try { - return e.qjs_call(fnPtr, shim.undefined.ptr, argc, argvPtr); - } finally { - if (argvPtr !== 0) e.wasm_free(argvPtr); - } -} diff --git a/packages/repl-engine/src/repl-store.ts b/packages/repl-engine/src/repl-store.ts deleted file mode 100644 index 247e7494..00000000 --- a/packages/repl-engine/src/repl-store.ts +++ /dev/null @@ -1,406 +0,0 @@ -/** - * The daemon's per-project REPL store — the roadmap doc's §Snapshots - * storage: "Snapshots and the call store live in the daemon's existing - * per-project store — a `repl/` subdirectory next to the workflow state - * under `workflowHomeDir()/projects//`". - * - * The layout reuses the workflow store-layout helpers verbatim - * (`workflowProjectPaths` from `@automatalabs/workflows` — the same - * helpers the mcp-server project registry uses, so the store key derives - * from the project directory exactly as the workflow engine's and one - * project has one repl store): - * - * ```text - * workflowHomeDir()/projects// - * project.json (the workflow engine's manifest: key → projectDir) - * runs/… (workflow run state — untouched) - * repl/ - * snapshot.bin the enveloped VM snapshot (see snapshot-envelope.ts) - * calls.jsonl the append-only call store (JsonlCallStore) - * ``` - * - * ## Snapshot-write mechanics (spec-owed decisions) - * - * - **Cadence**: the broker fires a state-changing boundary after each - * eval and after each settlement drain that changed VM state - * (`BrokerOptions.snapshotSink`); the daemon wires it to - * `snapshotWriter(workspace, wasm)`, the debounced writer this store - * provides. `boundary()` marks the workspace dirty; - * `flush()` — fired by the broker at the end of each serialized - * operation, the burst boundary — writes once per burst. A broker - * eval that first pumps settled calls and then drains the eval itself - * is therefore ONE atomic write, taken before the eval's promise - * resolves (the debounce knob: `SnapshotWriteOptions.debounceBursts`, - * default true; false writes synchronously at every boundary). The - * debounced gap is always covered by the call store: settlements are - * recorded BEFORE they settle, so a restore replays them from the - * store arm. - * - **Atomicity**: every write goes to `.tmp`, fsynced, - * then renamed over `snapshot.bin` (a kill at any moment leaves - * either the old complete snapshot or the new complete one — never a - * torn file), then the directory is fsynced (best-effort) so the - * rename itself is durable. The tmp file is fixed-name (single-writer - * discipline: one daemon per project, like the call store) and is - * removed on failure; a crash leaves it for the next write to - * overwrite. - * - **Failure posture**: a snapshot write that fails throws loudly (the - * previous snapshot file is untouched) and a corrupt or truncated - * snapshot file refuses loudly on load (`SnapshotEnvelopeError`, - * naming the file and the problem) — a single-shot error, never a - * silent pass and never a retry loop. A failed write also leaves the - * writer's dirty boundary IN PLACE, so the next flush retries the - * same state (phase-D review round 6: the boundary used to clear - * before the write, silently dropping a failed last-disconnect - * snapshot). The store stays usable: a fresh `writeSnapshot` replaces - * the bad file, or `reset()` clears the whole `repl/` directory (the - * `reset` tool's engine-side). - * - **Config knobs** (decided names): `ReplStoreOptions.persistenceRoot` - * (overrides `workflowHomeDir` — tests and `AGENTPRISM_PERSISTENCE_ROOT` - * parity), `ReplStoreOptions.env` (workflow-path env overrides), - * `ReplStoreOptions.snapshotWrite.debounceBursts` and - * `snapshotWrite.fsync` (defaults: true, true). - */ - -import { - closeSync, - existsSync, - fsyncSync, - mkdirSync, - openSync, - readFileSync, - readdirSync, - renameSync, - rmSync, - writeSync, -} from 'node:fs'; -import { join } from 'node:path'; - -import { workflowProjectPaths } from '@automatalabs/workflows'; - -import type { SnapshotSink } from './broker.js'; -import { - deserializeSnapshot, - serializeSnapshot, - SnapshotEnvelopeError, - wasmSha256Of, -} from './snapshot-envelope.js'; -import { JsonlCallStore } from './store.js'; -import type { ReplSnapshot, WasmInput } from './types.js'; -import type { Workspace } from './workspace.js'; - -/** The `repl/` subdirectory name next to the workflow state. */ -export const REPL_STORE_SUBDIR = 'repl'; -/** The enveloped snapshot file name inside the repl directory. */ -export const SNAPSHOT_FILENAME = 'snapshot.bin'; -/** The call-store log file name inside the repl directory. */ -export const CALL_STORE_FILENAME = 'calls.jsonl'; -/** The tmp file the atomic write stages into (fixed name — one daemon - * per project, the same single-writer discipline as the call store). */ -const TMP_SUFFIX = '.tmp'; - -/** The snapshot-write policy knobs (names decided here; see module docs). */ -export interface SnapshotWriteOptions { - /** - * Coalesce the state-changing boundaries of one drain burst (a broker - * eval's pump-drain + eval-drain) into a single atomic write at the - * burst boundary. Default true — the doc's "debounce within a single - * drain burst". When false, every boundary writes synchronously. - */ - debounceBursts?: boolean; - /** - * fsync the snapshot file (and, best-effort, its directory) before a - * write is acknowledged. Default true — a kill at any moment loses - * nothing that was acknowledged. - */ - fsync?: boolean; -} - -/** Options for opening a project's repl store. */ -export interface ReplStoreOptions { - /** - * Override for the workflow home root the store lives under (see - * `workflowHomeDir`; tests use it; the environment's - * `AGENTPRISM_PERSISTENCE_ROOT` wins when this is omitted). - */ - persistenceRoot?: string; - /** Injectable env map for the workflow-path helpers (tests). */ - env?: Record; - /** The snapshot-write policy knobs. */ - snapshotWrite?: SnapshotWriteOptions; -} - -/** A snapshot restored from the store, with its identity meta. */ -export interface RestoredReplSnapshot { - /** The raw VM snapshot, ready for `Workspace.restore`. */ - snapshot: ReplSnapshot; - /** The wasm binary sha256 the envelope recorded. */ - wasmSha256: string; - /** The envelope format version (validated against the engine's). */ - formatVersion: number; - /** When the envelope was written. */ - createdAtMs: number; -} - -/** Store-level counters (the status seam + the debounce tests). */ -export interface ReplStoreStats { - /** Successful atomic snapshot writes since open (or since reset). */ - snapshotWrites: number; -} - -/** - * One project's REPL store: the enveloped snapshot plus the call store, - * under `workflowHomeDir()/projects//repl`. All operations are - * synchronous — a state-changing boundary persists before the caller's - * promise resolves (the broker's sink contract). - */ -export class ReplWorkspaceStore { - /** The project directory this store belongs to. */ - readonly projectDir: string; - /** The `repl/` directory holding this store's files. */ - readonly replDir: string; - /** The enveloped snapshot file (`/snapshot.bin`). */ - readonly snapshotPath: string; - /** The call-store log file (`/calls.jsonl`). */ - readonly callStorePath: string; - - private readonly options: { debounceBursts: boolean; fsync: boolean }; - private callStoreInstance: JsonlCallStore | null = null; - private snapshotWriteCount = 0; - - private constructor(projectDir: string, replDir: string, options: { debounceBursts: boolean; fsync: boolean }) { - this.projectDir = projectDir; - this.replDir = replDir; - this.snapshotPath = join(replDir, SNAPSHOT_FILENAME); - this.callStorePath = join(replDir, CALL_STORE_FILENAME); - this.options = options; - } - - /** - * Open (and create, on first touch) the project's repl store. The - * daemon passes the VALIDATED project directory (its project registry - * realpaths it, exactly like the workflow tool's `projectDir`). - */ - static open(projectDir: string, options: ReplStoreOptions = {}): ReplWorkspaceStore { - const paths = workflowProjectPaths(projectDir, { - persistenceRoot: options.persistenceRoot, - env: options.env, - }); - const replDir = join(paths.rootDir, REPL_STORE_SUBDIR); - mkdirSync(replDir, { recursive: true }); - return new ReplWorkspaceStore(projectDir, replDir, { - debounceBursts: options.snapshotWrite?.debounceBursts ?? true, - fsync: options.snapshotWrite?.fsync ?? true, - }); - } - - /** True when an enveloped snapshot exists at the store path. */ - hasSnapshot(): boolean { - return existsSync(this.snapshotPath); - } - - /** - * Write a snapshot of the workspace's VM to disk: serialize into the - * identity envelope (wasm sha256 + format version + gzip) and replace - * the snapshot file atomically (tmp + rename + fsync). The wasm - * binary's hash is computed here, so the envelope always records the - * binary that actually laid out the memory. - */ - writeSnapshot(snapshot: ReplSnapshot, wasm: WasmInput): void { - const envelope = serializeSnapshot(snapshot, wasmSha256Of(wasm)); - this.writeAtomic(envelope); - this.snapshotWriteCount++; - } - - /** - * Load the enveloped snapshot and verify its identity against the - * binary the host is about to restore with. A wasm-hash mismatch - * REFUSES LOUDLY naming both hashes (never a restore into garbage — - * the doc's transfer lesson 5; the check runs INSIDE the envelope - * deserializer, between the header parse and the payload decode, so a - * snapshot recorded by another binary refuses as WASM_HASH_MISMATCH - * even when its payload would not deserialize here — a phase-D review - * regression: the comparison used to happen after the payload was - * interpreted, so an incompatible old payload failed as - * CORRUPT_PAYLOAD without naming the hashes); a version bump or a - * corrupt/truncated file refuses with `SnapshotEnvelopeError` naming - * the file and the problem. Single-shot: the error propagates to the - * caller, and the store stays usable (a fresh `writeSnapshot` or - * `reset()`). - */ - loadSnapshot(wasm: WasmInput): RestoredReplSnapshot { - if (!existsSync(this.snapshotPath)) { - throw new SnapshotEnvelopeError( - 'BAD_HEADER', - `no snapshot at ${this.snapshotPath} — the workspace was never snapshotted`, - { path: this.snapshotPath }, - ); - } - const envelope = deserializeSnapshot(readFileSync(this.snapshotPath), { - path: this.snapshotPath, - expectedWasmSha256: wasmSha256Of(wasm), - }); - return { - snapshot: envelope.snapshot, - wasmSha256: envelope.meta.wasmSha256, - formatVersion: envelope.meta.formatVersion, - createdAtMs: envelope.meta.createdAtMs, - }; - } - - /** - * The project's append-only call store (`calls.jsonl` — a durable - * `JsonlCallStore`, opened lazily on first use and closed by `close`/ - * `reset`). One store per project: forks of one snapshot mint - * overlapping call ids, so each project keeps its own ledger. The - * `repl/` directory is recreated when missing (after a `reset()` a - * fresh use self-heals, like the snapshot writer does). - */ - callStore(): JsonlCallStore { - mkdirSync(this.replDir, { recursive: true }); - this.callStoreInstance ??= JsonlCallStore.open(this.callStorePath); - return this.callStoreInstance; - } - - /** - * The debounced snapshot writer the daemon wires to the broker's - * state-changing-boundary sink (`BrokerOptions.snapshotSink`): every - * `boundary()` marks the workspace dirty; `flush()` — the broker's - * end-of-operation burst boundary — writes once per burst (when - * `debounceBursts` is false, every boundary writes synchronously - * instead). The write snapshots the LIVE workspace through the same - * `writeSnapshot` atomic path. - */ - snapshotWriter(workspace: Workspace, wasm: WasmInput): SnapshotSink { - const debounce = this.options.debounceBursts; - let dirty = false; - return { - boundary: () => { - // A workspace torn down mid-operation cannot be snapshotted — - // its VM is gone (see `flush`). - if (workspace.isDisposed) return; - if (!debounce) { - this.writeSnapshot(workspace.snapshot(), wasm); - return; - } - dirty = true; - }, - flush: () => { - if (!dirty) return; - // A workspace torn down mid-operation (a reset/dispose racing a - // parked restore-time loadSession whose reconcile lands late, for - // example) cannot be snapshotted — the VM is gone, and the state - // that owns this writer is being discarded anyway. The skip is a - // deliberate TEARDOWN no-op, not the retained-dirty failure - // posture below: the daemon's shutdown drain persists its - // settlements BEFORE the workspace is disposed, so the state's - // last good snapshot on disk is the persistence story (a - // late-landing op must not throw "operation on a disposed - // workspace" from inside the broker's end-of-op flush). - if (workspace.isDisposed) { - dirty = false; - return; - } - // The write happens BEFORE the boundary clears (phase-D review - // round 6): a failing write leaves the boundary dirty, so the - // next flush — the next drain burst, or the next disconnect's - // retried drain — retries the SAME state instead of silently - // dropping it (the old order cleared `dirty` first, so a failed - // last-disconnect snapshot was lost without a trace). The write's - // throw propagates to the broker's operation — the failure is - // loud at the surface that triggered the boundary. - this.writeSnapshot(workspace.snapshot(), wasm); - dirty = false; - }, - }; - } - - /** - * Teardown the store: close the call store and delete the `repl/` - * directory's contents (the `reset()` guest function's engine-side — - * the workspace's VM and stored state are dropped together). - * §6.1 [C]13: a REFUSED snapshot renamed aside - * (`snapshot.bin.refused-`) is NEVER deleted — auto-reset must - * not be silent data destruction — so the whole-directory wipe is - * entry-wise and PRESERVES the renamed-aside files (a `reset()` - * after an auto-reset keeps the refused snapshot for inspection). - */ - reset(): void { - this.close(); - if (existsSync(this.replDir)) { - for (const entry of readdirSync(this.replDir)) { - if (entry.startsWith(`${SNAPSHOT_FILENAME}.refused-`)) continue; - rmSync(join(this.replDir, entry), { recursive: true, force: true }); - } - } - this.snapshotWriteCount = 0; - } - - /** Close the call store's log file (idempotent; the store stays - * readable through a later `callStore()`). */ - close(): void { - this.callStoreInstance?.close(); - this.callStoreInstance = null; - } - - /** Store-level counters. */ - stats(): ReplStoreStats { - return { snapshotWrites: this.snapshotWriteCount }; - } - - /** The atomic replace: stage into `.tmp`, fsync, rename, - * best-effort directory fsync. The store directory is recreated when - * missing (after a `reset()`, a fresh write self-heals). Any failure - * removes the tmp file and throws — the previous snapshot file is - * untouched. */ - private writeAtomic(bytes: Uint8Array): void { - const tmp = `${this.snapshotPath}${TMP_SUFFIX}`; - let fd: number | undefined; - try { - mkdirSync(this.replDir, { recursive: true }); - fd = openSync(tmp, 'w'); - let written = 0; - while (written < bytes.length) { - written += writeSync(fd, bytes, written, bytes.length - written); - } - if (this.options.fsync) fsyncSync(fd); - closeSync(fd); - fd = undefined; - renameSync(tmp, this.snapshotPath); - if (this.options.fsync) this.fsyncDir(); - } catch (error) { - if (fd !== undefined) { - try { - closeSync(fd); - } catch { - // Best effort — the tmp file is removed below regardless. - } - } - try { - rmSync(tmp, { force: true }); - } catch { - // Best effort — the next write's open truncates a stale tmp. - } - throw error; - } - } - - /** Directory fsync so the rename itself is durable; best-effort - * (some platforms do not support opening directories for sync). */ - private fsyncDir(): void { - let dirFd: number | undefined; - try { - dirFd = openSync(this.replDir, 'r'); - fsyncSync(dirFd); - } catch { - // Best effort — the file fsync is the load-bearing one. - } finally { - if (dirFd !== undefined) { - try { - closeSync(dirFd); - } catch { - // Best effort. - } - } - } - } -} diff --git a/packages/repl-engine/src/snapshot-envelope.ts b/packages/repl-engine/src/snapshot-envelope.ts deleted file mode 100644 index 570d2a56..00000000 --- a/packages/repl-engine/src/snapshot-envelope.ts +++ /dev/null @@ -1,404 +0,0 @@ -/** - * The at-rest identity envelope for quickjs-wasi snapshots — transfer - * lesson 5 from the roadmap doc (docs/roadmap/repl-orchestrator.md): - * "quickjs-wasi snapshots are raw WASM linear memory — valid only against - * the byte-identical `quickjs.wasm` build. A package upgrade plus a disk - * snapshot = a restore into garbage with no diagnosis, unless the snapshot - * file itself records which binary laid it out." - * - * The envelope wraps the shim's own `serializeSnapshot()` output (the - * versioned binary: QJSS magic + version + extension metadata + raw - * memory — quickjs-wasi's documented at-rest form) in: - * - * ```text - * \n - * - * header = { - * "format": "repl-snapshot", — the envelope format name - * "formatVersion": 2, — the envelope format version - * "wasmSha256": "<64 hex>", — sha256 of the wasm binary that - * laid out this memory - * "createdAtMs": - * } - * ``` - * - * The identity check lives at restore: the recorded `wasmSha256` is - * compared against the hash of the binary the host is about to restore - * with, and a mismatch REFUSES LOUDLY naming both hashes — never a silent - * restore into garbage. The envelope format version is the second refusal - * axis: a bump (an incompatible envelope layout, or a guest-library surface - * the host can no longer serve) refuses old snapshots naming both - * versions. gzip is the compression (the doc's choice: JS runtimes - * decompress it natively; measured at ~7.9x on real snapshots). - * - * Snapshot compatibility therefore holds across daemon restarts and - * machines running the same quickjs-wasi package version; a version bump - * makes old snapshots refuse loudly instead of corrupting. Portability - * with the Rust harness is explicitly not a goal — different binary, - * different layout — and the envelope makes that a clean rejection rather - * than a surprise. - */ - -import { createHash } from 'node:crypto'; -import { gunzipSync, gzipSync } from 'node:zlib'; - -import { QuickJS, type Snapshot } from 'quickjs-wasi'; - -import type { ReplSnapshot, WasmInput, WasmModule } from './types.js'; - -/** The envelope format name (the header's `format` field). */ -export const SNAPSHOT_FORMAT = 'repl-snapshot'; - -/** - * The envelope format version. Bumped when the envelope layout or the - * snapshot payload format changes incompatibly (a quickjs-wasi upgrade, - * a guest-library surface the host cannot serve): an envelope carrying a - * different version refuses loudly naming both versions instead of - * attempting a restore. - */ -export const SNAPSHOT_FORMAT_VERSION = 3; - -/** The header is one JSON line; refuse anything longer as not-our-file. */ -const MAX_HEADER_BYTES = 4096; - -/** The envelope header as recorded at rest. */ -export interface SnapshotEnvelopeMeta { - format: typeof SNAPSHOT_FORMAT; - formatVersion: number; - /** Hex sha256 of the wasm binary whose layout produced the snapshot. */ - wasmSha256: string; - createdAtMs: number; -} - -/** The parsed envelope: the decompressed snapshot plus its identity meta. */ -export interface SnapshotEnvelope { - snapshot: ReplSnapshot; - meta: SnapshotEnvelopeMeta; -} - -/** The envelope failure vocabulary (see `SnapshotEnvelopeError`). */ -export type SnapshotEnvelopeErrorCode = - | 'BAD_HEADER' - | 'FORMAT_MISMATCH' - | 'VERSION_MISMATCH' - | 'CORRUPT_PAYLOAD' - | 'WASM_HASH_MISMATCH' - | 'RESTORE_CORRUPT'; - -/** - * A loud envelope failure. `WASM_HASH_MISMATCH` is raised by the restore - * path (the store's `loadSnapshot`), which compares the recorded hash - * against the running binary; `RESTORE_CORRUPT` is raised by - * `Workspace.restore` when a payload that PASSED every decode check - * cannot be materialized (or initialized) — the corruption class that no - * at-rest check can see (a structurally valid envelope whose VM header - * or memory content is garbage, phase-D review rejection); the other - * codes are raised by `deserializeSnapshot` itself. Every message names - * the offending file path (when one was given) and the recorded vs - * expected values, so a restore that refuses can never be mistaken for a - * silent pass. - */ -export class SnapshotEnvelopeError extends Error { - readonly code: SnapshotEnvelopeErrorCode; - /** The snapshot file path, when the failure happened at a path. */ - readonly path: string | undefined; - /** The value recorded in the envelope (version, hash, format…). */ - readonly recorded: string | undefined; - /** The value the host expected. */ - readonly expected: string | undefined; - - constructor( - code: SnapshotEnvelopeErrorCode, - message: string, - details: { path?: string; recorded?: string; expected?: string } = {}, - ) { - super(message); - this.name = 'SnapshotEnvelopeError'; - this.code = code; - this.path = details.path; - this.recorded = details.recorded; - this.expected = details.expected; - } -} - -/** - * A restore-time corruption refusal (`code: 'RESTORE_CORRUPT'`, part of - * the `SnapshotEnvelopeError` family so one containment catch covers the - * whole load path — decode AND materialization): the envelope's own - * checks passed (format, version, wasm hash, gzip, the shim's binary - * parse, the shape/bounds check), yet restoring the VM from the payload - * failed — `RuntimeError: memory access out of bounds` on a corrupted - * in-range VM header (a context/runtime/stack pointer patched to a - * wrong-but-in-bounds value), a guest surface that cannot be rehosted, a - * provenance registry that cannot bootstrap. This is exactly the doc's - * "restore into garbage" class, caught one step later than the envelope - * can see it: the refusal must be CONTAINED (recorded as a stable - * refusal by the daemon, never crash-looped, never retried into - * garbage). `Workspace.restore` raises it after disposing any partially - * created VM; the original failure's message travels inside the refusal - * so the tool result names the problem. - */ -export class SnapshotRestoreError extends SnapshotEnvelopeError { - constructor(message: string, details: { cause?: unknown } = {}) { - super('RESTORE_CORRUPT', message); - this.name = 'SnapshotRestoreError'; - if (details.cause !== undefined) { - // ES2022 `Error` cause (target-compatible; the declaration stays - // self-contained — no explicit `cause` field is declared, the - // option is the standard constructor one). - (this as Error & { cause?: unknown }).cause = details.cause; - } - } -} - -/** - * Serialize a raw VM snapshot into the at-rest identity envelope: the - * shim's versioned binary serialization, gzip-compressed, with a JSON - * header line carrying the format name, the envelope format version, the - * wasm-binary sha256 and the creation time. The envelope is what the - * per-project store persists (`ReplWorkspaceStore.writeSnapshot`); the - * `wasmSha256` is the binary's identity the restore path compares - * against (`wasmSha256Of`). - * - * Synchronous (gzip of a ~1.5 MB memory image is a few ms), so a state- - * changing boundary can persist before the caller's promise resolves. - */ -export function serializeSnapshot( - snapshot: ReplSnapshot, - wasmSha256: string, - options: { createdAtMs?: number } = {}, -): Uint8Array { - // The hash is the envelope's identity: a malformed value would corrupt - // the restore comparison — refuse at write time. - if (typeof wasmSha256 !== 'string' || !/^[0-9a-f]{64}$/.test(wasmSha256)) { - throw new Error(`serializeSnapshot: wasmSha256 must be a 64-char lowercase hex string (got ${JSON.stringify(wasmSha256)})`); - } - const serialized = QuickJS.serializeSnapshot(snapshot as unknown as Snapshot); - const gz = gzipSync(serialized); - const header: SnapshotEnvelopeMeta = { - format: SNAPSHOT_FORMAT, - formatVersion: SNAPSHOT_FORMAT_VERSION, - wasmSha256, - createdAtMs: options.createdAtMs ?? Date.now(), - }; - const head = Buffer.from(`${JSON.stringify(header)}\n`, 'utf8'); - const envelope = new Uint8Array(head.length + gz.length); - envelope.set(head, 0); - envelope.set(gz, head.length); - return envelope; -} - -/** - * Deserialize an at-rest envelope: split the header line, validate the - * format name and the format version (REFUSING LOUDLY — naming both - * versions — when a bump invalidated the file), verify the recorded - * wasm-binary hash against the binary the host is about to restore with - * (REFUSING LOUDLY — naming both hashes — BEFORE any payload - * interpretation, so an incompatible old payload can never masquerade - * as a corrupt file), then gunzip the payload and run the shim's binary - * deserializer. Every failure is a `SnapshotEnvelopeError` with a - * specific code; a corrupted or truncated file is a loud single-shot - * error, never a silent pass and never a retry loop. - * - * The identity check is an option (`expectedWasmSha256`) rather than a - * separate post-hoc step so it runs between the header parse and the - * payload decode: a snapshot recorded by another binary — whose raw - * memory layout is garbage to this binary — must refuse as - * `WASM_HASH_MISMATCH` naming both hashes, never as `CORRUPT_PAYLOAD` - * (phase-D review regression: the payload used to be gunzipped and - * passed through `QuickJS.deserializeSnapshot()` before the running - * hash was compared, so an incompatible old payload failed as - * CORRUPT_PAYLOAD without naming the hashes). - */ -export function deserializeSnapshot( - bytes: Uint8Array, - options: { path?: string; expectedWasmSha256?: string } = {}, -): SnapshotEnvelope { - const path = options.path; - const nl = bytes.indexOf(0x0a); - if (nl < 0) { - throw new SnapshotEnvelopeError( - 'BAD_HEADER', - `${label(path)}no envelope header line — expected a newline-terminated JSON header followed by the gzip payload`, - { path }, - ); - } - if (nl > MAX_HEADER_BYTES) { - throw new SnapshotEnvelopeError('BAD_HEADER', `${label(path)}envelope header exceeds ${MAX_HEADER_BYTES} bytes`, { - path, - }); - } - let header: unknown; - try { - header = JSON.parse(Buffer.from(bytes.subarray(0, nl)).toString('utf8')); - } catch (error) { - throw new SnapshotEnvelopeError('BAD_HEADER', `${label(path)}unparseable envelope header (${(error as Error).message})`, { - path, - }); - } - const meta = validateHeader(header, path); - if (options.expectedWasmSha256 !== undefined) { - // The recorded hash versus the running binary, BEFORE the payload is - // touched: a mismatched restore refuses naming both hashes even when - // the old binary's payload would not even parse here (review - // regression: the comparison used to happen after deserialization). - if (meta.wasmSha256 !== options.expectedWasmSha256) { - throw new SnapshotEnvelopeError( - 'WASM_HASH_MISMATCH', - `${label(path)}snapshot was laid out by wasm binary sha256 ${meta.wasmSha256}, but the running ` + - `binary hashes to ${options.expectedWasmSha256} — refusing to restore into garbage (a quickjs-wasi ` + - `upgrade invalidated this snapshot; recreate the workspace)`, - { path, recorded: meta.wasmSha256, expected: options.expectedWasmSha256 }, - ); - } - } - const payload = bytes.subarray(nl + 1); - let raw: Uint8Array; - try { - raw = gunzipSync(payload); - } catch (error) { - throw new SnapshotEnvelopeError( - 'CORRUPT_PAYLOAD', - `${label(path)}gzip payload corrupt or truncated (${(error as Error).message}) — the snapshot cannot be restored`, - { path }, - ); - } - let snapshot: unknown; - try { - snapshot = QuickJS.deserializeSnapshot(raw); - } catch (error) { - throw new SnapshotEnvelopeError( - 'CORRUPT_PAYLOAD', - `${label(path)}serialized snapshot corrupt or truncated (${(error as Error).message}) — the snapshot cannot be restored`, - { path }, - ); - } - if (!isReplSnapshot(snapshot)) { - throw new SnapshotEnvelopeError( - 'CORRUPT_PAYLOAD', - `${label(path)}deserialized snapshot has an unrecognized shape — the snapshot cannot be restored`, - { path }, - ); - } - return { snapshot, meta }; -} - -/** - * The sha256 (hex) of a wasm binary — the identity the envelope records - * and the restore path compares. Raw bytes hash directly; a compiled - * module hashes through the engine's module registry (populated by - * `loadShippedWasm` — the only producer of `WasmModule` values). A - * module the engine did not load cannot be hashed (its bytes are not - * recoverable from the compiled form); pass raw bytes instead. - */ -export function wasmSha256Of(wasm: WasmInput): string { - if (wasm instanceof ArrayBuffer) { - return sha256Hex(new Uint8Array(wasm)); - } - if (ArrayBuffer.isView(wasm)) { - return sha256Hex(new Uint8Array(wasm.buffer, wasm.byteOffset, wasm.byteLength)); - } - const recorded = moduleHashes.get(wasm); - if (recorded === undefined) { - throw new Error( - 'wasmSha256Of: cannot hash a WasmModule that was not produced by loadShippedWasm — pass the raw wasm bytes instead', - ); - } - return recorded; -} - -/** - * @internal Record a compiled module's binary hash — called by - * `loadShippedWasm` when it compiles the shipped binary. Not part of the - * published API (not re-exported from the index); `wasmSha256Of` reads - * the registry. - */ -export function noteWasmModuleHash(module: WasmModule, sha256HexHash: string): void { - moduleHashes.set(module, sha256HexHash); -} - -/** Compiled-module → binary-hash registry (see `noteWasmModuleHash`). */ -const moduleHashes = new WeakMap(); - -function sha256Hex(bytes: Uint8Array): string { - return createHash('sha256').update(bytes).digest('hex'); -} - -/** Validate the parsed header object; refuse loudly on any mismatch. */ -function validateHeader(raw: unknown, path: string | undefined): SnapshotEnvelopeMeta { - if (typeof raw !== 'object' || raw === null) { - throw new SnapshotEnvelopeError('BAD_HEADER', `${label(path)}envelope header is not an object`, { path }); - } - const h = raw as Record; - if (h.format !== SNAPSHOT_FORMAT) { - throw new SnapshotEnvelopeError( - 'FORMAT_MISMATCH', - `${label(path)}snapshot carries envelope format ${JSON.stringify(h.format)} — this engine only serves ${JSON.stringify(SNAPSHOT_FORMAT)}; refusing to restore`, - { path, recorded: typeof h.format === 'string' ? h.format : String(h.format) }, - ); - } - if (h.formatVersion !== SNAPSHOT_FORMAT_VERSION) { - throw new SnapshotEnvelopeError( - 'VERSION_MISMATCH', - `${label(path)}snapshot carries format version ${String(h.formatVersion)}, but this engine supports version ${SNAPSHOT_FORMAT_VERSION} — a format upgrade invalidated it; refusing to restore into garbage (recreate the workspace)`, - { path, recorded: String(h.formatVersion), expected: String(SNAPSHOT_FORMAT_VERSION) }, - ); - } - if (typeof h.wasmSha256 !== 'string' || !/^[0-9a-f]{64}$/.test(h.wasmSha256)) { - throw new SnapshotEnvelopeError('BAD_HEADER', `${label(path)}envelope header carries an invalid wasmSha256`, { - path, - }); - } - if (typeof h.createdAtMs !== 'number' || !Number.isFinite(h.createdAtMs)) { - throw new SnapshotEnvelopeError('BAD_HEADER', `${label(path)}envelope header carries an invalid createdAtMs`, { - path, - }); - } - return { - format: SNAPSHOT_FORMAT, - formatVersion: h.formatVersion as number, - wasmSha256: h.wasmSha256, - createdAtMs: h.createdAtMs, - }; -} - -/** The shim's deserialized snapshot must satisfy the engine's shape — - * and its pointers must be STRUCTURALLY SANE (phase-D review rejection: - * the check used to be type-only, so a valid gzip/QJSS payload with a - * corrupted in-range-format VM header — `contextPtr` patched to - * `0xfffffff0`, say — deserialized cleanly and then crashed the restore - * with `RuntimeError: memory access out of bounds`, an uncoded failure - * the daemon could not contain). A quickjs runtime/context pointer is a - * malloc'd offset into the snapshot's own linear memory: never 0, never - * negative, never at/above the memory end — a pointer outside (0, - * memory.length) cannot be a legitimate restored VM. The stack pointer - * is the wasm stack's top, likewise strictly inside the memory. These - * bounds cannot prove a payload GOOD (in-memory corruption stays - * invisible to any at-rest check — the restore-time containment in - * `Workspace.restore` catches that class), but they make the cheap, - * obvious header-corruption class a clean `CORRUPT_PAYLOAD` refusal at - * decode, before any VM exists. */ -function isReplSnapshot(value: unknown): value is ReplSnapshot { - if (typeof value !== 'object' || value === null) return false; - const v = value as ReplSnapshot; - return ( - v.memory instanceof Uint8Array && - typeof v.stackPointer === 'number' && - typeof v.runtimePtr === 'number' && - typeof v.contextPtr === 'number' && - Array.isArray(v.extensions) && - sanePointer(v.stackPointer, v.memory.byteLength) && - sanePointer(v.runtimePtr, v.memory.byteLength) && - sanePointer(v.contextPtr, v.memory.byteLength) - ); -} - -/** A quickjs-wasi VM pointer: a non-negative integer strictly inside the - * snapshot memory (malloc'd offsets are never 0 — see `isReplSnapshot`). */ -function sanePointer(ptr: number, memoryLength: number): boolean { - return Number.isInteger(ptr) && ptr > 0 && ptr < memoryLength; -} - -function label(path: string | undefined): string { - return path === undefined ? '' : `snapshot ${path}: `; -} diff --git a/packages/repl-engine/src/steering-table.ts b/packages/repl-engine/src/steering-table.ts deleted file mode 100644 index 3ff314c1..00000000 --- a/packages/repl-engine/src/steering-table.ts +++ /dev/null @@ -1,71 +0,0 @@ -import { ACP_EXTENSION_SUPPORT_MATRIX, SESSION_STEERING_METHOD } from '@automatalabs/acp-agents'; - -/** One documentation-only distribution-probe row. Runtime routing never reads this table. */ -export interface SteeringMechanismRow { - backend: string; - advertised: boolean; - mechanism: 'strict active-turn injection' | 'unsupported'; - disposition: 'supported' | 'typed-unsupported' | 'not-advertised'; - distProbe?: 'claude' | 'codex'; -} - -/** Derive the checked-in inventory from executable distribution probes. */ -export function steeringMechanismRows(): SteeringMechanismRow[] { - const rows: SteeringMechanismRow[] = []; - for (const row of ACP_EXTENSION_SUPPORT_MATRIX) { - if (row.method !== SESSION_STEERING_METHOD) continue; - const advertised = row.disposition === 'supported'; - rows.push({ - backend: row.agent, - advertised, - mechanism: advertised ? 'strict active-turn injection' : 'unsupported', - disposition: row.disposition, - distProbe: row.distProbe, - }); - } - return rows; -} - -const MECHANISM_CASES = `Runtime steering availability is read from the session's raw initialize metadata only: -\`initializeMeta.steering.supported === true\`. The distribution matrix above is documentation, -not a runtime router. - -| Case | Wire behavior | Result | -|---|---|---| -| ACP prompt in flight; raw steering advertised | one strict \`_session/steering\` request with \`idleBehavior: "promptRequired"\` | \`injected\`, or \`idle\` for \`promptRequired\` | -| ACP prompt in flight; raw steering not advertised | no request | \`unsupported\` | -| no ACP prompt in flight, including opening/extraction/repair gaps | no request | \`idle\` | -| steering transport/server failure | no prompt fallback | rejects \`AGENT_EXECUTION_ERROR\` | -| malformed response or \`startedNewTurn\` | cancel + fatal session lane | rejects non-recoverably | - -Future work is always explicit: \`handle.queue(prompt)\` creates a distinct, durable FIFO public -turn and the broker sends it through ordinary \`session/prompt\` only when it reaches the queue -head. Queueing never uses \`_session/steering\` or a backend-native queue.`; - -/** Generate the complete deterministic documentation artifact. */ -export function generateSteeringMechanismTable(): string { - const rows = steeringMechanismRows(); - const lines: string[] = []; - lines.push('# Per-backend steering mechanism table'); - lines.push(''); - lines.push(''); - lines.push(''); - lines.push('The installed-distribution inventory:'); - lines.push(''); - lines.push('| Backend | `_session/steering` | Strict steering behavior |'); - lines.push('|---|---|---|'); - for (const row of rows) { - const probe = row.distProbe !== undefined ? ` (probed: ${row.distProbe})` : ''; - const behavior = row.advertised - ? 'strict active-turn injection via `session.steer()`' - : 'unsupported (no steering wire request)'; - lines.push(`| ${row.backend} | ${row.advertised ? 'advertised' : 'NOT advertised'}${probe} | ${behavior} |`); - } - lines.push('| custom backend | whatever its raw initialize metadata advertises | strict active-turn injection when advertised; unsupported otherwise |'); - lines.push(''); - lines.push(MECHANISM_CASES); - return `${lines.join('\n')}\n`; -} diff --git a/packages/repl-engine/src/store.ts b/packages/repl-engine/src/store.ts deleted file mode 100644 index eeff9bf4..00000000 --- a/packages/repl-engine/src/store.ts +++ /dev/null @@ -1,594 +0,0 @@ -/** - * The results-by-call-id call store — the broker's append-only settlement - * ledger (the roadmap doc's transfer lesson 1: "results recorded by call - * ID before being settled into the guest"). This is the ONLY host-side - * state that must survive independently of snapshots: on restore, the - * snapshot's guest registry is read back and each outstanding call is - * reconciled — a call the store shows as completed settles from here, - * exactly once. - * - * Two implementations, mirroring the harness reference broker's store - * (`agentprism-rust/crates/broker/src/store.rs`): - * - * - `InMemoryCallStore` — volatile (tests, ephemeral hosts). - * - `JsonlCallStore` — a durable append-only JSON-lines file. Every - * mutation is one appended line, written and fsynced synchronously - * (dispatch handlers run inside a VM eval and cannot await); reopening - * replays the log and repairs a crash-torn final line instead of - * refusing the file (the doc's kill-at-any-point posture: torn tails - * are the normal lifecycle, not a recovery path — the harness's ledger - * IDs R55/R81). - * - * First-wins everywhere, mirroring the guest registry's settlement - * idempotence: `recordDispatched` keeps the original record for a known - * id, `recordCompleted` keeps the FIRST completion and reports whether - * THIS call newly recorded one. That is what makes the broker's - * record→settle→consume delivery loop safely retryable: a crash between - * the store write and the guest settlement leaves both sides idempotent, - * so the next delivery attempt settles the call exactly once. - * - * One workspace instance per file: no locking, no compaction. Forks of - * one snapshot mint overlapping call ids, so a multi-workspace host gives - * each workspace its own store file (the daemon's per-project `repl/` - * directory, a later phase's wiring). - */ - -import { - closeSync, - existsSync, - fstatSync, - fsyncSync, - ftruncateSync, - openSync, - readFileSync, - writeSync, -} from 'node:fs'; - -/** Which guest call produced a record. */ -export type CallKind = 'agent' | 'checkpoint' | 'queue' | 'steer' | 'cancel'; - -/** How a completed call settled. */ -export type CallOutcomeKind = 'resolve' | 'reject'; - -/** - * The settlement of a completed call: the outcome plus the JSON-safe - * value it settled with (the resolution value, the `{ name, message, - * code?, recoverable? }` rejection object, the steering outcome string, - * or the parsed checkpoint answer). - */ -export interface CallOutcome { - outcome: CallOutcomeKind; - value: unknown; - completedAtMs: number; -} - -/** One call's full record. */ -export interface CallRecord { - /** The stable guest-facing call id (`"c1"`, `"c2"`, …). */ - callId: string; - kind: CallKind; - /** Verbatim prompt (agent/queue), question (checkpoint), or control label. */ - detail: string; - /** Verbatim `optionsJson` string, or null. */ - optionsJson: string | null; - /** - * The agent call's backend-routing spec, VERBATIM including the guest's - * reserved `"default"` sentinel (agent calls only; null otherwise) — - * the phase-D review round-2 fix: backend identity/pool routing is - * persisted with the dispatch, so a restore (or a lazy re-attach of a - * settled handle) never routes by the CURRENT configured default and - * misses a still-resumable original session. `null` on legacy records - * (pre-attachment logs). - */ - modelSpec: string | null; - /** - * The RESOLVED backend id the call's session opened under (agent calls - * whose session opened; recorded by `recordAttached` alongside the - * session id, overwritten by a re-issue's new session). The re-attach - * routing pin: `loadSession` routes by this id (a backend id doubles as - * a model routing spec) instead of re-resolving the model spec against - * the current default backend. `null` for checkpoint/steer records, - * agent records whose session never opened, and legacy logs. - */ - backendId: string | null; - /** Founding reusable agent/session call id for queue, steer, and cancel records. */ - foundingCallId: string | null; - /** Workspace admission time and total ordering sequence. */ - admittedAtMs: number; - admissionSequence: number; - dispatchedAtMs: number; - /** Times this call was re-issued after being found lost (same call id). */ - reissues: number; - /** Present once the call completed (first completion wins). */ - completion: CallOutcome | null; - /** Backend ACP session id for agent records after open/reattach. */ - sessionId: string | null; - /** Queue admission, ACP handoff, and explicit-cancellation timestamps. */ - queuedAtMs: number | null; - handoffAtMs: number | null; - cancelledAtMs: number | null; -} - -/** - * The store seam. All operations are synchronous: dispatch handlers run - * inside a VM eval (a host callback cannot await), and the settlement - * pump records before settling — so the write must be durable before the - * function returns. - */ -export interface CallStore { - /** Idempotent per call id: a known id keeps its original record. */ - recordDispatched(record: CallRecord): void; - /** Record that a lost call was re-issued under the same id. Throws for - * an id the store has never seen (a dangling re-issue would corrupt - * the replay ledger). */ - recordReissued(callId: string, atMs: number): void; - /** - * Record the backend ACP session id an agent call's session opened - * under — the restore path's re-attach key (`sessionId` on the record) — - * plus the RESOLVED backend id that session belongs to (`backendId`, the - * re-attach routing pin: a restore or lazy re-attach routes by it - * instead of re-resolving the model spec against the current default - * backend). OVERWRITES: the record carries the CURRENT session — a - * re-issued call's new session replaces the lost one (the log keeps the - * history as appended lines, and replay applies them in order). Throws - * for an id the store has never seen dispatched. - */ - recordAttached(callId: string, sessionId: string, atMs: number, backendId?: string | null): void; - /** - * Record a completion. Returns `true` iff the completion was newly - * recorded; `false` when the call already had one (first-wins, no - * change). Throws for an id the store has never seen dispatched. - */ - recordCompleted(callId: string, outcome: CallOutcome): boolean; - /** Record the queue prompt's point-of-no-return handoff marker. */ - recordHandoff(callId: string, atMs: number): void; - /** Record explicit cancellation of a public turn. */ - recordCancelled(callId: string, atMs: number): void; - /** Record queue admission into the durable per-session FIFO. */ - recordQueued(callId: string, atMs: number): void; - lookup(callId: string): CallRecord | undefined; - /** Every record, in first-dispatch order. */ - all(): CallRecord[]; -} - -function unknownCall(callId: string): Error { - return new Error(`call store: no record for call ${callId}`); -} - -// ──────────────────────────────────────────────────────────────────────── -// In-memory -// ──────────────────────────────────────────────────────────────────────── - -/** Volatile store: a Map plus dispatch order. */ -export class InMemoryCallStore implements CallStore { - private readonly records = new Map(); - private readonly order: string[] = []; - - recordDispatched(record: CallRecord): void { - if (this.records.has(record.callId)) return; // idempotent — keep the original - this.order.push(record.callId); - // Normalize legacy records (pre-delivery-marker logs) onto the current shape. - this.records.set(record.callId, { - ...record, - sessionId: record.sessionId ?? null, - foundingCallId: record.foundingCallId ?? null, - admittedAtMs: record.admittedAtMs ?? record.dispatchedAtMs, - admissionSequence: record.admissionSequence ?? 0, - modelSpec: record.modelSpec ?? null, - backendId: record.backendId ?? null, - queuedAtMs: record.queuedAtMs ?? null, - handoffAtMs: record.handoffAtMs ?? null, - cancelledAtMs: record.cancelledAtMs ?? null, - }); - } - - recordReissued(callId: string, atMs: number): void { - const record = this.records.get(callId); - if (record === undefined) throw unknownCall(callId); - record.reissues += 1; - record.dispatchedAtMs = atMs; - } - - recordAttached(callId: string, sessionId: string, atMs: number, backendId?: string | null): void { - const record = this.records.get(callId); - if (record === undefined) throw unknownCall(callId); - record.sessionId = sessionId; - record.backendId = backendId ?? null; - void atMs; - } - - recordCompleted(callId: string, outcome: CallOutcome): boolean { - const record = this.records.get(callId); - if (record === undefined) throw unknownCall(callId); - if (record.completion !== null) return false; // first completion wins - record.completion = outcome; - return true; - } - - recordHandoff(callId: string, atMs: number): void { - const record = this.records.get(callId); - if (record === undefined) throw unknownCall(callId); - if (record.handoffAtMs !== null) return; - record.handoffAtMs = atMs; - } - - recordCancelled(callId: string, atMs: number): void { - const record = this.records.get(callId); - if (record === undefined) throw unknownCall(callId); - if (record.cancelledAtMs !== null) return; - record.cancelledAtMs = atMs; - } - - recordQueued(callId: string, atMs: number): void { - const record = this.records.get(callId); - if (record === undefined) throw unknownCall(callId); - if (record.queuedAtMs !== null) return; // first-wins - record.queuedAtMs = atMs; - } - - lookup(callId: string): CallRecord | undefined { - return this.records.get(callId); - } - - all(): CallRecord[] { - return this.order.map((id) => this.records.get(id)!).filter((r) => r !== undefined); - } -} - -// ──────────────────────────────────────────────────────────────────────── -// Durable JSON-lines store -// ──────────────────────────────────────────────────────────────────────── - -/** One appended line in the JSONL log. */ -type LogLine = - | { event: 'dispatched'; record: CallRecord } - | { event: 'reissued'; callId: string; atMs: number } - | { event: 'attached'; callId: string; sessionId: string; atMs: number; backendId?: string | null } - | { event: 'completed'; callId: string; outcome: CallOutcome } - | { event: 'handoff'; callId: string; atMs: number } - | { event: 'cancelled'; callId: string; atMs: number } - | { event: 'queued'; callId: string; atMs: number }; - -function isLogLine(value: unknown): value is LogLine { - if (typeof value !== 'object' || value === null) return false; - const v = value as { event?: unknown }; - if (v.event === 'dispatched') { - const r = (value as { record?: unknown }).record; - return ( - typeof r === 'object' && - r !== null && - typeof (r as CallRecord).callId === 'string' && - typeof (r as CallRecord).detail === 'string' && - ((r as CallRecord).optionsJson === null || typeof (r as CallRecord).optionsJson === 'string') && - typeof (r as CallRecord).dispatchedAtMs === 'number' && - typeof (r as CallRecord).reissues === 'number' && - ((r as CallRecord).completion === null || typeof (r as CallRecord).completion === 'object') - ); - } - if (v.event === 'reissued') { - return ( - typeof (value as { callId?: unknown }).callId === 'string' && - typeof (value as { atMs?: unknown }).atMs === 'number' - ); - } - if (v.event === 'attached') { - const a = value as { callId?: unknown; sessionId?: unknown; atMs?: unknown; backendId?: unknown }; - return ( - typeof a.callId === 'string' && - typeof a.sessionId === 'string' && - typeof a.atMs === 'number' && - (a.backendId === undefined || a.backendId === null || typeof a.backendId === 'string') - ); - } - if (v.event === 'completed') { - const o = (value as { outcome?: unknown }).outcome; - return ( - typeof o === 'object' && - o !== null && - ((o as CallOutcome).outcome === 'resolve' || (o as CallOutcome).outcome === 'reject') && - typeof (o as CallOutcome).completedAtMs === 'number' - ); - } - if (v.event === 'handoff' || v.event === 'cancelled') { - const d = value as { callId?: unknown; atMs?: unknown }; - return typeof d.callId === 'string' && typeof d.atMs === 'number'; - } - if (v.event === 'queued') { - const q = value as { callId?: unknown; atMs?: unknown }; - return typeof q.callId === 'string' && typeof q.atMs === 'number'; - } - return false; -} - -/** - * Durable store: an append-only JSON-lines file. Every mutation is one - * line, written and fsynced on write; opening replays the log into an - * in-memory index — repairing a crash-torn final line first (below). - * - * ## Torn tails are repaired; mid-log corruption is refused - * - * Every append is `\n` written in one call, so a process killed - * mid-append leaves exactly one artifact shape: a FINAL line with no - * terminating newline. That artifact must not make the session - * unopenable (kill-at-any-point is the normal lifecycle), so: - * - * - **Unterminated + unparseable** — only a crash mid-append produces - * this shape; the fragment's bytes are durably preserved in a - * `.torn-` sidecar next to the log, then the file is - * truncated to the last `\n` boundary. Truncating restores the file to - * "all complete records", so the NEXT append can never fuse onto the - * fragment and compound the damage. - * - **Unterminated + parseable** — the crash landed between a record's - * bytes and its newline; the record is complete, so it is KEPT and the - * missing terminator is written back. Truncating here would vaporize a - * real record (e.g. a completion whose result was already paid for). - * - **Terminated but unparseable, anywhere** — a line that HAS its - * newline was fully written, which is past the append discipline's only - * crash window; garbage there means external damage, and silently - * skipping records from the middle of the audit log would corrupt - * everything replayed after it. That stays a hard error. - * - * The repair runs in BYTE space before any UTF-8 decoding: a torn tail - * can split a multi-byte character (`detail` is verbatim prompt text), - * and decoding the whole file first would fail on the invalid tail and - * take every intact record with it. The sidecar is written and synced - * BEFORE the truncation, so a failure to preserve leaves the log - * untouched for the next attempt. - * - * ## Appends heal to the acknowledged prefix - * - * The store tracks `cleanLen`: the byte offset of the durably - * ACKNOWLEDGED prefix — every byte at or below it belongs to a record - * whose append fully succeeded. A failed `writeSync` can leave a partial - * line behind, and a retried append after it would fuse into one - * newline-terminated but unparseable line — turning a transient IO error - * into what the next open must treat as permanent corruption. Every - * append therefore starts from the acknowledged prefix: leftover partial - * bytes are truncated away first, then the line is written. - */ -export class JsonlCallStore implements CallStore { - private readonly filePath: string; - private fd: number; - /** Byte length of the durably acknowledged prefix (see module docs). */ - private cleanLen: number; - private readonly index = new InMemoryCallStore(); - - private constructor(path: string, fd: number, cleanLen: number) { - this.filePath = path; - this.fd = fd; - this.cleanLen = cleanLen; - } - - /** Open (or create) the log at `path` and replay it. */ - static open(path: string): JsonlCallStore { - let raw: Buffer = Buffer.alloc(0); - if (existsSync(path)) { - raw = readFileSync(path); - } - // The complete region: everything up to and including the last - // newline (the whole file when empty or newline-terminated). - const boundary = raw.length === 0 || raw[raw.length - 1] === 0x0a ? raw.length : raw.lastIndexOf(0x0a) + 1; - const index = new InMemoryCallStore(); - // Complete lines were written by JSON.stringify, so the region is - // valid UTF-8 by construction; anything else there is genuine - // corruption, refused loudly like any other. - const complete = raw.subarray(0, boundary).toString('utf8'); - let lineNo = 0; - for (const line of complete.split('\n')) { - lineNo++; - if (line.trim() === '') continue; - let parsed: unknown; - try { - parsed = JSON.parse(line); - } catch (error) { - throw new Error(`call store ${path}:${lineNo}: corrupt log line (${(error as Error).message})`); - } - if (!isLogLine(parsed)) { - throw new Error(`call store ${path}:${lineNo}: unrecognized log line`); - } - replay(index, parsed); - } - const tail = raw.subarray(boundary); - let terminateTail = false; - if (tail.length > 0) { - let parsedTail: LogLine | undefined; - try { - const candidate: unknown = JSON.parse(tail.toString('utf8')); - if (isLogLine(candidate)) parsedTail = candidate; - } catch { - parsedTail = undefined; - } - if (parsedTail !== undefined) { - // A complete record missing only its `\n`: keep it, and terminate - // it below so the next append starts its own line. - replay(index, parsedTail); - terminateTail = true; - } else { - // A crash artifact: preserve the fragment first (durably — the - // truncation below destroys the only other copy), then truncate - // to the last complete-record boundary. - preserveTornFragment(path, tail); - const fd = openSync(path, 'r+'); - try { - ftruncateSync(fd, boundary); - fsyncSync(fd); - } finally { - closeSync(fd); - } - } - } - const fd = openSync(path, 'a'); - if (terminateTail) { - writeSync(fd, '\n'); - fsyncSync(fd); - } - // Everything on disk right now is complete, replayed records: the - // acknowledged prefix is the whole file. - const cleanLen = getSize(fd); - const store = new JsonlCallStore(path, fd, cleanLen); - // Transfer the replayed index (records are first-wins copies — the - // replayed completions travel inside the dispatched records). - for (const record of index.all()) { - store.index.recordDispatched(record); - } - return store; - } - - /** The log file's path. */ - path(): string { - return this.filePath; - } - - recordDispatched(record: CallRecord): void { - if (this.index.lookup(record.callId) !== undefined) return; // idempotent - this.append({ event: 'dispatched', record }); - this.index.recordDispatched(record); - } - - recordReissued(callId: string, atMs: number): void { - // Validate against the index first so the log never carries a - // dangling re-issue. - if (this.index.lookup(callId) === undefined) throw unknownCall(callId); - this.append({ event: 'reissued', callId, atMs }); - this.index.recordReissued(callId, atMs); - } - - recordAttached(callId: string, sessionId: string, atMs: number, backendId?: string | null): void { - if (this.index.lookup(callId) === undefined) throw unknownCall(callId); - this.append({ event: 'attached', callId, sessionId, atMs, backendId: backendId ?? null }); - this.index.recordAttached(callId, sessionId, atMs, backendId); - } - - recordCompleted(callId: string, outcome: CallOutcome): boolean { - const existing = this.index.lookup(callId); - if (existing === undefined) throw unknownCall(callId); - if (existing.completion !== null) return false; // first completion wins - this.append({ event: 'completed', callId, outcome }); - return this.index.recordCompleted(callId, outcome); - } - - recordHandoff(callId: string, atMs: number): void { - const existing = this.index.lookup(callId); - if (existing === undefined) throw unknownCall(callId); - if (existing.handoffAtMs !== null) return; - this.append({ event: 'handoff', callId, atMs }); - this.index.recordHandoff(callId, atMs); - } - - recordCancelled(callId: string, atMs: number): void { - const existing = this.index.lookup(callId); - if (existing === undefined) throw unknownCall(callId); - if (existing.cancelledAtMs !== null) return; - this.append({ event: 'cancelled', callId, atMs }); - this.index.recordCancelled(callId, atMs); - } - - recordQueued(callId: string, atMs: number): void { - const existing = this.index.lookup(callId); - if (existing === undefined) throw unknownCall(callId); - if (existing.queuedAtMs !== null) return; // first-wins - this.append({ event: 'queued', callId, atMs }); - this.index.recordQueued(callId, atMs); - } - - lookup(callId: string): CallRecord | undefined { - return this.index.lookup(callId); - } - - all(): CallRecord[] { - return this.index.all(); - } - - /** Close the log file. Idempotent; the in-memory index stays readable. */ - close(): void { - if (this.fd === -1) return; - closeSync(this.fd); - this.fd = -1; - } - - /** True once `close()` ran (the log file is closed; any later write - * throws). The teardown-completeness probe (phase-D review round 8: - * a rejected disposal used to skip the store close — the daemon - * shutdown regression asserts the store really closed). */ - isClosed(): boolean { - return this.fd === -1; - } - - private append(line: LogLine): void { - // Retry safety for the delivery loop (which retries a failed - // completion write with the SAME outcome): a failed write can leave - // a PARTIAL line behind, and a retry appended after it would fuse - // both into one newline-terminated but unparseable line. Every - // append therefore starts from the acknowledged prefix: heal any - // leftover partial bytes first, then write; on failure, roll back - // (best effort — the pre-write heal covers a failed rollback too). - if (getSize(this.fd) !== this.cleanLen) { - ftruncateSync(this.fd, this.cleanLen); - } - const buf = Buffer.from(`${JSON.stringify(line)}\n`, 'utf8'); - let written = 0; - try { - while (written < buf.length) { - written += writeSync(this.fd, buf, written, buf.length - written); - } - fsyncSync(this.fd); - } catch (error) { - try { - ftruncateSync(this.fd, this.cleanLen); - } catch { - // Best effort — the next append's pre-write heal covers it. - } - throw error; - } - this.cleanLen += buf.length; - } -} - -/** Apply one replayed log line to the in-memory index. */ -function replay(index: InMemoryCallStore, line: LogLine): void { - if (line.event === 'dispatched') index.recordDispatched(line.record); - else if (line.event === 'reissued') index.recordReissued(line.callId, line.atMs); - else if (line.event === 'attached') index.recordAttached(line.callId, line.sessionId, line.atMs, line.backendId ?? null); - else if (line.event === 'completed') index.recordCompleted(line.callId, line.outcome); - else if (line.event === 'queued') index.recordQueued(line.callId, line.atMs); - else if (line.event === 'handoff') index.recordHandoff(line.callId, line.atMs); - else index.recordCancelled(line.callId, line.atMs); -} - -/** Current file size of an open fd. */ -function getSize(fd: number): number { - return fstatSync(fd).size; -} - -/** - * Preserve a torn tail's bytes in a sidecar next to the log — - * `.torn-` — written and synced BEFORE the log is - * truncated, so a failure here aborts the open with the log untouched: - * the fragment is never vaporized. A sidecar FILE rather than a log - * line, deliberately: the fragment is raw bytes and can end - * mid-UTF-8-character, which a text log cannot carry faithfully. - */ -function preserveTornFragment(path: string, fragment: Buffer): string { - const nowMs = Date.now(); - let attempt = 0; - for (;;) { - const suffix = attempt === 0 ? `${nowMs}` : `${nowMs}-${attempt}`; - const candidate = `${path}.torn-${suffix}`; - try { - const fd = openSync(candidate, 'wx'); - try { - writeSync(fd, fragment); - fsyncSync(fd); - } finally { - closeSync(fd); - } - return candidate; - } catch (error) { - if ((error as NodeJS.ErrnoException).code === 'EEXIST') { - attempt += 1; - continue; - } - throw error; - } - } -} diff --git a/packages/repl-engine/src/trapfree.ts b/packages/repl-engine/src/trapfree.ts deleted file mode 100644 index a03f9b8e..00000000 --- a/packages/repl-engine/src/trapfree.ts +++ /dev/null @@ -1,597 +0,0 @@ -/** - * Trap-free introspection primitives — the engine's read surface for guest - * state. - * - * Every function here drives the raw `qjs_*` exports of the quickjs-wasi - * shim and **never executes guest code**: own-property-DESCRIPTOR reads - * only (accessors are never invoked), engine-level brand checks (never - * `instanceof`, never prototype inspection, never `Symbol.toStringTag`), - * proxies guarded before any descriptor/key/prototype read (a proxy fires - * traps on all of them), and raw native conversions only on values already - * brand-checked as the matching primitive. - * - * This is the module the roadmap doc's transfer lesson R69 is enforced in: - * a guest `Object.prototype.value` pollution, a getter installed on - * `SyntaxError.prototype.name`, or a proxy whose every trap counts - * executions must not be able to influence anything the host reads from - * the realm. Failure modes degrade to `undefined`/`[]`/markers rather than - * throwing through quickjs-wasi's `JSException` constructor (which - * performs guest-visible `[[Get]]` reads of `name`/`message`/`stack` on - * the exception value before any host `catch` can intercept). - * - * Handle ownership: every function that returns a `JSValueHandle` hands - * ownership to the caller, who must dispose it. Every accessor `get`/`set` - * handle encountered on a descriptor is disposed here — a leaked accessor - * handle pins guest memory (review measured a 1 MiB VM exhausting after - * ~3,128 accessor-valued completions). - */ - -import { JSValueHandle, type QuickJS } from 'quickjs-wasi'; - -/** The raw WASM exports the engine drives (the type is not exported by the shim). */ -export type QuickJSExports = ReturnType; - -/** - * Trap-free own-data-property read, driven over the raw - * `qjs_get_own_property_descriptor` export — **never** through - * `JSValueHandle.getOwnPropertyDescriptor()`, whose failure path throws a - * `JSException` whose constructor performs guest-visible `[[Get]]` reads - * of `name`/`message`/`stack` on the exception value (review regression: - * a getter installed on `InternalError.prototype.name` would execute while - * a failing descriptor read was being reported). Here a failed read's - * exception value is taken out of the runtime and freed — no `JSException` - * is ever constructed — and the engine-created descriptor object's own - * data properties are read via raw `qjs_get_prop_value` (OrdinaryGet on - * own data properties: no guest code runs, even against a polluted - * `Object.prototype`). - * - * Returns `undefined` when the property is absent, the read failed, the - * property is an accessor (accessors are never invoked — their `get`/`set` - * handles are owned by the caller and are disposed here; a leaked accessor - * handle pins guest memory, which review measured exhausting a 1 MiB VM - * after ~3,128 accessor-valued completions), or the object is a proxy (a - * descriptor read would fire its `getOwnPropertyDescriptor` trap). The - * returned handle is owned by the caller and must be disposed. - */ -export function readOwnDataProperty(handle: JSValueHandle, key: string): JSValueHandle | undefined { - // Proxies fire traps on descriptor reads — this is the backstop guard; - // call sites guard too. Engine-level brand check, never a guest trap. - if (handle.isProxy) return undefined; - - const vm = handle.vm; - const e = vm._getExports(); - // `qjs_get_own_property_descriptor` takes the key as a JSValue (like the - // shim's own descriptor path, which passes `vm.newString(key)`), not as - // a C string — passing a `_writeString` pointer makes the C engine read - // raw bytes as a JSValue and report “no such property”. The key handle - // is engine-created and freed right after the call. - const keyHandle = vm.newString(key); - let descPtr: number; - try { - descPtr = e.qjs_get_own_property_descriptor(handle.ptr, keyHandle.ptr); - } finally { - keyHandle.dispose(); - } - if (descPtr === 0) return undefined; // no such own property - - const desc = new JSValueHandle(vm, descPtr); - try { - if (e.qjs_is_exception(desc.ptr) !== 0) { - // The C descriptor read failed (allocation failure edge). Take the - // exception value out of the runtime and free it; never construct - // quickjs-wasi's `JSException` (guest-visible getters would run in - // its constructor). The property reads as absent. - takeAndFreeException(e, vm); - return undefined; - } - // Data vs accessor: `qjs_has_own_property` (raw), which never walks - // the prototype — an accessor descriptor must not leak a polluted - // `Object.prototype.value` through the `value` read. - if (hasOwnRaw(e, vm, desc.ptr, 'value')) { - return getPropRaw(e, vm, desc.ptr, 'value'); - } - // Accessor descriptor: never invoke the accessors; free their owned - // handles so they don't pin guest memory. - getPropRaw(e, vm, desc.ptr, 'get')?.dispose(); - getPropRaw(e, vm, desc.ptr, 'set')?.dispose(); - return undefined; - } finally { - desc.dispose(); - } -} - -/** - * The full trap-free own-property-descriptor read the previewer needs: the - * value AND the `enumerable` flag, with accessors never invoked (their - * handles are freed here). Failure modes (exception, accessor, proxy) - * degrade to `undefined` exactly like `readOwnDataProperty`. - */ -export type OwnDescriptor = - | { kind: 'data'; value: JSValueHandle; enumerable: boolean } - | { kind: 'accessor'; enumerable: boolean }; - -export function readOwnDescriptor(handle: JSValueHandle, key: string): OwnDescriptor | undefined { - // Proxies fire traps on descriptor reads — never touch one. - if (handle.isProxy) return undefined; - - const vm = handle.vm; - const e = vm._getExports(); - const keyHandle = vm.newString(key); - let descPtr: number; - try { - descPtr = e.qjs_get_own_property_descriptor(handle.ptr, keyHandle.ptr); - } finally { - keyHandle.dispose(); - } - if (descPtr === 0) return undefined; - - const desc = new JSValueHandle(vm, descPtr); - try { - if (e.qjs_is_exception(desc.ptr) !== 0) { - takeAndFreeException(e, vm); - return undefined; - } - const enumerable = readFlag(e, vm, desc.ptr, 'enumerable'); - if (hasOwnRaw(e, vm, desc.ptr, 'value')) { - const value = getPropRaw(e, vm, desc.ptr, 'value'); - if (value === undefined) return undefined; - return { kind: 'data', value, enumerable }; - } - // Accessor: never invoke; free the owned get/set handles. - getPropRaw(e, vm, desc.ptr, 'get')?.dispose(); - getPropRaw(e, vm, desc.ptr, 'set')?.dispose(); - return { kind: 'accessor', enumerable }; - } finally { - desc.dispose(); - } -} - -/** - * Raw `hasOwnProperty` on an engine-created object (no prototype walk, no - * traps). - */ -export function hasOwnRaw(e: QuickJSExports, vm: QuickJS, objPtr: number, key: string): boolean { - const { ptr: keyPtr } = vm._writeString(key); - try { - return e.qjs_has_own_property(objPtr, keyPtr) !== 0; - } finally { - e.wasm_free(keyPtr); - } -} - -/** - * Raw own-property read on an engine-created object. `qjs_get_prop_value` - * is OrdinaryGet: on an engine-created plain object with own data - * properties no guest code can run — but the read itself can still fail - * (allocation edge), so the exception is taken out and freed and the read - * reports `undefined` instead of throwing. The returned handle is owned by - * the caller and must be disposed. - */ -export function getPropRaw( - e: QuickJSExports, - vm: QuickJS, - objPtr: number, - key: string, -): JSValueHandle | undefined { - // `qjs_get_prop_value` takes the key as a JSValue (the shim's `getProp` - // passes a handle), never as a C string. - const keyHandle = vm.newString(key); - let ptr: number; - try { - ptr = e.qjs_get_prop_value(objPtr, keyHandle.ptr); - } finally { - keyHandle.dispose(); - } - const handle = new JSValueHandle(vm, ptr); - if (e.qjs_is_exception(handle.ptr) !== 0) { - takeAndFreeException(e, vm); - handle.dispose(); - return undefined; - } - return handle; -} - -/** - * Take the runtime's current exception value out and free it. Used by the - * raw failure paths so a failed C call never leaves a sticky exception - * behind (the shim's own `keys()` leaves one, which would poison later - * operations) and never constructs a `JSException`. - */ -export function takeAndFreeException(e: QuickJSExports, vm: QuickJS): void { - // `qjs_get_exception` clears the runtime's slot; with no exception set it - // returns JS_UNDEFINED (pointer 0), so a non-zero pointer is a real - // exception value owned by us. - const excPtr = e.qjs_get_exception(); - if (excPtr !== 0) { - new JSValueHandle(vm, excPtr).dispose(); - } -} - -/** - * Trap-free own enumerable string keys (`Object.keys` semantics), driven - * over the raw `qjs_get_own_property_names` export with exception cleanup — - * the shim's `keys()` leaves a sticky runtime exception behind when the C - * call fails, which would poison later operations. Returns `[]` on failure. - */ -export function rawOwnKeys(handle: JSValueHandle): string[] { - const vm = handle.vm; - const e = vm._getExports(); - // ALL own string keys (enumerable or not) — the semantic equivalent of - // the guest's `Object.getOwnPropertyNames`: the provenance registry's - // maintenance pass and the manifest's user-binding diff must agree on - // the global key set, and the realm builtins (non-enumerable on - // globalThis) belong in it. The shim's plain names export is - // ENUM_ONLY (review probe: 16 keys on a fresh realm vs 70 for the - // guest's getOwnPropertyNames) — never use it for scope enumeration. - const keysPtr = e.qjs_get_own_property_names_all(handle.ptr); - const keysHandle = new JSValueHandle(vm, keysPtr); - if (e.qjs_is_exception(keysHandle.ptr) !== 0) { - takeAndFreeException(e, vm); - keysHandle.dispose(); - return []; - } - try { - const lenHandle = getPropRaw(e, vm, keysHandle.ptr, 'length'); - if (lenHandle === undefined) return []; - let len: number; - try { - len = lenHandle.toNumber(); - } finally { - lenHandle.dispose(); - } - const out: string[] = []; - for (let i = 0; i < len; i++) { - const keyPtr = e.qjs_get_prop_uint32(keysHandle.ptr, i); - const keyHandle = new JSValueHandle(vm, keyPtr); - if (e.qjs_is_exception(keyHandle.ptr) !== 0) { - takeAndFreeException(e, vm); - keyHandle.dispose(); - break; - } - try { - out.push(keyHandle.toString()); - } finally { - keyHandle.dispose(); - } - } - return out; - } finally { - keysHandle.dispose(); - } -} - -/** - * Trap-free `Reflect.ownKeys` listing (strings AND symbols, enumerable or - * not), with the materialized key array read back through own-property - * descriptors per the preview format's enumeration-fill rule (FORMAT.md - * §6): a hole in the materialized array is a binary contract violation and - * reports `corrupted: true` instead of a fabricated key list. - * - * The returned `keys` entries are in `Reflect.ownKeys` order: canonical - * array indices first, then string keys by insertion order, then symbols - * (symbols carry no name — the previewer only counts them for the overflow - * flag). `corrupted` callers must degrade to "list nothing, flag overflow". - */ -export interface OwnKey { - /** The property name for string keys; `undefined` for symbol keys. */ - name: string | undefined; - symbol: boolean; -} - -export function rawOwnKeysAll(handle: JSValueHandle): { keys: OwnKey[]; corrupted: boolean } { - const vm = handle.vm; - const e = vm._getExports(); - // Proxies fire the ownKeys trap — never reached here (callers guard with - // isProxy first), but the backstop guard costs nothing. - if (handle.isProxy) return { keys: [], corrupted: true }; - const keysPtr = e.qjs_get_own_property_keys(handle.ptr); - const keysHandle = new JSValueHandle(vm, keysPtr); - if (e.qjs_is_exception(keysHandle.ptr) !== 0) { - takeAndFreeException(e, vm); - keysHandle.dispose(); - return { keys: [], corrupted: true }; - } - try { - const lenHandle = getPropRaw(e, vm, keysHandle.ptr, 'length'); - if (lenHandle === undefined) return { keys: [], corrupted: true }; - let len: number; - try { - len = lenHandle.toNumber(); - } finally { - lenHandle.dispose(); - } - const keys: OwnKey[] = []; - for (let i = 0; i < len; i++) { - // Descriptor read of the materialized element — never a plain [[Get]] - // (FORMAT.md §6: under a broken fill a [[Get]] would fire a polluted - // prototype accessor and fabricate keys). A hole is a binary - // contract violation: degrade honestly. - const element = readOwnDescriptor(keysHandle, String(i)); - if (element === undefined || element.kind !== 'data') { - return { keys: [], corrupted: true }; - } - try { - if (element.value.isSymbol) { - keys.push({ name: undefined, symbol: true }); - } else if (element.value.isString) { - keys.push({ name: element.value.toString(), symbol: false }); - } else { - // The materialized array holds only strings and symbols. - return { keys: [], corrupted: true }; - } - } finally { - element.value.dispose(); - } - } - return { keys, corrupted: false }; - } finally { - keysHandle.dispose(); - } -} - -/** - * Brand-checked typed-array info via the raw `qjs_get_typed_array_buffer` - * export: byte length and bytes-per-element of the view. Returns - * `undefined` for values that are not typed arrays (including DataView — - * callers check `isDataView` first) and for failed reads (exception taken - * out and freed). `length` is the element count (`byteLength / bpe`). - */ -export interface TypedArrayInfo { - byteLength: number; - bytesPerElement: number; - length: number; -} - -export function typedArrayInfo(handle: JSValueHandle): TypedArrayInfo | undefined { - const vm = handle.vm; - const e = vm._getExports(); - const outPtr = e.wasm_malloc(12); - try { - // The export returns the view's backing ArrayBuffer as an OWNED JSValue - // (heap box) — or the exception sentinel box for non-views. Both are - // owned by us and both are disposed here: a leaked buffer handle pins - // the entire backing store of the view, so repeated previews without - // disposal accumulate WASM/QuickJS allocations (review measured the - // same class of leak exhausting small VMs). The exception path also - // takes the runtime exception the C read set, out and frees it. - const bufferPtr = e.qjs_get_typed_array_buffer(handle.ptr, outPtr, outPtr + 4, outPtr + 8); - if (bufferPtr === 0) return undefined; // allocation edge — nothing owned - const buffer = new JSValueHandle(vm, bufferPtr); - try { - if (e.qjs_is_exception(buffer.ptr) !== 0) { - // Not a typed-array view: free the sentinel box and clear the - // pending engine exception it set (mirrors the Rust reference - // implementation's typed_array_info). - takeAndFreeException(e, vm); - return undefined; - } - const view = new DataView(e.memory.buffer); - const byteLength = view.getUint32(outPtr + 4, true); - const bytesPerElement = view.getUint32(outPtr + 8, true); - if (bytesPerElement === 0) return undefined; - return { - byteLength, - bytesPerElement, - length: byteLength / bytesPerElement, - }; - } finally { - buffer.dispose(); - } - } finally { - e.wasm_free(outPtr); - } -} - -/** - * ArrayBuffer byte length via the raw `qjs_get_array_buffer` export - * (returns the data pointer and writes the byte length to the out slot). - * `undefined` for non-ArrayBuffers and failed reads. - */ -export function arrayBufferByteLength(handle: JSValueHandle): number | undefined { - const vm = handle.vm; - const e = vm._getExports(); - // Callers brand-check `isArrayBuffer` first (FORMAT.md §1: engine brand - // checks only). The export returns a RAW byte pointer into WASM linear - // memory — NOT a JSValue — so it must never be passed to - // `qjs_is_exception`: a guest-controlled buffer could begin with the - // exception tag and be misread as a failed read (review: a 16-byte - // buffer then rendered as `ArrayBuffer(0)`). A NULL return means not an - // ArrayBuffer, a detached buffer, or an allocation edge; the runtime - // exception slot is cleared defensively (the C read may set one — the - // Rust reference implementation does the same). - const outPtr = e.wasm_malloc(4); - try { - const dataPtr = e.qjs_get_array_buffer(handle.ptr, outPtr); - if (dataPtr === 0) { - takeAndFreeException(e, vm); - return undefined; - } - return new DataView(e.memory.buffer).getUint32(outPtr, true); - } finally { - e.wasm_free(outPtr); - } -} - -/** - * The `[[ProxyTarget]]` of a proxy, trap-free (read through the raw export, - * no traps). Returns `undefined` for revoked proxies and failed reads — - * never constructs quickjs-wasi's `JSException`. The returned handle is - * owned by the caller and must be disposed. - */ -export function readProxyTarget(handle: JSValueHandle): JSValueHandle | undefined { - const vm = handle.vm; - const e = vm._getExports(); - // The export returns an OWNED heap box: the target for a live proxy, or - // the exception sentinel for a revoked one (the C read also sets a - // pending runtime exception). Every non-success path disposes the box - // AND takes the runtime exception out — a leaked exception box - // accumulates on every revoked-proxy preview (review: repeated previews - // grew WASM/QuickJS allocations). - const targetPtr = e.qjs_get_proxy_target(handle.ptr); - if (targetPtr === 0) return undefined; // allocation edge — nothing owned - const target = new JSValueHandle(vm, targetPtr); - if (e.qjs_is_exception(target.ptr) !== 0) { - target.dispose(); - takeAndFreeException(e, vm); - return undefined; // revoked proxy - } - if (target.isUndefined || target.isNull) { - target.dispose(); - return undefined; // revoked - } - return target; // owned by the caller — must be disposed -} - -/** - * Render a guest value into host data without ever executing guest code: - * primitives via native conversions, objects via own enumerable - * data-property descriptor reads, brand checks for the engine-recognized - * object kinds (markers), depth/property caps and a cycle guard so - * adversarial shapes stay bounded. - * - * `maxLen` bounds the ARRAY read (default 256): the general preview read - * (a completion value, a provenance registry) truncates arrays past the - * cap with a `'[ArrayTruncated]'` marker. The COMPLETE read — the - * host-owned metadata surfaces (`readValueComplete`: the pending-call - * registry, the await log, the provenance registry's `read()` result) — - * lifts both caps: those surfaces are the frozen guest library's own - * metadata, never guest content, and truncating them leaks markers into - * the broker's id lists (phase-E review round 3: the 16 384-element cap - * truncated the pending registry and its marker mapped to `undefined`; - * the 256-property cap dropped bindings 256+ from the manifest's - * provenance). - * - * This is the conservative seed of the ObjectPreview rendering the tool - * result eventually carries (the full CDP-style previewer is - * `preview.ts`); everything read here is trap-free and bounded. - */ -export function readValue(handle: JSValueHandle, depth: number, seen: Set, maxLen = 256): unknown { - return readValueBounded(handle, depth, seen, maxLen, 256); -} - -/** - * The COMPLETE trap-free read for host-owned metadata surfaces — the - * guest-surface reads (the pending-call registry, the await log) and the - * provenance registry's `read()` result. Identical discipline to - * `readValue` (own-data-property descriptor reads, engine brand checks, - * cycle guard, depth bound) with NO array-length or object-key cap: the - * pending-call registry must report the WHOLE registry (phase-E review - * rejection: the 16 384-element array cap silently truncated the list - * and its `[ArrayTruncated]` marker leaked into the broker's id lists as - * an `undefined` hole) and the provenance registry must report every - * binding's origin (phase-E review rejection: the 256-property object - * cap dropped bindings 256+ from the manifest's provenance). These - * surfaces are the frozen guest library's own metadata (call ids, - * kinds, options strings, origin labels) — never guest-authored content - * — so the adversarial caps don't apply; the read is bounded by the VM's - * memory like the metadata itself. - */ -export function readValueComplete(handle: JSValueHandle, depth = 0, seen = new Set()): unknown { - return readValueBounded(handle, depth, seen, Number.POSITIVE_INFINITY, Number.POSITIVE_INFINITY); -} - -function readValueBounded( - handle: JSValueHandle, - depth: number, - seen: Set, - maxLen: number, - maxKeys: number, -): unknown { - if (handle.isUndefined) return undefined; - if (handle.isNull) return null; - - const t = handle.typeof; - switch (t) { - case 'boolean': - // JS_ToCString on a boolean primitive is native — no guest code runs. - return handle.toString() === 'true'; - case 'number': - return handle.toNumber(); - case 'string': - return handle.toString(); - case 'bigint': - return handle.toBigInt(); - case 'symbol': - return '[Symbol]'; - case 'function': - return '[Function]'; - default: - break; // objects - } - - // Engine-level brand checks: never fire traps, cannot be spoofed from - // guest JavaScript. Proxies are never touched further (descriptor reads - // would fire their traps). - if (handle.isProxy) return '[Proxy]'; - if (handle.isPromise) return '[Promise]'; - if (handle.isDate) return '[Date]'; - if (handle.isMap) return '[Map]'; - if (handle.isSet) return '[Set]'; - if (handle.isWeakMap) return '[WeakMap]'; - if (handle.isWeakSet) return '[WeakSet]'; - if (handle.isWeakRef) return '[WeakRef]'; - if (handle.isRegExp) return '[RegExp]'; - if (handle.isArrayBuffer) return '[ArrayBuffer]'; - - const ptr = handle.ptr; - if (seen.has(ptr)) return '[Circular]'; - if (depth >= 4) return '[Object]'; - - seen.add(ptr); - try { - if (handle.isArray) { - const lengthHandle = readOwnDataProperty(handle, 'length'); - const length = lengthHandle === undefined ? 0 : lengthHandle.toNumber(); - lengthHandle?.dispose(); - const out: unknown[] = []; - const count = Math.min(length, maxLen); - for (let i = 0; i < count; i++) { - const v = readOwnDataProperty(handle, String(i)); - if (v === undefined) continue; // sparse hole - try { - out.push(readValueBounded(v, depth + 1, seen, maxLen, maxKeys)); - } finally { - v.dispose(); - } - } - if (length > maxLen) out.push('[ArrayTruncated]'); - return out; - } - - const out: Record = {}; - let count = 0; - for (const key of rawOwnKeys(handle)) { - if (count >= maxKeys) { - out['[Truncated]'] = true; - break; - } - const v = readOwnDataProperty(handle, key); - if (v === undefined) continue; // accessor or deleted between reads - try { - out[key] = readValueBounded(v, depth + 1, seen, maxLen, maxKeys); - } finally { - v.dispose(); - } - count++; - } - return out; - } finally { - seen.delete(ptr); - } -} - -/** - * Read one boolean flag off an engine-created descriptor object (`value`/ - * `get`/`set`/`writable`/`enumerable`/`configurable`). `false` on any read - * failure — flags only matter in the false direction for the previewer - * (non-enumerable means "not listed"). - */ -function readFlag(e: QuickJSExports, vm: QuickJS, objPtr: number, key: string): boolean { - const handle = getPropRaw(e, vm, objPtr, key); - if (handle === undefined) return false; - try { - return handle.isBool ? handle.toBoolean() : false; - } finally { - handle.dispose(); - } -} diff --git a/packages/repl-engine/src/types.ts b/packages/repl-engine/src/types.ts deleted file mode 100644 index 93c58463..00000000 --- a/packages/repl-engine/src/types.ts +++ /dev/null @@ -1,94 +0,0 @@ -/** - * Self-contained public types for the engine's WebAssembly surface. - * - * The repo's tsconfig has no DOM lib (`lib: ["ES2022", "ESNext.Disposable"]`), - * and a consumer may have the same. In that configuration neither the - * `BufferSource` type nor the `WebAssembly` namespace is declared, and the - * ambient declarations this package compiles against - * (`wasm-ambient.d.ts`) are source-only — TypeScript does not emit input - * `.d.ts` files, so they are absent from the published package (which - * ships `dist` only). The public options therefore use the types below, - * which are fully self-contained: - * - * - `ArrayBuffer` / `ArrayBufferView` come from `lib.es5` — always - * present. - * - `WasmModule` is an **opaque** stand-in for `WebAssembly.Module` - * (see below). - * - * The published declaration graph (dist/vm.d.ts, dist/workspace.d.ts) - * references only these types plus `lib.es5` names, so a consumer with a - * non-DOM lib and `skipLibCheck: false` type-checks the package cleanly. - */ - -/** - * Opaque brand carried by `WasmModule`. Declared as a private unique - * symbol, never exported and never assigned at runtime: it exists purely - * so that no accidental value satisfies the type. A reviewer's negative - * probe proved the necessity: with `WasmModule` as an empty interface, - * `{ wasm: 42 }` type-checked (every non-null value satisfies an empty - * interface) and failed only at runtime. - */ -declare const wasmModuleBrand: unique symbol; - -/** - * A compiled WebAssembly module — the engine's opaque stand-in for - * `WebAssembly.Module`, declared locally so the published type graph does - * not depend on the consumer's lib. - * - * The type is branded, so the only way to obtain a `WasmModule` value is - * `loadShippedWasm()` (the engine compiles the shipped `quickjs.wasm` - * binary and brands the real `WebAssembly.Module` at that single - * boundary). Custom WASM is accepted as raw bytes instead: - * `ArrayBuffer | ArrayBufferView` satisfies `WasmInput` directly. - */ -export interface WasmModule { - /** @internal Opaque brand — never construct `WasmModule` values yourself. */ - readonly [wasmModuleBrand]: void; -} - -/** - * What the engine accepts wherever a WebAssembly binary or pre-compiled - * module is needed: raw bytes (the shipped `quickjs.wasm`), a view over - * bytes, or a compiled module (from `loadShippedWasm`). - */ -export type WasmInput = ArrayBuffer | ArrayBufferView | WasmModule; - -/** - * An extension record in a snapshot's metadata (name, memory/table base, - * init function) — the engine's self-contained stand-in for the shim's - * `SnapshotExtension`, kept in lockstep with quickjs-wasi's shape so a - * snapshot taken through the shim round-trips without conversion. - */ -export interface ReplSnapshotExtension { - /** Extension name as passed at create/restore time (e.g. `structured-clone`). */ - name: string; - /** Allocated base offset in linear memory for this extension's static data. */ - memoryBase: number; - /** Allocated base offset in the indirect function table. */ - tableBase: number; - /** Name of the init function exported by the extension. */ - initFn: string; -} - -/** - * A snapshot of a VM's full state (raw WASM linear memory plus runtime - * pointers) — the engine's self-contained stand-in for the shim's - * `Snapshot` type, so `ReplVm.restore` can be declared without naming - * quickjs-wasi types (a consumer with a non-DOM lib and `skipLibCheck: - * false` must type-check the published declarations cleanly). - * - * Structurally identical to the shim's `Snapshot`, so a snapshot produced - * through the shim (`vm.snapshot()`) satisfies it directly. - */ -export interface ReplSnapshot { - /** The raw WASM linear memory contents. */ - memory: Uint8Array; - /** The stack pointer value at snapshot time. */ - stackPointer: number; - /** Pointer to the JSRuntime in WASM memory. */ - runtimePtr: number; - /** Pointer to the JSContext in WASM memory. */ - contextPtr: number; - /** Metadata about loaded extensions (empty if none). */ - extensions: ReplSnapshotExtension[]; -} diff --git a/packages/repl-engine/src/vm.ts b/packages/repl-engine/src/vm.ts deleted file mode 100644 index a21bfa23..00000000 --- a/packages/repl-engine/src/vm.ts +++ /dev/null @@ -1,1400 +0,0 @@ -/** - * The QuickJS-in-WASM VM layer of the REPL engine. - * - * This module is the "runtime shim" tier of the roadmap doc's mapping - * table (docs/roadmap/repl-orchestrator.md): `quickjs-wasi` is used as-is, - * including the npm package's shipped `quickjs.wasm` binary (we never build - * our own binary — snapshot portability with the harness is explicitly not - * a goal, and the shipped binary keeps snapshots compatible across daemons - * running the same quickjs-wasi version). - * - * What this layer adds on top of the shim: - * - * - **Wasm loading** — the shipped binary, resolved through the package's - * own export map and compiled once per process into a - * `WebAssembly.Module`, which quickjs-wasi recommends reusing across - * VM instantiations. - * - **Eval with top-level `await`** — `EvalFlags.ASYNC`, the script-global - * REPL mode the harness pinned (R69): bindings persist on `globalThis`, - * sloppy mode, completion value = last expression, top-level `return` - * stays a syntax error (the parser's "return not in function" check is - * independent of the async flag). The eval returns a Promise whose - * fulfillment is an engine-created `{ value }` wrapper around the - * completion value. - * - **The job drain** — the same pending-job loop quickjs-wasi's - * built-in `executePendingJobs()` runs (its own implementation is - * exactly `while (qjs_is_job_pending()) qjs_execute_pending_job()`), - * surfaced as `drainJobs()`. Because the interrupt stays armed across - * the drain (the wasm interrupt import fires per bytecode instruction, - * including inside drained jobs), a runaway microtask loop is bounded - * by the same handler that bounds the eval itself. - * - **Per-VM `memoryLimit`** — passed straight through to - * `QuickJSOptions.memoryLimit` (quickjs-wasi built-in, `JS_SetMemoryLimit`). - * - **Per-eval and per-drain `interruptHandler`** — quickjs-wasi's - * `interruptHandler` is a per-VM create-time option; the engine composes - * per-operation semantics on top of that built-in by installing one - * VM-level handler that delegates to a mutable per-operation slot. The - * slot is armed for the duration of one eval **or one standalone - * settlement drain** and restored afterwards, so handlers never leak - * across operations — and a settlement drain that resumes a suspended - * continuation carries its own interrupt signal (a continuation left - * queued by an interrupted drain would otherwise run with no protection). - * - **Serialized VM operations** — `evalCode` performs **no `await`**: - * host execution is synchronous from the caller's perspective, so an - * eval cannot be interleaved with `dispose()` (review regression: the - * completion read used to yield through an already-settled host promise, - * and `const p = ws.eval('6*7'); ws.dispose(); await p` crashed reading - * nulled WASM exports) and the interrupt slot's save/restore cannot be - * reordered by concurrent evals (review regression: two overlapping - * evals restored out of nesting order and left a stale handler armed - * for later operations). An `opDepth` guard makes the serialization - * invariant structural, so a future host-callback path that tries to - * re-enter the VM fails loudly instead of corrupting the slot. - * - **A trap-free evaluation and error boundary** — the doc's mandatory - * rule (transfer lesson R69): never execute guest getters while - * rendering guest state. This is enforced structurally, not by review - * discipline, because two quickjs-wasi paths would otherwise violate it: - * - `QuickJS.evalCode()` wraps synchronous failures (parse errors) in a - * `JSException` whose **constructor** performs guest-visible `[[Get]]` - * reads of `name`/`message`/`stack` on the guest exception — before - * any host `catch` can intercept. The engine therefore never calls - * `evalCode()`: it drives the same raw `qjs_eval` export (via the - * package's public `_getExports()`/`_writeString()` accessors) and - * handles a synchronous exception itself, with own-property-descriptor - * reads only. - * - `JSValueHandle.getOwnPropertyDescriptor()` throws a `JSException` - * when the C descriptor read fails (allocation edge), and that - * constructor runs the same guest-visible getters. The engine's own - * descriptor path drives `qjs_get_own_property_descriptor` directly, - * takes a failed read's exception value out of the runtime and frees - * it (never constructing `JSException`), and reads the descriptor - * object's own data properties through raw `qjs_get_prop_value` — - * OrdinaryGet on an engine-created plain object, so even a polluted - * `Object.prototype` cannot run guest code. - * - `QuickJS.executePendingJobs()` renders a job error through - * `exc.toString()`, a JavaScript string conversion that **executes - * guest code** (`toString()`/`valueOf()`/`Symbol.toPrimitive`, proxy - * traps). The engine's drain uses the same built-in job loop over the - * raw exports but reads the failed job's exception trap-free and - * reports it as a `DrainJobError` carrying `EvalErrorInfo`. - * - Proxies are never touched: a proxy fires traps on descriptor, key, - * and prototype reads, so every descriptor read is guarded with - * `isProxy` first (engine-level brand check, spoof-proof), and a - * thrown proxy reports a trap-free marker. - * - Every handle the engine takes from the shim is disposed — including - * the exception value of a failed eval, a failed descriptor read, and - * the `get`/`set` handles of accessor descriptors — so failed paths - * never accumulate guest memory inside a long-lived VM (both leaks - * were measured during review: a 1 MiB VM exhausted after ~4,018 - * syntax errors / ~3,128 accessor completions). - */ - -import { createHash } from 'node:crypto'; -import { readFile } from 'node:fs/promises'; - -import { parse } from 'acorn'; -import { EvalFlags, JSValueHandle, QuickJS } from 'quickjs-wasi'; - -import { classifyError, type EvalErrorInfo } from './errors.js'; -import { noteWasmModuleHash } from './snapshot-envelope.js'; -import { readOwnDataProperty, readValue, takeAndFreeException } from './trapfree.js'; -import type { ReplSnapshot, WasmInput, WasmModule } from './types.js'; - -/** Options for creating a VM. */ -export interface ReplVmOptions { - /** - * WASM bytes or a pre-compiled module (`WasmInput` — a self-contained - * stand-in for `WebAssembly.Module | BufferSource`, see `types.ts`). - * Defaults to the `quickjs-wasi` package's shipped `quickjs.wasm` - * binary, resolved through the package export map and compiled once per - * process. - */ - wasm?: WasmInput; - /** - * Per-VM malloc limit in bytes (quickjs-wasi `memoryLimit` built-in). - * When exceeded, allocations fail and surface as - * `InternalError: out of memory` (an `EvalErrorInfo` with - * `outOfMemory: true`). Defaults to `ReplVm.DEFAULT_MEMORY_LIMIT`. - */ - memoryLimit?: number; -} - -/** The per-job CONTINUATION-LEASE seam (the eval-break targeting - * identity — see the broker and `guest-library.ts`): the drain loop - * reads the guest library's continuation lease before each job (the VM - * is idle between jobs) and clears it after a job that started with - * one, so the lease is set exactly while a suspended eval's - * continuation segment executes (and during the library's - * lease-setting reaction that immediately precedes it). `cell.current` - * is the host-side mirror of the CURRENT job's lease — an interrupt - * handler consulted DURING the job reads it to learn which eval's - * continuation (if any) is executing. */ -export interface ReplJobLease { - /** Read the current continuation-lease token (string) or undefined. */ - read(): string | undefined; - /** Clear the continuation lease (drain start, and after a job that - * started with one). */ - clear(): void; - /** The host-side mirror of the current job's lease, set before each - * job executes. */ - cell: { current: string | undefined }; -} - -/** Options for a single eval. */ -export interface ReplEvalOptions { - /** Filename used in guest stack traces. Defaults to `''`. */ - filename?: string; - /** - * Per-eval interrupt handler. The VM-level quickjs-wasi interrupt - * handler (a built-in consulted by the wasm interrupt import, roughly - * once per bytecode instruction) delegates to this slot while the eval - * and its drain are running. Return `true` to abort execution with - * `InternalError: interrupted` (`EvalErrorInfo.interrupted === true`). - * The slot is restored to its previous value afterwards, so the handler - * never leaks into later operations. - */ - interruptHandler?: () => boolean; - /** - * An interrupt handler consulted ONLY by the eval's OWN job drain — - * never by the eval code itself. The eval-break signal's direct-eval - * seam: a suspended eval's continuation can be resumed by a - * SYNCHRONOUS host-callback settlement (`checkpoint.answer` in a - * later eval resolves the checkpoint's deferred right there), and - * that resumed continuation executes inside the answering eval's own - * drain — an execution the settlement-drain handler cannot reach. - * The eval's own code deliberately never consults this handler: an - * unrelated eval's code must never be broken by a signal armed - * against another eval (the phase-E review rejection's leak). - */ - drainInterruptHandler?: () => boolean; - /** - * The per-job continuation-lease plumbing (see `ReplJobLease`): the - * drain loop maintains the lease mirror so the drain-phase interrupt - * handler can tell WHICH eval's continuation is executing. The broker - * passes its lease; a bare workspace passes none (no tracking). - */ - jobLease?: ReplJobLease; - /** - * Called exactly once per eval, AFTER the script's synchronous code - * phase and BEFORE the job drain. The code-phase boundary: the - * script's synchronous execution — including every synchronous host - * callback it made — has finished, and the drain phase (which runs - * the continuations the code phase queued) is next. The callback - * runs while the VM is idle (the code phase returned; no guest code - * is on the stack), so the caller may perform host-side VM reads. - */ - beforeDrain?: () => void; - /** - * Attach the engine's uncaught-rejection bridge when the eval SUSPENDS - * (its completion promise is still pending after the drain): the bridge - * — `p.then(undefined, err => console.error(err))` — routes a late - * rejection of the completion promise into the ordinary console bridge, - * so it surfaces as an error-level console line in the next tool - * result instead of vanishing (the doc's transfer lesson 3: "late - * uncaught rejections surface as error-level console lines in the next - * tool result"). Attached only on the pending arm; a completion that - * resolves is dropped exactly as a `.then` continuation's would be, and - * one that rejects within the drain is the eval's ordinary error - * outcome. Best-effort by contract: the bridge call runs guest code - * (`Promise.prototype.then`), which a hostile realm may have sabotaged - * — a throw there is taken out and freed and the pending outcome is - * reported unchanged (the harness's stance: a root that sabotages - * `Promise.prototype.then` is sabotaging only itself). - */ - rejectionBridge?: boolean; -} - -/** Options for a standalone settlement drain (`ReplVm.drainJobs`). */ -export interface ReplDrainOptions { - /** - * Interrupt handler armed for the duration of this drain. A suspended - * eval's handler is removed when the eval returns, so a later settlement - * drain that resumes a runaway continuation must carry its own signal — - * without one, the continuation would run with no interrupt protection. - * Return `true` to abort the drained job with - * `InternalError: interrupted` (surfaced as a `DrainJobError`). - */ - interruptHandler?: () => boolean; - /** - * The per-job continuation-lease plumbing (see `ReplJobLease`): the - * drain loop maintains the lease mirror so the drain's interrupt - * handler can tell WHICH eval's continuation is executing. The broker - * passes its lease; a bare workspace passes none (no tracking). - */ - jobLease?: ReplJobLease; -} - -/** - * The outcome of one eval, after the job drain: - * - * - `{ kind: 'value', value }` — the eval's completion promise fulfilled - * within the drain; `value` is the trap-free read of the completion - * value (a shallow read; the ObjectPreview rendering that the tool - * result eventually carries is a later phase's job). - * - `{ kind: 'pending' }` — the completion promise is still pending after - * the drain (a top-level `await` suspended on an unsettled promise). - * The harness's pinned shape: no fabricated value; the continuation - * resumes at settlement like a `.then`. - * - `{ kind: 'error', error }` — the eval threw (synchronously, via a - * rejected completion promise, or via a job error during the drain — - * the typical drain error is the per-eval interrupt firing inside a - * drained continuation). - */ -export type ReplEvalOutcome = - | { kind: 'value'; value: unknown } - | { kind: 'pending' } - | { kind: 'error'; error: EvalErrorInfo }; - -/** - * A job-drain failure, carrying trap-free error info. - * - * quickjs-wasi's own `executePendingJobs()` renders the failed job's - * exception through `exc.toString()` — a JavaScript string conversion - * that executes guest code (`toString`/`valueOf`/`Symbol.toPrimitive`, - * proxy traps). The engine's drain reads the exception value - * own-property-descriptor-wise instead, so the message is built from - * `EvalErrorInfo` data and no guest code runs while a drain error is - * reported. - */ -export class DrainJobError extends Error { - /** Trap-free structured information about the failed job's exception. */ - readonly info: EvalErrorInfo; - - constructor(info: EvalErrorInfo) { - super(`Job execution error: ${info.name}: ${info.message}`); - this.name = 'DrainJobError'; - this.info = info; - } -} - -/** - * The result of the trap-free eval call: a live promise handle, or a - * trap-free report of a synchronous (parse/compile) failure. - */ -type EvalResult = - | { kind: 'ok'; handle: JSValueHandle } - | { kind: 'error'; error: EvalErrorInfo }; - -// The shipped binary is compiled once per process and reused across VM -// instantiations (the pattern quickjs-wasi's README recommends). -let shippedModule: Promise | null = null; - -// The structured-clone extension (.so) is loaded once per process and -// attached to every VM: older guest libraries used it for the $N -// freezing path (deleted in 0.4.0), and the extension travels in -// snapshots, so the restore path must attach the same byte-identical -// artifact it was snapshotted with (quickjs-wasi restores extension -// memory against the descriptors it was created with). -let structuredCloneExtension: Promise | null = null; - -async function loadStructuredCloneExtension(): Promise { - structuredCloneExtension ??= (async () => { - // Resolved through the package export map, like the shipped wasm - // binary (`quickjs-wasi/structured-clone.so`). - const resolved = import.meta.resolve('quickjs-wasi/structured-clone.so'); - return readFile(new URL(resolved)); - })(); - return structuredCloneExtension; -} - -/** - * Load the `quickjs-wasi` package's shipped `quickjs.wasm` binary and - * compile it into a reusable module (typed as `WasmModule`, the opaque - * stand-in for `WebAssembly.Module` — see `types.ts`). - * - * The binary's sha256 is recorded against the compiled module in the - * envelope registry (`noteWasmModuleHash` — see `snapshot-envelope.ts`): - * the at-rest snapshot envelope records which binary laid out the VM - * memory, and the restore path compares that recorded hash against the - * module it restores with. `loadShippedWasm` is the ONLY producer of - * `WasmModule` values, so the registry covers every compiled module the - * engine can be asked to hash. - */ -export function loadShippedWasm(): Promise { - shippedModule ??= (async () => { - const resolved = import.meta.resolve('quickjs-wasi/quickjs.wasm'); - const bytes = await readFile(new URL(resolved)); - // The engine is the only producer of `WasmModule` values: the real - // `WebAssembly.Module` is branded opaque at this boundary so consumers - // cannot fabricate `WasmInput` values that would fail at runtime. - const module = (await WebAssembly.compile(bytes)) as unknown as WasmModule; - noteWasmModuleHash(module, createHash('sha256').update(bytes).digest('hex')); - return module; - })(); - return shippedModule; -} - -/** - * A QuickJS-in-WASM VM with the REPL engine's eval/drain semantics. - * - * One `ReplVm` backs exactly one workspace; the workspace owns its - * lifecycle (`create` → `evalCode`/`drainJobs` → `dispose`). - * - * All operations are serialized: `evalCode` and `drainJobs` are - * synchronous from the caller's perspective (no `await` between arming - * the interrupt slot and restoring it), so operations can never interleave - * with each other or with `dispose()`. - */ -export class ReplVm { - /** Default per-VM malloc limit when the caller configures none. */ - static readonly DEFAULT_MEMORY_LIMIT = 64 * 1024 * 1024; - - private readonly vm: QuickJS; - private readonly memoryLimitBytes: number; - private readonly interruptSlot: { current: (() => boolean) | null }; - /** Engine-owned throw-site capture helpers installed in the realm. */ - private readonly throwCaptureAvailable: boolean; - /** Active-operation depth; > 0 means a VM operation is running. */ - private opDepth = 0; - private disposed = false; - - private constructor( - vm: QuickJS, - memoryLimitBytes: number, - interruptSlot: { current: (() => boolean) | null }, - ) { - this.vm = vm; - this.memoryLimitBytes = memoryLimitBytes; - this.interruptSlot = interruptSlot; - // The bridge and previewer modules drive the shim through this - // module-scoped map (see `getVmShim`): the public type graph must stay - // free of quickjs-wasi types, because a consumer with a non-DOM lib - // and `skipLibCheck: false` type-checks the published declarations - // cleanly — quickjs-wasi's own declarations need DOM globals. - vmShims.set(this, vm); - this.throwCaptureAvailable = this.installThrowCapture(); - } - - /** - * Create a fresh VM. The wasm module (shipped binary by default) may be - * shared across VMs; each VM gets its own isolated runtime and context. - */ - static async create(options: ReplVmOptions = {}): Promise { - const wasm = options.wasm ?? (await loadShippedWasm()); - const memoryLimitBytes = options.memoryLimit ?? ReplVm.DEFAULT_MEMORY_LIMIT; - - // quickjs-wasi's `interruptHandler` is a per-VM create-time option; the - // per-operation semantics are composed on top of the built-in by - // delegating through a slot that `evalCode`/`drainJobs` arm per call - // (see the module docs). The slot object is captured by the VM-level - // closure, so it must be shared with the instance rather than assigned - // after creation. - const interruptSlot: { current: (() => boolean) | null } = { current: null }; - const vm = await QuickJS.create({ - wasm, - memoryLimit: memoryLimitBytes, - // The structured-clone extension ships with the quickjs-wasi package - // and is attached to every VM (older guest libraries used it for the - // deleted $N freezing path). The extension travels inside snapshots, - // so `restore()` attaches the same artifact — a restored pre-0.4.0 - // workspace still carries the extension. - extensions: [{ name: 'structured-clone', wasm: await loadStructuredCloneExtension() }], - interruptHandler: () => interruptSlot.current?.() ?? false, - }); - - return new ReplVm(vm, memoryLimitBytes, interruptSlot); - } - - /** - * Restore a VM from a quickjs-wasi snapshot (the same wasm build and the - * same structured-clone extension it was snapshotted with). Host - * callbacks are NOT restored by the shim — the caller re-registers them - * by name (the roadmap doc's restore path: a quickjs-wasi built-in) and - * then reconciles the in-VM pending-call registry through the guest - * library's reconciliation surface. - * - * Snapshot compatibility holds only across the same quickjs-wasi package - * version (the doc's rule: a version bump must refuse old snapshots - * loudly, never restore them silently); the at-rest identity envelope is - * a later phase's concern. - */ - static async restore(snapshot: ReplSnapshot, options: ReplVmOptions = {}): Promise { - const wasm = options.wasm ?? (await loadShippedWasm()); - const memoryLimitBytes = options.memoryLimit ?? ReplVm.DEFAULT_MEMORY_LIMIT; - - const interruptSlot: { current: (() => boolean) | null } = { current: null }; - const vm = await QuickJS.restore(snapshot, { - wasm, - memoryLimit: memoryLimitBytes, - extensions: [{ name: 'structured-clone', wasm: await loadStructuredCloneExtension() }], - interruptHandler: () => interruptSlot.current?.() ?? false, - }); - - return new ReplVm(vm, memoryLimitBytes, interruptSlot); - } - - - /** The configured malloc limit in bytes (per-VM, set at create time). */ - get memoryLimit(): number { - return this.memoryLimitBytes; - } - - /** True once `dispose()` has been called. */ - get isDisposed(): boolean { - return this.disposed; - } - - /** - * Run one script: evaluate with top-level-await semantics, drain the - * microtask/job queue, and report the completion. - * - * The returned promise is fulfilled **synchronously** — the body performs - * no `await`, so the eval cannot be interleaved with `dispose()` or with - * another eval (the interrupt-slot save/restore and the raw handle reads - * are atomic with respect to every other VM operation). The whole - * boundary is trap-free: a synchronous parse failure never passes through - * quickjs-wasi's `JSException` constructor (which performs guest-visible - * `[[Get]]` reads), a failed descriptor read never constructs one either, - * and a drain failure never passes through its `toString()`-based error - * rendering — see the module docs. - */ - async evalCode(code: string, options: ReplEvalOptions = {}): Promise { - const { outcome, completion } = this.evalCodeWithCompletion(code, options); - // The completion handle is owned by the caller of - // evalCodeWithCompletion — this public entry discards it, so it must - // dispose it (review regression: a resolved eval with - // `rejectionBridge: true` used to leak the handle — an adversarial - // 2 MiB VM probe died at eval ~19,346). Rejection bridging is - // decided by `options.rejectionBridge` alone (see - // `evalCodeWithCompletion`): a caller that wants the bridge does not - // thereby own a completion handle. - if (completion !== undefined) (completion as JSValueHandle).dispose(); - return outcome; - } - - /** - * The package-internal eval entry the broker layer drives: like - * `evalCode`, but the completion handle is returned alongside the - * shallow snapshot read (`completion`, OWNED BY THE CALLER — the - * caller must dispose it). For a RESOLVED eval it is the live - * completion-value handle the broker previews for the tool result's - * `result` line; for a PENDING eval (the completion suspended on a - * host call) it is the eval WRAPPER promise handle — the broker's - * active-eval tracking probe: the wrapper stays pending while the - * eval's continuation is in flight and settles when the continuation - * completes or is broken (phase-E review rejection: the pending - * completion used to be dropped, so the workspace had no host-side - * notion of "an eval is running" and the interrupt tool could not - * target it). For `error` outcomes `completion` is undefined and the - * caller owns nothing. The published type graph never names the - * handle type: this method is not re-exported from the package index - * (the bridge's `getVmShim` precedent), and `completion` is typed - * `unknown` so the declaration stays self-contained. - */ - evalCodeWithCompletion( - code: string, - options: ReplEvalOptions = {}, - ): { outcome: ReplEvalOutcome; completion?: unknown; interruptedInDrain?: boolean } { - this.assertAlive(); - this.assertNotReentrant(); - - // Rejection bridging and completion ownership are SEPARATE decisions - // (review regression: they used to be one flag, so a resolved eval - // leaked its completion wrapper and a discarded completion was never - // disposed): the bridge attaches to a SUSPENDED completion when - // `rejectionBridge` is set; the live completion handle is returned - // to this internal entry's caller on every resolved eval, and the - // caller (the broker, or `evalCode` which disposes it) owns it. - const attachBridge = options.rejectionBridge === true; - - // Arm the per-eval interrupt slot for the whole operation (eval + its - // drain). Because the body below is synchronous, this save/restore - // cannot be reordered by a concurrent eval: operations serialize. - const previousInterrupt = this.interruptSlot.current; - const evalHandler = options.interruptHandler ?? null; - this.interruptSlot.current = evalHandler; - this.opDepth++; - let handle: JSValueHandle | undefined; - try { - // A capture belongs to exactly one submitted eval. The generation - // lives inside the realm so it survives snapshot/restore; beginning - // an eval also invalidates any handled throw left by an older eval. - const throwGeneration = this.throwCaptureAvailable - ? this.beginThrowCaptureGeneration() - : undefined; - const evaluated = this.evalTrapFree( - throwGeneration === undefined ? code : instrumentThrownValues(code, throwGeneration), - options.filename ?? '', - EvalFlags.ASYNC, - ); - if (evaluated.kind === 'error') { - return { outcome: { kind: 'error', error: evaluated.error } }; - } - handle = evaluated.handle; - // The drain phase arms the eval's own handler PLUS the drain-phase - // extra handler (see `ReplEvalOptions.drainInterruptHandler`): a - // continuation resumed by a synchronous host-callback settlement - // (a checkpoint answer) executes here and must be breakable by the - // armed interrupt signal even though the eval's own code never - // consults it. - this.interruptSlot.current = - options.drainInterruptHandler === undefined - ? evalHandler - : evalHandler === null - ? options.drainInterruptHandler - : () => evalHandler() || options.drainInterruptHandler!(); - // The code-phase boundary (see `ReplEvalOptions.beforeDrain`): the - // script's synchronous execution — including every synchronous - // host callback it made — has finished, and the drain phase (which - // runs the continuations the code phase queued) is next. The VM is - // idle here, so the callback may touch the VM. - options.beforeDrain?.(); - try { - this.runDrain(options.jobLease); - } catch (e) { - if (e instanceof DrainJobError) { - // The drain was INTERRUPTED (the armed eval-break signal, or - // the per-eval deadline): the interrupted continuation's - // engine wrapper never settles (the quickjs interrupt aborts - // the async job without rejecting its promise), so the - // caller — the broker — must release its tracked running eval - // (exactly like the pump path's `noteInterruptedDrain`). The - // flag distinguishes this from a code-phase interrupt (the - // fresh eval's own code hitting the deadline), which affects - // no tracked eval. - return { outcome: { kind: 'error', error: e.info }, interruptedInDrain: true }; - } - throw e; // host-side failure, not a guest outcome — fail loudly - } - return this.readCompletion(handle, true, attachBridge, throwGeneration); - } finally { - // The wrapper is disposed here on every arm except the retained- - // pending one: `readCompletion` returns a DUP for the retained - // arm (the caller owns it) and disposes the original itself, so - // this dispose is either a no-op (the value arm already disposed - // it) or the genuine release (the error/DrainJobError arms). - handle?.dispose(); - this.interruptSlot.current = previousInterrupt; - this.opDepth--; - } - } - - /** - * Run the job drain loop: execute all pending microtask jobs (promise - * reactions, resumed top-level-await continuations). This is - * quickjs-wasi's built-in `executePendingJobs()` loop driven over the - * package's own exports (`qjs_is_job_pending` / `qjs_execute_pending_job` - * — the built-in's implementation is exactly that loop). - * - * A suspended eval's per-eval handler is removed when the eval returns; - * this standalone drain therefore arms **its own** interrupt signal for - * its duration (`options.interruptHandler`), so a continuation left - * queued by an interrupted eval — or resumed by host-side settlement - * (subagent calls in a later phase) — cannot run away unguarded. - * - * The one deliberate difference from the built-in: a failed job's - * exception is read trap-free and thrown as a `DrainJobError` instead of - * being rendered through `toString()`, which would execute guest code. - * Returns the number of jobs executed. - */ - drainJobs(options: ReplDrainOptions = {}): number { - this.assertAlive(); - this.assertNotReentrant(); - - const previousInterrupt = this.interruptSlot.current; - this.interruptSlot.current = options.interruptHandler ?? null; - this.opDepth++; - try { - // A standalone drain is a new execution boundary. If its first job - // throws an uninstrumented primitive, it must not inherit a capture - // left by a handled throw in an earlier eval. - this.clearThrowCapture(); - return this.runDrain(options.jobLease); - } finally { - this.interruptSlot.current = previousInterrupt; - this.opDepth--; - } - } - - /** Dispose the VM, releasing the WASM instance. Idempotent. */ - dispose(): void { - if (this.disposed) return; - this.disposed = true; - this.interruptSlot.current = null; - this.vm.dispose(); - } - - /** - * Write a live guest value into a realm GLOBAL slot by name — the - * engine's seam for `_` (the §4.4 result-history global: the previous - * eval's completion value, IPython-style; the broker sets it after - * every eval that resolved with a value). The value handle is - * BORROWED (the raw set dups it — the caller keeps ownership). Trap- - * free by construction: a plain `JS_SetProperty` on the global object - * runs no guest code. Called BETWEEN VM operations (never re-entrant - * — like every host-side VM read). - */ - setGlobal(name: string, value: unknown): void { - // `value` is typed `unknown` for the published declaration graph (see - // `evalCodeWithCompletion`'s completion — the public surface must stay - // free of quickjs-wasi types); the caller passes the live completion - // handle, borrowed (the raw set dups it). - this.assertAlive(); - const shim = getVmShim(this) as QuickJS; - shim.setProp(shim.global, name, value as JSValueHandle); - } - - /** Support for `using` declarations (Explicit Resource Management). */ - [Symbol.dispose](): void { - this.dispose(); - } - - /** - * Read a RETAINED suspended-eval completion wrapper after it settled - * (the broker's active-eval sweep calls this on a wrapper whose - * `promiseState` left 0 — the `_` result-history seam: a suspended - * eval that completed during a previous drain is the PREVIOUS eval, - * and its completion value becomes `_`). Fully synchronous and - * trap-free, like `readCompletion`: for a FULFILLED completion the - * unwrapped value handle (the raw `qjs_promise_result` ref's - * `{ value }` own-data property — never a `[[Get]]`) is returned - * OWNED BY THE CALLER; a REJECTED completion (the eval errored late - * — `_` stays unchanged, the error already rendered through the - * rejection bridge), a still-pending wrapper, or the pollution quirk - * (the wrapper with no own `value` — the wrapper itself is returned, - * so the caller always sees *a* value) never leaves the caller - * owning anything unexpected. Must be called between VM operations. - * Both the parameter and the result are typed `unknown` (the caller - * casts to the shim's handle type internally) so the published - * declaration graph stays free of quickjs-wasi types. - */ - readRetainedCompletion(handle: unknown): unknown { - this.assertAlive(); - const wrapper = handle as JSValueHandle; - const state = wrapper.promiseState; - if (state === 0) return undefined; // still pending — nothing to read - const e = this.vm._getExports(); - const resultPtr = e.qjs_promise_result(wrapper.ptr); - const result = new JSValueHandle(this.vm, resultPtr); - try { - if (state === 2) return undefined; // rejected — `_` stays unchanged - const valueHandle = readOwnDataProperty(result, 'value'); - if (valueHandle !== undefined) return valueHandle; - // The unexpected wrapper shape (the pollution quirk the README - // pins): return the wrapper itself — the caller sees *a* value. - return result.dup(); - } finally { - result.dispose(); - } - } - - /** Install immutable engine helpers that capture the stack at each - * instrumented `throw` while returning the exact thrown value. The - * identity-preserving return keeps guest catch semantics unchanged, - * including for primitives and proxies. */ - private installThrowCapture(): boolean { - const source = - '(() => {' + - 'var g = this;' + - `if (typeof g[${JSON.stringify(THROW_CAPTURE_GLOBAL)}] === "function" && ` + - `typeof g[${JSON.stringify(THROW_STACK_GLOBAL)}] === "function" && ` + - `typeof g[${JSON.stringify(THROW_BOUNDARY_GLOBAL)}] === "function") return true;` + - 'var same = Object.is; var generation = 0; var lastGeneration = -1; var lastValue; var lastStack;' + - `Object.defineProperty(g, ${JSON.stringify(THROW_CAPTURE_GLOBAL)}, {` + - 'value: function (value, captureGeneration) {' + - 'lastValue = value; lastStack = new Error().stack; lastGeneration = captureGeneration; return value; },' + - 'writable: false, enumerable: false, configurable: false });' + - `Object.defineProperty(g, ${JSON.stringify(THROW_STACK_GLOBAL)}, {` + - 'value: function (value, expectedGeneration) {' + - 'return same(lastValue, value) && ' + - '(expectedGeneration === undefined || lastGeneration === expectedGeneration) ? lastStack : undefined; },' + - 'writable: false, enumerable: false, configurable: false });' + - `Object.defineProperty(g, ${JSON.stringify(THROW_BOUNDARY_GLOBAL)}, {` + - 'value: function (beginEval, expectedGeneration) {' + - 'if (beginEval === true || expectedGeneration === undefined || lastGeneration === expectedGeneration) {' + - 'lastValue = undefined; lastStack = undefined; lastGeneration = -1;' + - '}' + - 'if (beginEval === true) generation++; return generation; },' + - 'writable: false, enumerable: false, configurable: false });' + - 'return true;' + - '})()'; - const installed = this.evalTrapFree(source, '', 0); - if (installed.kind === 'error') return false; - installed.handle.dispose(); - return true; - } - - /** Start one realm-persistent throw-capture generation. The helper - * clears the previous slot before returning the generation id, so a - * primitive handled by an older eval cannot match this eval's error. */ - private beginThrowCaptureGeneration(): number | undefined { - const begun = this.evalTrapFree( - `this[${JSON.stringify(THROW_BOUNDARY_GLOBAL)}](true)`, - '', - 0, - ); - if (begun.kind === 'error') return undefined; - try { - return begun.handle.isNumber ? begun.handle.toNumber() : undefined; - } finally { - begun.handle.dispose(); - } - } - - /** Clear a capture at a host execution boundary without minting a new - * eval generation. Best-effort: capture is supplemental metadata. */ - private clearThrowCapture(): void { - if (!this.throwCaptureAvailable) return; - const cleared = this.evalTrapFree( - `this[${JSON.stringify(THROW_BOUNDARY_GLOBAL)}](false)`, - '', - 0, - ); - if (cleared.kind === 'ok') cleared.handle.dispose(); - } - - /** Classify one thrown value trap-free, supplementing stackless values - * with the engine-captured throw-site stack. */ - private readErrorInfoWithCapturedStack( - handle: JSValueHandle, - expectedGeneration?: number, - ): EvalErrorInfo { - const info = readErrorInfo(handle); - if (info.stack === undefined) { - const captured = this.capturedThrowStack(handle, expectedGeneration); - if (captured !== undefined) info.stack = captured; - } - return info; - } - - /** Call the immutable engine helper with the thrown value BORROWED. - * Its captured `Object.is` performs identity comparison without any - * proxy traps or guest property reads. */ - private capturedThrowStack( - handle: JSValueHandle, - expectedGeneration?: number, - ): string | undefined { - if (!this.throwCaptureAvailable) return undefined; - const key = this.vm.newString(THROW_STACK_GLOBAL); - const fn = this.vm.getProp(this.vm.global, key); - const generation = expectedGeneration === undefined - ? undefined - : this.vm.newNumber(expectedGeneration); - try { - if (!fn.isFunction) return undefined; - const e = this.vm._getExports(); - const argv = e.wasm_malloc(8); - let resultPtr: number; - try { - new DataView(e.memory.buffer).setUint32(argv, handle.ptr, true); - new DataView(e.memory.buffer).setUint32( - argv + 4, - generation?.ptr ?? this.vm.undefined.ptr, - true, - ); - resultPtr = e.qjs_call(fn.ptr, this.vm.undefined.ptr, 2, argv); - } finally { - e.wasm_free(argv); - } - const result = new JSValueHandle(this.vm, resultPtr); - try { - if (e.qjs_is_exception(result.ptr) !== 0) { - takeAndFreeException(e, this.vm); - return undefined; - } - return result.isString ? result.toString() : undefined; - } finally { - result.dispose(); - } - } finally { - generation?.dispose(); - fn.dispose(); - key.dispose(); - } - } - - private assertAlive(): void { - if (this.disposed) { - throw new Error('ReplVm: eval/drain on a disposed VM'); - } - } - - /** - * VM operations are serialized (an op is synchronous and cannot nest). - * This guard makes that invariant structural: a future host-callback - * path that re-enters the VM mid-operation fails loudly here instead of - * corrupting the interrupt slot. - */ - private assertNotReentrant(): void { - if (this.opDepth > 0) { - throw new Error('ReplVm: reentrant VM operation (eval/drain while an operation is active)'); - } - } - - /** - * Evaluate one script through the raw `qjs_eval` export, never through - * `QuickJS.evalCode()`: quickjs-wasi's wrapper converts a synchronous - * failure into a `JSException` whose constructor performs guest-visible - * `[[Get]]` reads of `name`/`message`/`stack` on the guest exception — - * a getter installed on `SyntaxError.prototype.name` executes during - * error construction, before any host `catch` could intercept it. Here - * the exception value is read own-property-descriptor-wise (see - * `readErrorInfo`) and freed immediately, so the eval/error boundary - * cannot invoke guest getters. - */ - private evalTrapFree(code: string, filename: string, flags: number): EvalResult { - const e = this.vm._getExports(); - const codeStr = this.vm._writeString(code); - const fnStr = this.vm._writeString(filename); - const resultPtr = e.qjs_eval(codeStr.ptr, codeStr.len, fnStr.ptr, flags); - e.wasm_free(codeStr.ptr); - e.wasm_free(fnStr.ptr); - if (e.qjs_is_exception(resultPtr) !== 0) { - // Synchronous eval failure (parse/compile errors — with - // `EvalFlags.ASYNC`, runtime throws surface as a rejected completion - // promise instead). Mirror the shim's `throwIfException` ordering: - // take the exception value out of the runtime first, then free the - // exception-sentinel result. - const exc = new JSValueHandle(this.vm, e.qjs_get_exception()); - e.qjs_free_value(resultPtr); - try { - return { kind: 'error', error: readErrorInfo(exc) }; - } finally { - exc.dispose(); - } - } - return { kind: 'ok', handle: new JSValueHandle(this.vm, resultPtr) }; - } - - /** - * Read the completion of an already-drained eval promise. **Fully - * synchronous**: for a settled promise the result is taken straight from - * the runtime via the raw `qjs_promise_result` export — never through - * `resolvePromise()`, whose host promise yields through the microtask - * queue even when it is already settled. That yield is what let - * `dispose()` interleave with an in-flight eval (review regression: - * `const p = ws.eval('6*7'); ws.dispose(); await p` crashed with - * `TypeError: Cannot read properties of null (reading 'qjs_is_proxy')` - * once the WASM exports were nulled); a synchronous read makes the race - * structurally impossible. - * - * With `keepCompletion` the resolved arm returns the live completion - * VALUE handle (the engine-created `{ value }` wrapper unwrapped - * trap-free via its own descriptor — the broker previews the value, not - * the wrapper), owned by the caller; the wrapper itself is disposed on - * that path (review regression: it used to be retained, so every - * resolved eval leaked it — a 2 MiB VM died at eval ~19,346). The - * pending arm attaches the uncaught-rejection bridge first when - * `attachBridge` is set (see `ReplEvalOptions.rejectionBridge`) — - * bridging is independent of completion ownership, so a caller that - * discards the completion (public `evalCode`) still gets the bridge - * and never leaks the handle. - */ - private readCompletion( - handle: JSValueHandle, - keepCompletion: boolean, - attachBridge: boolean, - throwGeneration?: number, - ): { outcome: ReplEvalOutcome; completion?: unknown } { - try { - // 0 pending, 1 fulfilled, 2 rejected (quickjs-wasi built-in - // `promiseState`). - const state = handle.promiseState; - if (state === 0) { - if (attachBridge) this.attachRejectionBridge(handle, throwGeneration); - if (keepCompletion) { - // Retained-pending arm: the caller owns a DUP of the wrapper - // promise handle (the completion stays pending — the eval - // suspended on a host call; its continuation runs at a later - // settlement drain, and the wrapper settles when the - // continuation completes or is broken — the broker's - // active-eval probe). The original is disposed by the finally - // below like every other arm. - return { outcome: { kind: 'pending' }, completion: handle.dup() }; - } - return { outcome: { kind: 'pending' } }; - } - // For settled promises `qjs_promise_result` returns a new owned - // reference to the promise's result: the `{ value }` completion - // wrapper on fulfillment, the raw thrown value on rejection. - const resultPtr = this.vm._getExports().qjs_promise_result(handle.ptr); - const result = new JSValueHandle(this.vm, resultPtr); - // `result` is disposed in the finally unless the caller took - // ownership of it (the wrapper-as-completion fallback below). - let callerOwnsWrapper = false; - try { - if (state === 2) { - return { - outcome: { - kind: 'error', - error: this.readErrorInfoWithCapturedStack(result, throwGeneration), - }, - }; - } - // Trap-free unwrap of the engine-created `{ value }` wrapper — an - // own-data-property descriptor read, never `[[Get]]` (R69: a guest - // `Object.prototype.value` pollution must not be able to hijack - // eval results). When the wrapper shape is unexpected (the - // pollution quirk the README pins: the engine's [[Set]] silently - // no-ops, leaving the wrapper with no own `value`), the wrapper - // itself is read so the caller always sees *a* value. - const valueHandle = readOwnDataProperty(result, 'value'); - const snapshot = - valueHandle === undefined ? readValue(result, 0, new Set()) : readValue(valueHandle, 0, new Set()); - if (keepCompletion) { - if (valueHandle !== undefined) { - // The caller owns the unwrapped value handle; the wrapper is - // disposed by the finally below. - return { outcome: { kind: 'value', value: snapshot }, completion: valueHandle }; - } - callerOwnsWrapper = true; - return { outcome: { kind: 'value', value: snapshot }, completion: result }; - } - valueHandle?.dispose(); - return { outcome: { kind: 'value', value: snapshot } }; - } finally { - if (!callerOwnsWrapper) result.dispose(); - } - } finally { - handle.dispose(); - } - } - - /** - * The uncaught-rejection bridge for a suspended eval completion (the - * doc's transfer lesson 3): attach `p.then(undefined, renderer)` so a - * late rejection of the completion promise travels the ordinary - * console bridge — rendered as an error-level console line, delivered - * at the settlement drain's natural point — never a new intent-plane - * surface. The renderer produces the §4.6 uncaught-error line (the - * same shape the broker's `errorLine` renders for an in-eval error): - * the error name and message, the call-id/resolved-backend attribution - * when the error came from a subagent call, and the guest stack's top - * frames with line numbers in the submitted code (`at :line: - * col` — the guest library augments rejected registry calls with the - * CALL-SITE stack, see `settleCall` in guest-library.ts; a thrown - * Error's own stack carries the frames natively). Best-effort: the - * bridge script is evaluated with the engine's own trap-free eval, the - * call's result (including an exception result, whose runtime - * exception is taken out and freed) is disposed, and any failure - * leaves the pending outcome unchanged. The renderer never throws - * (its own guard swallows a hostile error value — a proxy's traps - * fire inside the renderer, never in the host's error path). - */ - private attachRejectionBridge( - handle: JSValueHandle, - throwGeneration?: number, - ): void { - const e = this.vm._getExports(); - let bridge: JSValueHandle | undefined; - try { - // Plain string concatenation (no template literal — the source - // must be exactly what the VM evaluates; every `\` below is an - // escaped backslash so the guest code receives the literal escape - // sequences). - const source = - '(p) => { p.then(undefined, (err) => { try {' + - 'var e = err;' + - 'var name = "Error";' + - 'var message = "undefined";' + - 'if (e !== null && e !== undefined) {' + - 'if (typeof e === "string") { message = e; }' + - 'else if (typeof e === "number" || typeof e === "boolean" || typeof e === "bigint") { message = String(e); }' + - 'else {' + - 'if (typeof e.name === "string") { name = e.name; }' + - 'if (typeof e.message === "string") { message = e.message; }' + - 'else if (typeof e.toString === "function") { try { message = e.toString(); } catch (_x) { message = "unreadable error"; } }' + - '} }' + - 'var line = name + ": " + message;' + - 'if (e !== null && e !== undefined && (typeof e === "object" || typeof e === "function")) {' + - 'var attrib = "";' + - 'if (typeof e.replCallId === "string") { attrib = "(call " + e.replCallId;' + - 'if (typeof e.replBackend === "string") { attrib += " on backend " + e.replBackend; } attrib += ")"; }' + - 'else if (typeof e.replBackend === "string") { attrib = "(on backend " + e.replBackend + ")"; }' + - 'if (attrib !== "") { line += " " + attrib; }' + - '}' + - 'var frames = [];' + - 'var stack = e !== null && e !== undefined && typeof e.stack === "string" ? e.stack : undefined;' + - `if (typeof stack !== "string" && typeof this[${JSON.stringify(THROW_STACK_GLOBAL)}] === "function") {` + - `stack = this[${JSON.stringify(THROW_STACK_GLOBAL)}](e, ${throwGeneration === undefined ? 'undefined' : String(throwGeneration)});` + - '}' + - 'if (typeof stack === "string") {' + - 'var parts = stack.split("\\n");' + - 'for (var i = 0; i < parts.length && frames.length < 8; i++) {' + - 'var m = /^\\s*at\\s+(?:(.+?)\\s+\\()?:(\\d+):(\\d+)\\)?\\s*$/.exec(parts[i]);' + - 'if (m !== null) {' + - 'frames.push(m[1] !== undefined ? " at " + m[1] + " (:" + m[2] + ":" + m[3] + ")" : " at :" + m[2] + ":" + m[3]);' + - '} } }' + - 'if (frames.length > 0) { line += "\\n" + frames.join("\\n"); }' + - 'console.error(line);' + - '} catch (_bridge) {} }); }'; - const evaluated = this.evalTrapFree(source, '', 0); - if (evaluated.kind === 'error') return; - bridge = evaluated.handle; - // Raw `qjs_call` with one borrowed argument (the completion promise); - // the result — including an exception result — is disposed here. - const argv = e.wasm_malloc(4); - let resultPtr: number; - try { - new DataView(e.memory.buffer).setUint32(argv, handle.ptr, true); - resultPtr = e.qjs_call(bridge.ptr, this.vm.undefined.ptr, 1, argv); - } finally { - e.wasm_free(argv); - } - const result = new JSValueHandle(this.vm, resultPtr); - try { - if (e.qjs_is_exception(result.ptr) !== 0) { - // The guest sabotaged `Promise.prototype.then`; the rejection - // bridge cannot be attached — the pending outcome is unchanged. - takeAndFreeException(e, this.vm); - } - } finally { - result.dispose(); - } - } finally { - bridge?.dispose(); - } - } - - /** - * The pending-job loop shared by `evalCode` (with the eval's armed - * handler) and `drainJobs` (with the drain's own armed handler). A - * failed job's exception is read trap-free and thrown as a - * `DrainJobError`; the drain stops at the first failure, so jobs queued - * after it remain pending for a later drain. - * - * With a `jobLease` (see `ReplJobLease`) the loop maintains the - * continuation-lease mirror: the guest lease is cleared at drain start - * (a stale lease left by an interrupted drain must never leak into this - * drain's first job), read before each job into `cell.current` (the - * interrupt handler consulted during the job reads the mirror), and - * cleared again after a job that started with one. The guest library's - * lease-setting reaction is registered on the WRAPPER promise itself - * — immediately BEFORE the await machinery's own reaction on the same - * wrapper (phase-E review rejection round 6: the 0.3.0 reaction ran on - * the awaited VALUE's settlement, so a sibling `q.then(...)` registered - * after the eval started awaiting `q` ran between the lease set and - * the continuation and consumed the armed signal) — so the wrapper's - * settlement queues the lease-setting job DIRECTLY BEFORE the - * machinery job that runs the eval's continuation segment: the job - * AFTER the lease-setting job starts with the lease set — it IS the - * segment — and the segment's end (the next loop iteration) clears - * it: the lease is set exactly while the segment executes, and no - * job in between can run with it set. - */ - private runDrain(lease?: ReplJobLease): number { - const e = this.vm._getExports(); - let count = 0; - lease?.clear(); - if (lease !== undefined) lease.cell.current = undefined; - while (e.qjs_is_job_pending() !== 0) { - // The current job's lease, read between jobs (the VM is idle). - const jobLease = lease?.read(); - if (lease !== undefined) lease.cell.current = jobLease; - const result = e.qjs_execute_pending_job(); - // The lease-carrying job ended: clear the guest lease so a later - // job (or a later drain) never starts under a stale lease. The - // mirror keeps the value until the next job's read; when this job - // fails, the interrupted-drain release reads it after the throw. - if (jobLease !== undefined) lease?.clear(); - if (result < 0) { - // The failed job's exception is the runtime's current exception. - // `qjs_get_exception` moves it out (the runtime's slot is cleared, - // exactly like the shim's `executePendingJobs`), so the handle owns - // the only reference; the VM stays usable after it is disposed. - const exc = new JSValueHandle(this.vm, e.qjs_get_exception()); - try { - throw new DrainJobError(this.readErrorInfoWithCapturedStack(exc)); - } finally { - exc.dispose(); - } - } - count++; - } - // No job is executing after a successful drain. In particular, a - // later eval's synchronous code must never observe the last job's - // continuation token. The error arm above intentionally bypasses - // this clear so interrupted-drain attribution can read the token. - if (lease !== undefined) lease.cell.current = undefined; - return count; - } -} - -const THROW_CAPTURE_GLOBAL = '__replCaptureThrownValueV2'; -const THROW_STACK_GLOBAL = '__replCapturedThrowStackV2'; -const THROW_BOUNDARY_GLOBAL = '__replThrowCaptureBoundaryV2'; -const THROW_INSTRUMENT_CACHE_MAX = 256; -const throwInstrumentCache = new Map(); -const THROW_FUNCTION_NODES = new Set([ - 'FunctionDeclaration', - 'FunctionExpression', - 'ArrowFunctionExpression', - 'StaticBlock', -]); - -/** Wrap each throw argument in an identity-preserving capture call. Point - * insertions add no newlines, so captured stacks retain submitted-code - * line numbers. Parse failures stay untouched for the VM to report. */ -function instrumentThrownValues(code: string, generation: number): string { - let instrumentation = throwInstrumentCache.get(code); - if (instrumentation === undefined) { - try { - const ast = parse(code, { - ecmaVersion: 'latest', - sourceType: 'script', - allowAwaitOutsideFunction: true, - }) as unknown as ThrowNode; - const sites: ThrowSite[] = []; - const catchSites: CatchSite[] = []; - collectThrowSites(ast, false, sites, catchSites); - instrumentation = { - sites, - catchSites, - nestedCaptureSafe: !containsPotentialCaptureShadow(ast), - }; - } catch { - instrumentation = null; - } - if (throwInstrumentCache.size >= THROW_INSTRUMENT_CACHE_MAX) throwInstrumentCache.clear(); - throwInstrumentCache.set(code, instrumentation); - } - if (instrumentation === null || instrumentation.sites.length === 0) return code; - const insertions: Array<{ pos: number; text: string }> = []; - for (const site of instrumentation.sites) { - if (site.inFunction && !instrumentation.nestedCaptureSafe) continue; - const capture = site.inFunction - ? THROW_CAPTURE_GLOBAL - : `this[${JSON.stringify(THROW_CAPTURE_GLOBAL)}]`; - insertions.push({ - pos: site.start, - text: `${capture}(`, - }); - insertions.push({ pos: site.end, text: `,${generation})` }); - } - // A catch means the throw was handled and can no longer be the - // uncaught value classified at this eval boundary. Clear only this - // eval's generation so concurrent suspended evals cannot erase each - // other's supplemental stack metadata. - for (const site of instrumentation.catchSites) { - if (site.inFunction && !instrumentation.nestedCaptureSafe) continue; - const boundary = site.inFunction - ? THROW_BOUNDARY_GLOBAL - : `this[${JSON.stringify(THROW_BOUNDARY_GLOBAL)}]`; - insertions.push({ - pos: site.start, - text: `${boundary}(false,${generation});`, - }); - } - insertions.sort((a, b) => b.pos - a.pos); - let instrumented = code; - for (const insertion of insertions) { - instrumented = - instrumented.slice(0, insertion.pos) + insertion.text + instrumented.slice(insertion.pos); - } - return instrumented; -} - -/** Nested code cannot use `this` as the realm object: methods, strict - * functions, and arrows all give it user-controlled semantics. Instead - * it resolves the immutable engine helper installed on the global object. - * If the submitted program mentions that reserved identifier, uses - * `with`, or can inject a local binding through direct eval, leave nested - * throws untouched rather than risk changing the value being thrown. */ -function containsPotentialCaptureShadow(node: ThrowNode | null | undefined): boolean { - if (node === null || node === undefined || typeof node !== 'object' || typeof node.type !== 'string') return false; - const record = node as unknown as Record; - if ( - node.type === 'Identifier' && - (record.name === THROW_CAPTURE_GLOBAL || record.name === THROW_BOUNDARY_GLOBAL) - ) return true; - if (node.type === 'WithStatement') return true; - if (node.type === 'CallExpression') { - const callee = record.callee as Record | null | undefined; - if (callee?.type === 'Identifier' && callee.name === 'eval') return true; - } - for (const key of Object.keys(node)) { - if (key === 'type' || key === 'start' || key === 'end' || key === 'loc' || key === 'range') continue; - const child = record[key]; - if (Array.isArray(child)) { - for (const item of child) { - if (containsPotentialCaptureShadow(item as ThrowNode)) return true; - } - } else if (child !== null && typeof child === 'object') { - if (containsPotentialCaptureShadow(child as ThrowNode)) return true; - } - } - return false; -} - -function collectThrowSites( - node: ThrowNode | null | undefined, - inFunction: boolean, - sites: ThrowSite[], - catchSites: CatchSite[], -): void { - if (node === null || node === undefined || typeof node !== 'object' || typeof node.type !== 'string') return; - if (node.type === 'ThrowStatement' && node.argument !== null && node.argument !== undefined) { - sites.push({ start: node.argument.start, end: node.argument.end, inFunction }); - } - if (node.type === 'CatchClause' && node.body !== null && node.body !== undefined) { - catchSites.push({ start: node.body.start + 1, inFunction }); - } - const childInFunction = inFunction || THROW_FUNCTION_NODES.has(node.type); - for (const key of Object.keys(node)) { - if (key === 'type' || key === 'start' || key === 'end' || key === 'loc' || key === 'range') continue; - const child = (node as unknown as Record)[key]; - if (Array.isArray(child)) { - for (const item of child) { - collectThrowSites(item as ThrowNode, childInFunction, sites, catchSites); - } - } else if (child !== null && typeof child === 'object') { - collectThrowSites(child as ThrowNode, childInFunction, sites, catchSites); - } - } -} - -interface ThrowSite { - start: number; - end: number; - inFunction: boolean; -} - -interface CatchSite { - start: number; - inFunction: boolean; -} - -interface ThrowInstrumentation { - sites: ThrowSite[]; - catchSites: CatchSite[]; - nestedCaptureSafe: boolean; -} - -interface ThrowNode { - type: string; - start: number; - end: number; - argument?: ThrowNode | null; - body?: ThrowNode | null; -} - -/** - * Trap-free error info: name/message/stack are read as own data - * properties; primitives thrown as values convert natively (strings and - * bigints as themselves, booleans as `'true'`/`'false'`, numbers as their - * native string form, symbols as the bare brand `Symbol`). Guest getters are - * never invoked while rendering the error. - * - * Two adversarial shapes are guarded before any descriptor/prototype - * inspection: a thrown **proxy** would fire traps on every descriptor and - * prototype read (review measured three traps from one thrown proxy), and - * an error whose **prototype is a proxy** (`Object.setPrototypeOf`) - * would fire its traps on the prototype's `name` read. Both report a - * trap-free marker instead: `[Proxy]` for a thrown proxy, a fallback - * `name` (`'Error'`) when the real name lives behind an accessor or a - * proxy and is therefore unreachable without running guest code. - */ -function readErrorInfo(handle: JSValueHandle): EvalErrorInfo { - // A proxy fires traps on descriptor, key, and prototype reads. Never - // touch one while rendering an error: report a trap-free marker. - if (handle.isProxy) return classifyError('Error', '[Proxy]'); - if (handle.isUndefined) return classifyError('Error', 'undefined'); - if (handle.isNull) return classifyError('Error', 'null'); - - const t = handle.typeof; - if (t !== 'object' && t !== 'function') { - let message: string; - switch (t) { - case 'string': - message = handle.toString(); - break; - case 'bigint': - message = handle.toBigInt().toString(); - break; - case 'symbol': - // FORMAT.md §1.1/§5.7: a symbol's description sits behind - // `qjs_get_symbol_description`, which invokes guest `Symbol.keyFor` - // — a forbidden seam (guest code would run, and a guest that - // replaces `Symbol.keyFor` could forge the classification). The - // bare brand is the only trap-free rendering. - message = 'Symbol'; - break; - default: - message = String(t === 'boolean' ? handle.toString() === 'true' : handle.toNumber()); - break; - } - return classifyError('Error', message); - } - - let nameHandle: JSValueHandle | undefined; - let messageHandle: JSValueHandle | undefined; - let stackHandle: JSValueHandle | undefined; - let replCallIdHandle: JSValueHandle | undefined; - let replBackendHandle: JSValueHandle | undefined; - let protoHandle: JSValueHandle | undefined; - try { - nameHandle = readOwnDataProperty(handle, 'name'); - messageHandle = readOwnDataProperty(handle, 'message'); - stackHandle = readOwnDataProperty(handle, 'stack'); - // The §4.6 error attribution (see `EvalErrorInfo`): the guest - // library stamps 'replCallId' onto every rejected registry call's - // Error and the host stamps 'replBackend' onto the rejection value - // it settles — both read here as own data strings, trap-free. - replCallIdHandle = readOwnDataProperty(handle, 'replCallId'); - replBackendHandle = readOwnDataProperty(handle, 'replBackend'); - let name = nameHandle && nameHandle.typeof === 'string' ? nameHandle.toString() : undefined; - if (name === undefined && handle.isError) { - // Error constructor names live on the error prototype (`name` is not - // an own property of error instances in quickjs-ng). Reading the - // prototype's own data property stays trap-free: getPrototypeOf fires - // no traps on real errors, and the descriptor read never invokes a - // getter. The prototype may itself be a proxy (errors are ordinary - // objects — `Object.setPrototypeOf` works), so it is guarded before - // inspection; an accessor `name` (guest-installed getter) reads as - // absent and the name falls back to `'Error'` — never invoked. - protoHandle = handle.getPrototypeOf(); - if ( - protoHandle && - !protoHandle.isNull && - !protoHandle.isUndefined && - !protoHandle.isProxy - ) { - const protoName = readOwnDataProperty(protoHandle, 'name'); - if (protoName) { - try { - if (protoName.typeof === 'string') name = protoName.toString(); - } finally { - protoName.dispose(); - } - } - } - } - const message = - messageHandle && messageHandle.typeof === 'string' ? messageHandle.toString() : ''; - const stack = - stackHandle && stackHandle.typeof === 'string' ? stackHandle.toString() : undefined; - const replCallId = - replCallIdHandle && replCallIdHandle.typeof === 'string' - ? replCallIdHandle.toString() - : undefined; - const replBackend = - replBackendHandle && replBackendHandle.typeof === 'string' - ? replBackendHandle.toString() - : undefined; - return classifyError(name ?? 'Error', message, stack, replCallId, replBackend); - } finally { - nameHandle?.dispose(); - messageHandle?.dispose(); - stackHandle?.dispose(); - replCallIdHandle?.dispose(); - replBackendHandle?.dispose(); - protoHandle?.dispose(); - } -} - -/** - * Module-scoped shim registry, populated by the `ReplVm` constructor (see - * there for why the public type graph must not name quickjs-wasi types). - * WeakMap: disposed VMs drop out with their instances. - */ -const vmShims = new WeakMap(); - -/** - * The underlying quickjs-wasi instance of a VM. **Internal** — the bridge - * and previewer modules cast this to the shim's `QuickJS`; not part of the - * package's public API (not re-exported from the index). The `unknown` - * return keeps quickjs-wasi types out of the published declarations. - */ -export function getVmShim(vm: ReplVm): unknown { - return vmShims.get(vm); -} diff --git a/packages/repl-engine/src/wasm-ambient.d.ts b/packages/repl-engine/src/wasm-ambient.d.ts deleted file mode 100644 index 2729c4c4..00000000 --- a/packages/repl-engine/src/wasm-ambient.d.ts +++ /dev/null @@ -1,18 +0,0 @@ -/** - * Minimal ambient types for the WebAssembly surface this package touches. - * - * The repo's base tsconfig lib is `ES2022 + ESNext.Disposable` (no DOM), so - * TypeScript provides neither the global `WebAssembly` namespace nor the - * `BufferSource` type here. `quickjs-wasi`'s own declarations reference - * both, but dependency declarations are skipped (`skipLibCheck`), while - * this package's checked code uses them directly — so this file declares - * just the pieces we use instead of pulling the entire DOM lib in. At - * runtime the `WebAssembly` global is provided by Node. - */ - -type BufferSource = ArrayBufferView | ArrayBuffer; - -declare namespace WebAssembly { - interface Module {} - function compile(bytes: BufferSource): Promise; -} diff --git a/packages/repl-engine/src/workspace.ts b/packages/repl-engine/src/workspace.ts deleted file mode 100644 index 22aa296c..00000000 --- a/packages/repl-engine/src/workspace.ts +++ /dev/null @@ -1,1109 +0,0 @@ -/** - * The workspace layer of the REPL engine. - * - * One VM per workspace. The workspace object owns the VM lifecycle: - * `create` (instantiate the VM), `evalCode` (eval + job drain), `drainJobs` - * (a settlement drain), and `dispose`. Workspaces are keyed by project - * directory, mirroring the daemon's project model that the `repl` tool - * (a later phase) addresses them by; the `WorkspaceRegistry` enforces the - * one-VM-per-workspace invariant — including under concurrent first - * touches, by deduplicating the in-flight creation promise rather than - * creating duplicate VMs and disposing losers (review regression: two - * concurrent `get('/same')` calls instantiated two VMs before one was torn - * down, violating the invariant and multiplying memory use). - */ - -import { JSValueHandle, type QuickJS } from 'quickjs-wasi'; - -import { - provenanceBootstrap, - provenanceRecord, - provenanceView, - baselineLexicalKeys, - type ProvenanceOrigin, - type ProvenanceView, -} from './provenance.js'; -import { SnapshotRestoreError } from './snapshot-envelope.js'; -import { ReplVm, getVmShim, loadShippedWasm, type ReplDrainOptions, type ReplEvalOptions, type ReplEvalOutcome, type ReplJobLease } from './vm.js'; -import type { ReplSnapshot, WasmInput } from './types.js'; -import { instrumentTopLevelAwaits } from './await-instrument.js'; -import { - GUEST_LEASE_GLOBAL, - clearContinuationLease, - installGuestBridge, - readContinuationLease, - readGuestSurface, - readRealmSlotTypeToken, - registerGuestHostCallbacks, - type ConsoleEvent, - type GuestBridgeHandlers, - type GuestCall, - type GuestSurface, -} from './bridge.js'; -import { rawLexicalKeys } from './global-lexical.js'; -import { headTailDescription, inspectGlobal, manifestBinding } from './preview.js'; -import { rawOwnKeys } from './trapfree.js'; - -/** One user binding of the workspace manifest (see `Workspace.manifest`). */ -export interface WorkspaceBinding { - name: string; - /** Structure-only token (type/shape/size — never value content), or - * `agent handle` for a live agent handle. */ - token: string; - /** The machine-readable structure-only type label (see preview.ts's - * `manifestTypeLabel`): `string`, `number`, `object`, `array`, - * `agent handle`, … — the structured manifest's type field, so a - * structured consumer never has to parse the token (phase-E review - * round 4: the type used to live only inside the formatted token). */ - type: string; - /** The trap-free byte-size estimate of the binding's value (the doc's - * manifest contract: every top-level binding reports name, type, AND - * size; 0 only for the unreadable accessor/sabotage cases). */ - sizeBytes: number; - /** The stable call id when the binding is an agent handle (the broker - * appends the live-handle status from the call store); null otherwise. */ - handleCallId: string | null; - /** The sanitized provenance label (`eval 3`, `worker c2`, `session - * restore`), or null when untracked. */ - provenance: string | null; - /** Wall clock of the provenance attribution (ms since epoch). */ - provenanceAtMs: number | null; -} - -/** The workspace manifest — `ls` for the data plane (see `Workspace.manifest`). */ -export interface WorkspaceManifest { - /** Every user top-level binding, sorted by name. */ - bindings: WorkspaceBinding[]; - /** The `$N` log-ref globals as a range — always empty since 0.4.0 - * (the `$N` capture system is deleted); the field stays for - * report-shape compatibility with the older manifest surface. */ - logs: { first: number | null; last: number | null; count: number }; - /** The registry's snapshot-durable eval counter. */ - evalSeq: number; -} - -export type { ReplDrainOptions, ReplEvalOptions, ReplEvalOutcome } from './vm.js'; -export type { WasmInput, WasmModule } from './types.js'; - -/** Options for creating a workspace (per-VM configuration). */ -export interface WorkspaceOptions { - /** - * WASM bytes or a pre-compiled module (`WasmInput` — a self-contained - * stand-in for `WebAssembly.Module | BufferSource`; the published type - * graph must not depend on the consumer's lib, see `types.ts`). - * Defaults to the `quickjs-wasi` package's shipped `quickjs.wasm` - * binary. - */ - wasm?: WasmInput; - /** - * Per-VM malloc limit in bytes. Resource limits are server - * configuration, invisible to the guest; the default is - * `ReplVm.DEFAULT_MEMORY_LIMIT` (64 MiB). - */ - memoryLimit?: number; - /** - * Host handlers for the guest bridge, installed at VM creation — the - * doc's injection discipline (the library and its `__host_*` callbacks - * are in place from the first eval on; a workspace never exposes the - * DSL as undefined). When omitted, the workspace installs its default - * **parking bridge**: agent/checkpoint/steer calls park (they pend in - * the guest registry, visible through `surface()`/`parkedCalls()`, and - * stay unsolved until a later phase attaches real backends — parking - * never fabricates a result). The one deliberate exception is - * `checkpoint.answer`: answering a parked question settles the matching - * pending checkpoint first-wins (the data plane interrupting the intent - * plane works even with no backends attached). Console events - * accumulate in `consoleEvents()`. A later phase that wires real - * backends swaps handlers via `registerGuestHostCallbacks` (the same - * re-registration the restore path uses). - */ - handlers?: GuestBridgeHandlers; -} - -/** - * A persistent JavaScript REPL workspace: one QuickJS-in-WASM VM plus the - * per-workspace policy around it. State persists between evals because it - * lives in the VM, not in a transcript. - */ -export class Workspace { - /** The project directory this workspace belongs to. */ - readonly projectDir: string; - /** The configured per-VM malloc limit in bytes. */ - readonly memoryLimit: number; - - private readonly vm: ReplVm; - private readonly consoleEventBuffer: ConsoleEvent[] = []; - private readonly parkedCallsBuffer = new Map(); - /** The fresh-realm baseline key set (the manifest's user-binding - * difference; captured once per process, see `provenance.ts`). */ - private readonly baselineKeysSet: Set; - /** The fresh-realm baseline GLOBAL LEXICAL key set (top-level - * `let`/`const`/`class` bindings the library itself carries — empty - * on the shipped library; see `provenance.ts`'s `baselineLexicalKeys`). */ - private readonly baselineLexicalKeysSet: Set; - /** The fresh-realm baseline TYPE TOKENS (name → trap-free `typeof` - * token of the pristine value): the manifest's changed-binding - * detector — a user REBINDING of a baseline global (`Math = 42`) - * changes the token, so the binding is listed with its provenance - * (phase-E review rejection: the baseline filter hid overwritten - * built-ins entirely). */ - private readonly baselineTypes = new Map(); - /** Parked CHECKPOINT calls only, by call id — the answer-delivery - * table of the parking bridge. Kept separate from `parkedCallsBuffer` - * so `checkpoint.answer` can settle a pending question first-wins - * without ever touching a parked agent/steer call that happens to - * share the id space (review regression: the bridge rejected every - * answer with `false`, leaving the original promise pending forever). */ - private readonly parkedCheckpointCalls = new Map(); - /** The parking bridge's per-call KIND (agent/steer/checkpoint) and - * per-call question text — the introspection handlers (`workspace()`/ - * `agents()`) serve them in the §4.5 shapes. */ - private readonly parkedKinds = new Map(); - private readonly parkedQuestions = new Map(); - /** The parking bridge's per-call AGENT data (the §4.5 agents() shape - * serves the real model spec and task — a parked call records what - * the guest asked for; never fabricated empties). */ - private readonly parkedModelSpecs = new Map(); - private readonly parkedTasks = new Map(); - /** The parking bridge's code-phase/current-eval `reset()` request. */ - private pendingReset = false; - /** The parking bridge's retained SUSPENDED-eval completions (one per - * suspended eval — `eval` keeps the wrapper; the drain's sweep reads - * the settled value into `_` and releases it). */ - private readonly suspendedCompletions = new Set(); - /** Continuation identity for suspended parking-bridge evals. */ - private readonly suspendedEvalTokens = new Map(); - /** Suspended eval completions whose own code/continuation called reset. */ - private readonly resetOwningCompletions = new Set(); - private parkingEvalTokenSeq = 0; - private currentParkingEvalToken: string | undefined; - private inParkingEval = false; - /** The guest continuation lease mirrored per drained job. */ - private readonly parkingJobLease: ReplJobLease = { - read: () => this.readContinuationLease(), - clear: () => this.clearContinuationLease(), - cell: { current: undefined }, - }; - private disposed = false; - - private constructor(projectDir: string, vm: ReplVm, baseline: Set) { - this.projectDir = projectDir; - this.vm = vm; - this.memoryLimit = vm.memoryLimit; - this.baselineKeysSet = baseline; - this.baselineLexicalKeysSet = new Set(); - } - - /** - * Create a workspace: instantiate its VM (defaulting to the shipped - * `quickjs.wasm` binary and the default memory limit), install the - * guest bridge — the version-marked library plus its `__host_*` - * callbacks — so the DSL (`agent`, `checkpoint`, `console`, the - * combinators) is live from the first eval on (the doc's injection - * discipline; see `WorkspaceOptions.handlers` for the default parking - * bridge), and bootstrap the per-binding provenance registry (the - * workspace manifest's provenance seam; a fresh workspace starts with - * the baseline `known` set and no origins — the first eval's - * maintenance pass attributes its bindings to `eval 1`). - */ - static async create(projectDir: string, options: WorkspaceOptions = {}): Promise { - const wasm = options.wasm ?? (await loadShippedWasm()); - const vm = await ReplVm.create({ wasm, memoryLimit: options.memoryLimit }); - const workspace = new Workspace(projectDir, vm, new Set()); - await installGuestBridge(vm, options.handlers ?? workspace.defaultHandlers()); - const [bootstrap, lexicalBaseline] = await Promise.all([provenanceBootstrap(vm, wasm), baselineLexicalKeys(wasm)]); - workspace.baselineKeysSet.clear(); - for (const key of bootstrap.baseline) workspace.baselineKeysSet.add(key); - for (const key of lexicalBaseline) workspace.baselineLexicalKeysSet.add(key); - for (const [name, token] of bootstrap.baselineTypes) workspace.baselineTypes.set(name, token); - return workspace; - } - - /** - * Restore a workspace from a quickjs-wasi snapshot: the VM is restored - * and the host callbacks are re-registered by name (`registerGuestHostCallbacks` - * — the guest library and its pending-call registry travel INSIDE the - * snapshot; the library is never re-evaluated). The - * provenance registry travels with the snapshot too; a PRE-PROVENANCE - * snapshot (whose library predates the registry) gets the registry - * installed by the host bootstrap and its pre-existing bindings are - * swept as `session restore` — "first seen at restore", never a - * guessed origin. This is the restore-path constructor the daemon - * layer (a later phase) uses with the identity-enveloped snapshots; it - * exists now so the settlement machinery (store → guest exactly-once - * delivery across a simulated crash) is testable at the workspace - * boundary. - * - * A payload that PASSED every envelope check (hash, version, gzip, - * shape, pointer bounds) but cannot be materialized — a corrupted - * in-range VM header (a pointer patched to a wrong-but-in-bounds - * value), garbage the shim's binary parse accepted, a guest surface - * that cannot be rehosted, a provenance registry that cannot - * bootstrap — REFUSES as `SnapshotRestoreError` (code - * `RESTORE_CORRUPT`, the envelope family's restore-time member), and - * any partially created VM is DISPOSED before the refusal propagates - * (phase-D review rejection: the callback/provenance initialization - * used to throw with the half-built VM still live, and the raw - * `RuntimeError` leaked past the daemon's `SnapshotEnvelopeError` - * containment, so every subsequent touch retried the restore into - * garbage). The refusal is single-shot and coded — the daemon records - * it as a stable refusal and never crash-loops. - */ - static async restore(projectDir: string, snapshot: ReplSnapshot, options: WorkspaceOptions = {}): Promise { - const wasm = options.wasm ?? (await loadShippedWasm()); - let vm: ReplVm; - try { - vm = await ReplVm.restore(snapshot, { wasm, memoryLimit: options.memoryLimit }); - } catch (error) { - // The VM never materialized (nothing to dispose): the shim's - // restore choked on the payload — a structurally valid envelope - // whose in-range header or memory is garbage. Raise the coded - // refusal with the underlying failure named, never a raw wasm - // `RuntimeError` (the daemon's containment catches the envelope - // family only). - throw new SnapshotRestoreError( - `restoring the workspace VM from the snapshot failed (${(error as Error)?.message ?? String(error)})`, // eslint-disable-line max-len - { cause: error }, - ); - } - const workspace = new Workspace(projectDir, vm, new Set()); - try { - registerGuestHostCallbacks(vm, options.handlers ?? workspace.defaultHandlers()); - const [bootstrap, lexicalBaseline] = await Promise.all([provenanceBootstrap(vm, wasm), baselineLexicalKeys(wasm)]); - workspace.baselineKeysSet.clear(); - for (const key of bootstrap.baseline) workspace.baselineKeysSet.add(key); - for (const key of lexicalBaseline) workspace.baselineLexicalKeysSet.add(key); - for (const [name, token] of bootstrap.baselineTypes) workspace.baselineTypes.set(name, token); - if (bootstrap.created) { - // The pre-provenance restore sweep: attribute bindings that existed - // before this host started tracking. - provenanceRecord(vm, { kind: 'restore' }); - } - return workspace; - } catch (error) { - // The VM EXISTS but cannot be rehosted/bootstraped: dispose it - // (a partial VM must never be left live — phase-D review - // rejection) and raise the same coded refusal. - vm.dispose(); - throw new SnapshotRestoreError( - `initializing the restored workspace failed (${(error as Error)?.message ?? String(error)})`, // eslint-disable-line max-len - { cause: error }, - ); - } - } - - /** - * Snapshot the workspace's VM: raw WASM linear memory plus runtime - * pointers (the quickjs-wasi snapshot). The guest library and the - * pending-call registry travel inside; the host callbacks do not - * (re-register by name after restore — `rehost`). The at-rest - * identity envelope (wasm hash + format version + gzip) is the daemon - * layer's wrap, a later phase; this is the raw snapshot seam. - */ - snapshot(): ReplSnapshot { - this.assertAlive(); - return (getVmShim(this.vm) as QuickJS).snapshot(); - } - - /** - * Evaluate a script in the workspace's VM: eval + job drain + completion - * report. See `ReplVm.evalCode` for the outcome shapes. The returned - * promise is fulfilled synchronously (the VM layer performs no `await`), - * so an eval cannot race `dispose()`. - * - * The §4.4 result-history global is maintained HERE (the workspace - * level owns `_` for its own evals — the broker sets it for its evals - * through the same `setGlobal` seam): a RESOLVED eval's completion - * value becomes `_` — an undefined completion (an empty poll) makes - * `_` undefined too: the previous eval's completion value IS - * undefined (the review probe: `42`, then `""`, then `_` must read - * undefined, never the stale 42). An error leaves `_` unchanged; a - * SUSPENDED eval retains its completion wrapper and `drainJobs`'s - * sweep sets `_` once the continuation settles. - * - * The parking bridge's `reset()` teardown runs AFTER the current eval - * completes (the doc's §4.5): a completed eval disposes now; a - * SUSPENDED eval keeps the workspace alive until its continuation - * settles at the drain (`drainJobs`'s sweep), then disposes — the - * continuation runs to completion first, and the eval that called - * reset() is the one whose completion owes the teardown. - */ - eval(code: string, options?: ReplEvalOptions): Promise { - this.assertAlive(); - // `cell.current` names a continuation only while that continuation's - // job executes. A completed earlier drain must never make this eval's - // synchronous code look like the earlier suspended eval. - this.parkingJobLease.cell.current = undefined; - const token = `w${++this.parkingEvalTokenSeq}`; - this.currentParkingEvalToken = token; - this.pendingReset = false; - let instrumented = code; - try { - const surface = this.surface(); - if (surface?.supportsContinuationLease === true) { - instrumented = instrumentTopLevelAwaits(code, token, { - wrapIterables: surface.supportsIterableLease === true, - }); - } - } catch { - // A legacy/broken surface keeps native await semantics; reset() - // during the current eval is still handled by the local flag. - } - const usesParkingJobLease = instrumented !== code; - this.inParkingEval = true; - let evaluated: ReturnType; - try { - evaluated = this.vm.evalCodeWithCompletion(instrumented, { - ...options, - jobLease: options?.jobLease ?? (usesParkingJobLease ? this.parkingJobLease : undefined), - }); - } finally { - this.inParkingEval = false; - this.parkingJobLease.cell.current = undefined; - } - const { outcome, completion } = evaluated; - const resetByThisEval = this.pendingReset; - this.pendingReset = false; - if (completion !== undefined) { - if (outcome.kind === 'value') { - try { - // Every resolved eval's completion value becomes `_` — even - // undefined (an empty script — the documented poll idiom — - // overwrites the older value with undefined). - this.setGlobal('_', completion); - } catch { - // A failed `_` write must not fail the eval that produced the - // value. - } - (completion as JSValueHandle).dispose(); - } else { - // A suspended eval: retain the wrapper — the drain's sweep reads - // the settled value into `_` and releases the handle. The - // reset-owning completion is tracked separately (a reset() the - // eval called owes its teardown only once THIS eval completes). - this.suspendedCompletions.add(completion as JSValueHandle); - this.suspendedEvalTokens.set(completion as JSValueHandle, token); - if (resetByThisEval) this.resetOwningCompletions.add(completion as JSValueHandle); - } - } - // The teardown for an eval that COMPLETED within this call runs now - // (its output above is already captured); a suspended eval's - // teardown runs at the drain (see `drainJobs`). Earlier suspended - // reset-owning evals remain tracked until their own completion. - if (outcome.kind !== 'pending' && resetByThisEval) { - this.dispose(); - } - return Promise.resolve(outcome); - } - - /** - * @internal Package-internal eval seam for the broker layer: like - * `eval`, but when the eval RESOLVED the live completion-value handle - * comes back alongside the shallow snapshot (`completion` — an opaque - * quickjs-wasi `JSValueHandle`, OWNED BY THE CALLER, who must dispose it - * after previewing; for a SUSPENDED eval (the completion pending on a - * host call) it is the eval wrapper promise handle — the caller's - * active-eval probe (the wrapper settles when the continuation - * completes or is broken; the caller owns and must dispose it); - * `undefined` for error outcomes). The broker - * previews it trap-free for the tool result's `result` line. Not part of - * the published API (not re-exported from the index); `completion` is - * typed `unknown` so the public declaration graph stays self-contained. - */ - evalWithCompletion( - code: string, - options?: ReplEvalOptions, - ): { outcome: ReplEvalOutcome; completion?: unknown; interruptedInDrain?: boolean } { - this.assertAlive(); - return this.vm.evalCodeWithCompletion(code, options); - } - - /** - * Re-register the four `__host_*` callbacks by name — the same - * re-registration the snapshot-restore path uses. This is the seam the - * broker (a later phase wires real backends) takes over a workspace - * with: the guest library and its pending-call registry are untouched - * (never re-injected), and the guest's `issueCall` looks the host - * function up by name at call time, so replacing the host-side - * trampoline routes every subsequent call to the new handler. Safe on a - * live VM and on a restored one. - */ - rehost(handlers: GuestBridgeHandlers): void { - this.assertAlive(); - registerGuestHostCallbacks(this.vm, handlers); - } - - /** - * Run the job drain loop (settle what can be settled). Used after an - * eval that suspended, when host-side settlement (subagent calls in a - * later phase) has made progress. Because a suspended eval's interrupt - * handler is no longer armed, the drain accepts its own per-drain - * `interruptHandler` so a resumed runaway continuation stays bounded. - * - * After the drain, the RETAINED-SUSPENDED-EVAL sweep runs: a - * continuation that completed during the drain is the PREVIOUS eval — - * its completion value becomes `_` — and a reset() the settled eval - * requested tears the workspace down now that the eval completed (the - * §4.5 host-side effect). - */ - drainJobs(options?: ReplDrainOptions): number { - this.assertAlive(); - const count = this.vm.drainJobs({ - ...options, - jobLease: - options?.jobLease ?? - (this.suspendedEvalTokens.size > 0 ? this.parkingJobLease : undefined), - }); - this.sweepSuspendedEvals(); - return count; - } - - /** - * One pass over the retained suspended-eval completions (the parking - * bridge's `_` / reset() bookkeeping): a settled wrapper's fulfilled - * value becomes `_` (a rejection — the eval errored late — leaves `_` - * unchanged), the handle is released, and when the reset-requesting - * eval's own completion settled the teardown runs (the workspace is - * disposed after its last eval completed). Runs after every drain; - * between VM operations only. - */ - private sweepSuspendedEvals(): void { - if (this.suspendedCompletions.size === 0) return; - let resetOwnerCompleted = false; - for (const completion of [...this.suspendedCompletions]) { - if (completion.promiseState === 0) continue; // still pending - this.suspendedCompletions.delete(completion); - this.suspendedEvalTokens.delete(completion); - if (this.resetOwningCompletions.delete(completion)) resetOwnerCompleted = true; - try { - const value = this.vm.readRetainedCompletion(completion) as JSValueHandle | undefined; - if (value !== undefined) { - try { - this.setGlobal('_', value); - } catch { - // A failed `_` write must not fail the drain. - } - value.dispose(); - } - } catch { - // Best-effort bookkeeping: a hostile completion shape must not - // break the drain. - } - completion.dispose(); - } - if (resetOwnerCompleted) { - // The reset-requesting eval completed (its continuation settled at - // this drain): the teardown runs now — the host-side effect the - // deleted `reset` action performed. - this.pendingReset = false; - this.dispose(); - } - } - - /** - * Teardown: dispose the VM and drop all stored state. In-flight work is - * cancelled by the caller (the tool layer); the VM itself is gone after - * this. Because every VM operation is synchronous, no operation can be - * in flight when this runs. - */ - dispose(): void { - if (this.disposed) return; - this.disposed = true; - for (const completion of this.suspendedCompletions) completion.dispose(); - this.suspendedCompletions.clear(); - this.suspendedEvalTokens.clear(); - this.resetOwningCompletions.clear(); - this.currentParkingEvalToken = undefined; - this.parkingJobLease.cell.current = undefined; - this.vm.dispose(); - } - - /** Support for `using` declarations (Explicit Resource Management). */ - [Symbol.dispose](): void { - this.dispose(); - } - - /** True once `dispose()` has been called. */ - get isDisposed(): boolean { - return this.disposed; - } - - /** - * @internal The continuation-lease READ seam (the broker's eval-break - * targeting identity — see `ReplJobLease` in vm.ts): reads the guest - * library's `__replLease` accessor between VM operations. Best-effort - * (a missing/broken accessor reads as `undefined`). - */ - readContinuationLease(): string | undefined { - this.assertAlive(); - return readContinuationLease(this.vm); - } - - /** - * @internal Write a live guest value into a realm global slot — the - * `_` seam (the broker sets the previous eval's completion value; see - * `ReplVm.setGlobal`). - */ - setGlobal(name: string, value: unknown): void { - this.assertAlive(); - this.vm.setGlobal(name, value); - } - - /** - * @internal Read a RETAINED suspended-eval completion wrapper after it - * settled (the broker's active-eval sweep, and this workspace's own - * parking-bridge sweep): the fulfilled completion value handle, - * owned by the caller (dispose after use), or undefined for a - * rejected/still-pending completion (`_` stays unchanged then). See - * `ReplVm.readRetainedCompletion`. Called between VM operations. - */ - readRetainedCompletion(completion: unknown): unknown { - this.assertAlive(); - return this.vm.readRetainedCompletion(completion as JSValueHandle); - } - - /** - * @internal The continuation-lease CLEAR seam (see - * `readContinuationLease`): the drain loop clears the lease at drain - * start and after every lease-carrying job. - */ - clearContinuationLease(): void { - this.assertAlive(); - clearContinuationLease(this.vm); - } - - /** - * The console events accumulated by the default parking bridge, in - * order (only populated when `options.handlers` was omitted — custom - * handlers own their events). Each event carries the guest-rendered - * ONE line per console.* call. - */ - consoleEvents(): readonly ConsoleEvent[] { - return this.consoleEventBuffer; - } - - /** - * The parked host calls of the default parking bridge, by call id - * (only populated when `options.handlers` was omitted). A later phase - * that attaches real backends settles these `GuestCall`s (or takes - * them over) — parking is the honest no-backend state, it never - * fabricates results. - * - * Parked checkpoint QUESTIONS are in this map too, but answers do NOT - * arrive through here: `checkpoint.answer` settles the matching - * pending checkpoint directly (see `defaultHandlers`) — the entry is - * removed from both maps on delivery, so this map only ever lists - * still-parked calls. - */ - parkedCalls(): ReadonlyMap { - return this.parkedCallsBuffer; - } - - /** - * The guest library's reconciliation surface — the host's door back - * into the pending-call registry (`pending`/`settle`/`stats`, used by - * `status` and by the post-restore reconciliation loop). `undefined` - * only when the bridge is not installed (it always is on workspaces - * created through this class). - */ - surface(): GuestSurface | undefined { - this.assertAlive(); - return readGuestSurface(this.vm); - } - - /** - * Content-free metadata for one realm global slot — the workspace- - * manifest seam (`status`): name, type, size; metadata, never content. - */ - inspectBinding(name: string): { kind: 'data' | 'accessor' | 'absent'; label: string; sizeBytes: number } { - this.assertAlive(); - return inspectGlobal(this.vm, name); - } - - /** - * One maintenance pass of the per-binding provenance registry (the - * workspace manifest's provenance seam): attribute new/rebound user - * bindings to the operation that created them. The broker drives this - * after each eval and each settlement drain; `Workspace.restore` sweeps - * pre-provenance snapshots itself. Orientation metadata only — never - * errors upward. - */ - provenanceRecord(origin: ProvenanceOrigin): void { - this.assertAlive(); - provenanceRecord(this.vm, origin); - } - - /** - * The sanitized provenance registry (see `provenance.ts`): which eval - * or worker settlement created/rebound each user binding, with the - * registry's eval counter. The manifest renderer's provenance seam. - */ - provenanceView(): ProvenanceView { - this.assertAlive(); - return provenanceView(this.vm); - } - - /** - * The workspace manifest — `ls` for the data plane (the roadmap doc's - * `status` manifest): every user top-level binding (fresh-realm - * baseline set difference — the guest library's own globals and the - * realm builtins are never listed — plus user bindings that SHADOW or - * OVERWRITE baseline globals: a lexical declaration always wins over - * the baseline (a user `const Math = 42` is listed with the lexical - * value's metadata), and a baseline global whose value's type token - * changed from the fresh-realm baseline has been rebinding by user - * code and is listed too — phase-E review round 5: the baseline - * filter used to remove both; a baseline global whose VALUE is no - * longer the pristine baseline object (a SAME-TYPE overwrite — - * `Math = { userOwned: true }` keeps the `object` token) is listed - * the same way through the registry's changed-known read — phase-E - * review rejection round 6: the token-only detector missed same-type - * replacements entirely), with its structure-only token - * (type/shape/size — metadata, never content), its provenance label - * (`via eval N` / `via worker cN` — null when untracked), and the - * live-handle call id when the binding is an agent handle (the caller - * — the broker — appends the handle status from the call store). The - * GLOBAL LEXICAL bindings (top-level `let`/`const`/`class` — the - * roadmap's canonical `const research = agent(...)` state) are - * enumerated too, through the engine's internal global-var object - * (see `global-lexical.ts`): lexical bindings are not global-object - * properties, and they SHADOW a same-named global-object property for - * identifier resolution, so a name present in both lists yields ONE - * binding — the lexical view (what the orchestrator's code sees). The - * `$N` log-ref globals are listed separately as a range, mirroring the - * harness manifest's logs breakdown. Trap-free throughout: descriptor - * reads only, accessors never invoked. - */ - manifest(): WorkspaceManifest { - this.assertAlive(); - const baseline = this.baselineKeys(); - const lexicalBaseline = this.baselineLexicalKeys(); - const lexicalKeys = new Set(rawLexicalStringKeys(this.vm)); - const names = unionNames(rawGlobalStringKeys(this.vm), rawLexicalStringKeys(this.vm)); - const view = provenanceView(this.vm); - const user = names.filter((name) => { - if (!baseline.has(name) && !lexicalBaseline.has(name)) return true; - // A GLOBAL LEXICAL binding SHADOWS a same-named baseline global - // for identifier resolution (a user `const Math = 42`): the - // binding the orchestrator's code sees is the user's, so the - // manifest lists it — the lexical view, the same rule as the - // one-binding-per-name union (phase-E review rejection: the - // baseline filter removed shadowing bindings entirely). - if (lexicalKeys.has(name) && !lexicalBaseline.has(name)) return true; - // A baseline GLOBAL REBINDING (a `Math = 42` assignment): the - // value's trap-free type token changed from the fresh-realm - // baseline — the user overwrote the built-in, and the manifest - // lists the overwrite like any other user binding (phase-E - // review rejection: overwritten built-ins were hidden). A - // SAME-TYPE overwrite (`Math = { userOwned: true }` — both - // values are objects, so the token cannot see it) is caught by - // the registry's changed-known list, which compares the current - // value against the ORIGINAL baseline value (SameValue — - // phase-E review rejection round 6: same-type overwrites stayed - // absent from the manifest with no provenance). - if (this.baselineChanged(name) || view.changed.has(name)) return true; - return false; - }); - const bindings: WorkspaceBinding[] = []; - for (const name of user) { - const info = manifestBinding(this.vm, name); - if (info === null) continue; - const origin = view.origins.get(name); - bindings.push({ - name, - token: info.token, - type: info.type, - sizeBytes: info.sizeBytes, - handleCallId: info.handleCallId, - provenance: origin === undefined ? null : origin.via, - provenanceAtMs: origin === undefined ? null : origin.at, - }); - } - bindings.sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0)); - return { - bindings, - // The `$N` capture system is deleted (0.4.0): the range is always - // empty. Kept for report-shape compatibility with the older - // manifest surface. - logs: { first: null, last: null, count: 0 }, - evalSeq: view.evalSeq, - }; - } - - private baselineKeys(): Set { - return this.baselineKeysSet; - } - - private baselineLexicalKeys(): Set { - return this.baselineLexicalKeysSet; - } - - /** Whether a baseline name's current type token differs from the - * fresh-realm baseline's (a user rebinding of the built-in). Trap- - * free: descriptor read only, never a `[[Get]]`. Same-type - * replacements (`Math = { userOwned: true }`) keep the token and - * are detected by the registry's changed-known list instead (see - * `manifest`) — the value identity the token cannot provide. */ - private baselineChanged(name: string): boolean { - const baselineType = this.baselineTypes.get(name); - if (baselineType === undefined) return false; - try { - return readRealmSlotTypeToken(this.vm, name) !== baselineType; - } catch { - return false; - } - } - - private defaultHandlers(): GuestBridgeHandlers { - const events = this.consoleEventBuffer; - const parked = this.parkedCallsBuffer; - const parkedCheckpoints = this.parkedCheckpointCalls; - const parkedKinds = this.parkedKinds; - const parkedQuestions = this.parkedQuestions; - const parkedModelSpecs = this.parkedModelSpecs; - const parkedTasks = this.parkedTasks; - const workspace = this; - return { - agent: (call, callId, modelSpec, task) => { - parked.set(callId, call); - parkedKinds.set(callId, 'agent'); - // The §4.5 agents() shape serves the call's REAL model spec and - // task (the parking bridge keeps them verbatim; a later broker - // that takes over the workspace reads them from the registry - // surface). - parkedModelSpecs.set(callId, modelSpec); - parkedTasks.set(callId, task); - }, - checkpoint: (call, callId, question, _optionsJson, answerJson) => { - if (answerJson !== null) { - // Answer mode: the orchestrator is delivering the user's answer - // for a parked checkpoint. Find the matching pending checkpoint - // (checkpoints are tracked SEPARATELY from parked agent/steer - // calls — a checkpoint.answer must never settle an agent call - // that shares the id space), parse the JSON answer, settle the - // call, and report delivery. First-wins: the entry is removed - // before settling, so a second delivery of the same id reports - // false (unknown or already-answered), exactly like the guest's - // idempotent settle-by-call-id. A malformed answer (a host-side - // contract violation — the guest only sends JSON.stringify - // output) rejects the call rather than parking the question - // forever. - const pending = parkedCheckpoints.get(callId); - if (pending === undefined) return false; - parkedCheckpoints.delete(callId); - parked.delete(callId); - parkedKinds.delete(callId); - parkedQuestions.delete(callId); - parkedModelSpecs.delete(callId); - parkedTasks.delete(callId); - let answer: unknown; - try { - answer = JSON.parse(answerJson); - } catch { - pending.reject( - new Error(`checkpoint ${callId}: answer was not valid JSON`), - ); - return true; - } - pending.resolve(answer); - return true; - } - // Question mode: park the call in both tables — the general - // parked-calls map (the no-backend state a later phase settles - // or takes over) and the checkpoint table (answer delivery). - parked.set(callId, call!); - parkedCheckpoints.set(callId, call!); - parkedKinds.set(callId, 'checkpoint'); - parkedQuestions.set(callId, question ?? ''); - return undefined; - }, - queue: (call, callId) => { - parked.set(callId, call); - parkedKinds.set(callId, 'queue'); - }, - steer: (call, callId) => { - parked.set(callId, call); - parkedKinds.set(callId, 'steer'); - }, - cancelSession: (call, callId) => { - parked.set(callId, call); - parkedKinds.set(callId, 'cancel'); - }, - cancelQueue: (call, callId) => { - parked.set(callId, call); - parkedKinds.set(callId, 'cancel'); - }, - console: (event) => { - events.push(event); - }, - // The eval-plane helpers under the parking bridge: `sleep` is a - // real host-side timer (the VM itself stays timer-free); the - // introspection pair serve the parking state in the doc's §4.5 - // shapes (no backends attached — diagnostics are all empty); - // `reset` marks the teardown the eval consumes after completing. - sleep: (call, ms) => { - const delay = Number.isFinite(ms) && ms > 0 ? Math.min(ms, 2 ** 31 - 1) : 0; - setTimeout(() => { - try { - call.resolve(undefined); - } catch { - // The workspace was disposed before the timer fired — the - // call is gone with it; nothing to settle. - } - }, delay); - }, - workspace: () => { - const manifest = workspace.manifest(); - const inFlightIds: string[] = []; - for (const id of parked.keys()) { - if (!inFlightIds.includes(id)) inFlightIds.push(id); - } - return JSON.stringify({ - bindings: manifest.bindings.map((b) => ({ - name: b.name, - type: b.type, - sizeBytes: b.sizeBytes, - provenance: b.provenance, - task: null, - ...(b.handleCallId !== null ? { callId: b.handleCallId } : {}), - ...(b.handleCallId !== null - ? { status: parked.has(b.handleCallId) ? 'pending' : 'settled' } - : {}), - })), - inFlight: inFlightIds, - checkpoints: [...parkedCheckpoints.keys()].map((id) => ({ - id, - question: headTailDescription(parkedQuestions.get(id) ?? '', 200), - })), - diagnostics: { reconcile: null, drainError: null, childrenClosed: false }, - }); - }, - agents: () => - JSON.stringify( - [...parked.entries()] - .filter(([callId]) => parkedKinds.get(callId) === 'agent') - .map(([callId]) => ({ - callId, - // The §4.5 shape: the call's REAL model spec and task (the - // parking bridge recorded them verbatim at issue — never - // fabricated empties). The task keeps the engine seam's - // retained 200-character metadata preview. - modelSpec: parkedModelSpecs.get(callId) ?? '', - task: metadataHeadTail(parkedTasks.get(callId) ?? '', 200), - state: 'opening', - supportsSteering: false, - queuedTurns: 0, - })), - ), - reset: () => { - const token = workspace.parkingJobLease.cell.current; - if ( - token !== undefined && - (!workspace.inParkingEval || token !== workspace.currentParkingEvalToken) - ) { - for (const [completion, evalToken] of workspace.suspendedEvalTokens) { - if (evalToken === token) { - workspace.resetOwningCompletions.add(completion); - return; - } - } - } - workspace.pendingReset = true; - }, - defaultBackend: () => undefined, - }; - } - - private assertAlive(): void { - if (this.disposed) { - throw new Error(`Workspace ${this.projectDir}: operation on a disposed workspace`); - } - } -} - -/** Retained task metadata preview: exactly `max` UTF-16 characters, - * matching the broker's existing agents() task formatting. */ -function metadataHeadTail(value: string, max: number): string { - if (value.length <= max) return value; - const keep = Math.max(0, max - 1); - const half = Math.floor(keep / 2); - return `${value.slice(0, half)}…${value.slice(value.length - (keep - half))}`; -} - -/** The realm global's string-key set, trap-free (raw own-key read). */ -function rawGlobalStringKeys(vm: ReplVm): string[] { - const shim = getVmShim(vm) as QuickJS; - return rawOwnKeys(shim.global); -} - -/** The realm's GLOBAL LEXICAL string-key set, trap-free (the internal - * global-var object's own keys — top-level `let`/`const`/`class` - * declarations; see `global-lexical.ts`). */ -function rawLexicalStringKeys(vm: ReplVm): string[] { - return rawLexicalKeys(vm); -} - -/** The union of two name lists, first-seen order (the manifest's - * binding namespace: global-object keys plus global lexical keys). */ -function unionNames(a: string[], b: string[]): string[] { - const seen = new Set(a); - const out = [...a]; - for (const name of b) { - if (!seen.has(name)) { - seen.add(name); - out.push(name); - } - } - return out; -} - -/** Options for a `WorkspaceRegistry`. */ -export interface WorkspaceRegistryOptions { - /** - * WASM bytes or module shared as the default by every workspace the - * registry creates (see `WasmInput` in `types.ts`). Defaults to the - * shipped `quickjs.wasm` binary. - */ - wasm?: WasmInput; - /** - * Default per-VM malloc limit in bytes for workspaces created without - * their own `memoryLimit` (per-workspace limits override this). - */ - memoryLimit?: number; - /** - * Default guest-bridge handlers for workspaces created without their - * own `handlers` (per-workspace handlers override this). See - * `WorkspaceOptions.handlers` for the default parking bridge. - */ - handlers?: GuestBridgeHandlers; -} - -/** An in-flight workspace creation, tracked so concurrent `get`s dedupe. */ -interface PendingCreate { - /** The creation promise every concurrent `get` for this key awaits. */ - promise: Promise; - /** - * Set by `dispose`/`disposeAll` to veto materialization: the created - * workspace is torn down without being registered, and the waiting - * caller's promise rejects. - */ - cancelled: boolean; -} - -/** - * Project-keyed registry of workspaces, enforcing one VM per workspace. - * This is the engine boundary the `repl` MCP tool (a later phase) calls: - * the tool resolves its `projectDir` argument through here, exactly like - * the daemon's project registry resolves project contexts for `workflow`. - */ -export class WorkspaceRegistry { - private readonly workspaces = new Map(); - private readonly pending = new Map(); - private readonly options: WorkspaceRegistryOptions; - - constructor(options: WorkspaceRegistryOptions = {}) { - this.options = options; - } - - /** - * Get the workspace for a project directory, creating it on first touch. - * The same project directory always yields the same workspace (and thus - * the same VM) for the lifetime of the registry. - * - * Concurrent first-touches of one key share a single in-flight creation - * promise: exactly one VM is instantiated, and both callers receive the - * same workspace. When concurrent callers pass different options, the - * first caller's options win (first-touch-wins is the registry's - * documented policy — the workspace exists once, so its configuration - * is decided once). - */ - async get(projectDir: string, options: WorkspaceOptions = {}): Promise { - const existing = this.workspaces.get(projectDir); - if (existing) return existing; - - // Deduplicate the in-flight creation itself — not just the completed - // result (review regression: two concurrent first-touches each ran a - // full `Workspace.create`, instantiating two VMs for one project). - const flight = this.pending.get(projectDir); - if (flight) return flight.promise; - - const pending: PendingCreate = { cancelled: false, promise: undefined! }; - const merged: WorkspaceOptions = { - wasm: options.wasm ?? this.options.wasm, - memoryLimit: options.memoryLimit ?? this.options.memoryLimit, - handlers: options.handlers ?? this.options.handlers, - }; - pending.promise = Workspace.create(projectDir, merged).then( - (created) => { - // Only remove our own entry: `dispose` may have removed it already - // and a later `get` may have installed a fresh one. - if (this.pending.get(projectDir) === pending) this.pending.delete(projectDir); - if (pending.cancelled) { - // `dispose`/`disposeAll` won the race: never materialize the - // workspace. Tear the fresh VM down immediately — no VM is left - // behind either way. - created.dispose(); - throw new Error(`Workspace ${projectDir}: creation cancelled by dispose`); - } - this.workspaces.set(projectDir, created); - return created; - }, - (error) => { - if (this.pending.get(projectDir) === pending) this.pending.delete(projectDir); - throw error; - }, - ); - this.pending.set(projectDir, pending); - return pending.promise; - } - - /** True when a workspace exists for this project directory. */ - has(projectDir: string): boolean { - return this.workspaces.has(projectDir); - } - - /** The number of live workspaces. */ - get size(): number { - return this.workspaces.size; - } - - /** The project directories with live workspaces. */ - keys(): string[] { - return [...this.workspaces.keys()]; - } - - /** - * Dispose and drop the workspace for a project directory. Returns true - * when a live workspace was disposed. When only a creation is in flight - * (no live workspace yet), it is cancelled: the created workspace is - * torn down without materializing and the waiting `get` caller's promise - * rejects — the invariant "dispose means the workspace is gone" holds - * even under the race, and a later `get` creates a fresh workspace. - */ - dispose(projectDir: string): boolean { - const workspace = this.workspaces.get(projectDir); - if (workspace) { - this.workspaces.delete(projectDir); - workspace.dispose(); - return true; - } - const flight = this.pending.get(projectDir); - if (flight) { - this.pending.delete(projectDir); - flight.cancelled = true; - } - return false; - } - - /** Dispose and drop every workspace; cancel every in-flight creation. */ - disposeAll(): void { - for (const workspace of this.workspaces.values()) { - workspace.dispose(); - } - this.workspaces.clear(); - for (const flight of this.pending.values()) { - flight.cancelled = true; - } - this.pending.clear(); - } -} diff --git a/packages/repl-engine/test/broker.test.ts b/packages/repl-engine/test/broker.test.ts deleted file mode 100644 index e68eda7d..00000000 --- a/packages/repl-engine/test/broker.test.ts +++ /dev/null @@ -1,2656 +0,0 @@ -/** - * Broker tests (phase C): the broker wires a workspace's guest bridge to - * ACP subagent sessions through acp-agents. Pins the doc's deliverables - * against a FAKE runner/session (a real backend needs live credentials — - * the capability negotiation and wire behavior are mocked structurally): - * - * - the eval tool-result shapes (resolved / suspended / rejected), - * - the agent call round trip with continuation-at-settlement, - * - exactly-once settlement, including a simulated crash between the - * store write and the guest settlement (both the live retry and the - * snapshot/restore + reconcile path), - * - late uncaught rejections surfacing as error-level console lines in - * the next tool result, - * - the checkpoint round trip (raise → previewed question → answer in a - * later eval → settlement within that eval), - * - steering outcome visibility: a backend WITH and WITHOUT the - * `_session/steering` extension (injected / queued / startedNewTurn / - * failed / cancelled / idle), - * - the concurrency cap (dispatch-time refusal, slot release on - * settlement), - * - trap-free result rendering and the output caps. - */ - -import assert from 'node:assert/strict'; -import { mkdtempSync, rmSync } from 'node:fs'; -import { tmpdir } from 'node:os'; -import { join } from 'node:path'; -import { test } from 'node:test'; - -import { - Broker, - InMemoryCallStore, - JsonlCallStore, - Workspace, - type BrokerOpenSessionOptions, - type BrokerLoadSessionOptions, - type BrokerPromptOptions, - type BrokerRunner, - type BrokerSession, - type BrokerTurn, - type CallOutcome, - type CallStore, - type ReplEvalResult, -} from '../src/index.js'; - -const PROJECT = '/tmp/repl-broker-project'; - -/** Let queued host microtasks (openSession continuations, readiness - * flags) run. */ -async function tick(): Promise { - await new Promise((resolve) => setTimeout(resolve, 0)); -} - -/** A fake held-open ACP session: the test drives turns and steer calls. - * Models acp-agents' preflight/handoff split: when `released` is set, - * prompt() rejects WITHOUT firing the handoff acknowledgment (the - * backend prompt is never invoked — the async pre-handoff rejection - * the review probe hit); otherwise it accepts the prompt. The seam - * order models the FIXED acp-agents contract (the crash-boundary - * regression): the backend prompt is registered (invoked) FIRST, and - * the handoff acknowledgment fires only after it — so a delivered - * marker recorded by the broker can never precede the hand-off. The - * `dieBeforeHandoff`/`dieAfterHandoff` modes simulate a process death - * in the seam itself. */ -class FakeSession implements BrokerSession { - readonly sessionId: string; - initializeMeta: Readonly> | undefined; - readonly prompts: Array<{ content: string; promptMeta?: Record; resolve: (turn: BrokerTurn) => void; reject: (error: unknown) => void }> = []; - readonly steers: Array<{ content: string; promptMeta?: Record; resolve: (outcome: unknown) => void; reject: (error: unknown) => void }> = []; - cancelled = 0; - releases = 0; - /** When true, prompt() rejects pre-handoff (released session) — the - * handoff acknowledgment never fires and the backend is never - * invoked. */ - isReleased = false; - /** Crash-boundary simulation: the prompt is invoked (registered) and - * then the process dies BEFORE the handoff acknowledgment fires — - * the only interval the fixed seam leaves between "the backend - * received the prompt" and "the delivered marker is durable". The - * steer must stay undelivered-in-the-store: reconcile re-queues it - * (never lost). */ - dieBeforeHandoff = false; - /** Crash-boundary simulation: the prompt is invoked, the handoff - * acknowledgment fires (the delivered marker is durable), and then - * the process dies — reconcile must never replay it (never - * duplicated). */ - dieAfterHandoff = false; - stopReason = 'end_turn'; - readonly texts: string[] = []; - /** The assistant text of each COMPLETED turn (the result-shaping - * source the broker reads — the prompt texts are in `texts`). */ - readonly completedTexts: string[] = []; - - constructor(readonly openedWith: BrokerOpenSessionOptions) { - this.sessionId = `fake-session-${FakeSession.nextId++}`; - this.initializeMeta = { steering: { supported: true } }; - } - - static nextId = 0; - - prompt(content: string, opts: BrokerPromptOptions = {}): Promise { - if (this.isReleased) { - // The async pre-handoff rejection: the promise rejects and the - // handoff acknowledgment is NEVER fired (acp-agents' preflight: - // released session, aborted signal, prompt-in-flight). - return Promise.reject(new Error('InteractiveSession has been released')); - } - this.texts.push(content); - return new Promise((resolve, reject) => { - // The backend prompt is invoked (registered) FIRST — the seam - // order of the fixed acp-agents contract; the handoff - // acknowledgment fires only after it. - this.prompts.push({ content, promptMeta: opts.promptMeta, resolve, reject }); - if (this.dieBeforeHandoff) { - // The prompt reached the backend, but the process died before - // the acknowledgment — the delivered marker was never recorded. - reject(new Error('process died in the hand-off seam')); - return; - } - if (this.dieAfterHandoff) { - // The acknowledgment fires (the broker durably records the - // delivered marker), then the process dies. - opts.onHandoff?.(); - reject(new Error('process died after the delivered marker')); - return; - } - // The handoff acknowledgment — the fake's model of acp-agents - // invoking the backend prompt once every preflight passed. - opts.onHandoff?.(); - }); - } - - steer(content: string, opts: BrokerPromptOptions = {}): Promise { - return new Promise((resolve, reject) => { - this.steers.push({ content, promptMeta: opts.promptMeta, resolve, reject }); - }); - } - - /** The re-attach seam (phase D), mirroring the REAL acp-agents adapter: - * resolves IMMEDIATELY with a scripted loaded-turn outcome when one is - * set (the replay made the completed-while-down turn observable), - * parks otherwise (the still-running-at-load case). */ - readonly loadedTurns: Array<{ resolve: (turn: BrokerTurn) => void; reject: (error: unknown) => void }> = []; - /** The seam's scripted loaded-turn outcome. Null parks the seam. */ - loadedTurnTextValue: string | null = null; - awaitCurrentTurn(): Promise { - if (this.loadedTurnTextValue !== null) { - return Promise.resolve({ stopReason: this.stopReason, text: this.loadedTurnTextValue }); - } - return new Promise((resolve, reject) => { - this.loadedTurns.push({ resolve, reject }); - }); - } - - cancel(): Promise { - this.cancelled++; - // The real session settles the in-flight turn with stopReason - // "cancelled"; the fake mirrors that. - for (const pending of this.prompts.splice(0)) { - pending.resolve({ stopReason: 'cancelled', text: '' }); - } - return Promise.resolve(); - } - - release(): Promise { - this.releases++; - return Promise.resolve(); - } - - currentTurnText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ''; - } - - /** The §5 [C]12 fold, mirroring the real acp-client state: EVERY - * assistant message chunk joins with "\n\n" (multi-chunk turns - * record their chunk arrays; whole-text turns fold to themselves). */ - readonly chunkedTurns: string[][] = []; - foldedTurnText(): string { - const chunks = this.chunkedTurns[this.chunkedTurns.length - 1]; - if (chunks !== undefined) return chunks.join('\n\n'); - return this.currentTurnText(); - } - - finalMessageText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ''; - } - - rawStructuredOutput(): unknown { - return undefined; - } - - /** The broker's openSession lands asynchronously; wait for it. */ - completeTurn(text: string): void { - const pending = this.prompts.shift(); - assert.ok(pending, 'a prompt turn must be in flight'); - this.completedTexts.push(text); - pending.resolve({ stopReason: this.stopReason, text }); - } - - /** Complete the in-flight turn from MULTIPLE assistant message chunks - * (the acp-client textChunks model): `currentTurnText()` glues them - * separator-free while `foldedTurnText()` joins with "\n\n" — the - * broker's no-schema result must read the FOLD. */ - completeTurnChunked(chunks: string[]): void { - const pending = this.prompts.shift(); - assert.ok(pending, 'a prompt turn must be in flight'); - this.completedTexts.push(chunks.join('')); - this.chunkedTurns.push(chunks); - pending.resolve({ stopReason: this.stopReason, text: chunks.join('') }); - } - - failTurn(error: unknown): void { - const pending = this.prompts.shift(); - assert.ok(pending, 'a prompt turn must be in flight'); - pending.reject(error); - } - - completeSteer(outcome: string): void { - const pending = this.steers.shift(); - assert.ok(pending, 'a steer wire call must be in flight'); - pending.resolve({ outcome }); - } - - failSteer(error: unknown): void { - const pending = this.steers.shift(); - assert.ok(pending, 'a steer wire call must be in flight'); - pending.reject(error); - } -} - -/** A fake runner: opens fake sessions, records what was requested. The - * steering capability is negotiated at initialize — the fake models that - * by stamping every session it opens from `supportsSteering` (the - * broker captures the capability ONCE, at session open). */ -class FakeRunner implements BrokerRunner { - readonly sessions: FakeSession[] = []; - readonly openedWith: BrokerOpenSessionOptions[] = []; - readonly loadedWith: BrokerLoadSessionOptions[] = []; - supportsSteering = true; - /** The re-attach capability gate (phase D): models acp-agents' - * `supportsLoadSession` — a backend that does not advertise - * session/load rejects the load BEFORE any wire request. */ - supportsLoadSession = true; - /** The scripted loaded-turn outcome for loadSession-created sessions - * (the real adapter resolves the seam from the session/load replay). - * Null parks the seam (the still-running-at-load case). */ - loadedTurnText: string | null = null; - /** Open failures to inject (each one rejects openSession once). */ - failNextOpens = 0; - /** Load failures to inject (each one rejects loadSession once). */ - failNextLoads = 0; - /** When true, loadSession PARKS (the caller releases it through - * `releaseParkedLoad`) — the delayed-load probe for the §4.2 - * mint-time addressability of lazy re-attach turns (a queue on a - * drained settled handle, or a restore whose queue rebuild - * re-attaches the founding session). */ - parkLoads = false; - readonly parkedLoads: Array<{ - opts: BrokerLoadSessionOptions; - resolve: (session: FakeSession) => void; - reject: (error: unknown) => void; - }> = []; - - /** Resolve the oldest parked load with a fresh session (the - * loadSession-shaped creation — capabilities and the scripted - * loaded-turn outcome stamp like the normal path). */ - releaseParkedLoad(): FakeSession { - const parked = this.parkedLoads.shift(); - assert.ok(parked, 'a loadSession call must be parked'); - const session = new FakeSession(parked.opts); - session.initializeMeta = this.supportsSteering ? { steering: { supported: true } } : {}; - session.loadedTurnTextValue = this.loadedTurnText; - this.sessions.push(session); - parked.resolve(session); - return session; - } - disposeCalls = 0; - /** Extra registered custom backends (appended to the known list). */ - extraBackends: string[] = []; - /** The static config-option vocabularies the runner publishes (the - * §4.1 admission seam). Empty/absent = dynamic (undefined). */ - staticConfigOptions: Record = {}; - /** configOptions keys the backend rejects at open (the dynamic- - * vocabulary late failure the [C]5 fallback covers). */ - failConfigKeys: Set = new Set(); - /** The backend's own error message for a rejected config option - * (defaults to a key-free message). Lets tests model backends whose - * late error names an accepted sibling while omitting the actual - * rejected key — the [C]5 message-short-circuit defect. */ - failConfigMessage = 'invalid config option'; - /** Session mode ids rejected independently of configOptions. */ - failModes: Set = new Set(); - - listBackends(): string[] { - return ['claude', 'codex', 'opencode', 'pi', ...this.extraBackends]; - } - - defaultBackendId(): string { - return 'claude'; - } - - knownConfigOptionIds(backendId: string): string[] | undefined { - return this.staticConfigOptions[backendId]; - } - - async openSession(opts: BrokerOpenSessionOptions): Promise { - if (opts.mode !== undefined && this.failModes.has(opts.mode)) { - throw new Error(`ACP agent (pi) cannot apply session mode "${opts.mode}" (advertised modes: none)`); - } - if (this.failNextOpens > 0) { - this.failNextOpens--; - throw new Error('spawn failed'); - } - if (opts.configOptions !== undefined) { - for (const key of Object.keys(opts.configOptions)) { - if (this.failConfigKeys.has(key)) { - throw new Error(this.failConfigMessage); - } - } - } - const session = new FakeSession(opts); - session.initializeMeta = this.supportsSteering ? { steering: { supported: true } } : {}; - this.sessions.push(session); - this.openedWith.push(opts); - return session; - } - - async loadSession(opts: BrokerLoadSessionOptions): Promise { - this.loadedWith.push(opts); - if (!this.supportsLoadSession) { - // The acp-agents capability gate (capabilities.ts): a backend - // that omits session/load rejects before any wire request. - throw new Error('backend does not advertise session/load (loadSession capability gate)'); - } - if (this.failNextLoads > 0) { - this.failNextLoads--; - throw new Error('session not found at the backend'); - } - if (this.parkLoads) { - return new Promise((resolve, reject) => { - this.parkedLoads.push({ opts, resolve, reject }); - }); - } - const session = new FakeSession(opts); - session.initializeMeta = this.supportsSteering ? { steering: { supported: true } } : {}; - session.loadedTurnTextValue = this.loadedTurnText; - this.sessions.push(session); - return session; - } - - async dispose(): Promise { - this.disposeCalls++; - } - - last(): FakeSession { - assert.ok(this.sessions.length > 0, 'a session must have been opened'); - return this.sessions[this.sessions.length - 1]; - } -} - -/** Create a workspace + attached broker with a fake runner. */ -async function setup(options: { - maxConcurrentAgents?: number; - runner?: FakeRunner; - store?: ConstructorParameters[1]['store']; - interruptHandler?: () => boolean; -} = {}) { - const ws = await Workspace.create(PROJECT); - const runner = options.runner ?? new FakeRunner(); - const broker = await Broker.attach(ws, { - runner, - store: options.store, - maxConcurrentAgents: options.maxConcurrentAgents, - interruptHandler: options.interruptHandler, - }); - return { ws, broker, runner }; -} - -function output(r: ReplEvalResult): string[] { - return r.output; -} - -/** Dispatch one agent call and wait until its session is open with the - * initial turn in flight. */ -async function dispatchAgent( - broker: Broker, - runner: FakeRunner, - code = 'const pi = agent("pi/deepseek-v4-flash-max", "research X"); "started"', -): Promise { - const r = await broker.eval(code); - assert.ok(r.result !== undefined, `dispatch eval must complete: ${JSON.stringify(r)}`); - await tick(); - assert.equal(runner.sessions.length, 1); - assert.equal(runner.last().prompts.length, 1, 'the initial turn is in flight'); -} - -// ──────────────────────────────────────────────────────────────────────── -// Eval shapes -// ──────────────────────────────────────────────────────────────────────── - -test('eval shapes: resolved reports the previewed value; suspended lists pending ids with no fabricated value; rejected reports the error line', async () => { - const { ws, broker, runner } = await setup(); - - // Resolved: the completion value is previewed (trap-free, see the - // accessor test below). - const resolved = await broker.eval('6 * 7'); - assert.equal(resolved.result, '42'); - assert.equal(resolved.pending.length, 0); - assert.deepEqual(resolved.output, []); - assert.deepEqual(resolved.completed, []); - - // Microtask-only awaits resolve within the drain. - const micro = await broker.eval('await Promise.all([1, 2]).then(([a, b]) => a + b)'); - assert.equal(micro.result, '3'); - - // Object completion values render the §4.4 depth-limited repr - // (nested collections expand to depth 2; deeper levels collapse). - const obj = await broker.eval('({ sections: [{ title: "Auth flow" }], n: 3 })'); - assert.equal(obj.result, '{sections: [{…}], n: 3}'); - - // Rejected: the error renders as an error-level line; result is absent. - const rejected = await broker.eval('throw new Error("boom")'); - assert.equal(rejected.result, undefined); - assert.ok(output(rejected).some((line) => line.includes('Error: boom'))); - assert.equal(rejected.pending.length, 0); - - // Top-level return stays a syntax error (the doc's pinned shape). - const syntax = await broker.eval('return 1'); - assert.equal(syntax.result, undefined); - assert.ok(output(syntax).some((line) => line.startsWith('SyntaxError'))); - - // Suspended: no fabricated value, the pending call ids are listed. - const suspended = await broker.eval('const r = await agent("pi/deepseek-v4-flash-max", "task"); "done:" + r'); - assert.equal(suspended.result, undefined, 'no fabricated value'); - assert.deepEqual(suspended.pending, ['c1']); - assert.deepEqual(output(suspended), []); - - // Started-not-awaited handles list their pending id too. - const started = await broker.eval('const second = agent("pi/x", "other"); "ok"'); - assert.equal(started.result, 'ok'); - assert.deepEqual(started.pending, ['c1', 'c2']); - - // Settle both; the suspended eval's continuation runs at settlement. - runner.sessions[0].completeTurn('hello'); - runner.sessions[1].completeTurn('world'); - await tick(); - const pumped = await broker.pump(); - assert.deepEqual(pumped, ['c1', 'c2']); - await ws.dispose(); -}); - -test('the fused-eval seam: waitForCalls reports the suspended eval\'s completion — kind/result attributed by the continuation token, with the late-error and still-pending arms', async () => { - const { ws, broker, runner } = await setup(); - // The suspended eval completes during the wait's pumps: the wait - // reports the completion's kind and repr, attributed to THIS eval via - // its continuation token (a concurrent client's eval can never steal - // the attribution). - const r1 = await broker.eval('const r = await agent("pi/x", "task"); "done:" + r'); - assert.equal(r1.kind, 'pending'); - const waiting = broker.waitForCalls(undefined, 5000, r1.evalToken); - await tick(); - runner.last().completeTurn('hello'); - const waited = await waiting; - assert.equal(waited.drained, true); - assert.equal(waited.result.kind, 'value'); - assert.equal(waited.result.result, 'done:hello'); - assert.equal(waited.result.evalToken, undefined, 'wait results carry no token of their own'); - // The settled eval's completion became `_` on the way. - assert.equal((await broker.eval('_')).result, 'done:hello'); - // A LATE ERROR: the suspended eval rejects during the wait's pumps — - // the wait reports kind 'error' (the rendering is in the output - // lines) and no result. - const r2 = await broker.eval('const q = agent("pi/x", "task2"); await q; "never"'); - const waiting2 = broker.waitForCalls(undefined, 5000, r2.evalToken); - await tick(); - runner.last().completeTurn(''); - const waited2 = await waiting2; - assert.equal(waited2.result.kind, 'error'); - assert.equal(waited2.result.result, undefined); - assert.ok( - output(waited2.result).some((line) => line.includes('no assistant output')), - `the late error rendered in the wait's output: ${output(waited2.result).join('\n')}`, - ); - // STILL PENDING: the eval awaits a call that never settles within - // the bound — kind stays 'pending' with the in-flight ids. - const r3 = await broker.eval('const t = agent("pi/x", "never-settles"); await t'); - const waiting3 = broker.waitForCalls(undefined, 200, r3.evalToken); - const waited3 = await waiting3; - assert.equal(waited3.result.kind, 'pending', 'the eval is still suspended at the bound'); - assert.ok(waited3.result.pending.length > 0, 'the in-flight ids are reported'); - assert.equal(waited3.result.result, undefined, 'no completion value while suspended'); - await ws.dispose(); -}); - -test('§5 [C]12: the no-schema result fold joins assistant message chunks with "\n\n" — multi-chunk answers are never glued', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-broker-chunks-')); - const storePath = join(dir, 'calls.jsonl'); - const { ws, broker, runner } = await setup({ store: JsonlCallStore.open(storePath) }); - // The LIVE path: the awaited call's turn completes in THREE assistant - // message chunks (the acp-client textChunks model). The broker's - // no-schema result must read the FOLD — "\n\n" between every chunk — - // never the separator-free glue ("…won't modify any files.TypeScript - // files under…"). - const r1 = await broker.eval('const p = agent("pi/x", "chunked"); await p'); - assert.equal(r1.kind, 'pending'); - await tick(); - runner.last().completeTurnChunked(['First chunk.', 'Second chunk.', 'Third chunk.']); - await tick(); - await broker.pump(); - const got = await broker.eval('await p'); - assert.equal(got.result, 'First chunk.\n\nSecond chunk.\n\nThird chunk.', 'the §5 fold, not the glue'); - await ws.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('§3.1: the fused pump reports the finished shape the moment THIS eval settles — an unrelated long-running call elsewhere in the workspace never holds the finished shape to the bound (review finding)', async () => { - const { ws, broker, runner } = await setup(); - // An unrelated long-running call from an EARLIER eval (never awaited) - // stays pending for the whole wait. - await broker.eval('const slow = agent("pi/x", "long research"); "started"'); - await tick(); - assert.equal(runner.sessions.length, 1, 'the unrelated session opened'); - // THIS eval awaits its own call; the wait pumps with a long bound. - const r1 = await broker.eval('const mine = await agent("pi/x", "my task"); "mine:" + mine'); - assert.equal(r1.kind, 'pending'); - const waiting = broker.waitForCalls(undefined, 5000, r1.evalToken); - await tick(); - assert.equal(runner.sessions.length, 2, 'the eval\'s own session opened'); - // Only the eval's own call settles — the unrelated one is still in - // flight. The wait must return the finished shape IMMEDIATELY, not - // pump until the unrelated call drains (and never to the bound). - const startedAt = Date.now(); - runner.last().completeTurn('my answer'); - const waited = await waiting; - assert.equal(waited.drained, true, 'the wait reported drained'); - assert.equal(waited.result.kind, 'value', 'the finished shape'); - assert.equal(waited.result.result, 'mine:my answer'); - assert.ok(Date.now() - startedAt < 2000, `prompt return: ${Date.now() - startedAt} ms`); - // The unrelated call was untouched and is still pending (the wait - // did NOT wait for it). - assert.deepEqual(waited.result.pending, ['c1'], 'the unrelated call stays in flight'); - // And it still settles normally afterwards. - runner.sessions[0].completeTurn('slow result'); - await tick(); - await broker.pump(); - assert.equal((await broker.eval('await slow')).result, 'slow result'); - await ws.dispose(); -}); - -test('a suspended eval continues at settlement like a .then: its output lands in the next tool result', async () => { - const { ws, broker, runner } = await setup(); - const r1 = await broker.eval('const r = await agent("pi/x", "task"); console.log("got", r); "done:" + r'); - assert.equal(r1.kind, 'pending'); - assert.match(r1.evalToken ?? '', /^e\d+$/, 'the continuation token rides the eval result (the fused-eval seam)'); - assert.deepEqual(r1.pending, ['c1']); - await tick(); - runner.last().completeTurn('hello'); - await tick(); - await broker.pump(); - const r2 = await broker.eval('"probe"'); - assert.ok( - output(r2).some((line) => line === 'got hello'), - `continuation output: ${output(r2).join('\n')}`, - ); - await ws.dispose(); -}); - -test('a late uncaught rejection of a suspended eval surfaces as an error-level console line in the next tool result', async () => { - const { ws, broker, runner } = await setup(); - const r1 = await broker.eval('const p = agent("pi/x", "research"); await p; "never"'); - assert.deepEqual(r1.pending, ['c1']); - await tick(); - // The worker fails AFTER the eval returned: the completion promise - // rejects late, and the rejection bridge routes it into the console - // bridge (error-level, $N-frozen) instead of vanishing. - runner.last().failTurn(new Error('research failed')); - await tick(); - await broker.pump(); - const r2 = await broker.eval('"probe"'); - const errorLines = output(r2).filter((line) => line.startsWith('error: ')); - assert.equal(errorLines.length, 1, `one error line, got: ${output(r2)}`); - assert.ok(errorLines[0].includes('Error: research failed'), errorLines[0]); - await ws.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Agent options and result shaping -// ──────────────────────────────────────────────────────────────────────── - -test('agent options: the §4.1 option bag (schema, cwd, configOptions, mode) maps onto the runner (cwd default, label, runId, keepSession, retainSessionLog)', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent( - broker, - runner, - 'const pi = agent("pi/deepseek-v4-flash-max", "research X", { configOptions: { thinking: true }, cwd: "/tmp/elsewhere", mode: "read-only" }); "ok"', - ); - const opened = runner.openedWith[0]; - assert.equal(opened.model, 'pi/deepseek-v4-flash-max'); - assert.equal(opened.cwd, '/tmp/elsewhere'); - assert.deepEqual(opened.configOptions, { thinking: true }); - assert.equal(opened.mode, 'read-only'); - assert.equal(opened.label, 'repl:c1', 'the label is broker-owned (no guest label key)'); - assert.equal(opened.runId, 'c1'); - assert.equal(opened.keepSession, true); - assert.equal(opened.retainSessionLog, true); - await ws.dispose(); -}); - -test('agent options: every known undefined-valued key is absent and dispatches', async () => { - const { ws, broker, runner } = await setup(); - const options = [ - '{ schema: undefined }', - '{ cwd: undefined }', - '{ configOptions: undefined }', - '{ mode: undefined }', - '{ cwd: "/tmp", schema: undefined }', - '{ configOptions: { thinkingLevel: undefined } }', - ]; - for (const option of options) { - const result = await broker.eval(`agent("pi/x", "t", ${option}); "started"`); - assert.equal(result.result, 'started', `${option} is admitted`); - } - await tick(); - assert.equal(runner.sessions.length, options.length, 'every known-key case reaches backend dispatch'); - assert.ok(runner.sessions.every((session) => session.prompts.length === 1), 'every initial turn is in flight'); - assert.equal(runner.openedWith[0].schema, undefined); - assert.equal(runner.openedWith[1].cwd, PROJECT); - assert.equal(runner.openedWith[2].configOptions, undefined); - assert.equal(runner.openedWith[3].mode, undefined); - assert.equal(runner.openedWith[4].cwd, '/tmp'); - assert.deepEqual(runner.openedWith[5].configOptions, {}); - await ws.dispose(); -}); - -test('agent options: a relative cwd and unknown keys refuse the call with recoverable: false; the unknown-key error ENUMERATES the valid keys', async () => { - const { ws, broker, runner } = await setup(); - const r1 = await broker.eval('await agent("pi/x", "t", { cwd: "relative" }).catch(e => e.code + "/" + e.recoverable)'); - assert.equal(r1.result, 'SCRIPT_VALIDATION_ERROR/false'); - const r2 = await broker.eval('await agent("pi/x", "t", { bogus: 1 }).catch(e => e.code + "|" + e.message)'); - assert.equal(r2.result, 'SCRIPT_VALIDATION_ERROR|agent options: unknown option "bogus" (valid options: schema, cwd, configOptions, mode)'); - const r3 = await broker.eval('await agent("pi/x", "t", { label: "nope" }).catch(e => e.code)'); - assert.equal(r3.result, 'SCRIPT_VALIDATION_ERROR', 'the deleted label/meta/promptMeta/tier/toolNames keys are unknown options'); - const r4 = await broker.eval('await agent("pi/x", "t", { schema: 42 }).catch(e => e.code)'); - assert.equal(r4.result, 'SCRIPT_VALIDATION_ERROR'); - const r5 = await broker.eval( - 'await agent("pi/x", "t", { bogus: undefined }).catch(e => e.code + "|" + e.message)', - ); - assert.equal( - r5.result, - 'SCRIPT_VALIDATION_ERROR|agent options: unknown option "bogus" (valid options: schema, cwd, configOptions, mode)', - 'an undefined value cannot make the unknown key disappear before host admission validation', - ); - const r6 = await broker.eval( - 'await agent("pi/x", "t", { bogus: function () {} }).catch(e => e.code + "|" + e.message)', - ); - assert.equal( - r6.result, - 'SCRIPT_VALIDATION_ERROR|agent options: unknown option "bogus" (valid options: schema, cwd, configOptions, mode)', - 'a function value cannot make the unknown key disappear before host admission validation', - ); - const r7 = await broker.eval( - 'await agent("pi/x", "t", { bogus: Symbol("s") }).catch(e => e.code + "|" + e.message)', - ); - assert.equal( - r7.result, - 'SCRIPT_VALIDATION_ERROR|agent options: unknown option "bogus" (valid options: schema, cwd, configOptions, mode)', - 'a symbol value cannot make the unknown key disappear before host admission validation', - ); - const r8 = await broker.eval( - 'await agent("pi/x", "t", { bogus: 10n }).catch(e => e.code + "|" + e.message)', - ); - assert.equal( - r8.result, - 'SCRIPT_VALIDATION_ERROR|agent options: unknown option "bogus" (valid options: schema, cwd, configOptions, mode)', - 'a bigint value cannot bypass the unknown-key schema error before host admission validation', - ); - assert.equal(runner.sessions.length, 0, 'no invalid option bag reaches backend dispatch'); - await ws.dispose(); -}); - -test('queue/steer options: undefined promptMeta is absent and an undefined unknown key still rejects durably', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - runner.last().completeTurn('done'); - await tick(); - await broker.pump(); - - const rejected = await broker.eval( - 'await pi.steer("redirect", { bogus: undefined }).catch(e => e.code + "|" + e.message)', - ); - assert.equal( - rejected.result, - 'SCRIPT_VALIDATION_ERROR|steer options: unknown option "bogus"', - 'an undefined value cannot make an unknown steer key disappear before host admission validation', - ); - assert.equal(runner.last().prompts.length, 0, 'the invalid steer options do not start a turn'); - assert.equal(runner.last().steers.length, 0, 'the invalid steer options do not reach live steering'); - - const accepted = await broker.eval('pi.queue("next", { promptMeta: undefined }); "started"'); - assert.equal(accepted.result, 'started', 'undefined promptMeta is admitted as an absent known option'); - await tick(); - assert.equal(runner.last().prompts.length, 1, 'queue dispatches a new turn'); - assert.equal(runner.last().prompts[0].content, 'next'); - - runner.last().completeTurn('follow-up result'); - await tick(); - await broker.pump(); - await ws.dispose(); -}); - -test('queue admission validation refusals keep minted ids and durable rejections without entering the live FIFO', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - const refused = await broker.eval(` - const badPrompt = pi.queue(42); - const badKey = pi.queue("later", { schema: {} }); - const badMeta = pi.queue("later", { promptMeta: [] }); - JSON.stringify({ - ids: [badPrompt.id, badKey.id, badMeta.id], - codes: await Promise.all([badPrompt, badKey, badMeta].map((q) => q.catch((e) => e.code))), - }) - `); - assert.equal(refused.result, '{"ids":["c2","c3","c4"],"codes":["SCRIPT_VALIDATION_ERROR","SCRIPT_VALIDATION_ERROR","SCRIPT_VALIDATION_ERROR"]}'); - for (const callId of ['c2', 'c3', 'c4']) { - const record = broker.store().lookup(callId)!; - assert.equal(record.kind, 'queue'); - assert.equal(record.completion!.outcome, 'reject', `${callId} refusal is durable`); - assert.equal(record.handoffAtMs, null); - assert.ok(!broker.liveAgents().some((agent) => agent.callId === callId), `${callId} never entered the live FIFO projection`); - } - runner.last().completeTurn('founding done'); - await tick(); - await broker.pump(); - await tick(); - assert.equal(runner.last().prompts.length, 0, 'no refused queue item was sent later'); - await ws.dispose(); -}); - -test('§4.1 admission validation: an unknown backend segment rejects SYNCHRONOUSLY, naming the segment and enumerating the known backends — never a silent route to the default backend', async () => { - const runner = new FakeRunner(); - runner.extraBackends = ['browser']; - const { ws, broker } = await setup({ runner }); - const unknown = await broker.eval('await agent("watson/deep-v4", "t").catch(e => e.message)'); - assert.equal( - unknown.result, - 'unknown backend "watson" in model spec "watson/deep-v4" (known backends: browser, claude, codex, opencode, pi)', - ); - assert.equal(runner.sessions.length, 0, 'nothing was spawned'); - // A registered custom backend joins the known vocabulary. - const custom = await broker.eval('await agent("browser/x", "t").catch(e => e.message)'); - assert.equal(custom.result, undefined); - await tick(); - assert.equal(runner.sessions.length, 1, 'the custom backend dispatched'); - await ws.dispose(); -}); - -test('§4.1 admission validation: reserved configOptions.model rejects synchronously with REPL-native modelSpec guidance', async () => { - const { ws, broker, runner } = await setup(); - const rejected = await broker.eval( - 'await agent("pi/openai/model", "t", { configOptions: { model: "openai/other" } }).catch(e => e.message)', - ); - assert.equal( - rejected.result, - 'configOptions option "model" with authored value "openai/other" is reserved; use the first modelSpec argument instead', - ); - assert.equal(runner.sessions.length, 0); - await ws.dispose(); -}); - -test('§4.1 admission validation: configOptions keys validate against the resolved backend\'s known vocabulary where it is knowable; the [C]5 fallback names the offending key when the vocabulary is dynamic', async () => { - const { ws, broker, runner } = await setup(); - // Knowable vocabulary (the runner seam publishes it): a typo'd key - // fails in milliseconds, naming the key and the valid alternatives. - runner.staticConfigOptions = { pi: ['thinkingLevel', 'effort'] }; - const typo = await broker.eval('await agent("pi/x", "t", { configOptions: { thinkinglevel: "high" } }).catch(e => e.message)'); - assert.equal( - typo.result, - 'configOptions: unknown option "thinkinglevel" for backend "pi" (known options: thinkingLevel, effort)', - ); - assert.equal(runner.sessions.length, 0, 'nothing was spawned'); - // Dynamic vocabulary (the seam returns undefined): admitted, and the - // late failure names the offending key. - runner.staticConfigOptions = {}; - runner.failConfigKeys = new Set(['thinkinglevel']); - const late = await broker.eval('const p = await agent("pi/x", "t", { configOptions: { thinkinglevel: "high" } }).catch(e => e.name + ": " + e.message); console.log("got", p); "done"'); - assert.equal(late.result, undefined, 'the late rejection arrives after the eval suspended'); - await tick(); - await broker.pump(); - const probe = await broker.eval('"probe"'); - assert.ok( - output(probe).some((line) => line === 'got ConfigOptionsError: backend pi rejected the call\'s configOptions — offending key "thinkinglevel" (backend error: invalid config option)'), - output(probe).join('\n'), - ); - // Multiple dynamic keys still identify the ACTUAL rejected key. The - // accepted sibling must not be reported as merely one of several - // candidates (the round-4 review repro: { good: true, bad: true } - // rendered "offending key among: good, bad"). - runner.failConfigKeys = new Set(['bad']); - const multi = await broker.eval( - 'const m = await agent("pi/x", "t", { configOptions: { good: true, bad: true } }).catch(e => e.name + ": " + e.message); console.log("multi", m); "done"', - ); - assert.equal(multi.result, undefined); - await tick(); - await broker.pump(); - const multiProbe = await broker.eval('"probe"'); - const multiLine = output(multiProbe).find((line) => line.startsWith('multi ConfigOptionsError')); - assert.ok(multiLine !== undefined, output(multiProbe).join('\n')); - assert.ok(multiLine.includes('offending key "bad"'), multiLine); - assert.ok(!multiLine.includes('among'), multiLine); - assert.ok(!multiLine.includes('"good"'), `the accepted key is not accused: ${multiLine}`); - await ws.dispose(); -}); - -test('§4.1 [C]5: a multi-key late configOptions error whose backend message NAMES AN ACCEPTED SIBLING still isolates and names the actual rejected key — the message hit never skips the prefix probes', async () => { - const { ws, broker, runner } = await setup(); - // Dynamic vocabulary: admitted. The backend rejects `bad`, and its - // own late error names the ACCEPTED sibling `good` while omitting - // the rejected key — the round-6 review repro: { good: true, - // bad: true } + "accepted option good; another config option is - // invalid" emitted that vague message verbatim because ANY submitted - // key appearing in the message short-circuited the isolation. - runner.failConfigKeys = new Set(['bad']); - runner.failConfigMessage = 'accepted option good; another config option is invalid'; - const late = await broker.eval( - 'const m = await agent("pi/x", "t", { configOptions: { good: true, bad: true } }).catch(e => e.name + ": " + e.message); console.log("got", m); "done"', - ); - assert.equal(late.result, undefined, 'the late rejection arrives after the eval suspended'); - await tick(); - await broker.pump(); - const probe = await broker.eval('"probe"'); - const line = output(probe).find((l) => l.startsWith('got ConfigOptionsError')); - assert.ok(line !== undefined, output(probe).join('\n')); - assert.ok(line.includes('offending key "bad"'), `the actual rejected key is named, never the vague backend message alone: ${line}`); - assert.ok(!line.includes('offending key "good"'), `the accepted sibling is never accused: ${line}`); - assert.ok(!line.includes('among'), line); - // The backend's own error is still reported verbatim — but as the - // quoted backend error inside the conforming attribution, not as the - // whole answer. - assert.ok(line.includes('accepted option good; another config option is invalid'), line); - assert.ok(line.includes('backend pi'), `the late error names the resolved backend: ${line}`); - await ws.dispose(); -}); - -test('the structured-output schema is validated by acp-agents\' own ladder (parse → validate → re-prompt → SCHEMA_NONCOMPLIANCE)', async () => { - const { ws, broker, runner } = await setup(); - const started = await broker.eval( - 'const p = agent("pi/x", "research", { schema: { type: "object", properties: { answer: { type: "string" } }, required: ["answer"] } }); "started"', - ); - assert.equal(started.result, 'started'); - await tick(); - // The worker's final message is JSON: the ladder's native/prose - // extraction validates it. - runner.last().completeTurn('{"answer": "42"}'); - await tick(); - await broker.pump(); - const got = await broker.eval('await p'); - assert.deepEqual(got.result, "{answer: '42'}"); - await ws.dispose(); -}); - -test('a schema miss re-prompts (the ladder), then rejects SCHEMA_NONCOMPLIANCE when exhausted', async () => { - const { ws, broker, runner } = await setup(); - await broker.eval( - 'const p = agent("pi/x", "research", { schema: { type: "object", properties: { answer: { type: "string" } }, required: ["answer"] } }); "started"', - ); - await tick(); - const session = runner.last(); - // First turn: unparseable prose — the ladder re-prompts (the default - // 2 retries; the guest-visible maxSchemaRetries key is deleted with - // the §4.1 option narrowing). - session.completeTurn('let me think about this...'); - await tick(); - assert.equal(session.prompts.length, 1, 'one repair turn was sent'); - session.completeTurn('still not json'); - await tick(); - assert.equal(session.texts.length, 3, 'two repair turns total (the default budget)'); - session.completeTurn('still not json either'); - await tick(); - await broker.pump(); - const r = await broker.eval('await p.catch(e => e.code + "/" + e.recoverable)'); - assert.equal(r.result, 'SCHEMA_NONCOMPLIANCE/false'); - await ws.dispose(); -}); - -test('an empty worker result rejects with the recoverable AGENT_EMPTY_OUTPUT', async () => { - const { ws, broker, runner } = await setup(); - await broker.eval('const p = agent("pi/x", "task"); "started"'); - await tick(); - runner.last().completeTurn(' '); - await tick(); - await broker.pump(); - const r = await broker.eval('await p.catch(e => e.code + "/" + e.recoverable)'); - assert.equal(r.result, 'AGENT_EMPTY_OUTPUT/true'); - await ws.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Exactly-once settlement -// ──────────────────────────────────────────────────────────────────────── - -test('exactly-once settlement: a crash between the store write and the guest settlement is healed by the next pump (record→settle→consume, both first-wins)', async () => { - const { ws, broker, runner } = await setup(); - const r1 = await broker.eval('const p = agent("pi/x", "task"); p.then((v) => console.log("settled:", v)); "started"'); - assert.equal(r1.result, 'started'); - await tick(); - runner.last().completeTurn('final'); - await tick(); - // Simulate the crash window: the pump's RECORD step completed (the - // store durably holds the completion) but the process died before the - // GUEST settlement. The store is the authority; the guest still has - // c1 pending. - broker.store().recordCompleted('c1', { outcome: 'resolve', value: 'final', completedAtMs: Date.now() }); - assert.deepEqual(broker.workspace.surface()!.pending().map((e) => e.id), ['c1']); - // The next pump re-delivers: the store write is first-wins (no - // change), the guest settles exactly once. - const pumped = await broker.pump(); - assert.deepEqual(pumped, ['c1']); - const r2 = await broker.eval('await p'); - assert.equal(r2.result, 'final'); - // The continuation fired exactly once (one joined line per call). - const settledMarkers = output(r2).filter((line) => line === 'settled: final'); - assert.equal(settledMarkers.length, 1, output(r2).join('\n')); - // The guest is idempotent: a second settlement of c1 is a no-op. - assert.equal(broker.workspace.surface()!.settle('c1', 'resolve', 'again'), false); - // The store kept the FIRST completion. - assert.equal(broker.store().lookup('c1')!.completion!.value, 'final'); - await ws.dispose(); -}); - -test('exactly-once settlement across a crash: the snapshot\'s registry is reconciled from the store after restore (settle-from-store arm)', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-broker-store-')); - const storePath = join(dir, 'calls.jsonl'); - const { ws, broker, runner } = await setup({ store: JsonlCallStore.open(storePath) }); - const r1 = await broker.eval('const p = agent("pi/x", "task"); p.then((v) => console.log("settled:", v)); "started"'); - assert.equal(r1.result, 'started'); - await tick(); - // Snapshot the live VM: the guest registry (with c1 pending) travels. - const snapshot = ws.snapshot(); - // The worker completes; the pump's RECORD step runs (the store durably - // holds the completion) — then the process crashes before the guest - // settlement. - runner.last().completeTurn('final'); - await tick(); - broker.store().recordCompleted('c1', { outcome: 'resolve', value: 'final', completedAtMs: Date.now() }); - await broker.dispose(); - ws.dispose(); - - // Restore: a fresh workspace over the snapshot, a fresh broker over - // the same store, a fresh runner (the old process is gone). - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - // The three-way reconciliation, store arm: completed-while-down calls - // settle from the store — exactly once. - const report = await broker2.reconcile(); - assert.deepEqual(report.settledFromStore, ['c1']); - assert.deepEqual(report.leftPending, []); - // The guest continuation fired exactly once, with the stored result. - const r2 = await broker2.eval('"probe"'); - const settledLines = output(r2).filter((line) => line === 'settled: final'); - assert.equal(settledLines.length, 1, output(r2).join('\n')); - assert.equal((await broker2.eval('await p')).result, 'final'); - // A second reconcile has nothing left to settle. - const report2 = await broker2.reconcile(); - assert.deepEqual(report2.settledFromStore, []); - assert.deepEqual(report2.leftPending, []); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Checkpoints -// ──────────────────────────────────────────────────────────────────────── - -test('checkpoint round trip: raised → the §4.3 output line (PLAIN question, the double-quote fix) → answered in a later eval → settlement within that eval', async () => { - const { ws, broker } = await setup(); - const raised = await broker.eval('const q = checkpoint("What color?"); "raised"'); - assert.deepEqual(raised.checkpoints, [{ id: 'c1', question: 'What color?' }], 'the question is plain head+tail metadata text — never a double-JSON-quoted form'); - assert.deepEqual(raised.pending, ['c1']); - assert.ok(output(raised).some((line) => line === 'checkpoint c1: What color?'), output(raised).join('\n')); - // The question crosses previewed/truncated — never verbatim and - // unbounded (the retained 200-char metadata preview, §7). - const longQ = await broker.eval('const q2 = checkpoint("a".repeat(300)); "raised"'); - assert.equal(longQ.checkpoints.length, 2); - const longQuestion = longQ.checkpoints[1].question; - assert.ok(longQuestion.startsWith('a'.repeat(120)), longQuestion); - assert.ok(longQuestion.includes('chars elided'), longQuestion); - assert.ok(!longQuestion.includes('a'.repeat(200)), 'the verbatim question never crosses unbounded'); - // The continuation rides the answer's own eval drain. - const answered = await broker.eval('checkpoint.answer("c1", "blue"); "delivered"'); - assert.equal(answered.result, 'delivered'); - assert.deepEqual(answered.checkpoints, [{ id: 'c2', question: longQuestion }]); - const r = await broker.eval('await q'); - assert.equal(r.result, 'blue'); - // The answer was recorded in the store BEFORE the settlement (the - // exactly-once discipline applies to answers too). - assert.equal(broker.store().lookup('c1')!.completion!.value, 'blue'); - assert.equal(broker.store().lookup('c1')!.completion!.outcome, 'resolve'); - // Unknown / already-answered ids report false; nothing new pends. - const unknown = await broker.eval('checkpoint.answer("cX", 1)'); - assert.equal(unknown.result, 'false'); - const again = await broker.eval('checkpoint.answer("c1", "green")'); - assert.equal(again.result, 'false'); - await ws.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Steering -// ──────────────────────────────────────────────────────────────────────── - -test('active advertised steering sends one strict extension request and no replacement prompt', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - // The steer is a wire call: the eval suspends until it resolves. - const steered = await broker.eval('const o = await pi.steer("go deeper", { promptMeta: { trace: "t1", steering: { user: "kept", idleBehavior: "startNewTurn" } } }); console.log("steer-outcome", o); "done"'); - assert.equal(steered.result, undefined); - assert.deepEqual(steered.pending, ['c1', 'c2']); - await tick(); - assert.equal(runner.last().steers.length, 1); - assert.equal(runner.last().steers[0].content, 'go deeper'); - assert.deepEqual(runner.last().steers[0].promptMeta, { - trace: 't1', - steering: { user: 'kept', idleBehavior: 'promptRequired' }, - }); - assert.equal(runner.last().prompts.length, 1, 'steering did not send an additional session/prompt'); - runner.last().completeSteer('injected'); - await tick(); - await broker.pump(); - const r = await broker.eval('"probe"'); - assert.ok(output(r).some((line) => line === 'steer-outcome injected'), output(r).join('\n')); - await ws.dispose(); -}); - -test('promptRequired resolves idle while failed rejects steering_failed; neither starts a turn', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - await broker.eval('const promptRequired = pi.steer("late"); "issued"'); - await tick(); - runner.last().completeSteer('promptRequired'); - await tick(); - await broker.pump(); - assert.equal((await broker.eval('await promptRequired')).result, 'idle'); - assert.equal(runner.last().prompts.length, 1, 'only the founding prompt remains in flight'); - - await broker.eval('const failedSteer = pi.steer("retry"); "issued"'); - await tick(); - runner.last().completeSteer('failed'); - await tick(); - await broker.pump(); - assert.equal( - (await broker.eval('await failedSteer.catch(e => e.code + "/" + e.recoverable + "/" + e.details.reason)')).result, - 'AGENT_EXECUTION_ERROR/true/steering_failed', - ); - assert.equal(runner.last().prompts.length, 1, 'failed steering did not create another prompt'); - await ws.dispose(); -}); - -test('active steering WITHOUT the raw advertisement resolves unsupported and never queues or prompts', async () => { - const { ws, broker, runner } = await setup(); - runner.supportsSteering = false; // negotiated at initialize - await dispatchAgent(broker, runner); - const unsupported = await broker.eval('const o = await pi.steer("go deeper"); "outcome:" + o'); - assert.equal(unsupported.result, 'outcome:unsupported'); - assert.deepEqual(unsupported.completed, ['c2'], 'the unsupported steer settled synchronously'); - await tick(); - assert.equal(runner.last().steers.length, 0, 'no _session/steering wire call'); - runner.last().completeTurn('first pass'); - await tick(); - await broker.pump(); - await tick(); - assert.equal(runner.last().prompts.length, 0, 'unsupported steering never becomes a future turn'); - await ws.dispose(); -}); - -test('idle steering is lossy idle with no wire request; queue creates an addressable future turn with its own answer', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - runner.last().completeTurn('done'); - await tick(); - await broker.pump(); - const idle = await broker.eval('const o = await pi.steer("more"); "outcome:" + o'); - assert.equal(idle.result, 'outcome:idle'); - await tick(); - assert.equal(runner.last().prompts.length, 0, 'idle steering sends no prompt'); - assert.equal(runner.last().steers.length, 0, 'idle steering sends no extension request'); - - const queued = await broker.eval('const q = pi.queue("more"); console.log("queue-id", q.id); const queuedAnswer = await q; console.log("answer", queuedAnswer); "done"'); - assert.equal(queued.result, undefined, 'the queued turn is in flight until its answer settles'); - await tick(); - assert.equal(runner.last().prompts.length, 1); - assert.equal(runner.last().prompts[0].content, 'more'); - const liveAgents = broker.liveAgents(); - assert.ok(liveAgents.some((a) => a.callId === 'c3' && a.state === 'running' && a.queuedTurns === 0), `liveAgents: ${JSON.stringify(liveAgents)}`); - runner.last().completeTurn('more results'); - await tick(); - const pumped = await broker.pump(); - const probe = await broker.eval('"probe"'); - const queueOutput = [...output(queued), ...pumped, ...output(probe)]; - assert.ok(queueOutput.some((line) => line === 'queue-id c3'), queueOutput.join('\n')); - assert.ok(queueOutput.some((line) => line === 'answer more results'), queueOutput.join('\n')); - assert.equal(broker.store().lookup('c3')!.kind, 'queue'); - assert.equal(broker.store().lookup('c3')!.completion!.value, 'more results'); - await ws.dispose(); -}); - -test('a failed queued turn rejects that queued handle with attributed call id and backend', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - runner.last().completeTurn('done'); - await tick(); - await broker.pump(); - const evaled = await broker.eval('const o = await pi.queue("do more").catch(e => e.message + "|" + e.replCallId + "|" + e.replBackend); console.log("got", o); "done"'); - assert.equal(evaled.result, undefined, 'the queued handle suspends until its turn settles'); - await tick(); - runner.last().failTurn(new Error('backend exploded')); - await tick(); - await broker.pump(); - const probe = await broker.eval('"probe"'); - assert.ok(output(probe).some((line) => line === 'got backend exploded|c2|pi'), output(probe).join('\n')); - await ws.dispose(); -}); - -test('steering wire failures reject AGENT_EXECUTION_ERROR and never start a replacement turn', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - await broker.eval('const o = await pi.steer("go deeper").catch(e => e.code + "/" + e.details.reason); console.log("steer-outcome", o); "done"'); - await tick(); - runner.last().failSteer(new Error('backend gone')); - await tick(); - await broker.pump(); - const r = await broker.eval('"probe"'); - assert.ok(output(r).some((line) => line === 'steer-outcome AGENT_EXECUTION_ERROR/steering_request_failed'), output(r).join('\n')); - assert.equal(runner.last().prompts.length, 1, 'only the founding prompt was ever sent'); - await ws.dispose(); -}); - -test('a steer in the same eval as dispatch resolves idle and is not retained through opening', async () => { - const { ws, broker, runner } = await setup(); - const r = await broker.eval('const pi = agent("pi/x", "task"); const o = await pi.steer("same eval"); "outcome:" + o'); - assert.equal(r.result, 'outcome:idle'); - await tick(); - assert.equal(runner.last().prompts.length, 1, 'only the initial turn is in flight'); - runner.last().completeTurn('first pass'); - await tick(); - await broker.pump(); - await tick(); - assert.equal(runner.last().prompts.length, 0, 'opening-time steering never becomes a later prompt'); - await ws.dispose(); -}); - -test('cancel with a turn in flight: the handle resolves "cancelled" and the cancelled call rejects with AGENT_CANCELLED', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - const cancelled = await broker.eval('const o = await pi.cancel(); console.log("cancel-outcome", o); "done"'); - assert.equal(cancelled.result, undefined); - await tick(); - assert.equal(runner.last().cancelled, 1, 'ACP session/cancel went out'); - await broker.pump(); - const r = await broker.eval('"probe"'); - assert.ok(output(r).some((line) => line === 'cancel-outcome cancelled'), output(r).join('\n')); - // The call itself rejects with the machine-readable cancellation — - // and the rejection is RECOVERABLE (review regression: it used to be - // recoverable: false, which the guest combinators treat as a halt - // signal — cancelling one worker then aborted the surrounding - // parallel()/pipeline()). One call's cancellation must never abort - // the orchestration owning it. - const call = await broker.eval('await pi.catch((e) => e.code + "/" + e.recoverable)'); - assert.equal(call.result, 'AGENT_CANCELLED/true'); - assert.equal(runner.last().cancelled, 1, 'a second cancel of the idle session is a no-op'); - const idle = await broker.eval('await pi.cancel()'); - assert.equal(idle.result, 'idle'); - await ws.dispose(); -}); - -test('a queued turn failure rejects its own handle and leaves the session usable', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - await broker.eval('const q = pi.queue("go deeper"); const o = await q.catch(e => e.message); console.log("queue-error", o); "done"'); - runner.last().completeTurn('first pass'); - await tick(); - await broker.pump(); - await tick(); - runner.last().failTurn(new Error('worker crashed')); - await tick(); - await broker.pump(); - const r = await broker.eval('"probe"'); - assert.ok(output(r).some((line) => line === 'queue-error worker crashed'), output(r).join('\n')); - assert.equal(broker.store().lookup('c2')!.completion!.outcome, 'reject'); - await ws.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Concurrency cap -// ──────────────────────────────────────────────────────────────────────── - -test('§4.1: dispatches above the concurrency cap QUEUE in dispatch order for the next free slot — never a rejection', async () => { - const { ws, broker, runner } = await setup({ maxConcurrentAgents: 2 }); - // Two calls admitted; the third and fourth QUEUE (their guest - // promises stay pending — the natural `parallel(items.map(...))` - // idiom never loses work). - const queued = await broker.eval('const a = agent("pi/x", "a"); const b = agent("pi/x", "b"); const c = agent("pi/x", "c"); const d = agent("pi/x", "d"); "started"'); - assert.equal(queued.result, 'started'); - assert.equal(runner.sessions.length, 2, 'only two sessions were opened'); - assert.deepEqual(queued.pending, ['c1', 'c2', 'c3', 'c4'], 'the queued dispatches stay pending in dispatch order'); - assert.equal(broker.store().lookup('c3')!.kind, 'agent', 'queued dispatches get their store record at admission time'); - // Settle c1: its slot frees and the QUEUE head (c3) dispatches first. - await tick(); - runner.sessions[0].completeTurn('a done'); - await tick(); - await broker.pump(); - await tick(); - assert.equal(runner.sessions.length, 3, 'the queue head dispatched'); - assert.equal(runner.sessions[2].openedWith.model, 'pi/x'); - assert.equal(runner.sessions[2].texts[0], 'c', 'dispatch order preserved'); - // Settle c2: c4 dispatches next (FIFO). - runner.sessions[1].completeTurn('b done'); - await tick(); - await broker.pump(); - await tick(); - assert.equal(runner.sessions.length, 4); - assert.equal(runner.sessions[3].texts[0], 'd'); - // All four resolve with their answers. - runner.sessions[2].completeTurn('c done'); - runner.sessions[3].completeTurn('d done'); - await tick(); - await broker.pump(); - const got = await broker.eval('[await a, await b, await c, await d].join(",")'); - assert.equal(got.result, 'a done,b done,c done,d done'); - await ws.dispose(); -}); - -test('§4.2: the interrupt cancels a QUEUED dispatch by its addressable id (AGENT_CANCELLED, recorded durably)', async () => { - const { ws, broker, runner } = await setup({ maxConcurrentAgents: 1 }); - await broker.eval('const a = agent("pi/x", "a"); const b = agent("pi/x", "b"); "started"'); - assert.deepEqual(broker.workspace.surface()!.pending().map((e) => e.id), ['c1', 'c2']); - const outcome = await broker.cancelCall('c2'); - assert.equal(outcome, 'cancelled'); - const r = await broker.eval('await b.catch((e) => e.code + "/" + e.recoverable)'); - assert.equal(r.result, 'AGENT_CANCELLED/true'); - assert.equal(broker.store().lookup('c2')!.completion!.outcome, 'reject'); - assert.equal(runner.sessions.length, 1, 'the queued call never dispatched'); - await ws.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Rendering -// ──────────────────────────────────────────────────────────────────────── - -test('trap-free result rendering: accessor properties render as (…) and never fire; Object.prototype.value pollution cannot hijack the result', async () => { - const { ws, broker } = await setup(); - // An accessor-valued completion renders the accessor marker — the - // getter never runs. - const accessor = await broker.eval('let fires = 0; const o = { get x() { fires++; return 1; } }; o'); - assert.equal(accessor.result, '{x: (…)}'); - assert.equal((await broker.eval('fires')).result, '0'); - // The R69 regression shape: a `value` getter on Object.prototype must - // not fabricate or hijack the completion preview. - await broker.eval('Object.defineProperty(Object.prototype, "value", { get() { throw new Error("hijacked"); } })'); - const polluted = await broker.eval('({ a: 1 })'); - // The own-descriptor read fires no getter: either the honest preview - // or the engine's documented degraded shape — never "hijacked". - assert.ok(!JSON.stringify(polluted).includes('hijacked')); - assert.ok(!output(polluted).some((l) => l.includes('hijacked'))); - await ws.dispose(); -}); - -test('output lines are the §4.4 one-line reprs (one joined line per console.* call, levels prefixed); §7: NO output caps — output above the DELETED ceilings ships whole', async () => { - const { ws, broker } = await setup(); - const r = await broker.eval('console.log({ a: 1 }, "text"); console.error("boom"); "done"'); - assert.equal(output(r)[0], '{a: 1} text'); - assert.equal(output(r)[1], 'error: boom'); - assert.equal(r.kind, 'value'); - // §7: the engine applies NO caps to guest output — the Python - // posture (an agent CAN flood its own context). A flood ABOVE BOTH - // deleted ceilings — 4500 lines, >50 KB of bytes (the v1 caps: - // 4000 lines / 50 000 bytes — now deleted with the cap apparatus) - // ships verbatim. Reintroducing the caps would truncate this flood; - // the assertions below must stay green. - const big = await broker.eval('for (let i = 0; i < 4500; i++) console.log("line", i, "padding", "x".repeat(20)); "done"'); - assert.equal(big.output.length, 4500); - assert.equal(output(big)[4499], 'line 4499 padding xxxxxxxxxxxxxxxxxxxx'); - assert.ok( - big.output.reduce((sum, line) => sum + Buffer.byteLength(line, 'utf8') + 1, 0) > 50_000, - 'the flood exceeds the deleted byte cap', - ); - // A DIRECT console string above the deleted 49 488-char emission - // budget ships WHOLE (no upper bound — the Python posture). - const whole = 'w'.repeat(60_000); - const direct = await broker.eval(`console.log(${JSON.stringify(whole)}); "done"`); - assert.equal(output(direct)[0], whole, 'a 60 000-char direct console string ships whole'); - // A string COMPLETION value above the same deleted budget renders - // whole too (§4.4: direct strings print whole — no upper bound). - const resultWhole = await broker.eval(`"r".repeat(60000)`); - assert.equal(resultWhole.result, 'r'.repeat(60_000), 'a 60 000-char string completion value renders whole'); - await ws.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Review-regression suite (phase C review round 1) -// ──────────────────────────────────────────────────────────────────────── - -test('review 1: the store\'s FIRST completion is authoritative — a newer live outcome never settles the guest against it', async () => { - const { ws, broker, runner } = await setup(); - await broker.eval('const p = agent("pi/x", "task"); p.then((v) => console.log("settled:", v)); "started"'); - await tick(); - // A previous crash-retry already recorded the FIRST completion in the - // store. The live outcome is newer AND different — the store wins. - broker.store().recordCompleted('c1', { outcome: 'resolve', value: 'stored-first', completedAtMs: Date.now() }); - runner.last().completeTurn('live-second'); - await tick(); - const pumped = await broker.pump(); - assert.deepEqual(pumped, ['c1']); - const r = await broker.eval('await p'); - assert.equal(r.result, 'stored-first'); - assert.equal(broker.store().lookup('c1')!.completion!.value, 'stored-first'); - assert.equal(broker.store().lookup('c1')!.completion!.outcome, 'resolve'); - await ws.dispose(); -}); - -test('the cap is absolute for queued turns: a queued handle stays addressable and starts when a slot frees', async () => { - const { ws, broker, runner } = await setup({ maxConcurrentAgents: 1 }); - // c1 opens and settles (idle). - await broker.eval('const a = agent("pi/x", "a"); "started"'); - await tick(); - runner.sessions[0].completeTurn('a done'); - await tick(); - await broker.pump(); - // c2 opens and settles (idle) — admitted because c1's slot was freed. - await broker.eval('const b = agent("pi/x", "b"); "started"'); - await tick(); - runner.sessions[1].completeTurn('b done'); - await tick(); - await broker.pump(); - const bSession = runner.sessions[1]; - // c3 now holds the workspace's ONLY slot. - await broker.eval('const c = agent("pi/x", "c"); "started"'); - await tick(); - // A queued turn on the idle b handle must NOT start while c3 is - // in flight — the cap gates turn starts, not just dispatches (review - // regression: idle-handle turns used to start unconditionally, - // so a cap-1 workspace ran two subagent turns concurrently). The - // queue() owns its answer and stays pending until that turn runs. - const queued = await broker.eval('const q = b.queue("more"); console.log("queue-id", q.id); const o = await q; console.log("outcome", o); "done"'); - assert.equal(queued.result, undefined, 'the queued handle suspends until its turn runs'); - assert.deepEqual(queued.pending, ['c3', 'c4'], 'c4 stays pending'); - await tick(); - assert.equal(bSession.prompts.length, 0, 'no follow-up turn started under cap pressure'); - // The queued turn is VISIBLE while pending: minted at enqueue, listed - // in agents() with the honest `queued` state (the review probe: the - // pending id was absent from agents() before its delivery started). - const queuedAgents = broker.liveAgents(); - assert.ok( - queuedAgents.some((a) => a.callId === 'c4' && a.state === 'queued' && a.task === 'more'), - `the queued turn is addressable before it starts: ${JSON.stringify(queuedAgents)}`, - ); - // c3 settles; its slot frees; the queued follow-up starts its turn — - // and settles with the TURN'S ANSWER. - runner.sessions[2].completeTurn('c done'); - await tick(); - await broker.pump(); - await tick(); - assert.equal(bSession.prompts.length, 1, 'the queued follow-up started once a slot freed'); - assert.equal(bSession.prompts[0].content, 'more'); - bSession.completeTurn('more results'); - await tick(); - await broker.pump(); - const probe = await broker.eval('"probe"'); - assert.ok(output(probe).some((line) => line === 'outcome more results'), output(probe).join('\n')); - assert.equal(broker.store().lookup('c4')!.completion!.value, 'more results'); - await ws.dispose(); -}); - -test('FIFO queued turns respect the cap across sessions — one queued turn at a time under cap 1', async () => { - const runner = new FakeRunner(); - runner.supportsSteering = false; - const { ws, broker } = await setup({ maxConcurrentAgents: 1, runner }); - // Two subagents, both settled and idle. - await broker.eval('const a = agent("pi/x", "a"); "started"'); - await tick(); - runner.sessions[0].completeTurn('a done'); - await tick(); - await broker.pump(); - await broker.eval('const b = agent("pi/x", "b"); "started"'); - await tick(); - runner.sessions[1].completeTurn('b done'); - await tick(); - await broker.pump(); - // c3 now holds the workspace's ONLY slot. - await broker.eval('const c = agent("pi/x", "c"); "started"'); - await tick(); - // An explicit queued turn on each idle session (cap exhausted). - await broker.eval('a.queue("more-a"); "queued"'); - await broker.eval('b.queue("more-b"); "queued"'); - assert.equal(runner.sessions[0].prompts.length, 0); - assert.equal(runner.sessions[1].prompts.length, 0); - // c settles → its slot frees → EXACTLY ONE queued delivery starts. - runner.sessions[2].completeTurn('c done'); - await tick(); - await broker.pump(); - await tick(); - const withDelivery = [runner.sessions[0], runner.sessions[1]].filter((s) => s.prompts.length === 1); - assert.equal(withDelivery.length, 1, 'exactly one delivery turn runs under cap 1'); - // It completes → the second delivery starts (the kick fires on every - // freed slot, including the delivery turn's own end). - withDelivery[0].completeTurn('delivered'); - await tick(); - await broker.pump(); - await tick(); - const remaining = [runner.sessions[0], runner.sessions[1]].filter((s) => s.prompts.length === 1); - assert.equal(remaining.length, 1, 'the second queued turn started after the first ended'); - await ws.dispose(); -}); - -test('queued turns are durable: prompt, founding session, FIFO admission, and handoff marker', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-broker-steer-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await dispatchAgent(broker, runner); - const queued = await broker.eval('const q = pi.queue("go deeper", { promptMeta: { trace: "t1" } }); "queued:" + q.id'); - assert.equal(queued.result, 'queued:c2'); - // The payload + founding session id live in the store (crash-durable — - // a crash before delivery loses nothing). - const queueRecord = broker.store().lookup('c2')!; - assert.equal(queueRecord.kind, 'queue'); - assert.equal(queueRecord.foundingCallId, 'c1'); - assert.ok(queueRecord.optionsJson!.includes('go deeper'), 'the payload survives in the record'); - assert.notEqual(queueRecord.queuedAtMs, null); - assert.equal(queueRecord.handoffAtMs, null, 'not handed off before the founding turn settles'); - assert.equal(queueRecord.completion, null, 'the queue handle owns its later answer'); - // The founding turn completes; the delivery turn starts — the prompt - // is handed to the backend, and the delivered marker is recorded only - // after that hand-off (the true point of no return: replay after it - // would duplicate delivery, but a crash BEFORE it must not make a - // restore skip a steer that was never delivered). - runner.last().completeTurn('first pass'); - await tick(); - await broker.pump(); - assert.notEqual(broker.store().lookup('c2')!.handoffAtMs, null, 'handoff marker recorded at the point of no return'); - await tick(); - assert.equal(runner.last().prompts[0].content, 'go deeper'); - runner.last().completeTurn('deeper results'); - await tick(); - await broker.pump(); - // A later reconcile must not recreate a completed queue item. - const report = await broker.reconcile(); - assert.deepEqual(report.reQueuedUndelivered, []); - assert.equal(runner.last().prompts.length, 0, 'no duplicate delivery turn after reconcile'); - await ws.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('a crash before queued-turn handoff restores the durable queue item exactly once', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-broker-steer-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - const r1 = await broker.eval('const pi = agent("pi/x", "task"); const q = pi.queue("go deeper"); "queue:" + q.id'); - assert.equal(r1.result, 'queue:c2'); - await tick(); - // Simulated crash: snapshot before any settlement; dispose without - // delivering anything. The store durably holds the queued turn. - const snapshot = ws.snapshot(); - await broker.dispose(); - ws.dispose(); - // Restore: a fresh workspace + broker over the same store. - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - // c1's loaded turn observably completed while we were down (the real - // adapter resolves the seam from the session/load replay) — the - // re-attach arm awaits it INLINE during reconcile now. - runner2.loadedTurnText = 'loaded turn'; - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - // The reconcile rebuilds the undelivered queue (the founding call is - // still pending — its session re-attach delivers the queued turn - // at open, the same merge path the same-eval test pins). - const report = await broker2.reconcile(); - assert.deepEqual(report.settledFromStore, []); - assert.deepEqual(report.reattached, ['c1'], 'the founding call re-attaches to its recorded backend session'); - assert.deepEqual(report.leftPending, []); - assert.deepEqual(report.reQueuedUndelivered, ['c2']); - // Idempotent: a second reconcile does not double-queue. - const report2 = await broker2.reconcile(); - assert.deepEqual(report2.reQueuedUndelivered, []); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('queued-handle cancellation targets exactly one pending queue item and is durable without ACP traffic', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - const issued = await broker.eval('const q = pi.queue("go deeper"); const qOutcome = q.catch(e => e.code); q.id'); - assert.equal(issued.result, 'c2'); - const cancellation = await broker.eval('const qCancel = q.cancel(); "requested"'); - assert.equal(cancellation.result, 'requested'); - await tick(); - await broker.pump(); - assert.equal((await broker.eval('await qCancel')).result, 'cancelled'); - assert.notEqual(broker.store().lookup('c2')!.cancelledAtMs, null, 'the exact queue cancellation is durable'); - assert.equal(broker.store().lookup('c2')!.completion!.outcome, 'reject'); - assert.equal(await broker.eval('await qOutcome').then((result) => result.result), 'AGENT_CANCELLED'); - assert.equal(runner.last().cancelled, 0, 'pending queue cancellation sends no ACP cancel'); - assert.equal(runner.last().prompts.length, 1, 'only the founding prompt exists'); - const report = await broker.reconcile(); - assert.deepEqual(report.reQueuedUndelivered, [], 'a cancelled queue item is never restored'); - await ws.dispose(); -}); - -test('the queue handoff marker is recorded only after session.prompt reaches the backend seam', async () => { - const inner = new InMemoryCallStore(); - let sessionRef: FakeSession | undefined; - let markers = 0; - const store = new class implements CallStore { - recordDispatched(r: Parameters[0]): void { - inner.recordDispatched(r); - } - recordReissued(callId: string, atMs: number): void { - inner.recordReissued(callId, atMs); - } - recordAttached(callId: string, sessionId: string, atMs: number): void { - inner.recordAttached(callId, sessionId, atMs); - } - recordCompleted(callId: string, outcome: CallOutcome): boolean { - return inner.recordCompleted(callId, outcome); - } - recordHandoff(callId: string, atMs: number): void { - assert.ok( - sessionRef !== undefined && sessionRef.prompts.length > 0, - 'the prompt hand-off must precede the durable handoff marker', - ); - markers++; - inner.recordHandoff(callId, atMs); - } - recordCancelled(callId: string, atMs: number): void { - inner.recordCancelled(callId, atMs); - } - recordQueued(callId: string, atMs: number): void { - inner.recordQueued(callId, atMs); - } - lookup(callId: string) { - return inner.lookup(callId); - } - all() { - return inner.all(); - } - }(); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store, runner }); - await dispatchAgent(broker, runner); - sessionRef = runner.last(); - await broker.eval('pi.queue("go deeper"); "queued"'); - runner.last().completeTurn('first pass'); - await tick(); - await broker.pump(); - assert.equal(markers, 1, 'the handoff marker was recorded exactly once, after the wire handoff'); - assert.equal(runner.last().prompts.length, 1, 'the queued content became the next turn'); - await ws.dispose(); -}); - -test('a queued prompt refused before handoff has no handoff marker and rejects its own handle durably', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - await dispatchAgent(broker, runner); - const session = runner.last(); - await broker.eval('const q = pi.queue("go deeper"); const queueOutcome = q.catch(e => e.message); q.id'); - session.isReleased = true; - session.completeTurn('first pass'); - await tick(); - await broker.pump(); - await tick(); - await broker.pump(); - const record = broker.store().lookup('c2')!; - assert.equal(record.handoffAtMs, null, 'a backend refusal before handoff never writes the marker'); - assert.equal(record.completion!.outcome, 'reject'); - assert.equal((await broker.eval('await queueOutcome')).result, 'InteractiveSession has been released'); - const report = await broker.reconcile(); - assert.deepEqual(report.reQueuedUndelivered, [], 'a durably rejected public turn is not replayed'); - await ws.dispose(); -}); - -test('three queued turns on one busy session run in exact mint-order FIFO with distinct answers', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - const session = runner.last(); - const issued = await broker.eval('const q1 = pi.queue("one"); const q2 = pi.queue("two"); const q3 = pi.queue("three"); [q1.id, q2.id, q3.id].join(",")'); - assert.equal(issued.result, 'c2,c3,c4'); - const projected = broker.liveAgents(); - assert.equal(projected.find((agent) => agent.callId === 'c1')!.queuedTurns, 3); - assert.deepEqual(projected.filter((agent) => agent.callId !== 'c1').map((agent) => [agent.callId, agent.state, agent.queuedTurns]), [ - ['c2', 'queued', 0], - ['c3', 'queued', 0], - ['c4', 'queued', 0], - ]); - session.completeTurn('first pass'); - await tick(); - await broker.pump(); - await tick(); - assert.equal(session.prompts[0].content, 'one'); - session.completeTurn('answer one'); - await tick(); - await broker.pump(); - await tick(); - assert.equal(session.prompts[0].content, 'two'); - session.completeTurn('answer two'); - await tick(); - await broker.pump(); - await tick(); - assert.equal(session.prompts[0].content, 'three'); - session.completeTurn('answer three'); - await tick(); - await broker.pump(); - const answers = await broker.eval('JSON.stringify(await Promise.all([q1, q2, q3]))'); - assert.equal(answers.result, '["answer one","answer two","answer three"]'); - await ws.dispose(); -}); - -test('every queued session/prompt carries its queue call id and preserves caller metadata with the host value winning', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - const session = runner.last(); - await broker.eval('pi.queue("go deeper", { promptMeta: { trace: "t1", "@automatalabs/agentprism": { user: "kept", replCallId: "wrong" } } }); "queued"'); - session.completeTurn('first pass'); - await tick(); - await broker.pump(); - await tick(); - assert.deepEqual(session.prompts[0].promptMeta, { - trace: 't1', - '@automatalabs/agentprism': { user: 'kept', replCallId: 'c2' }, - }); - await ws.dispose(); -}); - -test('review 3: a failing store write during checkpoint.answer leaves the checkpoint pending — a later answer retry succeeds', async () => { - const inner = new InMemoryCallStore(); - const store = new class implements CallStore { - failNextCompletion = false; - recordDispatched(r: Parameters[0]): void { - inner.recordDispatched(r); - } - recordReissued(callId: string, atMs: number): void { - inner.recordReissued(callId, atMs); - } - recordAttached(callId: string, sessionId: string, atMs: number): void { - inner.recordAttached(callId, sessionId, atMs); - } - recordCompleted(callId: string, outcome: CallOutcome): boolean { - if (this.failNextCompletion) { - this.failNextCompletion = false; - throw new Error('disk full'); - } - return inner.recordCompleted(callId, outcome); - } - recordHandoff(callId: string, atMs: number): void { - inner.recordHandoff(callId, atMs); - } - recordCancelled(callId: string, atMs: number): void { - inner.recordCancelled(callId, atMs); - } - recordQueued(callId: string, atMs: number): void { - inner.recordQueued(callId, atMs); - } - lookup(callId: string) { - return inner.lookup(callId); - } - all() { - return inner.all(); - } - }(); - const { ws, broker } = await setup({ store }); - await broker.eval('const q = checkpoint("What color?"); "raised"'); - // The answer's record write fails: the host callback throws, the guest - // answer call fails, and the checkpoint stays PENDING (it must not be - // consumed before its answer is durable — review regression). - store.failNextCompletion = true; - const failed = await broker.eval('checkpoint.answer("c1", "blue")'); - assert.equal(failed.result, undefined); - assert.ok(output(failed).some((l) => l.includes('disk full')), output(failed).join('\n')); - assert.deepEqual((await broker.eval('"probe"')).checkpoints.map((c) => c.id), ['c1']); - assert.equal(broker.store().lookup('c1')!.completion, null, 'nothing was recorded'); - // The retry (with the store healthy again) delivers the answer. - const ok = await broker.eval('checkpoint.answer("c1", "blue"); "delivered"'); - assert.equal(ok.result, 'delivered'); - const r = await broker.eval('await q'); - assert.equal(r.result, 'blue'); - assert.equal(broker.store().lookup('c1')!.completion!.value, 'blue'); - await ws.dispose(); -}); - -test('review 4: the guest schema reaches session creation — the native schema channels are configured, not validated blind', async () => { - const { ws, broker, runner } = await setup(); - const schema = { type: 'object', properties: { answer: { type: 'string' } }, required: ['answer'] }; - await dispatchAgent( - broker, - runner, - `const p = agent("pi/x", "task", { schema: ${JSON.stringify(schema)} }); "ok"`, - ); - assert.deepEqual(runner.openedWith[0].schema, schema, 'the schema rides openSession (session/new channel + per-turn channels)'); - await ws.dispose(); -}); - -test('review 5: the concurrency cap validates — default 6, over-ceiling clamps to 6, invalid values throw at attach', async () => { - const { ws, broker } = await setup(); - assert.equal(broker.maxConcurrentAgents, 6, 'the doc-settled default'); - await ws.dispose(); - // Over the ceiling: clamped (the six-per-workspace maximum is absolute). - const { ws: ws2, broker: broker2 } = await setup({ maxConcurrentAgents: 7 }); - assert.equal(broker2.maxConcurrentAgents, 6); - await ws2.dispose(); - // Invalid values are programming errors at attach time. - for (const bad of [NaN, 0, -1, 1.5, Infinity]) { - await assert.rejects( - async () => Broker.attach(await Workspace.create(PROJECT), { runner: new FakeRunner(), maxConcurrentAgents: bad }), - new RegExp('maxConcurrentAgents must be an integer'), - ); - } - // At the ceiling: six sessions open, the seventh dispatch QUEUES for - // the next free slot (the §4.1 rule — never a rejection). - const { ws: ws3, broker: broker3, runner: runner3 } = await setup(); - const r = await broker3.eval('for (let i = 0; i < 7; i++) agent("pi/x", "t" + i); "started"'); - assert.equal(r.result, 'started'); - await tick(); - assert.equal(runner3.sessions.length, 6, 'six sessions opened — never seven at once'); - assert.deepEqual(r.pending.length, 7, 'the seventh stays pending (queued)'); - // Settle one: the queued seventh dispatches. - runner3.sessions[0].completeTurn('done'); - await tick(); - await broker3.pump(); - await tick(); - assert.equal(runner3.sessions.length, 7, 'the queued dispatch started once a slot freed'); - await ws3.dispose(); -}); - -test('review 6: an agent call whose session never opens releases its concurrency slot (cap 1 — the next dispatch is admitted)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ maxConcurrentAgents: 1, runner }); - runner.failNextOpens = 1; - await broker.eval('const p = agent("pi/x", "boom"); "started"'); - await tick(); // openSession rejects - await broker.pump(); // the rejection settles; the slot MUST be released - assert.equal(broker.store().lookup('c1')!.completion!.outcome, 'reject'); - const r = await broker.eval('await p.catch((e) => e.message)'); - assert.ok(String(r.result).includes('spawn failed'), String(r.result)); - const admitted = await broker.eval('const p2 = agent("pi/x", "ok"); "started"'); - assert.equal(admitted.result, 'started', 'the second dispatch was admitted after the failed open'); - await tick(); - assert.equal(runner.sessions.length, 1); - await ws.dispose(); -}); - -test('same-eval steering of a call whose session never opens resolves idle and is never retained', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - runner.failNextOpens = 1; - const r1 = await broker.eval('const pi = agent("pi/x", "task"); const o = await pi.steer("same eval"); "outcome:" + o'); - assert.equal(r1.result, 'outcome:idle'); - await tick(); - await broker.pump(); - const r2 = await broker.eval('"probe"'); - assert.ok(!output(r2).some((l) => l.includes('same eval')), output(r2).join('\n')); - assert.equal(broker.store().lookup('c2')!.completion!.value, 'idle'); - const report = await broker.reconcile(); - assert.deepEqual(report.reQueuedUndelivered, []); - await ws.dispose(); -}); - -test('review 8: an interrupted continuation keeps the already-settled ids in the eval\'s completed list', async () => { - let checks = 0; - const { ws, broker, runner } = await setup({ interruptHandler: () => ++checks > 20000 }); - await broker.eval('const p = agent("pi/x", "task"); p.then(() => { let i = 0; while (true) i++; }); "started"'); - await tick(); - runner.last().completeTurn('final'); - await tick(); - // The eval's OWN pump delivers c1, then its drain runs the runaway - // continuation, which the interrupt handler breaks: the eval returns - // the ids it settled before the drain failed (review regression: - // completed used to come back empty). §6.2: the drain failure itself - // leaves the eval result surface — it is RETAINED under - // workspace().diagnostics, never rendered as an output line. - const r = await broker.eval('"probe"'); - assert.deepEqual(r.completed, ['c1'], 'the settled id survives the drain failure'); - assert.ok( - output(r).every((l) => !l.includes('interrupted') && !l.includes('Job execution error')), - `the drain failure left the surface: ${output(r).join('\n')}`, - ); - const diag = await broker.eval('workspace().diagnostics.drainError === null ? "none" : workspace().diagnostics.drainError.message'); - assert.ok( - String(diag.result ?? '').includes('interrupted') || String(diag.result ?? '').includes('Job execution error'), - `the failure is retained in diagnostics: ${diag.result}`, - ); - // pump() still propagates the drain failure as its public contract. - checks = 0; - await broker.eval('const p2 = agent("pi/x", "task2"); p2.then(() => { let i = 0; while (true) i++; }); "started"'); - await tick(); - runner.last().completeTurn('final2'); - await tick(); - await assert.rejects(() => broker.pump(), (e: unknown) => - (e as Error).message.includes('Job execution error'), - ); - await ws.dispose(); -}); - -test('review 8b: the broker-level interrupt handler bounds a DIRECT runaway eval (the default ReplEvalOptions handler)', async () => { - let checks = 0; - const { ws, broker } = await setup({ interruptHandler: () => ++checks > 1000 }); - // No settlement involved: the eval itself runs away. The broker's - // configured handler must bound it (review regression: it used to - // apply only to settlement drains, so a direct runaway eval could - // hang the workspace indefinitely). - const r = await broker.eval('let i = 0; while (true) i++;'); - assert.equal(r.result, undefined); - assert.ok(output(r).some((l) => l.includes('InternalError: interrupted')), output(r).join('\n')); - assert.ok(checks > 1000, 'the broker-level handler fired'); - // The broker default is a floor: a per-eval handler overrides it - // (the broker's own closure must not fire while the override runs). - const brokerChecks = checks; - let perEval = 0; - const r2 = await broker.eval('let j = 0; while (true) j++;', { interruptHandler: () => ++perEval > 1000 }); - assert.ok(output(r2).some((l) => l.includes('InternalError: interrupted')), output(r2).join('\n')); - assert.ok(perEval > 1000, 'the per-eval handler fired'); - assert.equal(checks, brokerChecks, 'the broker default did not fire under the per-eval override'); - await ws.dispose(); -}); - -test('review 9: verify/judgePanel reviewers route through the RUNNER\'S DEFAULT BACKEND id — a real registered segment, never a validation bypass', async () => { - const { ws, broker, runner } = await setup(); - // verify() resolves its reviewers through '__host_default_backend' (no - // per-call model in the DSL options) — the fake's default backend id. - await broker.eval('const v = verify("some claim", { reviewers: 2 }); "started"'); - await tick(); - assert.equal(runner.openedWith.length, 2); - assert.ok( - runner.openedWith.every((o) => o.model === 'claude'), - 'the reviewers carry the REAL default backend id (admission-validated like any agent() call)', - ); - for (const session of runner.sessions) session.completeTurn('{"real": true, "reason": "ok"}'); - await tick(); - await broker.pump(); - const v = await broker.eval('await v'); - assert.equal(v.result, '{real: true, realCount: 2, total: 2, votes: [{…}, {…}]}'); - // judgePanel() graders route the same way. - await broker.eval('const jp = judgePanel(["a", "b"], { judges: 2 }); "started"'); - await tick(); - assert.equal(runner.openedWith.length, 6); - assert.ok(runner.openedWith.slice(2).every((o) => o.model === 'claude')); - // The deleted v1 sentinel is NOT a registered backend: a bare - // `agent("default", …)` rejects at admission, naming the segment and - // enumerating the known backends — never a silent route to the - // default backend. - const refused = await broker.eval('const m = await agent("default", "x").catch((e) => e.message); m'); - assert.equal( - refused.result, - 'unknown backend "default" in model spec "default" (known backends: claude, codex, opencode, pi)', - ); - await ws.dispose(); -}); - -test('review 10: dispose releases every session the broker opened, even with a host-injected runner', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - await dispatchAgent(broker, runner); - runner.last().completeTurn('done'); - await tick(); - await broker.pump(); - assert.equal(runner.sessions.length, 1); - await broker.dispose(); - assert.equal(runner.sessions[0].releases, 1, 'the dedicated session was released'); - assert.equal(runner.disposeCalls, 0, 'the injected runner itself is the host\'s to dispose'); - await ws.dispose(); -}); - -test('review round 3: the eval result\'s pending list reports the WHOLE guest registry — 300 parked checkpoints list all 300 ids, dense and in order (phase-E review round 3: the trap-free surface read capped arrays at 256 elements and its [ArrayTruncated] marker leaked into the id list as an undefined hole — the structured tool output\'s pending field silently truncated)', async () => { - const { ws, broker } = await setup(); - const r = await broker.eval('for (let i = 0; i < 300; i++) checkpoint("q-" + i); "asked"'); - assert.equal(r.pending.length, 300, 'every pending call id is listed'); - assert.equal(r.pending[0], 'c1'); - assert.equal(r.pending[255], 'c256'); - assert.equal(r.pending[256], 'c257', 'no cap truncation at the 256th entry'); - assert.equal(r.pending[299], 'c300'); - assert.ok(r.pending.every((id, index) => id === `c${index + 1}`), 'dense, in registry order, no holes'); - assert.equal(r.pending.length, r.checkpoints.length); - await ws.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// The eval-plane redesign surface (§4.4 `_`, §4.5 workspace()/agents()/ -// reset(), §4.6 error rendering, §4.7 sleep) -// ──────────────────────────────────────────────────────────────────────── - -test('§4.4: `_` holds the previous eval\'s completion value (IPython-style) — the sole result-history global; an error leaves it unchanged', async () => { - const { ws, broker } = await setup(); - await broker.eval('1 + 1'); - const second = await broker.eval('_ * 10'); - assert.equal(second.result, '20'); - // `_` advances with every resolved eval (IPython semantics). - const third = await broker.eval('({ tagged: _ })'); - assert.equal(third.result, '{tagged: 20}'); - // An error does not move `_` (IPython behavior). - await broker.eval('throw new Error("no")'); - const afterError = await broker.eval('_'); - assert.equal(afterError.result, '{tagged: 20}', 'the failed eval left `_` unchanged'); - // `_` is an ordinary writable global: bindings are the memory. - await broker.eval('_ = "mine"'); - assert.equal((await broker.eval('_')).result, 'mine'); - await ws.dispose(); -}); - -test('§4.6: an uncaught eval error renders name + message + the guest stack\'s top frames with line numbers in the submitted code', async () => { - const { ws, broker } = await setup(); - const r = await broker.eval('const boom = () => { throw new TypeError("bad shape"); };\nboom();'); - assert.equal(r.result, undefined); - const line = output(r).find((l) => l.startsWith('TypeError')); - assert.ok(line !== undefined, output(r).join('\n')); - assert.ok(line.includes('TypeError: bad shape'), line); - // The stack's top frames carry LINE NUMBERS in the submitted code - // (the eval's filename is the VM default ``). - assert.match(line, /at boom \(:1:\d+\)/, line); - assert.match(line, /:2:\d+/, line); - const primitive = await broker.eval('const marker = 1;\nthrow "primitive boom";'); - const primitiveLine = output(primitive).find((l) => l.startsWith('Error: primitive boom')); - assert.ok(primitiveLine !== undefined, output(primitive).join('\n')); - assert.match(primitiveLine, /:2:\d+/, primitiveLine); - await ws.dispose(); -}); - -test('§4.6: an uncaught error from a subagent call names the call id and the resolved backend', async () => { - const { ws, broker, runner } = await setup(); - // Await a call that rejects; the rejection is uncaught in the eval. - const r = await broker.eval('await agent("pi/x", "research"); "never"'); - assert.equal(r.result, undefined); - await tick(); - runner.last().failTurn(new Error('research failed')); - await tick(); - await broker.pump(); - const probe = await broker.eval('"probe"'); - const line = output(probe).find((l) => l.includes('Error: research failed')); - assert.ok(line !== undefined, output(probe).join('\n')); - assert.ok(line.includes('(call c1 on backend pi)'), line); - // [C]10: the late rejection rendering ALSO carries the guest stack's - // frames with LINE NUMBERS in the submitted code (the review probe - // reproduced only the bare name/message line). - assert.match(line, /:1:\d+/, line); - await ws.dispose(); -}); - -test('§4.7: sleep(ms) is a guest helper over a HOST-side timer — the eval suspends and its continuation resumes at the next settlement drain', async () => { - const { ws, broker } = await setup(); - const started = Date.now(); - const r = await broker.eval('const t0 = Date.now(); await sleep(30); console.log("elapsed", Date.now() - t0); "slept"'); - assert.equal(r.result, undefined, 'the eval suspended on the sleep'); - // The host timer settles within the window; the pump drains the - // continuation. - await new Promise((resolve) => setTimeout(resolve, 80)); - await broker.pump(); - const probe = await broker.eval('"probe"'); - const line = output(probe).find((l) => l.startsWith('elapsed')); - assert.ok(line !== undefined, output(probe).join('\n')); - const elapsed = Number(/elapsed (\d+)/.exec(line)![1]); - assert.ok(elapsed >= 20 && elapsed < 5000, `elapsed ${elapsed}`); - assert.ok(Date.now() - started >= 20, 'wall clock advanced'); - await ws.dispose(); -}); - -test('workspace() returns honest status and agents() lists addressable queued turns', async () => { - const { ws, broker, runner } = await setup(); - // c1 fails; c2 succeeds. Both are agent-handle bindings. - await broker.eval('const boom = agent("pi/x", "boom"); const ok = agent("pi/x", "ok"); "started"'); - await tick(); - runner.sessions[0].failTurn(new Error('nope')); - await tick(); - await broker.pump(); - runner.sessions[1].completeTurn('fine'); - await tick(); - await broker.pump(); - const w = await broker.eval('const w = workspace(); w.bindings.filter((b) => b.name === "boom" || b.name === "ok").map((b) => b.name + ":" + (b.status ?? "-")).join(",")'); - assert.equal(w.result, 'boom:failed,ok:settled', 'the honest failed status for the rejected handle call'); - const shape = await broker.eval('(() => { const w = workspace(); return JSON.stringify({ keys: Object.keys(w).sort(), diag: Object.keys(w.diagnostics).sort(), empty: w.inFlight.length === 0 }); })()'); - assert.deepEqual(JSON.parse(shape.result!), { keys: ['bindings', 'checkpoints', 'diagnostics', 'inFlight'], diag: ['childrenClosed', 'drainError', 'reconcile', 'reconcileNotes'], empty: true }); - // agents(): a queued turn gets its own addressable entry. - await broker.eval('const o = await ok.queue("more"); console.log("queued-answer", o); "done"'); - await tick(); - const a = await broker.eval('const a = agents(); a.filter((x) => x.callId === "c3").map((x) => x.state + "|" + x.task)[0]'); - assert.equal(a.result, 'running|more'); - runner.sessions[1].completeTurn('the answer'); - await tick(); - await broker.pump(); - const after = await broker.eval('"probe"'); - assert.ok(output(after).some((l) => l === 'queued-answer the answer'), output(after).join('\n')); - await ws.dispose(); -}); - -test('§4.5/§7: workspace().checkpoints keeps the retained 200-character question preview', async () => { - const { ws, broker } = await setup(); - await broker.eval('checkpoint("q".repeat(300)); "asked"'); - const retained = broker.checkpointSummaries()[0].question; - const guest = await broker.eval('workspace().checkpoints[0].question'); - assert.equal(guest.result, retained, 'workspace() exposes the same retained preview as checkpoint summaries'); - assert.notEqual(guest.result, 'q'.repeat(300), 'the raw question cannot bypass the metadata preview'); - await ws.dispose(); -}); - -test('§4.5: reset() tears the workspace down after the current eval completes (the host-side effect the deleted reset action performed)', async () => { - const { ws, broker } = await setup(); - const r = await broker.eval('console.log("bye"); reset(); "done"'); - assert.equal(r.result, 'done'); - assert.ok(output(r).includes('bye'), 'the eval that called reset() completed normally first'); - // After the eval, the workspace is gone (the VM disposed). - assert.equal(ws.isDisposed, true, 'reset() tore the workspace down after the eval completed'); - await assert.rejects(async () => broker.eval('1 + 1'), /disposed|alive/); - await ws.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Review-fix regressions: `_` after a late completion, reset() after a -// suspended eval, strict mid-turn steering, queued-turn schema answers, -// admission-refusal attribution, and the late-rejection stack frames -// ──────────────────────────────────────────────────────────────────────── - -test('§4.4: `_` updates when a SUSPENDED eval completes during a pump — the settled previous eval is the previous eval', async () => { - const { ws, broker } = await setup(); - const r = await broker.eval('await sleep(10); 42'); - assert.equal(r.result, undefined, 'the eval suspended on the sleep'); - await new Promise((resolve) => setTimeout(resolve, 50)); - await broker.pump(); - const probe = await broker.eval('_'); - assert.equal(probe.result, '42', 'the late completion value became `_` (the review probe: undefined before the fix)'); - // An empty poll (the documented eval("") idiom) COMPLETES with - // undefined — `_` becomes undefined: the previous eval's completion - // value IS undefined (the review probe: `42`, then an empty eval, - // then `_` must read undefined, never the stale 42). - await broker.eval('"tagged"'); - await broker.eval(''); - assert.equal((await broker.eval('_')).result, 'undefined'); - // The overwrite happens for the IN-CALL undefined completion too: - // `42`, then `undefined;` — `_` reads undefined. - await broker.eval('42'); - await broker.eval('undefined'); - assert.equal((await broker.eval('_')).result, 'undefined'); - await ws.dispose(); -}); - -test('§4.5: reset() in a SUSPENDED eval tears the workspace down BEFORE any later guest code — the reset-owning eval completed at the pump', async () => { - const { ws, broker } = await setup(); - const r = await broker.eval('reset(); await sleep(50); console.log("finished"); "done-after-sleep"'); - assert.equal(r.result, undefined, 'the eval suspended on the sleep'); - assert.equal(ws.isDisposed, false, 'the workspace is ALIVE while the reset eval is still in flight'); - // The host timer fires; the next operation's pump runs the continuation - // to completion. The reset-owning eval COMPLETED — its teardown is owed - // BEFORE the new eval's submitted code runs: the new eval rejects on - // the disposed workspace and its code never executes (the review - // probe: the later eval returned "probe-ran" before the disposal). - await new Promise((resolve) => setTimeout(resolve, 90)); - await assert.rejects(async () => broker.eval('"probe-ran"'), /disposed/); - assert.equal(ws.isDisposed, true, 'the teardown ran before the later eval\'s code'); - await ws.dispose(); -}); - -test('startedNewTurn is a fatal steering protocol violation, never a queued result', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - // The founding turn is in flight: the steer is a live mid-turn - // injection. A strict backend must never start a public turn here. - await broker.eval('const o = pi.steer("redirect"); "steered"'); - await tick(); - runner.last().completeSteer('startedNewTurn'); - await tick(); - await broker.pump(); - const r = await broker.eval('await o.catch(e => e.code + "/" + e.recoverable + "/" + e.details.reason)'); - assert.equal(r.result, 'AGENT_EXECUTION_ERROR/false/steering_started_new_turn'); - assert.equal(runner.last().cancelled, 1, 'the lane-fatal response triggers best-effort ACP cancellation'); - assert.equal(runner.last().prompts.length, 0, 'no replacement public turn was started'); - await ws.dispose(); -}); - -test('queue on a schema handle resolves the inherited schema-validated object', async () => { - const { ws, broker, runner } = await setup(); - await broker.eval( - 'const h = agent("pi/x", "orig", { schema: { type: "object", properties: { n: { type: "number" } }, required: ["n"] } }); "started"', - ); - await tick(); - runner.last().completeTurn('{"n": 7}'); - await tick(); - await broker.pump(); - const founding = await broker.eval('await h'); - assert.equal(founding.result, '{n: 7}'); - // The queued turn mints its own call id and resolves with the turn's - // SCHEMA-VALIDATED answer (not raw text). - await broker.eval('const f = h.queue("more"); "started"'); - await tick(); - runner.last().completeTurn('{"n": 42}'); - await tick(); - await broker.pump(); - const got = await broker.eval('await f'); - assert.equal(got.result, '{n: 42}'); - await ws.dispose(); -}); - -test('§4.6: a synchronous admission refusal with a RESOLVED backend names the backend in the uncaught-error rendering (call id + backend)', async () => { - const { ws, broker } = await setup(); - const r = await broker.eval('await agent("pi/x", "t", { bogus: 1 })'); - assert.equal(r.result, undefined); - const line = output(r).find((l) => l.startsWith('WorkflowError')); - assert.ok(line !== undefined, output(r).join('\n')); - assert.ok(line.includes('unknown option "bogus"'), line); - assert.ok(line.includes('(call c1 on backend pi)'), line); - await ws.dispose(); -}); - -test('§4.6: cancelling a QUEUED dispatch stamps the resolved backend on the rejection (call id + backend on every known-backend rejection path)', async () => { - const { ws, broker } = await setup({ maxConcurrentAgents: 1 }); - await broker.eval('const a = agent("pi/x", "first"); "started"'); - await tick(); - // The second dispatch queues above the cap; the interrupt cancels it - // by id — the rejection carries the call id and its resolved backend. - await broker.eval('const q = agent("pi/y", "queued").catch((e) => e.replBackend + "/" + e.replCallId); "started"'); - const outcome = await broker.cancelCall('c2'); - assert.equal(outcome, 'cancelled'); - const r = await broker.eval('await q'); - assert.equal(r.result, 'pi/c2', 'the queued-dispatch cancellation rejection names the resolved backend'); - await ws.dispose(); -}); - -test('§4.5: workspace().diagnostics carries the retained reconcile summary, the RETAINED drain error, and childrenClosed through both state transitions', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-broker-diag-')); - const storePath = join(dir, 'calls.jsonl'); - const store = JsonlCallStore.open(storePath); - let interruptDrains = false; - const { ws, broker, runner } = await setup({ store, interruptHandler: () => interruptDrains }); - // A reconcile report is retained under diagnostics. - await broker.reconcile(); - const d1 = await broker.eval('workspace().diagnostics.reconcile === null ? "null" : typeof workspace().diagnostics.reconcile'); - assert.equal(d1.result, 'object'); - // No drain error yet; children open before the client-presence drain. - const d2 = await broker.eval('workspace().diagnostics.drainError === null ? "null" : workspace().diagnostics.drainError.name'); - assert.equal(d2.result, 'null', 'no drain error yet'); - const closed0 = await broker.eval('workspace().diagnostics.childrenClosed'); - assert.equal(closed0.result, 'false', 'children open before the client-presence drain'); - // A FAILED settlement drain RETAINS its error under diagnostics: a - // runaway guest continuation interrupted mid-drain (the broker-level - // interrupt handler fires inside the pump's settlement drain) — the - // §6.2 demotion: the failure leaves the eval result surface and - // lives here; the pump reports it honestly and the VM stays usable. - await broker.eval('agent("pi/x", "task").then(() => { let j = 0; while (true) j++; }); "started"'); - await tick(); - runner.last().completeTurn('done'); - await tick(); - interruptDrains = true; - await assert.rejects( - () => broker.pump(), - (error: unknown) => (error as Error).name === 'DrainJobError', - 'the runaway continuation interrupts the settlement drain', - ); - interruptDrains = false; - const retained = await broker.eval( - 'workspace().diagnostics.drainError === null ? "null" : workspace().diagnostics.drainError.name + ":" + workspace().diagnostics.drainError.message', - ); - assert.equal(retained.result, 'InternalError:interrupted', 'the failed settlement drain is RETAINED under diagnostics'); - // childrenClosed reflects the client-presence drain: the settled - // session releases and the latch flips. - const drained = await broker.drainForDisconnect(200); - assert.equal(drained, true, 'the drain completed within its bound'); - const closed1 = await broker.eval('workspace().diagnostics.childrenClosed'); - assert.equal(closed1.result, 'true', 'childrenClosed flips after the client-presence drain'); - await broker.dispose(); - ws.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('a pending queued turn is targetable by exact interrupt id: durable AGENT_CANCELLED and no ACP request', async () => { - const { ws, broker, runner } = await setup({ maxConcurrentAgents: 1 }); - // c1 settles (idle); c2 takes the only slot. - await broker.eval('const a = agent("pi/x", "a"); "started"'); - await tick(); - runner.sessions[0].completeTurn('a done'); - await tick(); - await broker.pump(); - await broker.eval('const busy = agent("pi/x", "busy"); "started"'); - await tick(); - // The queued turn on the idle a-handle waits behind the cap (c3) — its - // guest promise stays pending, and the turn is addressable NOW. - const steered = await broker.eval('const o = await a.queue("more").catch(e => e.code); console.log("outcome", o); "done"'); - assert.equal(steered.result, undefined, 'the queued handle suspends until its turn runs'); - await tick(); - assert.ok(broker.liveAgents().some((a) => a.callId === 'c3' && a.state === 'queued'), 'c3 is listed while queued'); - assert.equal(runner.sessions[0].prompts.length, 0, 'no turn started under cap pressure'); - // The interrupt cancels the queued turn by its own id — durable marker, - // recoverable rejection, and the turn leaves agents(). - assert.equal(await broker.cancelCall('c3'), 'cancelled'); - assert.equal(broker.store().lookup('c3')!.completion!.outcome, 'reject'); - assert.notEqual(broker.store().lookup('c3')!.cancelledAtMs, null, 'the cancellation is recorded durably'); - const probe = await broker.eval('"probe"'); - assert.ok(output(probe).some((l) => l === 'outcome AGENT_CANCELLED'), output(probe).join('\n')); - assert.ok(!broker.liveAgents().some((a) => a.callId === 'c3'), 'the cancelled queued turn left agents()'); - // The freed slot is NOT consumed by the cancelled turn; the session - // stays idle with no delivery turn. - assert.equal(runner.sessions[0].prompts.length, 0); - await ws.dispose(); -}); - -test('cancelling an active queued turn renders the uncaught error with call id and backend', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - runner.last().completeTurn('done'); - await tick(); - await broker.pump(); - // An uncaught await of the queued turn: the cancellation rejection - // renders through the rejection bridge in the next tool result. - await broker.eval('const f = pi.queue("long job"); await f; "unreachable"'); - await tick(); - assert.equal(await broker.cancelCall('c2'), 'cancelled'); - await tick(); - await broker.pump(); - const probe = await broker.eval('"probe"'); - const line = output(probe).find((l) => l.includes('turn c2 was cancelled')); - assert.ok(line !== undefined, output(probe).join('\n')); - assert.ok(line.includes('(call c2 on backend pi)'), `the rendering names the call id and the resolved backend: ${line}`); - await ws.dispose(); -}); - -test('a malformed steering outcome rejects with invalid_steering_response and triggers the fatal-lane procedure', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - await broker.eval('const o = pi.steer("redirect"); "steered"'); - await tick(); - // A backend outcome outside the strict protocol is lane-fatal. - runner.last().completeSteer('surprise'); - await tick(); - await broker.pump(); - assert.equal( - (await broker.eval('await o.catch(e => e.code + "/" + e.recoverable + "/" + e.details.reason)')).result, - 'AGENT_EXECUTION_ERROR/false/invalid_steering_response', - ); - assert.equal(runner.last().cancelled, 1); - assert.equal(runner.last().prompts.length, 0, 'malformed steering never creates a prompt'); - await ws.dispose(); -}); - -test('steering controls are wire-serialized and block the next queue turn across the prior turn boundary', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - const session = runner.last(); - await broker.eval( - 'const s1 = pi.steer("first control"); const s2 = pi.steer("second control"); const afterControls = pi.queue("future turn"); "issued"', - ); - await tick(); - assert.deepEqual(session.steers.map((steer) => steer.content), ['first control']); - - session.completeSteer('injected'); - await tick(); - await broker.pump(); - assert.deepEqual(session.steers.map((steer) => steer.content), ['second control']); - - session.completeTurn('founding answer'); - await tick(); - await broker.pump(); - assert.equal(session.prompts.length, 0, 'the queued turn waits for the unresolved second control'); - - session.completeSteer('promptRequired'); - await tick(); - await broker.pump(); - await tick(); - assert.equal((await broker.eval('await s1')).result, 'injected'); - assert.equal((await broker.eval('await s2')).result, 'idle'); - assert.equal(session.prompts[0]?.content, 'future turn'); - session.completeTurn('future answer'); - await tick(); - await broker.pump(); - assert.equal((await broker.eval('await afterControls')).result, 'future answer'); - await ws.dispose(); -}); - -test('the oldest eligible admission wins a free slot across founding calls and queued turns', async () => { - const { ws, broker, runner } = await setup({ maxConcurrentAgents: 1 }); - await broker.eval('const a = agent("pi/x", "a"); const aq = a.queue("older queued turn"); const b = agent("pi/x", "newer founding turn"); "issued"'); - await tick(); - const first = runner.sessions[0]; - first.completeTurn('a answer'); - await tick(); - await broker.pump(); - await tick(); - assert.equal(first.prompts[0]?.content, 'older queued turn', 'the older eligible queue head wins before b opens'); - assert.equal(runner.sessions.length, 1); - first.completeTurn('queued answer'); - await tick(); - await broker.pump(); - await tick(); - assert.equal(runner.sessions.length, 2, 'the newer founding call starts after the older queue turn'); - runner.sessions[1].completeTurn('b answer'); - await tick(); - await broker.pump(); - assert.equal((await broker.eval('await aq')).result, 'queued answer'); - await ws.dispose(); -}); - -test('an ineligible older queue head does not block eligible work on another session', async () => { - const { ws, broker, runner } = await setup({ maxConcurrentAgents: 2 }); - await broker.eval('const a = agent("pi/x", "a"); const blocked = a.queue("wait behind a"); const b = agent("pi/x", "b"); "issued"'); - await tick(); - assert.equal(runner.sessions.length, 2, 'b starts in the free slot despite the older ineligible queue item'); - assert.equal(runner.sessions[0].prompts.length, 1); - assert.equal(runner.sessions[1].prompts.length, 1); - runner.sessions[0].completeTurn('a answer'); - runner.sessions[1].completeTurn('b answer'); - await tick(); - await broker.pump(); - await tick(); - assert.equal(runner.sessions[0].prompts[0]?.content, 'wait behind a'); - runner.sessions[0].completeTurn('blocked answer'); - await tick(); - await broker.pump(); - assert.equal((await broker.eval('await blocked')).result, 'blocked answer'); - await ws.dispose(); -}); - -test('an active queued turn that ignores cancellation for 5 seconds makes the lane fatal', { timeout: 10_000 }, async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - const session = runner.last(); - session.completeTurn('done'); - await tick(); - await broker.pump(); - await broker.eval('const wedged = pi.queue("wedged"); const later = pi.queue("must fail"); "issued"'); - await tick(); - session.cancel = async () => { session.cancelled++; }; - assert.equal(await broker.cancelCall('c2'), 'cancelled'); - await new Promise((resolve) => setTimeout(resolve, 5_100)); - await broker.pump(); - const later = broker.store().lookup('c3')!.completion; - assert.equal(later?.outcome, 'reject'); - assert.equal( - (later?.value as { details?: { reason?: string } }).details?.reason, - 'cancellation_not_honored', - ); - assert.equal(session.releases, 1, 'the unusable session was quarantined/released'); - await ws.dispose(); -}); - -test('§4.1 [C]5: an independently failing diagnostic reopen preserves the original mode error and never blames configOptions', async () => { - const { ws, broker, runner } = await setup(); - runner.failModes = new Set(['default']); - const late = await broker.eval( - 'const p = await agent("pi/openai/model", "t", { mode: "default", configOptions: { thinkingLevel: "high" } }).catch(e => e.name + ": " + e.message); console.log("got", p); "done"', - ); - assert.equal(late.result, undefined, 'the late rejection arrives after the eval suspended'); - await tick(); - await broker.pump(); - const probe = await broker.eval('"probe"'); - const line = output(probe).find((l) => l.startsWith('got Error')); - assert.ok(line !== undefined, output(probe).join('\n')); - assert.ok(line.includes('cannot apply session mode "default" (advertised modes: none)'), line); - assert.ok(!line.includes('ConfigOptionsError'), line); - assert.ok(!line.includes('offending key'), line); - assert.ok(!line.includes('thinkingLevel'), line); - await ws.dispose(); -}); - -test('the interrupt targets an active queued turn by its own call id', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - runner.last().completeTurn('done'); - await tick(); - await broker.pump(); - const evaled = await broker.eval('const o = await pi.queue("long job").catch(e => e.code + "/" + e.recoverable); console.log("got", o); "done"'); - assert.equal(evaled.result, undefined, 'the queued turn is in flight'); - await tick(); - assert.ok(broker.liveAgents().some((a) => a.callId === 'c2' && a.state === 'running')); - // The interrupt (cancel by id) targets the TURN, not the founding call. - assert.equal(await broker.cancelCall('c2'), 'cancelled'); - await tick(); - await broker.pump(); - const probe = await broker.eval('"probe"'); - assert.ok(output(probe).some((l) => l === 'got AGENT_CANCELLED/true'), output(probe).join('\n')); - // The founding call's session stays usable (idle — the founding call - // itself was settled long before). - assert.equal(await broker.cancelCall('c1'), 'idle'); - await ws.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Review-rejection regressions (eval-plane redesign, review round 3): -// guest cancellation of cap-queued dispatches, queued-turn addressability -// during delayed lazy re-attachment, restored answer-mode queues without -// an attached founding session, cancellation with queued answer-mode -// siblings, reset ownership racing an unrelated suspended eval, and -// verbatim long modelSpec values in agents(). -// ──────────────────────────────────────────────────────────────────────── - -test('§4.1/§4.2: the guest handle\'s cancel() reaches a founding dispatch QUEUED above the cap — AGENT_CANCELLED, recorded durably, never a late prompt', async () => { - const { ws, broker, runner } = await setup({ maxConcurrentAgents: 1 }); - await broker.eval('const a = agent("pi/x", "a"); const b = agent("pi/y", "b"); "started"'); - assert.equal(runner.sessions.length, 1, 'only a dispatched — b waits in the dispatch queue'); - // The handle's OWN cancel (the review probe: h.cancel() fell through - // to `failed` while the supposedly-cancelled queued dispatch later - // opened and prompted when a slot freed). - const steered = await broker.eval('const o = await b.cancel(); "cancelled:" + o'); - assert.equal(steered.result, 'cancelled:cancelled', 'the handle cancel reports the honest cancelled outcome'); - const record = broker.store().lookup('c2')!; - assert.equal(record.completion!.outcome, 'reject', 'the queued dispatch settled durably'); - assert.equal((record.completion!.value as { code?: string }).code, 'AGENT_CANCELLED'); - const got = await broker.eval('await b.catch((e) => e.code + "/" + e.recoverable)'); - assert.equal(got.result, 'AGENT_CANCELLED/true', 'the queued founding call rejects recoverable'); - // Free the slot: the cancelled queued dispatch must never open a - // session or prompt. - await tick(); - runner.sessions[0].completeTurn('a done'); - await tick(); - await broker.pump(); - await tick(); - assert.equal(runner.sessions.length, 1, 'the cancelled queued dispatch never dispatched'); - assert.equal(runner.sessions[0].prompts.length, 0); - await ws.dispose(); -}); - -test('a queue on a drained settled handle is visible and interrupt-cancelable while lazy reattachment is in flight', async () => { - const { ws, broker, runner } = await setup(); - await dispatchAgent(broker, runner); - runner.last().completeTurn('done'); - await tick(); - await broker.pump(); - // The client-presence drain releases every child (the founding - // session is gone — queue re-attaches it lazily). - assert.equal(await broker.drainForDisconnect(200), true); - // Park the lazy load: the turn exists while the load is in flight. - runner.parkLoads = true; - const evaled = await broker.eval('const o = await pi.queue("more").catch(e => e.code); console.log("got", o); "done"'); - assert.equal(evaled.result, undefined, 'the queued handle suspends until its turn answers'); - await tick(); - assert.equal(runner.parkedLoads.length, 1, 'the lazy re-attach load is parked'); - // The review probe: during the delayed load the minted call was - // omitted from agents() and interrupt returned `none`. The turn must - // be visible and targetable from mint time. - const agents = broker.liveAgents(); - assert.ok( - agents.some((a) => a.callId === 'c2' && a.state === 'queued' && a.task === 'more' && a.modelSpec === 'pi/deepseek-v4-flash-max'), - `the minted turn is listed while the load is in flight: ${JSON.stringify(agents)}`, - ); - assert.equal(await broker.cancelCall('c2'), 'cancelled', 'interrupt targets the loading turn'); - // The load lands: the cancelled turn must never start — it settles - // with the recoverable AGENT_CANCELLED, cancelled durably. - const loaded = runner.releaseParkedLoad(); - await tick(); - await broker.pump(); - await tick(); - assert.equal(loaded.prompts.length, 0, 'the cancelled turn never prompted the re-attached session'); - const record = broker.store().lookup('c2')!; - assert.equal(record.completion!.outcome, 'reject'); - assert.notEqual(record.cancelledAtMs, null, 'the cancellation is recorded durably'); - const probe = await broker.eval('"probe"'); - assert.ok(output(probe).some((l) => l === 'got AGENT_CANCELLED'), output(probe).join('\n')); - assert.ok(!broker.liveAgents().some((a) => a.callId === 'c2'), 'the cancelled turn left agents()'); - await ws.dispose(); -}); - -test('restore: queued turns on a settled handle remain addressable through reattachment and deliver their own answers', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-broker-restore-queued-followup-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner, maxConcurrentAgents: 1 }); - // c1 opens + settles (idle); c2 takes the ONLY slot. - await broker.eval('const a = agent("pi/x", "a"); "started"'); - await tick(); - runner.sessions[0].completeTurn('a done'); - await tick(); - await broker.pump(); - await broker.eval('const b = agent("pi/x", "b"); "started"'); - await tick(); - // Two queued turns on the idle a-handle wait under the cap (c3, c4) — - // their promises stay pending; the store records the queued markers. - await broker.eval('const o3 = await a.queue("three").catch(e => e.code); console.log("got3", o3); "done"'); - await broker.eval('const o4 = await a.queue("four").catch(e => e.code); console.log("got4", o4); "done"'); - assert.equal(runner.sessions[0].prompts.length, 0, 'no delivery under cap pressure'); - assert.notEqual(broker.store().lookup('c3')!.queuedAtMs, null, 'the queued marker is durable'); - assert.notEqual(broker.store().lookup('c4')!.queuedAtMs, null); - // Simulated crash: snapshot + dispose with both queued turns pending. - const snapshot = ws.snapshot(); - await broker.dispose(); - ws.dispose(); - // Restore over the same store. - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - runner2.parkLoads = true; - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath), maxConcurrentAgents: 1 }); - // Reconcile parks on c2's re-attach load (the first parked load). - const reconciling = broker2.reconcile(); - await tick(); - assert.equal(runner2.parkedLoads.length, 1, 'c2\'s re-attach load is parked'); - const reattachedC2 = runner2.releaseParkedLoad(); - await reconciling; - await tick(); - // The rebuild scheduled c1's lazy re-attach (the second parked load): - // the re-queued turns are registered with NO attached session — the - // review probe: agents() omitted them and interrupt returned `none`. - assert.equal(runner2.parkedLoads.length, 1, 'the founding session\'s lazy re-attach is parked'); - const agents = broker2.liveAgents(); - assert.ok( - agents.some((a) => a.callId === 'c3' && a.state === 'queued' && a.task === 'three'), - `the restored queued turn c3 is listed while its session re-attaches: ${JSON.stringify(agents)}`, - ); - assert.ok( - agents.some((a) => a.callId === 'c4' && a.state === 'queued' && a.task === 'four'), - `the restored queued turn c4 is listed while its session re-attaches: ${JSON.stringify(agents)}`, - ); - // Interrupt targets the queued turn while the load is parked. - assert.equal(await broker2.cancelCall('c3'), 'cancelled'); - assert.equal(broker2.store().lookup('c3')!.completion!.outcome, 'reject'); - assert.notEqual(broker2.store().lookup('c3')!.cancelledAtMs, null, 'the cancellation is durable'); - assert.ok(!broker2.liveAgents().some((a) => a.callId === 'c3'), 'the cancelled turn left agents()'); - // The load lands; c4 merges into the rebuilt session's queue and - // waits for capacity (c2 still holds the only slot). - const loadedFounding = runner2.releaseParkedLoad(); - await tick(); - assert.equal(loadedFounding.prompts.length, 0, 'no delivery while the cap is exhausted'); - assert.ok( - broker2.liveAgents().some((a) => a.callId === 'c4' && a.state === 'queued'), - 'c4 waits attached and queued behind the cap', - ); - // c2's re-attached loaded turn completes; its slot frees and the - // queued turn delivers — settling the restored guest promise - // with the TURN'S ANSWER. - assert.equal(reattachedC2.loadedTurns.length, 1, 'the re-attach observes the loaded turn'); - reattachedC2.loadedTurns[0].resolve({ stopReason: 'end_turn', text: 'b done' }); - await tick(); - await broker2.pump(); - await tick(); - assert.equal(loadedFounding.prompts.length, 1, 'the queued turn started once a slot freed'); - assert.equal(loadedFounding.prompts[0].content, 'four'); - loadedFounding.completeTurn('four results'); - await tick(); - await broker2.pump(); - // The restored SUSPENDED evals resumed at their settlements: the - // cancelled turn's continuation printed its recoverable rejection and - // the delivered turn's continuation printed the TURN'S ANSWER (the - // §4.2 promise semantics survive the restore). - const probe = await broker2.eval('"probe"'); - assert.ok(output(probe).some((l) => l === 'got3 AGENT_CANCELLED'), output(probe).join('\n')); - assert.ok(output(probe).some((l) => l === 'got4 four results'), output(probe).join('\n')); - assert.equal(broker2.store().lookup('c4')!.completion!.value, 'four results', 'the delivered turn recorded its answer'); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('cancelling one active queued turn preserves its later FIFO sibling', async () => { - const { ws, broker, runner } = await setup({ maxConcurrentAgents: 1 }); - // c1 opens + settles (idle); c2 takes the only slot. - await broker.eval('const a = agent("pi/x", "a"); "started"'); - await tick(); - runner.sessions[0].completeTurn('a done'); - await tick(); - await broker.pump(); - await broker.eval('const busy = agent("pi/x", "busy"); "started"'); - await tick(); - // Two queued turns wait behind the cap (c3, c4). - await broker.eval('const o3 = await a.queue("three").catch(e => e.code); console.log("got3", o3); "done"'); - await broker.eval('const o4 = await a.queue("four").catch(e => e.code); console.log("got4", o4); "done"'); - await tick(); - assert.ok(broker.liveAgents().some((a) => a.callId === 'c3' && a.state === 'queued')); - assert.ok(broker.liveAgents().some((a) => a.callId === 'c4' && a.state === 'queued')); - // c2 settles: the kick starts c3's delivery turn (in flight). - runner.sessions[1].completeTurn('busy done'); - await tick(); - await broker.pump(); - await tick(); - assert.equal(runner.sessions[0].prompts.length, 1, 'the first queued turn is active'); - // The interrupt cancels only active c3. c4 remains pending and starts - // after the backend acknowledges the cancellation settlement. - assert.equal(await broker.cancelCall('c3'), 'cancelled'); - await tick(); - await broker.pump(); - await tick(); - const c4 = broker.store().lookup('c4')!; - assert.equal(c4.completion, null, 'the later sibling remains live'); - assert.equal(runner.sessions[0].prompts.length, 1); - assert.equal(runner.sessions[0].prompts[0].content, 'four'); - runner.sessions[0].completeTurn('four results'); - await tick(); - await broker.pump(); - const probe = await broker.eval('"probe"'); - assert.ok(output(probe).some((l) => l === 'got3 AGENT_CANCELLED'), output(probe).join('\n')); - assert.ok(output(probe).some((l) => l === 'got4 four results'), output(probe).join('\n')); - await ws.dispose(); -}); - -test('§4.5: reset() ownership is the reset-calling eval ALONE — an unrelated suspended eval never gates the teardown (and a continuation-called reset() attributes through the continuation token)', async () => { - const { ws, broker } = await setup(); - // The reset-owning eval suspends on a sleep; an UNRELATED eval - // suspends on a longer one. The review probe: the unrelated eval's - // suspension joined the owning set, so completing the reset-calling - // eval left the workspace running guest code while the unrelated - // eval stayed suspended. - const resetEval = await broker.eval('reset(); await sleep(50); console.log("reset-finished"); "reset-result"'); - assert.equal(resetEval.result, undefined, 'the reset-owning eval suspended'); - const unrelated = await broker.eval('await sleep(500); console.log("unrelated-finished"); "unrelated-result"'); - assert.equal(unrelated.result, undefined, 'the unrelated eval suspended'); - assert.equal(ws.isDisposed, false, 'the workspace is alive while the reset eval is in flight'); - // The reset-owning eval's sleep fires; the unrelated eval is STILL - // suspended. The teardown is owed the moment the reset-calling eval - // completes — the next eval must reject on the disposed workspace - // (never run its code, never wait for the unrelated eval). - await new Promise((resolve) => setTimeout(resolve, 90)); - await assert.rejects(async () => broker.eval('"probe-ran"'), /disposed/); - assert.equal(ws.isDisposed, true, 'the teardown depended only on the reset-calling eval'); - await ws.dispose(); -}); - -test('§4.5: reset() called from a RESUMED continuation (`await sleep(…); reset()`) attributes to the reset-calling eval — the teardown runs at the pump that completes it', async () => { - const { ws, broker } = await setup(); - const r = await broker.eval('await sleep(30); reset(); console.log("continuation-reset"); "done-after-reset"'); - assert.equal(r.result, undefined, 'the eval suspended on the sleep'); - assert.equal(ws.isDisposed, false, 'the workspace is alive while the eval is in flight'); - await new Promise((resolve) => setTimeout(resolve, 80)); - // The pump runs the continuation (which calls reset()); the sweep - // releases the completed wrapper and the serialized-op post-hook - // tears the workspace down — before any later guest code. - await broker.pump(); - assert.equal(ws.isDisposed, true, 'the continuation-called reset() tore the workspace down at the completing pump'); - await assert.rejects(async () => broker.eval('1 + 1'), /disposed|alive/); - await ws.dispose(); -}); - -test('§4.5: reset() after an immediately-resolved local await (the eval\'s OWN drain) is still THIS eval\'s request — the teardown follows that eval\'s completion', async () => { - const { ws, broker } = await setup(); - // The continuation of `await Promise.resolve(0)` runs in the eval's - // OWN drain phase — the reset() call there belongs to THIS eval (the - // per-eval flag, discriminated by the continuation token), never to - // whichever eval's token the lease mirror held last. - const r = await broker.eval('await Promise.resolve(0); reset(); "done-after-local-await"'); - assert.equal(r.result, 'done-after-local-await'); - assert.equal(ws.isDisposed, true, 'the own-drain reset() tore the workspace down after the eval completed'); - await assert.rejects(async () => broker.eval('1 + 1'), /disposed|alive/); - await ws.dispose(); -}); - -test('agents() carries modelSpec verbatim for session and queued-turn entries', async () => { - const { ws, broker, runner } = await setup(); - const spec = 'pi/' + 'x'.repeat(500); - await broker.eval(`const h = agent(${JSON.stringify(spec)}, "task"); "started"`); - await tick(); - const sessionEntry = await broker.eval( - `(() => { const a = agents()[0]; return a.modelSpec.length + ":" + a.modelSpec.slice(0, 3) + ":" + a.modelSpec.slice(-3); })()`, - ); - assert.equal(sessionEntry.result, '503:pi/:xxx', 'the session entry carries the whole 503-char spec'); - // The queued-turn entry renders the founding session's spec - // verbatim too (the review probe read a 200-char preview back). - runner.last().completeTurn('done'); - await tick(); - await broker.pump(); - await broker.eval('const f = h.queue("more"); "started"'); - await tick(); - const turnEntry = await broker.eval( - `(() => { const t = agents().find((a) => a.callId === "c2"); return t.modelSpec.length + ":" + (t.modelSpec === ${JSON.stringify(spec)}); })()`, - ); - assert.equal(turnEntry.result, '503:true', 'the queued-turn entry carries the whole spec verbatim'); - await ws.dispose(); -}); diff --git a/packages/repl-engine/test/eval-break-channel.test.ts b/packages/repl-engine/test/eval-break-channel.test.ts deleted file mode 100644 index 1e406510..00000000 --- a/packages/repl-engine/test/eval-break-channel.test.ts +++ /dev/null @@ -1,218 +0,0 @@ -/** - * Eval-break channel tests (phase-F review rounds 2–4 — the out-of-band - * interrupt delivery): the worker-thread relay's HTTP endpoint arms a - * shared-memory flag that the probe consumes with the arm-after-start - * rule. The rule is ordered by a SHARED MONOTONIC ARM-SEQUENCE COUNTER - * (round 3: the wall-clock comparison was replaced — a break arriving - * in the same millisecond as the execution's start was consumed as - * stale and permanently lost; a sequence has no resolution window), so - * a break armed after the execution began breaks it — down to the same - * instant — and a stale break (armed while the workspace was idle) is - * consumed-and-dropped on first observation and never breaks a later - * execution. Slots grow on demand (no project-count ceiling) and are - * released by `unregister` (round 3: the old fixed 64-slot array threw - * on the 65th registered project and never released slots). - * - * Round 4 — registration is ACKNOWLEDGED and slot assignments are - * GENERATION-FENCED: `register` resolves only once the worker applied - * the key→slot mapping (a first interrupt can never 404 against an - * unapplied mapping), and a stale arm for a RELEASED incarnation of a - * slot (the worker still held the old key's mapping when its `/break` - * landed) can never break the workspace that reuses the slot — the - * arm carries the old generation, which the new key's consume drops. - */ - -import assert from 'node:assert/strict'; -import { test } from 'node:test'; - -import { createEvalBreakChannel, type EvalBreakChannel } from '../src/index.js'; - -async function armViaHttp(channel: EvalBreakChannel, key: string): Promise { - const response = await fetch(`${await channel.breakUrl()}`, { - method: 'POST', - headers: { 'content-type': 'application/json' }, - body: JSON.stringify({ key }), - }); - return response.status; -} - -async function armOkViaHttp(channel: EvalBreakChannel, key: string): Promise { - const status = await armViaHttp(channel, key); - assert.equal(status, 204, 'the relay arms registered keys'); -} - -test('the channel arms via its HTTP endpoint and the probe consumes with the arm-after-start rule', async () => { - const channel = createEvalBreakChannel(); - try { - await channel.register('ws-a'); - // Unknown keys are refused by the relay (404) and by the probe. - assert.equal(await armViaHttp(channel, 'ws-nope'), 404); - assert.equal(channel.consumeBreak('ws-nope', channel.executionStartMarker()), false); - - // A break armed BEFORE the execution began (a stale flag — the - // workspace was idle when the interrupt was fired) is consumed-and- - // dropped: the probe returns false and the flag is gone. - await armOkViaHttp(channel, 'ws-a'); - const staleSince = channel.executionStartMarker(); // the execution began AFTER the arm - assert.equal(channel.consumeBreak('ws-a', staleSince), false, 'stale break dropped'); - assert.equal(channel.consumeBreak('ws-a', staleSince), false, 'the stale flag was consumed — nothing lingers'); - - // A break armed AFTER the execution began breaks it — exactly once - // (the consume-on-observation semantics). The marker is read BEFORE - // the arm: the arm's sequence strictly follows it, so the break is - // delivered even when both fall within the same wall-clock - // millisecond (phase-F review round 3: the old Date.now() - // comparison required armedAt > executionStartMs and lost a same-ms - // break as stale). - const t0 = channel.executionStartMarker(); - await armOkViaHttp(channel, 'ws-a'); - assert.equal(channel.consumeBreak('ws-a', t0), true, 'the running execution breaks'); - assert.equal(channel.consumeBreak('ws-a', t0), false, 'consumed on first observation'); - - // clearBreak drops an armed flag without consuming it. - await armOkViaHttp(channel, 'ws-a'); - channel.clearBreak('ws-a'); - assert.equal(channel.consumeBreak('ws-a', channel.executionStartMarker()), false, 'cleared flags never fire'); - - // A break armed AFTER a later execution began breaks THAT execution - // (the per-execution marker, not a global one). - const t2 = channel.executionStartMarker(); - await armOkViaHttp(channel, 'ws-a'); - assert.equal(channel.consumeBreak('ws-a', t2 - 1), true, 'armed after the execution start'); - } finally { - await channel.dispose(); - } -}); - -test('the channel is per-key: arming one workspace never breaks another', async () => { - const channel = createEvalBreakChannel(); - try { - await channel.register('ws-a'); - await channel.register('ws-b'); - const t0 = channel.executionStartMarker(); - await armOkViaHttp(channel, 'ws-a'); - assert.equal(channel.consumeBreak('ws-a', t0), true, 'the armed workspace breaks'); - assert.equal(channel.consumeBreak('ws-b', t0), false, 'the sibling workspace never fires'); - // A stale arm for one key does not consume another key's later arm. - const t1 = channel.executionStartMarker(); - await armOkViaHttp(channel, 'ws-b'); - assert.equal(channel.consumeBreak('ws-b', t1), true); - } finally { - await channel.dispose(); - } -}); - -test('re-registration is idempotent and the slot table survives it', async () => { - const channel = createEvalBreakChannel(); - try { - await channel.register('ws-a'); - await channel.register('ws-a'); - const t0 = channel.executionStartMarker(); - await armOkViaHttp(channel, 'ws-a'); - assert.equal(channel.consumeBreak('ws-a', t0), true, 'the re-registered key still arms'); - } finally { - await channel.dispose(); - } -}); - -test('registration is ACKNOWLEDGED: after `await register` the relay never 404s for the key (round 4: the fire-and-forget registration let a first interrupt hit an unapplied mapping)', async () => { - const channel = createEvalBreakChannel(); - try { - // The ack gate: the promise resolves only once the worker APPLIED - // the mapping — an immediately-following arm must succeed. - const registration = channel.register('ws-ack'); - await registration; - assert.equal(await armViaHttp(channel, 'ws-ack'), 204, 'the acked mapping arms immediately'); - // Idempotent re-registration resolves against the live mapping - // (the ack is already settled — no second round trip needed). - await channel.register('ws-ack'); - // A pending registration whose channel dies rejects instead of - // hanging (awaiting brokers degrade to the deadline bound). The - // outcome handler is attached BEFORE the dispose so the rejection - // is never unhandled. - const channel2 = createEvalBreakChannel(); - const dying = channel2.register('ws-doomed'); - const outcome = dying.then( - () => 'resolved', - (error: Error) => `rejected: ${error.message}`, - ); - await channel2.dispose(); - assert.match(await outcome, /rejected: .*disposed/, 'the pending ack rejects when the channel dies'); - } finally { - await channel.dispose(); - } -}); - -test('released slots are GENERATION-FENCED: a late arm for the released key can never break the workspace that reuses the slot (round 4)', async () => { - const channel = createEvalBreakChannel(); - try { - await channel.register('old'); - channel.unregister('old'); - // The re-registration is NOT awaited: the worker may still hold the - // released 'old'→slot mapping when its `/break` arrives (the - // register message for 'new' is in flight) — exactly the round-4 - // stale-arm window. Whichever interleaving wins, the new key must - // never observe the break: - // - the arm lands before the worker applies 'new' → it writes the - // OLD generation into the reused slot → 'new's consume drops it; - // - the worker already applied 'new' → the arm 404s (no flag). - const reRegistration = channel.register('new'); - const t0 = channel.executionStartMarker(); - const lateArm = await armViaHttp(channel, 'old'); - assert.ok([204, 404].includes(lateArm), `the late arm either lands (204) or 404s: ${lateArm}`); - // Let the re-registration settle (the ack may still be in flight) - // and consume repeatedly: any flag the late arm wrote is - // generation-dropped, and nothing lingers for a later execution. - await reRegistration; - for (let i = 0; i < 5; i++) { - assert.equal(channel.consumeBreak('new', t0), false, 'the reused slot never fires for the new key'); - } - // The new key still arms and breaks normally under its own - // generation once it is registered and acked. - const t1 = channel.executionStartMarker(); - await armOkViaHttp(channel, 'new'); - assert.equal(channel.consumeBreak('new', t1), true, 'the re-registered key arms again'); - } finally { - await channel.dispose(); - } -}); - -test('slots GROW beyond the initial capacity (no project-count ceiling) and unregister RELEASES them for reuse', async () => { - const channel = createEvalBreakChannel(); - try { - // Register more keys than the initial slot capacity: the shared - // buffer grows on demand instead of refusing (phase-F review round - // 3: the old fixed 64-slot channel threw on capacity exhaustion — - // the roadmap defines per-project workspaces with no project-count - // cap). - const keys = Array.from({ length: 40 }, (_, i) => `ws-${String(i).padStart(2, '0')}`); - for (const key of keys) await channel.register(key); - const t0 = channel.executionStartMarker(); - for (const key of keys) { - await armOkViaHttp(channel, key); - assert.equal(channel.consumeBreak(key, t0), true, `${key} arms and breaks after growth`); - } - // An unregistered key is refused again... - channel.unregister('ws-17'); - const t1 = channel.executionStartMarker(); - assert.equal(await armViaHttp(channel, 'ws-17'), 404, 'the unregistered key is unknown to the relay'); - assert.equal(channel.consumeBreak('ws-17', t1), false, 'the unregistered key never fires'); - // ...and its slot is REUSED by the next registration (the released - // slot's stale flag was cleared with it — the reused slot never - // fires for the new key without a fresh arm). - await channel.register('ws-17'); - const t2 = channel.executionStartMarker(); - assert.equal(channel.consumeBreak('ws-17', t2), false, 'the reused slot carries no stale flag'); - await armOkViaHttp(channel, 'ws-17'); - assert.equal(channel.consumeBreak('ws-17', t2), true, 'the re-registered key arms again'); - // Idempotent unregister / unregister of an unknown key are no-ops. - channel.unregister('ws-17'); - channel.unregister('ws-17'); - channel.unregister('never-registered'); - const t3 = channel.executionStartMarker(); - await armOkViaHttp(channel, 'ws-00'); - assert.equal(channel.consumeBreak('ws-00', t3), true, 'the other keys keep working across the unregisters'); - } finally { - await channel.dispose(); - } -}); diff --git a/packages/repl-engine/test/eval-break.test.ts b/packages/repl-engine/test/eval-break.test.ts deleted file mode 100644 index 3a3f2fd3..00000000 --- a/packages/repl-engine/test/eval-break.test.ts +++ /dev/null @@ -1,1140 +0,0 @@ -/** - * Phase-E review rejection round 2 regression suite, pinned at the - * engine boundary: the `interrupt` tool's two paths against a REAL - * currently-executing (in-flight) eval, and the `wait` tool's chain - * behavior under concurrency. - * - * 1. `waitForCalls` RELEASES the broker serialization chain between its - * pumps: a concurrent `cancelCall` (and `armEvalBreak`) completes - * mid-wait instead of queueing behind the whole bounded poll (up to - * 120 s) — an interrupt can cancel or break while the wait is still - * pumping, and the wait's very next pump observes the result. - * 2. The no-id interrupt (`armEvalBreak`) breaks a RUNNING runaway - * eval that is executing ACROSS drains: an eval whose body loops - * over subagent calls (yielding between iterations) is in flight - * while the wait pumps it; the interrupt lands mid-flight; the - * wait's next pump resumes the loop's next iteration and the quickjs - * interrupt handler breaks it MID-RUN. (The old daemon test only - * exercised a suspended continuation resumed later — the signal was - * armed against an eval that had never executed.) - * 3. The eval-break signal rides a direct eval's OWN drain too: a - * suspended eval's continuation resumed by a SYNCHRONOUS - * host-callback settlement (`checkpoint.answer` in a later eval) - * executes inside that eval's drain — where the old - * settlement-drain-only signal was blind, so the runaway burned the - * eval deadline instead of being broken by the interrupt. The - * interrupted drain releases the tracked eval (no stale arm target). - * - * Round 3 (the carried review's defects) adds: - * 4. The signal is keyed to the armed target's CONTINUATION, not to - * whichever drain runs next: an unrelated finite eval whose own - * drain executes real bytecode (polling the interrupt handler many - * times) is neither broken nor consumes the signal, and an - * unrelated settlement drain (a call no tracked eval awaits) does - * not fire it either — the armed state survives unrelated drains - * intact and breaks the target at its actual next execution. - * 5. A no-id interrupt TERMINATES every running eval it cannot arm — an - * in-flight eval suspended on nothing resumable (a never-settling - * local promise: no pending host call, no pending sleep) is - * RELEASED (its tracked continuation dropped, the token-keyed seam - * recording the termination) — `refused-idle` is honest only when - * NOTHING is running (the review defect: the eval was still - * running, so refusing was neither a break nor an honest idle - * refusal). A pending `sleep` keeps an eval armable: its host - * timer's settlement drain resumes the continuation like a host - * call's, so the armed signal breaks it mid-run there. - * 6. `waitForCalls` sleeps only for the REMAINING wait budget: a short - * `timeoutMs` returns in that budget, never a fixed 50 ms poll - * overshoot (~51 ms for every sub-50 ms timeout). - * - * Round 5 (the carried review's defects) adds: - * 7. The signal is keyed to the armed eval's CONTINUATION, not to - * settled call ids: an unawaited sibling `.then` registered BEFORE - * the target's await runs FIRST in the settlement drain (before the - * lease-setting reaction) — it can neither fire nor consume the - * signal, and the target's own continuation (the job after the - * reaction) is the execution broken mid-run. - * 8. Indirect waits are targetable: `await Promise.all([q])` arms and - * the continuation breaks when q settles (the identity is the - * promise graph, not a logged call-id list). - * 9. A zero `timeoutMs` wait still performs ONE immediately available - * state read: an idle workspace drains (`drained: true`), and a - * pending call's surface reads as pending (the old code returned - * unacquired with the deadline already past). - * 10. The instrumenter is HYGIENIC: a guest lexical `__replAwait` - * shadow cannot change the program's semantics (the injected seam - * is `this["__replAwait"]` — the keyword base is unshadowable, and - * no helper binding is injected into the persistent global lexical - * record). - * - * Round 6 (the carried review's defects) adds: - * 11. The lease is associated with the ACTUAL CONTINUATION JOB, not - * the next job: a sibling `q.then(...)` registered AFTER the - * target's await runs after the wrapper's settlement but BEFORE the - * lease-setting reaction (the reaction now rides the WRAPPER - * itself, immediately before the await machinery's own) — the - * sibling completes (siblingDone), the target's continuation is the - * job broken mid-run (targetDone never happens). - * 12. The for-await ITERABLE wrap preserves the iterable protocol: a - * `for await` loop over `[1, 2]` (or an async generator, or an - * awaited iterable) iterates normally through the broker — the - * 0.3.0 wrap returned a promise, making every loop throw - * `TypeError: not a function` — and a running loop remains - * breakable mid-iteration through the per-iteration lease. - * - * Round 7 (the reviewer's rejection of the previous attempt) adds: - * 13. The for-await iterable wrap passes SYNC iterable values through - * AsyncFromSyncIteratorContinuation semantics: `for await (const x - * of [Promise.resolve(1), 2])` yields the RESOLVED `[1, 2]` through - * the broker — never the promise objects. - * 14. The await instrumentation is semantically isolated from guest - * Promise sabotage: replacing `Promise.prototype.then` does not - * change the instrumented `await 40` (still `40`), and the - * continuation-lease targeting keeps working under the mutation. - * 15. The continuation-lease availability check is VERSION-GATED: a - * RESTORED 0.3.0 library (whose lease-setting reaction still runs - * on the awaited VALUE's settlement — the carried sibling-reaction - * interrupt-targeting defect) reports `supportsContinuationLease: - * true` but is served WITHOUT instrumentation — the flag alone would - * have re-armed the original defect on a supported older snapshot. - * A running eval under such a library can never be armed (§3.2): - * the no-id interrupt TERMINATES it (released), never refuses. - * - * All suites disable the per-eval deadline (`evalTimeoutMs: 0`), so - * the ONLY thing that can break a runaway here is the armed signal — - * a regression hangs the operation and the test's watchdog fails it. - */ - -import assert from 'node:assert/strict'; -import { test } from 'node:test'; - -import { - Broker, - Workspace, - ReplVm, - type BrokerLoadSessionOptions, - type BrokerOpenSessionOptions, - type BrokerPromptOptions, - type BrokerRunner, - type BrokerSession, - type BrokerTurn, -} from '../src/index.js'; -import { buildGuestLibrarySource } from '../src/guest/guest-library.js'; -import { getVmShim } from '../src/vm.js'; -import type { JSValueHandle, QuickJS } from 'quickjs-wasi'; - -const PROJECT = '/tmp/repl-eval-break-project'; - -async function tick(): Promise { - await new Promise((resolve) => setTimeout(resolve, 0)); -} - -/** The fake held-open ACP session (the same shape as broker.test.ts's). */ -class FakeSession implements BrokerSession { - readonly sessionId: string; - initializeMeta: Readonly> | undefined; - readonly prompts: Array<{ content: string; resolve: (turn: BrokerTurn) => void; reject: (error: unknown) => void }> = []; - releases = 0; - cancelCalls = 0; - stopReason = 'end_turn'; - readonly completedTexts: string[] = []; - - constructor(readonly openedWith: BrokerOpenSessionOptions | BrokerLoadSessionOptions) { - this.sessionId = `fake-session-${FakeSession.nextId++}`; - this.initializeMeta = { steering: { supported: true } }; - } - - static nextId = 0; - - prompt(content: string, opts: BrokerPromptOptions = {}): Promise { - return new Promise((resolve, reject) => { - this.prompts.push({ content, resolve, reject }); - opts.onHandoff?.(); - }); - } - - steer(content: string): Promise { - return new Promise((_, reject) => reject(new Error('steer not used in this suite'))); - } - - awaitCurrentTurn(): Promise { - return new Promise(() => {}); - } - - cancel(): Promise { - this.cancelCalls++; - for (const pending of this.prompts.splice(0)) { - pending.resolve({ stopReason: 'cancelled', text: '' }); - } - return Promise.resolve(); - } - - release(): Promise { - this.releases++; - return Promise.resolve(); - } - - currentTurnText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ''; - } - - finalMessageText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ''; - } - - rawStructuredOutput(): unknown { - return undefined; - } - - completeTurn(text: string): void { - const pending = this.prompts.shift(); - assert.ok(pending, 'a prompt turn must be in flight'); - this.completedTexts.push(text); - pending.resolve({ stopReason: this.stopReason, text }); - } -} - -class FakeRunner implements BrokerRunner { - readonly sessions: FakeSession[] = []; - - listBackends(): string[] { - return ['claude', 'codex', 'opencode', 'pi']; - } - - defaultBackendId(): string { - return 'claude'; - } - - async openSession(opts: BrokerOpenSessionOptions): Promise { - const session = new FakeSession(opts); - this.sessions.push(session); - return session; - } - - async loadSession(opts: BrokerLoadSessionOptions): Promise { - const session = new FakeSession(opts); - this.sessions.push(session); - return session; - } - - async dispose(): Promise {} - - last(): FakeSession { - assert.ok(this.sessions.length > 0, 'a session must exist'); - return this.sessions[this.sessions.length - 1]; - } -} - -async function setup(options: { runner?: BrokerRunner; evalTimeoutMs?: number } = {}): Promise<{ - ws: Workspace; - broker: Broker; -}> { - const ws = await Workspace.create(PROJECT); - const broker = await Broker.attach(ws, { - runner: options.runner, - evalTimeoutMs: options.evalTimeoutMs ?? 0, // the deadline is DISABLED: only the armed signal can break a runaway - }); - return { ws, broker }; -} - -function output(result: { output: string[] }): string[] { - return result.output; -} - -/** §6.2: an interrupted drain leaves the eval result surface (the v1 - * "interrupted" output line is deleted) — the failure is RETAINED under - * workspace().diagnostics.drainError instead. Assert the retention. */ -async function assertInterruptedRetained(broker: Broker, label: string): Promise { - const probe = await broker.eval( - 'workspace().diagnostics.drainError === null ? "none" : ' + - 'workspace().diagnostics.drainError.name + ":" + workspace().diagnostics.drainError.message', - ); - const message = String(probe.result ?? ''); - assert.ok( - message.includes('interrupted') || message.includes('Job execution error'), - `${label}: the interrupted drain is retained under diagnostics — got ${message}`, - ); -} - -/** Race a broker operation against a watchdog: a regression that leaves - * the runaway unbroken (the deadline is disabled in this suite) must - * FAIL the test, not hang the run. */ -async function bounded(label: string, promise: Promise, timeoutMs = 5000): Promise { - let timer: ReturnType | undefined; - const watch = new Promise((_, reject) => { - timer = setTimeout(() => reject(new Error(`watchdog: ${label} did not settle in ${timeoutMs} ms`)), timeoutMs); - }); - try { - return await Promise.race([promise, watch]); - } finally { - if (timer !== undefined) clearTimeout(timer); - } -} - -// ── 1. waitForCalls releases the chain between pumps ─────────────────── - -test('review round 2: a concurrent cancelCall completes MID-WAIT (the wait does not hold the broker chain across its sleeps) and the wait\'s next pump observes the cancelled settlement', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - await broker.eval('const p = agent("pi/x", "task"); "started"'); - await tick(); - // The wait starts pumping c1 (a bounded poll: pump + sleep + re-poll). - const waiting = broker.waitForCalls(['c1'], 30_000); - await tick(); - // The interrupt lands while the wait is in flight: cancelCall must run - // NOW (between the wait's pumps), not queue behind the whole 30 s - // poll. The old code serialized the entire wait, so this call could - // not complete until the wait finished or timed out — by which point - // the target could already have completed. - const outcome = await bounded('cancelCall mid-wait', broker.cancelCall('c1')); - assert.equal(outcome, 'cancelled', 'the live session was cancelled mid-wait'); - // The wait's very next pump delivers the cancelled settlement: the - // wait reports drained with the call completed (the cancel settles the - // call as the recoverable AGENT_CANCELLED — the pump observes it). - const { result, drained } = await bounded('wait after the mid-wait cancel', waiting); - assert.equal(drained, true, 'the cancelled call drained the wait'); - assert.ok(result.completed.includes('c1'), `completed: ${result.completed.join(', ')}`); - await broker.dispose(); - ws.dispose(); -}); - -// ── 2. armEvalBreak lands mid-wait and breaks an EXECUTING runaway ───── - -test('review round 2: the no-id interrupt breaks an EXECUTING runaway eval — an eval looping over subagent calls is in flight while a wait pumps it; the interrupt arms mid-wait and the wait\'s very next pump breaks the loop\'s next iteration MID-RUN (quickjs interrupt handler)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - // The running eval: a runaway whose body keeps EXECUTING across - // drains (each iteration does real work, fires the next subagent call - // and suspends — the eval is in flight the whole time, never - // completing). A suspended-continuation test (the old one) armed the - // signal against an eval that had never executed; this eval is - // mid-run — being pumped by a live wait — when the interrupt lands. - // (The per-iteration work matters: quickjs's interrupt counter only - // polls the handler on a bytecode budget, so a bare `await agent()` - // chunk can complete without a poll — a genuinely executing runaway - // is doing work, and that work is what the handler breaks.) - const a = await broker.eval( - 'const s = agent("pi/x", "task"); await s; for (;;) { let x = 0; for (let i = 0; i < 200000; i++) x += i; await agent("pi/x", "again"); }', - ); - assert.ok(a.pending.includes('c1'), `pending: ${a.pending.join(', ')}`); - await tick(); - // The wait starts pumping the eval's first call; the interrupt lands - // WHILE THE WAIT IS IN FLIGHT — the old waitForCalls held the broker - // serialization chain across its whole bounded poll, so this arm - // could not be processed until the wait finished or timed out (up to - // 120 s), by which point the target could already have completed. - const waiting1 = broker.waitForCalls(['c1'], 30_000); - await tick(); - const armed = await bounded('armEvalBreak mid-wait', broker.armEvalBreak()); - assert.equal(armed, true, 'the RUNNING eval was targeted while the wait pumped it'); - // The first settlement: the wait's very next pump resumes the loop's - // next iteration — and the armed signal breaks it MID-RUN (with the - // deadline disabled, only the signal can break it). The break is the - // wait's own drain error — honest output in the wait's result — and - // the settlement that resumed the broken iteration is still reported. - runner.sessions[0].completeTurn('resumed'); - const waited1 = await bounded('wait#1 after the mid-run break', waiting1); - assert.equal(waited1.drained, true, 'the settled call drained the wait'); - assert.ok(waited1.result.completed.includes('c1'), `completed: ${waited1.result.completed.join(', ')}`); - // §6.2: the break is the wait's own drain error — demoted to - // workspace().diagnostics, never rendered in the wait's output lines. - await assertInterruptedRetained(broker, 'the executing runaway was broken mid-run'); - assert.ok( - output(waited1.result).every((line) => !line.includes('interrupted')), - `the drain failure left the wait result surface: ${output(waited1.result).join('\n')}`, - ); - // The broken eval is released and the signal was consumed: the next - // eval runs normally, and a later arm REFUSES (no stale target — the - // interrupted continuation's wrapper never settles, so only the - // interrupted-drain release could have cleared it). - const after = await broker.eval('6 * 7'); - assert.equal(after.result, '42'); - assert.equal(await broker.armEvalBreak(), false, 'nothing is tracked after the break'); - await broker.dispose(); - ws.dispose(); -}); - -// ── 3. The signal rides a direct eval's own drain (checkpoint.answer) ── - -test('review round 2: a suspended eval\'s continuation resumed by checkpoint.answer inside a LATER eval\'s own drain is broken mid-run by the armed signal — and the interrupted drain releases the tracked eval', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - // eval A suspends on a checkpoint; its continuation is a runaway loop. - const a = await broker.eval('const q = checkpoint("go?"); await q; while (true) {}'); - assert.ok(a.pending.includes('c1'), `pending: ${a.pending.join(', ')}`); - // The interrupt arms against the running eval. - assert.equal(await broker.armEvalBreak(), true); - // eval B answers the checkpoint: the answer is a SYNCHRONOUS - // host-callback settlement — the continuation is resumed inside B's - // OWN drain, an execution the old settlement-drain-only signal was - // blind to (the runaway would burn the eval deadline instead of being - // broken by the interrupt). With the deadline disabled, only the - // armed signal can break it. - const b = await bounded( - 'eval B answering the checkpoint', - broker.eval('checkpoint.answer("c1", "go"); "answered"'), - ); - assert.equal(b.result, undefined, 'the answering eval was interrupted before producing a result'); - // The interruption happens in the ANSWERING eval's own drain, so it - // renders as the eval's own §4.6 error outcome — not the demoted - // retained-drain-error line. - assert.ok( - output(b).some((line) => line.includes('interrupted')), - `the resumed runaway was broken mid-run in the answering eval's drain: ${output(b).join('\n')}`, - ); - // The interrupted continuation's wrapper never settles — the - // interrupted-drain release (not the sweep) is what frees the tracked - // eval: a later arm refuses, and the next eval runs normally. - assert.equal(await broker.armEvalBreak(), false, 'the broken eval is no longer tracked'); - const after = await broker.eval('6 * 7'); - assert.equal(after.result, '42'); - // The signal was consumed by the running eval's execution: an - // UNRELATED eval's own code was never broken (the phase-E review - // rejection's targeting discipline) — the next arm still targets a - // genuinely running eval and breaks IT mid-run. - const c = await broker.eval( - 'const p2 = agent("pi/x", "task2"); await p2; for (;;) { let y = 0; for (let j = 0; j < 200000; j++) y += j; await agent("pi/x", "again2"); }', - ); - assert.ok(c.pending.includes('c2'), `pending: ${c.pending.join(', ')}`); - assert.equal(await broker.armEvalBreak(), true, 'a later running eval is targetable'); - runner.last().completeTurn('resumed'); - // Let the turn-resolution microtasks land (the task's readiness flag - // is set by a promise continuation) before the pumping eval runs. - await tick(); - const d = await broker.eval('"probe"'); - await assertInterruptedRetained(broker, 'the second runaway was broken by the second arm'); - assert.ok( - output(d).every((line) => !line.includes('interrupted')), - `the drain failure left the result surface (the second runaway was broken by the second arm): ${output(d).join('\n')}`, - ); - await broker.dispose(); - ws.dispose(); -}); - -// ── 4. The signal is keyed to the armed target's continuation ────────── - -test('review round 3: an UNRELATED finite eval whose own drain executes real bytecode neither consumes the eval-break signal nor is broken by it — the armed state survives and breaks the target at its actual next execution (the carried review defect: every later eval\'s drain installed the drainInterruptHandler, so an unrelated finite eval B was interrupted and noteInterruptedDrain cleared A\'s tracking while A\'s checkpoint stayed pending and uninterruptible)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - // eval A suspends on a checkpoint; its continuation is a runaway loop. - const a = await broker.eval('const q = checkpoint("go?"); await q; while (true) {}'); - assert.ok(a.pending.includes('c1'), `pending: ${a.pending.join(', ')}`); - // The interrupt arms against the running eval (its resume key: c1). - assert.equal(await broker.armEvalBreak(), true); - // Unrelated finite eval B whose DRAIN executes real bytecode — a - // microtask with a 200k-iteration loop polls the quickjs interrupt - // handler many times. The carried defect: B's own drain installed the - // armed drainInterruptHandler unconditionally, so the FIRST poll - // fired it — B was interrupted mid-run, and the interrupted-drain - // release cleared A's tracking (c1 stayed pending and UNINTERRUPTIBLE). - const b = await bounded( - 'unrelated finite eval with a bytecode-heavy drain', - broker.eval('Promise.resolve().then(() => { let x = 0; for (let i = 0; i < 200000; i++) x += i; globalThis.bDone = true; });'), - ); - assert.ok(b.result !== undefined, `the unrelated eval completed normally, never interrupted: ${output(b).join('\n')}`); - assert.equal((await broker.eval('bDone')).result, 'true', 'the drain ran the microtask to completion'); - // The armed state SURVIVED B: answering c1 resumes A's runaway in the - // answering eval's own drain, and the still-armed signal breaks it - // MID-RUN — the exact execution the interrupt targeted. - const c = await bounded( - 'eval C answering the checkpoint after the unrelated drain', - broker.eval('checkpoint.answer("c1", "go"); "answered"'), - ); - // The answering eval's own drain interruption renders as its own §4.6 - // error outcome (not the demoted retained-drain-error line). - assert.ok( - output(c).some((line) => line.includes('interrupted')), - `the armed signal survived the unrelated drain and broke the target: ${output(c).join('\n')}`, - ); - // The broken target was released: a later arm refuses. - assert.equal(await broker.armEvalBreak(), false, 'the broken eval is no longer tracked'); - await broker.dispose(); - ws.dispose(); -}); - -test('review round 3: an UNRELATED settlement drain (a call no tracked eval awaits) neither fires nor consumes the eval-break signal — the armed state survives and breaks the target at its actual next execution', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - // A RESOLVED eval leaves a pending agent call whose continuation does - // real bytecode when it settles (a .then with a 200k-iteration loop): - // its settlement drain polls the interrupt handler many times. NO - // tracked eval awaits c1 (the founding eval resolved). - await broker.eval('const p = agent("pi/x", "task").then((v) => { let x = 0; for (let i = 0; i < 200000; i++) x += i; return v; }); "started"'); - await tick(); - // eval A suspends on a checkpoint; its continuation is a runaway loop. - const a = await broker.eval('const q = checkpoint("go?"); await q; while (true) {}'); - assert.ok(a.pending.includes('c2'), `pending: ${a.pending.join(', ')}`); - assert.equal(await broker.armEvalBreak(), true) - // The unrelated call settles: the pump's drain runs c1's bytecode- - // heavy continuation, polling the armed handler — which must NOT fire - // (c1 is not one of the armed target's resume keys; the drain does - // not belong to the target). The carried defect fired on any drain. - runner.sessions[0].completeTurn('unrelated'); - await tick(); - const probe = await bounded('probe eval after the unrelated settlement', broker.eval('"probe"')); - assert.ok( - probe.result !== undefined && probe.result.includes('probe'), - `the unrelated settlement did not fire the signal: ${output(probe).join('\n')}`, - ); - // The armed state SURVIVED the unrelated settlement: answering c2 - // resumes A's runaway in the answering eval's own drain and the - // still-armed signal breaks it mid-run. - const c = await bounded('eval C answering the checkpoint', broker.eval('checkpoint.answer("c2", "go"); "answered"')); - // The answering eval's own drain interruption renders as its own §4.6 - // error outcome (not the demoted retained-drain-error line). - assert.ok( - output(c).some((line) => line.includes('interrupted')), - `the armed signal survived the unrelated settlement drain: ${output(c).join('\n')}`, - ); - assert.equal(await broker.armEvalBreak(), false, 'the broken eval is no longer tracked'); - await broker.dispose(); - ws.dispose(); -}); - -// ── 5. Nothing breakable → refuse without arming ─────────────────────── - -test('round 3 amended (§3.2): a no-id interrupt on an eval suspended on NOTHING RESUMABLE (a never-settling local promise — no pending host call, no pending sleep) TERMINATES it — the tracked continuation is released, the interrupt is never a refusal while an eval is running, and a later arm honestly refuses only when nothing is tracked', async () => { - const { ws, broker } = await setup(); - // The eval suspends (its completion stays pending) with ZERO pending - // host calls and no sleep: no execution can ever queue its - // continuation, so there is nothing to break mid-run. Arming would be - // dead weight that lingers until reset — but the eval IS running, so - // the interrupt must terminate it instead of refusing (the review - // defect: `refused-idle` for a running eval is neither a break nor an - // honest idle refusal). - const a = await broker.eval('await new Promise(() => {}); "never"'); - assert.equal(a.result, undefined, 'the eval suspended — no completion value'); - assert.deepEqual(a.pending, [], 'no pending host call'); - const released = await bounded('armEvalBreak terminating the suspended eval', broker.armEvalBreak()); - assert.equal(released, true, 'a running eval is terminated — the interrupt never refuses it'); - // The tracked continuation was RELEASED: nothing is running any more, - // so the next arm honestly refuses (the ONLY permitted refusal). - assert.equal(await broker.armEvalBreak(), false, 'nothing is tracked after the release — honest idle refusal'); - // The release terminated the eval (its continuation can never run): - // a later eval runs normally and the released eval's "never" never - // becomes the completion value. - const after = await broker.eval('6 * 7'); - assert.equal(after.result, '42'); - await broker.dispose(); - ws.dispose(); -}); - -test('round 3 amended (§3.2): an eval suspended on a pending SLEEP stays ARMABLE — the sleep settlement drain resumes the continuation and the armed signal breaks a runaway continuation mid-run there', async () => { - const { ws, broker } = await setup(); - // `sleep` is a HOST timer (§4.7), not a registry call: the eval - // suspends with an empty pending surface, but its continuation IS - // resumable (the timer's settlement drain runs it), so arming is - // never dead weight — the signal breaks it at that execution. - const a = await broker.eval('await sleep(120); while (true) {}'); - assert.equal(a.result, undefined, 'the eval suspended — no completion value'); - assert.deepEqual(a.pending, [], 'no pending host call (sleep is not a registry call)'); - assert.equal(await broker.armEvalBreak(), true, 'the sleep-suspended eval is armable — its continuation WILL execute'); - // The sleep timer fires, and the next operation's pump (a probe - // eval's own pump) settles it and resumes the runaway continuation — - // the armed signal breaks it mid-run there. The break lands in the - // pump phase, so the probe eval itself runs clean and the - // interruption is retained under workspace().diagnostics (§6.2). - await new Promise((resolve) => setTimeout(resolve, 200)); - const probe = await bounded('probe eval resuming the sleep continuation', broker.eval('"probe"')); - assert.equal(probe.result, 'probe', 'the probe eval runs normally — the break landed in its pump drain'); - await assertInterruptedRetained(broker, 'the armed signal broke the sleep-resumed continuation mid-run'); - assert.equal(await broker.armEvalBreak(), false, 'the broken eval is no longer tracked'); - await broker.dispose(); - ws.dispose(); -}); - -// ── 6. The wait sleeps only for the remaining budget ─────────────────── - -test('review round 3: waitForCalls respects the REMAINING wait budget — a 10 ms timeout returns in ~10 ms, never the fixed 50 ms poll overshoot (~51 ms for every sub-50 ms timeout: the carried review defect)', async () => { - const { ws, broker } = await setup(); - // A parked checkpoint keeps c1 pending forever (no runner needed): - // each wait pumps (nothing ready), sleeps, and must return at its - // deadline. The same parked call is reused across every sample — it - // never settles, so each `waitForCalls(['c1'], 10)` is an independent - // bounded poll. - const raised = await broker.eval('const q = checkpoint("go?"); "raised"'); - assert.ok(raised.pending.includes('c1'), `pending: ${raised.pending.join(', ')}`); - // DEFECT SIGNATURE: a FIXED ~51 ms poll overshoot on EVERY sub-50 ms - // timeout — a DETERMINISTIC floor (the old code slept an unconditional - // 50 ms per pump regardless of the remaining budget, so a 10 ms wait - // always returned in ~51 ms). Load noise is a DIFFERENT distribution: - // an intermittent inflated sample (the CI flake measured 67 ms once) - // riding an otherwise ~10 ms wait. A single-sample threshold cannot - // separate the two — one 67 ms load spike is indistinguishable from - // the ~51 ms defect floor — so naked widening would only destroy the - // test's power. Instead measure the wait N times (N = 8) and assert - // the MINIMUM is well under the 50 ms defect floor: the defect - // inflates EVERY sample to ~51 ms so its min is ~51 ms (caught), while - // a healthy 10 ms wait has a true ~10 ms floor and load noise is - // intermittent, so across N samples at least one lands near the floor - // and the min stays ~10 ms (passes under arbitrary load). min-of-N - // cleanly discriminates a deterministic floor from intermittent noise. - const SAMPLES = 8; - const elapsedSamples: number[] = []; - for (let i = 0; i < SAMPLES; i += 1) { - const started = Date.now(); - const { result, drained } = await bounded('bounded 10 ms wait', broker.waitForCalls(['c1'], 10)); - elapsedSamples.push(Date.now() - started); - assert.equal(drained, false, 'the parked checkpoint never settles — "still running"'); - assert.deepEqual(result.pending, ['c1'], 'the pending ids are reported'); - } - const minElapsed = Math.min(...elapsedSamples); - // 40 ms is 10 ms below the ~51 ms deterministic defect floor and ~4x - // the ~10 ms healthy floor: the defect (every sample ~51 ms) can never - // produce a min under 40 ms, while a healthy wait's least-contended - // sample lands near 10 ms even when sibling samples are load-inflated - // (measured: min stayed 10–12 ms under 72 CPU-bound workers on 48 - // cores). The min-of-N floor, not any single sample, is the discriminator. - assert.ok( - minElapsed < 40, - `the min of ${SAMPLES} 10 ms waits was ${minElapsed} ms (samples: ${elapsedSamples.join(', ')}); the fixed ~51 ms overshoot is gone`, - ); - await broker.dispose(); - ws.dispose(); -}); - -// ── 7. Round 4: the armed identity is the calls the eval AWAITS ──────── - -test('review round 4: an UNAWAITED SIBLING call (c2.then with a bytecode-heavy continuation) neither fires nor consumes the eval-break signal — settling c2 runs its own .then to completion, the awaited c1 stays pending, the armed target stays tracked (a later arm still returns true), and the target breaks at c1\'s actual settlement (the carried defect: every call an eval CREATED was a resume key, so settling the unawaited sibling interrupted its unrelated heavy .then, left c1 pending, and made the next arm refuse)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - // The eval AWAITS c1 but only TENS c2: `await c1` is a top-level await - // (instrumented — recorded as a resume key); `c2.then(...)` is not an - // await (never recorded). The carried defect treated every call the - // eval created as a resume key. - const a = await broker.eval( - 'const c1 = agent("pi/x", "one"); const c2 = agent("pi/x", "two"); const heavy = c2.then((v) => { let x = 0; for (let i = 0; i < 200000; i++) x += i; return "heavy:" + v; }); await c1; while (true) {}', - ); - assert.ok(a.pending.includes('c1'), `pending: ${a.pending.join(', ')}`); - assert.ok(a.pending.includes('c2'), `pending: ${a.pending.join(', ')}`); - assert.equal(await broker.armEvalBreak(), true, 'the running eval is targetable (it awaits c1)'); - // The UNAWAITED sibling settles first: its drain runs c2's OWN heavy - // .then continuation — 200k iterations polling the armed interrupt - // handler — and must COMPLETE (the drain does not belong to the - // target: c2 is not one of its resume keys). - runner.sessions[1].completeTurn('sibling'); - await tick(); - const probe = await bounded('probe after the unawaited sibling settled', broker.eval('await heavy')); - assert.ok( - probe.result !== undefined && probe.result.includes('heavy:sibling'), - `the unawaited sibling's own .then ran to completion, never interrupted: ${output(probe).join('\n')}`, - ); - assert.ok( - !output(probe).some((line) => line.includes('interrupted')), - `no execution was interrupted by the sibling's settlement: ${output(probe).join('\n')}`, - ); - // The armed state SURVIVED the unrelated settlement: the target is - // still tracked and c1 is still pending (the carried defect: the - // signal was consumed and the tracked eval released, so this arm - // returned false and c1 became uninterruptible). - assert.equal(await broker.armEvalBreak(), true, 'the target is still tracked after the sibling settlement'); - assert.ok((await broker.eval('"still-pending"')).pending.includes('c1'), 'c1 is still pending'); - // The awaited call settles: the eval\'s continuation (the runaway - // loop) executes and the still-armed signal breaks it MID-RUN. - runner.sessions[0].completeTurn('resumed'); - await tick(); - const broken = await bounded('probe after the awaited call settled', broker.eval('"after"')); - await assertInterruptedRetained(broker, "the awaited call's settlement resumed the runaway continuation and the armed signal broke it"); - assert.ok( - output(broken).every((line) => !line.includes('interrupted')), - `the drain failure left the result surface (the awaited call's settlement resumed the runaway continuation and the armed signal broke it): ${output(broken).join('\n')}`, - ); - assert.equal(await broker.armEvalBreak(), false, 'the broken eval is no longer tracked'); - await broker.dispose(); - ws.dispose(); -}); - -test('review round 4: a RUNNING eval awaiting an EARLIER eval\'s binding remains targetable — `await p` on a promise a previous eval created logs the call as THIS eval\'s resume key (the carried defect: an eval\'s resume keys were the calls it CREATED, so this eval had none and the arm refused)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - // eval 1 creates the call and resolves; the binding outlives the eval. - const first = await broker.eval('const p = agent("pi/x", "earlier"); "started"'); - assert.ok(first.pending.includes('c1'), `pending: ${first.pending.join(', ')}`); - // eval 2 awaits the EARLIER binding: its continuation is queued by - // c1's settlement — exactly the execution the interrupt must be able - // to break. - const second = await broker.eval('await p; while (true) {}'); - assert.ok(second.pending.includes('c1'), `pending: ${second.pending.join(', ')}`); - assert.equal(await broker.armEvalBreak(), true, 'an eval awaiting an earlier binding is targetable'); - // Settling c1 resumes eval 2's continuation (the runaway loop): the - // armed signal breaks it mid-run. - runner.last().completeTurn('resumed'); - await tick(); - const probe = await bounded('probe after settling the earlier binding', broker.eval('"probe"')); - await assertInterruptedRetained(broker, 'the resumed continuation was broken mid-run'); - assert.ok( - output(probe).every((line) => !line.includes('interrupted')), - `the drain failure left the result surface (the resumed continuation was broken mid-run): ${output(probe).join('\n')}`, - ); - assert.equal(await broker.armEvalBreak(), false, 'the broken eval is no longer tracked'); - await broker.dispose(); - ws.dispose(); -}); - -// ── 8. Round 4: the wait's chain ACQUISITION is deadline-bounded ─────── - -test('review round 4: waitForCalls\'s chain acquisition is bounded by the wait deadline — a bounded wait queued behind a long chain hold (the client-presence drain pumping a slow turn) returns at its bound, not behind the drain (the carried defect: a 20 ms wait behind a 250 ms eval took ~253 ms because the acquisition enqueued with no deadline)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - // A slow turn keeps the client-presence drain pumping: the drain is - // ONE serialized op, so its internal pumps/sleeps HOLD the broker - // serialization chain for the whole bounded run (the event loop stays - // free — the drain sleeps between pumps). - const a = await broker.eval('const p = agent("pi/x", "slow"); await p; "done"'); - assert.ok(a.pending.includes('c1'), `pending: ${a.pending.join(', ')}`); - const draining = broker.drainForDisconnect(2000, () => false); - await tick(); - // The 30 ms wait's acquisitions must race the REMAINING budget: with - // the chain held by the drain, the wait reports "still running" at - // its bound instead of queueing behind the drain. - const started = Date.now(); - const { result, drained } = await bounded('bounded wait behind the drain', broker.waitForCalls(['c1'], 30)); - const elapsed = Date.now() - started; - assert.equal(drained, false, 'the slow turn never settled within the wait bound — "still running"'); - assert.deepEqual(result.pending, ['c1'], 'the target ids are reported (none observed settled)'); - // DEFECT SIGNATURE: an unbounded chain acquisition makes the 30 ms - // wait QUEUE behind the drain and return only when the drain releases - // the chain — historically ~253 ms (a 20 ms wait behind a 250 ms - // drain), and in THIS test ~2000 ms (the drain's full bound, since the - // turn is settled only AFTER this measurement). Healthy the wait - // returns at its ~30 ms bound. The two distributions do NOT overlap - // (measured healthy worst case 40 ms under 72 CPU-bound workers on 48 - // cores; the CI flake was 81 ms, just over the old 80 ms epsilon), - // so — unlike the ~51 ms floor defect above, whose load noise overlaps - // the floor — a generous single-sample ceiling discriminates unambiguously - // without weakening the test. 150 ms is ~5x the ~30 ms bound (ample - // headroom over the 81 ms flake) yet far below the ≥253 ms defect: a - // wait that honors its bound cannot reach 150 ms, and the - // queued-behind-the-drain defect cannot come in under it. - assert.ok(elapsed < 150, `the 30 ms wait returned at its bound (${elapsed} ms), not behind the drain`); - // The drain still completes its work once the turn settles: the turn - // drains to completion and the children release (the doc's graceful - // drain is unaffected by the wait's bounded acquisition). - await new Promise((resolve) => setTimeout(resolve, 150)); - runner.sessions[0].completeTurn('slow-done'); - assert.equal(await bounded('drain completion', draining), true, 'the drain drained the turn to completion'); - await broker.dispose(); - ws.dispose(); -}); - -// ── 9. Round 5: the armed identity is the continuation, not settled ids ─ - -test('review round 5: an UNAWAITED SIBLING reaction registered BEFORE the target\'s await runs FIRST in the settlement drain — it can neither fire nor consume the eval-break signal, and the target\'s OWN continuation (the job after the lease-setting reaction) is the execution broken mid-run (the carried defect: settling q interrupted the sibling job, cleared the arm, and let the target continuation run later unbroken)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - // `q.then(sibling)` is registered BEFORE `await q`: the reactions on - // q are [sibling, lease-setting-reaction] in registration order, so - // the settlement drain runs the sibling's bytecode-heavy continuation - // FIRST — before any lease is set. The carried defect's signal was - // keyed to settled call ids: the drain "belonged" to the target, the - // FIRST poll fired on the sibling's job, the arm was consumed, and - // the target's own continuation ran later with no protection. - const a = await broker.eval( - 'const q = agent("pi/x", "one"); const sibling = q.then((v) => { let x = 0; for (let i = 0; i < 200000; i++) x += i; return "sibling:" + v; }); await q; while (true) {}', - ); - assert.ok(a.pending.includes('c1'), `pending: ${a.pending.join(', ')}`); - assert.equal(await broker.armEvalBreak(), true, 'the running eval is targetable'); - // The awaited call settles: the drain runs [sibling, the target's - // continuation]. The sibling job must COMPLETE (no lease is set yet — - // the lease-setting reaction runs after it), and the target's own - // continuation is the job broken mid-run. - runner.sessions[0].completeTurn('resumed'); - await tick(); - const probe = await bounded('probe after settling the awaited call', broker.eval('await sibling')); - assert.ok( - probe.result !== undefined && probe.result.includes('sibling:resumed'), - `the sibling continuation ran to completion, never interrupted: ${output(probe).join('\n')}`, - ); - await assertInterruptedRetained(broker, "the target's own continuation was broken mid-run (not the sibling's job)"); - assert.ok( - output(probe).every((line) => !line.includes('interrupted')), - `the drain failure left the result surface (the target's own continuation was broken mid-run (not the sibling's job)): ${output(probe).join('\n')}`, - ); - // The broken eval was released (the interrupted job's continuation - // lease named it exactly): a later arm refuses. - assert.equal(await broker.armEvalBreak(), false, 'the broken eval is no longer tracked'); - await broker.dispose(); - ws.dispose(); -}); - -test('review round 5: an INDIRECT wait is targetable — `await Promise.all([q]); while (true) {}` arms (the 0.2.0 log refused it: the awaited value is the combinator\'s promise, not a registry promise) and the armed signal breaks the continuation mid-run when q settles (the identity is the promise graph)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - // The eval awaits Promise.all([q]) — an indirect chain whose - // settlement is q's. The continuation lease rides the combinator - // promise's settlement, exactly like a direct call's. - const a = await broker.eval('const q = agent("pi/x", "one"); await Promise.all([q]); while (true) {}'); - assert.ok(a.pending.includes('c1'), `pending: ${a.pending.join(', ')}`); - assert.equal(await broker.armEvalBreak(), true, 'an eval awaiting an indirect chain is targetable'); - // Settling q resolves the Promise.all promise: the lease-setting - // reaction runs, then the target's continuation (the runaway loop) — - // broken mid-run by the armed signal. - runner.sessions[0].completeTurn('resumed'); - await tick(); - const probe = await bounded('probe after settling the indirect chain', broker.eval('"probe"')); - await assertInterruptedRetained(broker, "the indirect chain's continuation was broken mid-run"); - assert.ok( - output(probe).every((line) => !line.includes('interrupted')), - `the drain failure left the result surface (the indirect chain's continuation was broken mid-run): ${output(probe).join('\n')}`, - ); - assert.equal(await broker.armEvalBreak(), false, 'the broken eval is no longer tracked'); - const after = await broker.eval('6 * 7'); - assert.equal(after.result, '42', 'the workspace stays usable'); - await broker.dispose(); - ws.dispose(); -}); - -// ── 10. Round 5: zero-timeout waits perform an immediate state read ─── - -test('review round 5: a ZERO-timeout wait still performs ONE immediately available state read — an idle workspace reports drained (the carried defect: the chain acquisition returned unacquired with the deadline already past, so even an immediately readable state reported "still running")', async () => { - const { ws, broker } = await setup(); - const { result, drained } = await bounded('zero-timeout idle wait', broker.waitForCalls(undefined, 0)); - assert.equal(drained, true, 'an idle workspace drains immediately — the pending read was immediately available'); - assert.deepEqual(result.pending, [], 'the empty pending surface was read'); - assert.deepEqual(result.completed, []); - await broker.dispose(); - ws.dispose(); -}); - -test('review round 5: a ZERO-timeout wait on a workspace with a PENDING call reports the call as pending and "still running" (the carried defect: the unacquired acquisition reported an empty pending list)', async () => { - const { ws, broker } = await setup(); - const raised = await broker.eval('const q = checkpoint("go?"); "raised"'); - assert.ok(raised.pending.includes('c1'), `pending: ${raised.pending.join(', ')}`); - const { result, drained } = await bounded('zero-timeout pending wait', broker.waitForCalls(['c1'], 0)); - assert.equal(drained, false, 'the parked checkpoint never settles — "still running"'); - assert.deepEqual(result.pending, ['c1'], 'the pending surface was read immediately'); - await broker.dispose(); - ws.dispose(); -}); - -// ── 11. Round 5: the instrumenter is hygienic ────────────────────────── - -test('review round 5: the top-level-await instrumenter is HYGIENIC — a guest lexical `__replAwait` shadow cannot change the program\'s semantics (the 0.2.0 transform inserted the guest-resolvable identifier `__replAwait`, so `{ const __replAwait = () => 7; globalThis.seen = await Promise.resolve(42); }` yielded 7 instead of 42; the injected seam is now `this["__replAwait"]` — the keyword base is unshadowable)', async () => { - const { ws, broker } = await setup(); - const r = await broker.eval('{ const __replAwait = () => 7; globalThis.seen = await Promise.resolve(42); } "done"'); - assert.equal(r.result, 'done', `the eval completed normally: ${output(r).join('\n')}`); - const seen = await broker.eval('seen'); - assert.equal(seen.result, '42', 'the REAL library seam ran — the guest shadow changed nothing'); - // The shadowing identifier stays usable as the guest declared it. - const shadow = await broker.eval('{ const __replAwait = () => 7; globalThis.seen2 = __replAwait(); } "s"'); - assert.equal(shadow.result, 's'); - const seen2 = await broker.eval('seen2'); - assert.equal(seen2.result, '7', 'the guest\'s own shadowed identifier keeps its semantics'); - // The transform injects NO persistent helper binding (a top-level - // const would redeclare on the loop idiom): the same code runs again. - const again = await broker.eval('{ const __replAwait = () => 7; globalThis.seen3 = await Promise.resolve(9); } "again"'); - assert.equal(again.result, 'again', `the loop idiom does not redeclare: ${output(again).join('\n')}`); - const seen3 = await broker.eval('seen3'); - assert.equal(seen3.result, '9'); - await broker.dispose(); - ws.dispose(); -}); - -// ── 12. Round 6: the lease is the continuation job, not the next job ─── - -test('review round 6: a sibling `q.then(...)` registered AFTER the target\'s await neither fires nor consumes the eval-break signal — the lease is set only by the WRAPPER reaction (immediately before the await machinery\'s own), so the sibling job completes and the target\'s OWN continuation is the job broken mid-run (the carried defect: the 0.3.0 lease-setting reaction ran on the awaited VALUE\'s settlement, so the sibling job ran between the lease set and the continuation, consumed the armed signal, and the target\'s continuation completed later UNPROTECTED — the siblingDone:false / targetDone:true repro)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - // `await q` is evaluated FIRST (the wrap's resolve reaction is - // registered on q at that moment); the sibling `q.then(...)` is - // registered LATER, by a deferred microtask (so the reactions on q - // are [wrap-resolve, sibling] in registration order). Settlement - // queues [wrap-resolve, sibling] and the wrap-resolve job queues the - // WRAPPER's [lease-setting, machinery] reactions AFTER the sibling — - // the sibling job runs with NO lease set (it can neither fire nor - // consume the armed signal), and the machinery job — the target's - // actual continuation — starts with the lease set and is broken - // mid-run. The 0.3.0 ordering set the lease inside the wrap-resolve - // job: the sibling job then started with the lease set, the drain - // attributed it, the interrupt broke the SIBLING, and the target's - // continuation ran later with the arm consumed (targetDone). - const a = await broker.eval( - 'const q = agent("pi/x", "one"); const deferred = Promise.resolve().then(() => q.then((v) => { let x = 0; for (let i = 0; i < 200000; i++) x += i; return "sibling:" + v; })); await q; while (true) {}', - ); - assert.ok(a.pending.includes('c1'), `pending: ${a.pending.join(', ')}`); - assert.equal(await broker.armEvalBreak(), true, 'the running eval is targetable'); - // The awaited call settles: the drain runs [wrap-resolve, sibling, - // lease-setting, continuation]. The sibling job must COMPLETE — with - // the carried defect it was the job right after the lease set, so it - // was interrupted instead and the deferred sibling promise never - // settled (this probe would hang and the watchdog would fail). - runner.sessions[0].completeTurn('resumed'); - await tick(); - const probe = await bounded('probe after settling the awaited call', broker.eval('await deferred')); - assert.ok( - probe.result !== undefined && probe.result.includes('sibling:resumed'), - `the sibling reaction ran to completion, never interrupted: ${output(probe).join('\n')}`, - ); - await assertInterruptedRetained(broker, "the target's own continuation was broken mid-run (not the sibling's job)"); - assert.ok( - output(probe).every((line) => !line.includes('interrupted')), - `the drain failure left the result surface (the target's own continuation was broken mid-run (not the sibling's job)): ${output(probe).join('\n')}`, - ); - // The broken eval was released (the interrupted job's continuation - // lease named it exactly): a later arm refuses — the target never - // completed (an interrupted continuation's wrapper never settles). - assert.equal(await broker.armEvalBreak(), false, 'the broken eval is no longer tracked'); - await broker.dispose(); - ws.dispose(); -}); - -// ── 13. Round 6: for-await iterables keep the iterable protocol ──────── - -test('review round 6: the for-await ITERABLE wrap preserves the iterable protocol — `for await (const x of [1, 2])` iterates normally through the broker (the carried defect: the 0.3.0 instrumenter wrapped the iterable in `__replAwait`, whose promise result made the loop throw `TypeError: not a function` instead of iterating)', async () => { - const { ws, broker } = await setup(); - const r = await broker.eval( - 'globalThis.forAwaitSum = 0; for await (const x of [1, 2]) { globalThis.forAwaitSum += x; } "iterated"', - ); - assert.equal(r.result, 'iterated', `the for-await loop completed normally: ${output(r).join('\n')}`); - const sum = await broker.eval('forAwaitSum'); - assert.equal(sum.result, '3', 'the loop iterated [1, 2] — the iterable protocol is preserved'); - // An ASYNC-GENERATOR iterable still iterates across drains (each - // iteration's `next()`-result await rides the wrap's lease-wrapped - // promises). - const g = await broker.eval( - 'globalThis.g = (async function* () { yield 10; yield 20; })(); globalThis.genSum = 0; for await (const x of g) { globalThis.genSum += x; } "gen"', - ); - assert.equal(g.result, 'gen', `the async-generator loop completed: ${output(g).join('\n')}`); - const genSum = await broker.eval('genSum'); - assert.equal(genSum.result, '30', 'the async generator yielded 10 then 20'); - // `for await (const x of await y)`: the iterable IS an awaited - // expression — the instrumenter skips the iterable wrap (the loop - // iterates the unwrapped value), so the shape keeps its semantics. - const nested = await broker.eval( - 'globalThis.nestedSum = 0; for await (const x of await Promise.resolve([3])) { globalThis.nestedSum += x; } "nested"', - ); - assert.equal(nested.result, 'nested', `the awaited-iterable shape completed: ${output(nested).join('\n')}`); - const nestedSum = await broker.eval('nestedSum'); - assert.equal(nestedSum.result, '3', 'the awaited iterable [3] iterated once'); - await broker.dispose(); - ws.dispose(); -}); - -test('review round 6: a RUNNING for-await loop is breakable mid-iteration — the iterable wrap sets the continuation lease per iteration, so the armed signal breaks the loop\'s continuation exactly like any other awaited segment', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - // The loop's body awaits a host call every iteration: the eval is in - // flight across drains (the loop never completes) and the interrupt - // arms against it. The per-iteration lease (set by the iterable - // wrap's next()-result reactions, immediately before the loop's - // continuation job) makes the armed signal fire while the loop's own - // continuation executes — with the 0.3.0 wrap the eval threw - // `TypeError: not a function` at the first iteration (the wrap - // returned a promise, not an async iterable) and was never targetable. - // The per-iteration work matters exactly like the other runaway - // suites: quickjs's interrupt counter only polls the handler on a - // bytecode budget, so a bare `await agent()` chunk can complete - // without a poll — the work loop is what the handler breaks mid-run. - const a = await broker.eval( - 'const gen = (async function* () { for (;;) { yield 1; } })(); globalThis.ticks = 0; for await (const x of gen) { globalThis.ticks++; let y = 0; for (let i = 0; i < 200000; i++) y += i; await agent("pi/x", "tick"); } "done"', - ); - assert.ok(a.pending.includes('c1'), `pending: ${a.pending.join(', ')}`); - assert.equal(await broker.armEvalBreak(), true, 'the running for-await eval is targetable'); - // The awaited call settles: the loop's continuation (the next - // iteration's body) executes with the lease set and the armed signal - // breaks it MID-RUN. - runner.sessions[0].completeTurn('tick'); - await tick(); - const probe = await bounded('probe after settling the loop iteration', broker.eval('"probe"')); - await assertInterruptedRetained(broker, "the loop's continuation was broken mid-run"); - assert.ok( - output(probe).every((line) => !line.includes('interrupted')), - `the drain failure left the result surface (the loop's continuation was broken mid-run): ${output(probe).join('\n')}`, - ); - assert.equal(await broker.armEvalBreak(), false, 'the broken eval is no longer tracked'); - await broker.dispose(); - ws.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Round 7 — the reviewer's rejection of the previous attempt -// ──────────────────────────────────────────────────────────────────────── - -test('review round 7: the instrumented for-await over a SYNC iterable yields the RESOLVED values through the broker — `for await (const x of [Promise.resolve(1), Promise.resolve(2)])` collects `[1, 2]`, never promise objects (the reviewer\'s repro: the result wrapper resolved with the RAW iterator result, and because the wrapper is an ASYNC iterable the machinery used the value as-is — the promise object leaked through Broker instead of `1`)', async () => { - const { ws, broker } = await setup(); - const r = await broker.eval( - 'globalThis.round7sync = []; for await (const x of [Promise.resolve(1), Promise.resolve(2)]) { globalThis.round7sync.push(x); } "iterated"', - ); - assert.equal(r.result, 'iterated', `the loop completed: ${output(r).join('\n')}`); - const kinds = await broker.eval('round7sync.map((x) => typeof x).join(",")'); - assert.equal(kinds.result, 'number,number', 'the loop saw the RESOLVED numbers, never promise objects'); - const sum = await broker.eval('round7sync[0] + round7sync[1]'); - assert.equal(sum.result, '3', 'the resolved values are `1` and `2`'); - await broker.dispose(); - ws.dispose(); -}); - -test('review round 7: the instrumented top-level await is semantically isolated from guest Promise sabotage — replacing `Promise.prototype.then` does not change `await 40` (the reviewer\'s repro: the instrumented await returned `99` where the native evaluation returned `40`), and the continuation-lease targeting keeps working under the mutation (the lease-setting reaction rides the captured pristine `then`)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - // The reviewer's exact repro at the broker boundary: with the - // guest-resolvable `.then` in the mirroring machinery, the replaced - // prototype hijacked the instrumented await; the captured pristine - // `then` keeps the mirror native. - const r = await broker.eval( - 'Promise.prototype.then = function () { return 99; }; const x = await Promise.resolve(40); globalThis.round7x = x; "done"', - ); - assert.equal(r.result, 'done', `the eval completed: ${output(r).join('\n')}`); - const x = await broker.eval('round7x'); - assert.equal(x.result, '40', 'the instrumented await mirrors the native value under the replaced prototype'); - // The lease plumbing still works under the mutation: an eval - // suspended on a call is targetable, and the armed signal breaks the - // continuation (the job after the lease-setting reaction) mid-run — - // the same end-to-end shape as the round-6 sibling regression, with - // the prototype replaced. - const a = await broker.eval( - 'const q = agent("pi/x", "research"); const out = await q; let y = 0; for (let i = 0; i < 200000; i++) y += i; globalThis.round7out = out; "waiting"', - ); - assert.ok(a.pending.includes('c1'), `pending: ${a.pending.join(', ')}`); - assert.equal(await broker.armEvalBreak(), true, 'the suspended eval is targetable despite the replaced prototype'); - runner.sessions[0].completeTurn('result'); - await tick(); - const probe = await bounded('probe after settling the mutated-prototype eval', broker.eval('"probe"')); - await assertInterruptedRetained(broker, 'the continuation was broken mid-run under the mutation'); - assert.ok( - output(probe).every((line) => !line.includes('interrupted')), - `the drain failure left the result surface (the continuation was broken mid-run under the mutation): ${output(probe).join('\n')}`, - ); - assert.equal(await broker.armEvalBreak(), false, 'the broken eval is no longer tracked'); - await broker.dispose(); - ws.dispose(); -}); - -/** - * The emulated 0.3.0 library copy (review round 7): the shipped source - * with the version marker at 0.3.0, the 0.3.0 LEASE-SET ORDERING (the - * lease-setting reaction runs on the awaited VALUE's settlement — the - * carried sibling-reaction interrupt-targeting defect: a sibling - * `q.then(...)` registered after the eval started awaiting `q` runs - * between the lease set and the continuation, consumes the armed - * signal, and the target's continuation runs later unprotected), and no - * iterable-lease capability. The broker's version gate must refuse to - * instrument/arm on this copy even though it reports - * `supportsContinuationLease: true` — the flag alone would have passed - * the pre-round-7 check. - */ -function guestLibrary030Source(): string { - const source = buildGuestLibrarySource('0.3.0'); - // The 0.3.0 `__replAwait` wrapper (the exact form the round-6 - // rejection described): the lease is set inside the job that resolves - // the wrapper, whose reactions are registered on the awaited VALUE. - const wrapper030 = `return new Promise(function (resolve, reject) { - Promise.resolve(value).then( - function (v) { - try { - setContinuationLease(token); - } catch (_e) {} - resolve(v); - }, - function (e) { - try { - setContinuationLease(token); - } catch (_e) {} - reject(e); - }, - ); - });`; - // The 0.3.1 wrapper the shipped source carries (the lease-setting - // reaction rides the WRAPPER promise itself, registered before the - // await machinery's own reaction). - const wrapper031 = `var wrapper = new P(function (resolve, reject) { - try { - pThen.call(PResolve(value), resolve, reject); - } catch (e) { - reject(e); - } - }); - pThen.call( - wrapper, - function () { - try { - setContinuationLease(token); - } catch (_e) {} - }, - function () { - try { - setContinuationLease(token); - } catch (_e) {} - }, - ); - return wrapper;`; - const patched = source.replace(wrapper031, wrapper030); - if (patched === source) { - throw new Error('round-7 fixture: could not patch the 0.3.1 wrapper into the 0.3.0 ordering'); - } - // The 0.3.0 copy has no iterable-lease capability. - const flagged = patched.replace('supportsIterableLease: true,', 'supportsIterableLease: false,'); - if (flagged === patched) { - throw new Error('round-7 fixture: could not patch the iterable-lease flag'); - } - return flagged; -} - -test('review round 7: a RESTORED 0.3.0 library is served WITHOUT instrumentation and the eval-break interrupt TERMINATES the running eval — the continuation-lease availability check is VERSION-GATED on 0.3.1 (the reviewer\'s finding: the 0.3.0 copy reports `supportsContinuationLease: true` but its helper still carries the sibling-reaction interrupt-targeting defect, so the flag alone re-armed the original defect on a supported older snapshot) and §3.2 never refuses a running eval: the unkeyable tracked continuation is released instead', async () => { - const runner = new FakeRunner(); - // Build an emulated 0.3.0 library copy, install it in a bare VM (with - // the four __host_* globals, exactly like the pre-snapshot host the - // fixture stands in for), snapshot it, and restore the workspace: the - // restored copy is served as-is (the doc's older-library rule — never - // re-injected). - const vm = await ReplVm.create(); - const shim = getVmShim(vm) as QuickJS; - const noopHost = (_args: unknown[]): JSValueHandle | undefined => undefined; - for (const name of ['__host_agent', '__host_checkpoint', '__host_agent_steer', '__host_console']) { - const fnHandle = shim.newFunction(name, noopHost); - shim.setProp(shim.global, name, fnHandle); - fnHandle.dispose(); - } - const installed = await vm.evalCode(guestLibrary030Source()); - assert.equal(installed.kind, 'value', `the emulated 0.3.0 library installed: ${JSON.stringify(installed).slice(0, 200)}`); - // ReplVm does not expose snapshot(); the shim does (the workspace - // layer's own snapshot() is exactly this call). - const snapshot = shim.snapshot(); - vm.dispose(); - const ws = await Workspace.restore(PROJECT, snapshot); - const broker = await Broker.attach(ws, { runner, evalTimeoutMs: 0 }); - try { - const surface = ws.surface()!; - assert.equal(surface.version, '0.3.0', 'the restored copy reports its own version'); - assert.equal(surface.supportsContinuationLease, true, 'the 0.3.0 copy reports the flag — the OLD gate would have accepted it'); - assert.equal(surface.supportsIterableLease, false, 'the 0.3.0 copy has no iterable-lease capability'); - // An eval suspends on a call — in flight, and in principle - // targetable (the exact shape the pre-gate arm would have armed). - // The sibling `.then` is registered AFTER the await: with the 0.3.0 - // lease-set ordering it would run between the lease set and the - // continuation (and consume the armed signal); with NO - // instrumentation it runs natively and never sees a lease. - const a = await broker.eval( - 'globalThis.round7sibling = null; const q = agent("pi/x", "research"); const x = await q; q.then(() => { globalThis.round7sibling = __replLease; }); globalThis.round7done = x; "waiting"', - ); - assert.ok(a.pending.includes('c1'), `pending: ${a.pending.join(', ')}`); - // §3.2: the eval IS running — a no-id interrupt never refuses a - // running eval. The 0.3.0 copy cannot key the signal to the - // continuation (the version gate: arming would re-arm the - // sibling-reaction targeting defect), so the running eval is - // TERMINATED instead — its tracked continuation released. - assert.equal( - await broker.armEvalBreak(), - true, - 'the running eval is terminated on the restored 0.3.0 copy — released, never refused (the version gate)', - ); - // Nothing is tracked any more: the next no-id interrupt is the - // honest idle refusal — the ONLY permitted refusal. - assert.equal(await broker.armEvalBreak(), false, 'nothing is tracked after the release — honest idle refusal'); - // The released eval's guest job still completes natively when the - // call settles (its continuation is unreachable by any armed - // signal) — the sibling reaction and the continuation both run, - // exactly like an un-instrumented workspace. - runner.sessions[0].completeTurn('result'); - await tick(); - const probe = await bounded('probe after settling the 0.3.0-copy eval', broker.eval('"probe"')); - assert.equal(probe.result, 'probe', `the workspace stays healthy: ${output(probe).join('\n')}`); - const sibling = await broker.eval('round7sibling === undefined ? "unset" : round7sibling'); - assert.equal(sibling.result, 'unset', 'no instrumentation ran — the sibling never observed a continuation lease (the 0.3.0 lease-set defect was never re-armed)'); - const done = await broker.eval('round7done'); - assert.equal(done.result, 'result', 'the continuation settled natively with the turn text'); - } finally { - await broker.dispose(); - ws.dispose(); - } -}); diff --git a/packages/repl-engine/test/fixtures/types-consumer/consumer.ts b/packages/repl-engine/test/fixtures/types-consumer/consumer.ts deleted file mode 100644 index 410d543c..00000000 --- a/packages/repl-engine/test/fixtures/types-consumer/consumer.ts +++ /dev/null @@ -1,596 +0,0 @@ -/** - * Type-check fixture simulating a published-package consumer. - * - * Imports the engine's PUBLIC declarations (`dist`) under the - * repository's non-DOM lib (the tsconfig.base lib `ES2022` + - * `ESNext.Disposable` — the latter is required by the public - * `[Symbol.dispose]` methods, same as quickjs-wasi's own) with - * `skipLibCheck: false` and no ambient `@types`. The public type graph - * must be fully self-contained: every type the API references must be - * declared inside the package itself. - * - * Regression (review): `ReplVmOptions.wasm`, `WorkspaceOptions.wasm` and - * `WorkspaceRegistryOptions.wasm` referenced `BufferSource` / - * `WebAssembly.Module`, whose only declarations lived in an unpublished - * source ambient (`src/wasm-ambient.d.ts`, never emitted — the published - * package ships `dist` only). This configuration failed with seven - * missing-type errors across `dist/vm.d.ts` and `dist/workspace.d.ts`. - * - * Note: this file is intentionally NOT part of the package's own - * typecheck (the package tsconfig covers src only); it is compiled by - * `test/public-types.test.ts` against the built `dist` declarations. - */ -import { - ReplVm, - Workspace, - WorkspaceRegistry, - loadShippedWasm, - DrainJobError, - GuestCall, - GuestLibraryInstallError, - installGuestBridge, - registerGuestHostCallbacks, - readGuestSurface, - readRealmSlot, - inspectGlobal, - renderCollapsed, - formatByteSize, - formatNumber, - escapeString, - stringDescription, - shortString, - headTailDescription, - isCanonicalIndex, - GUEST_LIBRARY_VERSION, - GUEST_SURFACE_KEY, - GUEST_VERSION_GLOBAL, - HOST_AGENT, - HOST_CHECKPOINT, - HOST_CONSOLE, - HOST_STEER, - MAX_PREVIEW_PROPERTIES, - MAX_COLLAPSED_CHARS, - type ReplVmOptions, - type ReplEvalOptions, - type ReplEvalOutcome, - type WorkspaceOptions, - type WorkspaceRegistryOptions, - type EvalErrorInfo, - type WasmInput, - type WasmModule, - type ReplSnapshot, - type GuestBridgeHandlers, - type GuestSurface, - type GuestSurfaceEntry, - type ConsoleEvent, - type ConsoleLevel, - type RealmSlot, - type ObjectPreview, - type PropertyPreview, - type PreviewType, - type PreviewSubtype, -} from '../../../dist/index.js'; - -async function exercise(): Promise { - // The shipped binary compiles to the self-contained `WasmModule` type - // and round-trips into every option position that used to name the - // (undeclared) `WebAssembly.Module` / `BufferSource` globals. - const module: WasmModule = await loadShippedWasm(); - const vmOptions: ReplVmOptions = { wasm: module, memoryLimit: 1024 }; - const vm = await ReplVm.create(vmOptions); - const evalOptions: ReplEvalOptions = { - filename: 'consumer.js', - interruptHandler: () => false, - }; - const outcome: ReplEvalOutcome = await vm.evalCode('1 + 1', evalOptions); - if (outcome.kind === 'error') { - const info: EvalErrorInfo = outcome.error; - info.interrupted satisfies boolean; - info.outOfMemory satisfies boolean; - } - const wsOptions: WorkspaceOptions = { wasm: module }; - const ws = await Workspace.create('/tmp/project', wsOptions); - const registryOptions: WorkspaceRegistryOptions = { wasm: module, memoryLimit: 4096 }; - const registry = new WorkspaceRegistry(registryOptions); - const got: Workspace = await registry.get('/tmp/project', wsOptions); - const bytes = new Uint8Array([0, 97, 115, 109]); - const asBytes: WasmInput = bytes; - const asArrayBuffer: WasmInput = new ArrayBuffer(4); - let maybeDrainError: DrainJobError | undefined; - if (maybeDrainError) { - maybeDrainError.info satisfies EvalErrorInfo; - } - ws.dispose(); - got.dispose(); - registry.disposeAll(); - vm.dispose(); -} - -async function exercisePhaseB(): Promise { - // Phase B public surface: the guest-library bridge, the previewer, the - // caps — every exported declaration must be checkable by a consumer with - // a non-DOM lib and skipLibCheck: false (no quickjs-wasi types leak). - const bridgeVm = await ReplVm.create(); - const handlers: GuestBridgeHandlers = { - agent: (call: GuestCall, callId: string, modelSpec: string, task: string, optionsJson: string | null) => { - callId satisfies string; - modelSpec satisfies string; - task satisfies string; - optionsJson satisfies string | null; - call.resolve({ ok: true }); - }, - checkpoint: (call, callId, question, optionsJson, answerJson) => { - callId satisfies string; - question satisfies string | null; - optionsJson satisfies string | null; - answerJson satisfies string | null; - if (answerJson !== null) return true; - call?.resolve('answered'); - return undefined; - }, - queue: (call, callId, sessionId, payloadJson) => { - callId satisfies string; - sessionId satisfies string; - payloadJson satisfies string | null; - call.resolve('queued answer'); - }, - steer: (call, callId, sessionId, payloadJson) => { - callId satisfies string; - sessionId satisfies string; - payloadJson satisfies string | null; - call.resolve('injected'); - }, - cancelSession: (call, callId, sessionId) => { - callId satisfies string; - sessionId satisfies string; - call.resolve('idle'); - }, - cancelQueue: (call, callId, queueCallId) => { - callId satisfies string; - queueCallId satisfies string; - call.resolve('idle'); - }, - console: (event: ConsoleEvent) => { - event.level satisfies ConsoleLevel; - event.line satisfies string; - }, - sleep: (call: GuestCall, ms: number) => { - ms satisfies number; - call.resolve(undefined); - }, - workspace: () => '{}', - agents: () => '[]', - reset: () => undefined, - defaultBackend: () => undefined, - }; - await installGuestBridge(bridgeVm, handlers); - registerGuestHostCallbacks(bridgeVm, handlers); - const surface: GuestSurface | undefined = readGuestSurface(bridgeVm); - if (surface) { - surface.version satisfies string; - const pending: GuestSurfaceEntry[] = surface.pending(); - pending satisfies GuestSurfaceEntry[]; - const settled: boolean = surface.settle('c1', 'resolve', 42); - settled satisfies boolean; - const stats = surface.stats(); - stats.pendingCalls satisfies number; - } - const slot: RealmSlot = readRealmSlot(bridgeVm, 'agent'); - slot satisfies RealmSlot; - const meta = inspectGlobal(bridgeVm, 'agent'); - meta.kind satisfies 'data' | 'accessor' | 'absent'; - meta.label satisfies string; - meta.sizeBytes satisfies number; - const preview: ObjectPreview = { - type: 'object', - subtype: 'array', - description: 'Array(3)', - overflow: false, - properties: [{ name: '0', type: 'number', value: '1' } satisfies PropertyPreview], - }; - const t: PreviewType = preview.type; - const st: PreviewSubtype | undefined = preview.subtype; - renderCollapsed(preview) satisfies string; - formatByteSize(48000) satisfies string; - formatNumber(-0) satisfies string; - escapeString('a"b') satisfies string; - stringDescription('x'.repeat(300)) satisfies string; - shortString('y') satisfies string; - headTailDescription('z', 120) satisfies string; - isCanonicalIndex('0') satisfies boolean; - MAX_PREVIEW_PROPERTIES satisfies number; - MAX_COLLAPSED_CHARS satisfies number; - GUEST_LIBRARY_VERSION satisfies string; - GUEST_SURFACE_KEY satisfies string; - GUEST_VERSION_GLOBAL satisfies string; - HOST_AGENT satisfies string; - HOST_CHECKPOINT satisfies string; - HOST_CONSOLE satisfies string; - HOST_STEER satisfies string; - bridgeVm.dispose(); -} - -// Phase C: the broker, the call store, and the eval tool-result shape — -// all self-contained (no acp-agents / quickjs-wasi / shared-types types -// in the published declaration graph; the fixture's non-DOM lib and -// `skipLibCheck: false` compile the whole surface). -import { - Broker, - JsonlCallStore, - InMemoryCallStore, - DEFAULT_MAX_CONCURRENT_AGENTS, - type BrokerOptions, - type BrokerRunner, - type BrokerSession, - type BrokerTurn, - type BrokerOpenSessionOptions, - type BrokerLoadSessionOptions, - type BrokerPromptOptions, - type SteeringOutcomeValue, - type ReplEvalResult, - type CheckpointSummary, - type CheckpointInfo, - type LiveAgentInfo, - type ReconcileReport, - type CallStore, - type CallRecord, - type CallOutcome, - type CallKind, - type CallOutcomeKind, -} from '../../../dist/index.js'; - -function brokerTyping(ws: Workspace, opts: BrokerOptions): Promise { - return Broker.attach(ws, opts); -} - -function storeTyping(store: CallStore): void { - store.recordDispatched({ - callId: 'c1', - kind: 'agent', - detail: 'task', - optionsJson: null, - modelSpec: 'pi/x', - backendId: 'pi', - foundingCallId: null, - admittedAtMs: 1, - admissionSequence: 1, - dispatchedAtMs: 1, - reissues: 0, - completion: null, - sessionId: null, - queuedAtMs: null, - handoffAtMs: null, - cancelledAtMs: null, - }); - store.recordReissued('c1', 2); - store.recordQueued('c1', 4); - store.recordAttached('c1', 'backend-session-1', 5, 'pi'); - store.recordCompleted('c1', { outcome: 'resolve', value: { ok: true }, completedAtMs: 3 }) satisfies boolean; - store.recordHandoff('c1', 4); - store.recordCancelled('c1', 5); - store.lookup('c1') satisfies CallRecord | undefined; - store.all() satisfies CallRecord[]; -} - -function brokerSurfaceTyping(ws: Workspace): void { - const options: BrokerOptions = { - runner: { - listBackends() { - return ['claude', 'pi']; - }, - defaultBackendId() { - return 'claude'; - }, - async openSession(_opts: BrokerOpenSessionOptions): Promise { - return { - sessionId: 's1', - backendId: 'pi', - initializeMeta: { steering: { supported: true } }, - async prompt(_content: string, _opts?: BrokerPromptOptions): Promise { - return { stopReason: 'end_turn', text: 'ok' }; - }, - async steer(_content: string, _opts?: BrokerPromptOptions): Promise { - return { outcome: 'injected' }; - }, - async cancel(): Promise {}, - async release(): Promise {}, - currentTurnText(): string { - return ''; - }, - finalMessageText(): string { - return ''; - }, - rawStructuredOutput(): unknown { - return undefined; - }, - async awaitCurrentTurn(): Promise { - return { stopReason: 'end_turn', text: 'loaded' }; - }, - }; - }, - async loadSession(_opts: BrokerLoadSessionOptions): Promise { - return { - sessionId: 's1', - async prompt(): Promise { - return { stopReason: 'end_turn', text: 'ok' }; - }, - async steer(): Promise { - return { outcome: 'injected' }; - }, - async cancel(): Promise {}, - async release(): Promise {}, - currentTurnText(): string { - return ''; - }, - finalMessageText(): string { - return ''; - }, - rawStructuredOutput(): unknown { - return undefined; - }, - }; - }, - async dispose(): Promise {}, - }, - store: new InMemoryCallStore(), - maxConcurrentAgents: 6, - evalTimeoutMs: 30_000, - interruptHandler: () => false, - snapshotSink: { - boundary: (_kind: SnapshotBoundaryKind) => {}, - flush: () => {}, - }, - } satisfies BrokerOptions; - void brokerTyping(ws, options); - DEFAULT_MAX_CONCURRENT_AGENTS satisfies number; - const outcome: SteeringOutcomeValue = 'unsupported'; - outcome satisfies string; - const recordKind: CallKind = 'steer'; - recordKind satisfies string; - const outcomeKind: CallOutcomeKind = 'reject'; - outcomeKind satisfies string; - const summary: CheckpointSummary = { id: 'c1', question: 'What color?' }; - summary satisfies { id: string; question: string }; - const info: CheckpointInfo = { id: 'c1', question: 'x', optionsJson: null, raisedAtMs: 1 }; - const live: LiveAgentInfo = { - callId: 'c1', - modelSpec: 'pi/x', - task: 't', - state: 'running', - supportsSteering: true, - queuedTurns: 0, - }; - const report: ReconcileReport = { - settledFromStore: ['c1'], - reattached: [], - reissued: [], - failedLost: [], - requeuedCheckpoints: [], - leftPending: [], - reQueuedUndelivered: [], - }; - info satisfies CheckpointInfo; - live satisfies LiveAgentInfo; - report satisfies ReconcileReport; - const fileStore = JsonlCallStore.open('/tmp/consumer-calls.jsonl'); - fileStore.path() satisfies string; - fileStore.close(); - const evalResult: ReplEvalResult = { - output: ['42'], - kind: 'value', - result: '42', - evalToken: 'e1', - pending: [], - checkpoints: [], - completed: [], - }; - evalResult satisfies ReplEvalResult; - const callOutcome: CallOutcome = { outcome: 'resolve', value: 'x', completedAtMs: 1 }; - callOutcome satisfies CallOutcome; - void storeTyping; - void options; - void brokerSurfaceTyping; -} - -// Phase D: enveloped snapshots, the per-project store, and the restore -// path — all self-contained (no node types, no quickjs-wasi / workflows -// types in the published declaration graph; the fixture's non-DOM lib, -// `types: []` and `skipLibCheck: false` compile the whole surface). -import { - SNAPSHOT_FORMAT, - SNAPSHOT_FORMAT_VERSION, - serializeSnapshot, - deserializeSnapshot, - wasmSha256Of, - SnapshotEnvelopeError, - ReplWorkspaceStore, - REPL_STORE_SUBDIR, - SNAPSHOT_FILENAME, - CALL_STORE_FILENAME, - GUEST_PROVENANCE_KEY, - manifestBinding, - baselineGlobalKeys, - provenanceBootstrap, - provenanceRecord, - provenanceView, - isValidOriginLabel, - DEFAULT_EVAL_TIMEOUT_MS, - type SnapshotEnvelopeMeta, - type SnapshotEnvelope, - type SnapshotEnvelopeErrorCode, - type ReplStoreOptions, - type SnapshotWriteOptions, - type RestoredReplSnapshot, - type ReplStoreStats, - type SnapshotSink, - type SnapshotBoundaryKind, - type WorkspaceManifest, - type WorkspaceBinding, - type WorkspaceManifestReport, - type WorkspaceManifestBinding, - type ProvenanceOrigin, - type ProvenanceView, - type OriginRecord, - type BaselineKeys, -} from '../../../dist/index.js'; - -async function phaseDSurfaceTyping(): Promise { - SNAPSHOT_FORMAT satisfies string; - SNAPSHOT_FORMAT_VERSION satisfies number; - REPL_STORE_SUBDIR satisfies string; - SNAPSHOT_FILENAME satisfies string; - CALL_STORE_FILENAME satisfies string; - GUEST_PROVENANCE_KEY satisfies string; - DEFAULT_EVAL_TIMEOUT_MS satisfies number; - - const storeOptions: ReplStoreOptions = { - persistenceRoot: '/tmp/persist', - env: { AGENTPRISM_PERSISTENCE_ROOT: '/tmp/persist' }, - snapshotWrite: { debounceBursts: true, fsync: true }, - }; - const writeOptions: SnapshotWriteOptions = {}; - writeOptions.debounceBursts satisfies boolean | undefined; - writeOptions.fsync satisfies boolean | undefined; - const store = ReplWorkspaceStore.open('/tmp/project', storeOptions); - store.projectDir satisfies string; - store.replDir satisfies string; - store.snapshotPath satisfies string; - store.callStorePath satisfies string; - store.hasSnapshot() satisfies boolean; - const sink: SnapshotSink = { - boundary: (kind: SnapshotBoundaryKind) => { - kind satisfies 'eval' | 'settlement'; - }, - flush: () => {}, - }; - void sink; - // The snapshot writer is wired to a REAL workspace and wasm input — the - // phase-D review round-2 fixture rule: the writer's type surface must be - // exercised against real engine objects, never fake substitutes. - const realWorkspace = await Workspace.create('/tmp/project'); - const realWasm: WasmInput = new Uint8Array([0, 97, 115, 109]); - const fromWriter: SnapshotSink = store.snapshotWriter(realWorkspace, realWasm); - fromWriter.boundary('eval'); - store.stats() satisfies ReplStoreStats; - store.close(); - store.reset(); - - const snapshot: ReplSnapshot = { - memory: new Uint8Array(4), - stackPointer: 0, - runtimePtr: 0, - contextPtr: 0, - extensions: [], - }; - const envelope: Uint8Array = serializeSnapshot(snapshot, 'a'.repeat(64), { createdAtMs: 1 }); - envelope satisfies Uint8Array; - const parsed: SnapshotEnvelope = deserializeSnapshot(envelope, { path: '/tmp/snapshot.bin' }); - parsed.snapshot.runtimePtr satisfies number; - parsed.meta satisfies SnapshotEnvelopeMeta; - parsed.meta.wasmSha256 satisfies string; - parsed.meta.formatVersion satisfies number; - const hash: string = wasmSha256Of(new Uint8Array([1, 2, 3])); - hash satisfies string; - const restored: RestoredReplSnapshot = store.loadSnapshot(new Uint8Array([0, 97, 115, 109])); - restored.snapshot satisfies ReplSnapshot; - restored.wasmSha256 satisfies string; - restored.formatVersion satisfies number; - restored.createdAtMs satisfies number; - const errorCode: SnapshotEnvelopeErrorCode = 'WASM_HASH_MISMATCH'; - errorCode satisfies string; - const err = new SnapshotEnvelopeError('VERSION_MISMATCH', 'boom', { - path: '/tmp/snapshot.bin', - recorded: '2', - expected: '1', - }); - err.code satisfies SnapshotEnvelopeErrorCode; - err.path satisfies string | undefined; - - // The manifest + provenance + drain surface (phase-D review round 2). - const manifest: WorkspaceManifest = realWorkspace.manifest(); - manifest.evalSeq satisfies number; - manifest.logs satisfies { first: number | null; last: number | null; count: number }; - const binding: WorkspaceBinding = manifest.bindings[0]; - binding.name satisfies string; - binding.token satisfies string; - binding.handleCallId satisfies string | null; - binding.provenance satisfies string | null; - binding.provenanceAtMs satisfies number | null; - realWorkspace.provenanceRecord({ kind: 'eval' }); - realWorkspace.provenanceRecord({ kind: 'settlement', callIds: ['c1'] }); - realWorkspace.provenanceRecord({ kind: 'restore' }); - const provView: ProvenanceView = realWorkspace.provenanceView(); - provView.evalSeq satisfies number; - provView.origins satisfies Map; - const origin: ProvenanceOrigin = { kind: 'settlement', callIds: ['c1'] }; - origin satisfies ProvenanceOrigin; - const baseline: Promise = baselineGlobalKeys(realWasm); - baseline satisfies Promise; - const freshVm = await ReplVm.create(); - const bootstrapped: Promise<{ created: boolean; baseline: BaselineKeys }> = provenanceBootstrap(freshVm, realWasm); - void bootstrapped; - freshVm.dispose(); - isValidOriginLabel('eval 1') satisfies boolean; - const bindingToken = manifestBinding(freshVm, 'x'); - bindingToken?.token satisfies string | undefined; - bindingToken?.handleCallId satisfies string | null | undefined; - realWorkspace.dispose(); - - void writeOptions; - void fromWriter; - void restored; -} - -// The broker's phase-D round-2 additions: the enriched manifest, the -// client-presence drain, the eval deadline, and the cancel outcomes. -async function brokerManifestAndDrainTyping(): Promise { - const broker = await Broker.attach(await Workspace.create('/tmp/project')); - broker.isDrained satisfies boolean; - broker.busySessionCount() satisfies number; - broker.inFlightIds() satisfies string[]; - const drained: Promise = broker.drainForDisconnect(60_000); - drained satisfies Promise; - const cancelOutcome: Promise<'cancelled' | 'idle' | 'failed' | 'none'> = broker.cancelCall('c1'); - cancelOutcome satisfies Promise; - const report: WorkspaceManifestReport = broker.workspaceManifest(); - report.evalSeq satisfies number; - report.inFlight satisfies string[]; - report.checkpoints satisfies CheckpointInfo[]; - report.logs satisfies { first: number | null; last: number | null; count: number }; - const manifestBindingEntry: WorkspaceManifestBinding = report.bindings[0]; - manifestBindingEntry.name satisfies string; - manifestBindingEntry.token satisfies string; - manifestBindingEntry.provenance satisfies string | null; - manifestBindingEntry.provenanceAtMs satisfies number | null; - await broker.dispose(); -} - -// The self-contained snapshot stand-in round-trips as a type. -function snapshotTyping(snapshot: ReplSnapshot): ReplSnapshot { - return snapshot; -} - -// Negative cases — the public boundary must reject accidental values. -// Review regression: `WasmModule` was an empty interface, so `{ wasm: 42 }` -// type-checked (every non-null value satisfies an empty interface) and -// failed only at runtime. The branded opaque module type must make all of -// these compile-time errors; the `@ts-expect-error` directives fail the -// fixture build if any line stops erroring. - -// @ts-expect-error a bare number is not a wasm input -const badNumber: WasmInput = 42; - -// @ts-expect-error a bare object is not a compiled module (opaque brand) -const badModule: WasmModule = {}; - -// @ts-expect-error a string is not a wasm input -const badString: WasmInput = 'quickjs.wasm'; - -// @ts-expect-error `{ wasm: 42 }` must not type-check as VM options -const badVmOptions: ReplVmOptions = { wasm: 42 }; - -// @ts-expect-error `{ wasm: 42 }` must not type-check as workspace options -const badWsOptions: WorkspaceOptions = { wasm: 42 }; - -// @ts-expect-error a boolean is not a wasm input -const badBoolean: WasmInput = true; diff --git a/packages/repl-engine/test/fixtures/types-consumer/tsconfig.json b/packages/repl-engine/test/fixtures/types-consumer/tsconfig.json deleted file mode 100644 index 5df6732d..00000000 --- a/packages/repl-engine/test/fixtures/types-consumer/tsconfig.json +++ /dev/null @@ -1,13 +0,0 @@ -{ - "compilerOptions": { - "target": "ES2022", - "lib": ["ES2022", "ESNext.Disposable"], - "module": "NodeNext", - "moduleResolution": "NodeNext", - "strict": true, - "skipLibCheck": false, - "noEmit": true, - "types": [] - }, - "files": ["consumer.ts"] -} diff --git a/packages/repl-engine/test/guest-library.test.ts b/packages/repl-engine/test/guest-library.test.ts deleted file mode 100644 index b4c19d39..00000000 --- a/packages/repl-engine/test/guest-library.test.ts +++ /dev/null @@ -1,1709 +0,0 @@ -/** - * Phase B tests: the guest library, the host bridge, and the console - * bridge ($N freezing, settlement, reconciliation surface, snapshot - * travel). Combinators are exercised over a mocked `__host_agent` that - * settles synchronously (the eval's own drain completes the awaited - * results) or on demand (started-not-awaited handles). - */ - -import assert from 'node:assert/strict'; -import { test } from 'node:test'; - -import { - GUEST_LIBRARY_VERSION, - GUEST_SURFACE_KEY, - GUEST_VERSION_GLOBAL, - HOST_AGENT, - HOST_CHECKPOINT, - HOST_CONSOLE, - HOST_QUEUE, - HOST_QUEUE_CANCEL, - HOST_SESSION_CANCEL, - HOST_STEER, - GuestLibraryInstallError, - ReplVm, - installGuestBridge, - readGuestSurface, - readRealmSlot, - registerGuestHostCallbacks, - type ConsoleEvent, - type ConsoleLevel, - type GuestBridgeHandlers, - type GuestCall, -} from '../src/index.js'; -import { buildGuestLibrarySource } from '../src/guest/guest-library.js'; -import { getVmShim } from '../src/vm.js'; -import type { JSValueHandle, QuickJS, HostFunction } from 'quickjs-wasi'; - -// ──────────────────────────────────────────────────────────────────────── -// Mock host -// ──────────────────────────────────────────────────────────────────────── - -/** A scripted agent/control resolution: resolve with `value` or reject with - * `error` (exactly one of the two). */ -type Scripted = { resolveWith?: unknown; rejectWith?: unknown; assertPrompt?: string }; - -interface MockBridge { - handlers: GuestBridgeHandlers; - /** Every console event crossing the bridge, in order. */ - events: ConsoleEvent[]; - /** Agent calls, in issue order. */ - agentCalls: Array<{ call: GuestCall; callId: string; modelSpec: string; task: string; optionsJson: string | null }>; - /** Durable queued-turn calls, in issue order. */ - queueCalls: Array<{ call: GuestCall; callId: string; sessionId: string; payloadJson: string | null }>; - /** Strict active-turn steering calls, in issue order. */ - steerCalls: Array<{ call: GuestCall; callId: string; sessionId: string; payloadJson: string | null }>; - /** Reusable-session handle cancellation calls, in issue order. */ - sessionCancelCalls: Array<{ call: GuestCall; callId: string; sessionId: string }>; - /** Exact queued-turn cancellation calls, in issue order. */ - queueCancelCalls: Array<{ call: GuestCall; callId: string; queueCallId: string }>; - /** Checkpoint questions, keyed by call id (for answer delivery). */ - pendingCheckpoints: Map; - /** Scripted resolutions for agent/steer calls, consumed in order. */ - script: Scripted[]; -} - -function mockBridge(): MockBridge { - const bridge: MockBridge = { - handlers: { - agent: (call, callId, modelSpec, task, optionsJson) => { - bridge.agentCalls.push({ call, callId, modelSpec, task, optionsJson }); - settleScripted(bridge, call); - }, - checkpoint: (call, callId, question, optionsJson, answerJson) => { - if (answerJson !== null) { - // Answer mode: settle the original pending checkpoint. - const pending = bridge.pendingCheckpoints.get(callId); - if (!pending) return false; - bridge.pendingCheckpoints.delete(callId); - pending.resolve(JSON.parse(answerJson)); - return true; - } - bridge.pendingCheckpoints.set(callId, call!); - return undefined; - }, - queue: (call, callId, sessionId, payloadJson) => { - bridge.queueCalls.push({ call, callId, sessionId, payloadJson }); - settleScripted(bridge, call); - }, - steer: (call, callId, sessionId, payloadJson) => { - bridge.steerCalls.push({ call, callId, sessionId, payloadJson }); - settleScripted(bridge, call); - }, - cancelSession: (call, callId, sessionId) => { - bridge.sessionCancelCalls.push({ call, callId, sessionId }); - settleScripted(bridge, call); - }, - cancelQueue: (call, callId, queueCallId) => { - bridge.queueCancelCalls.push({ call, callId, queueCallId }); - settleScripted(bridge, call); - }, - console: (event) => { - bridge.events.push(event); - }, - sleep: (call, ms) => { - setTimeout(() => { - try { - call.resolve(undefined); - } catch { - // vm disposed mid-sleep — nothing to settle. - } - }, ms); - }, - workspace: () => '{}', - agents: () => '[]', - reset: () => undefined, - defaultBackend: () => 'claude', - }, - events: [], - agentCalls: [], - queueCalls: [], - steerCalls: [], - sessionCancelCalls: [], - queueCancelCalls: [], - pendingCheckpoints: new Map(), - script: [], - }; - return bridge; -} - -function settleScripted(bridge: MockBridge, call: GuestCall): void { - const s = bridge.script.shift(); - if (s === undefined) { - // No script: park the call; the test settles it later. - return; - } - if ('resolveWith' in s) call.resolve(s.resolveWith); - else if ('rejectWith' in s) call.reject(s.rejectWith); - // else: `{}` — parked (the test settles it later). -} - -async function createGuest(): Promise<{ vm: ReplVm; bridge: MockBridge }> { - const vm = await ReplVm.create(); - const bridge = mockBridge(); - await installGuestBridge(vm, bridge.handlers); - return { vm, bridge }; -} - -/** - * Install the guest library at an arbitrary version, with minimal - * host functions — simulates the OLDER host that snapshotted a workspace - * (the doc's evolution discipline: a host must serve snapshots carrying - * older library versions than the one it currently injects, and the - * resident version stays authoritative). The minimal surface is enough - * for the discipline assertions: agent/queue/steer/cancel/checkpoint park (their - * registry entries pend), answer mode reports false, console events - * bridge into the mock's event list. - */ -async function installGuestLibraryAtVersion( - vm: ReplVm, - version: string, - bridge: MockBridge, -): Promise { - const shim = getVmShim(vm) as QuickJS; - const hostFn = ( - fn: (args: Array) => JSValueHandle | undefined, - ): HostFunction => { - return function (this: JSValueHandle, ...args: JSValueHandle[]): JSValueHandle { - const strs = args.map((a) => (a.isString ? a.toString() : null)); - return fn(strs) ?? shim.undefined; - }; - }; - const callbacks: Array<[string, (args: Array) => JSValueHandle | undefined]> = [ - [HOST_AGENT, () => undefined], - [HOST_CHECKPOINT, (args) => (args.length >= 4 ? shim.false : undefined)], - [HOST_QUEUE, () => undefined], - [HOST_STEER, () => undefined], - [HOST_SESSION_CANCEL, () => undefined], - [HOST_QUEUE_CANCEL, () => undefined], - [ - HOST_CONSOLE, - (args) => { - const level = args[0]; - const payload = args[1] !== null ? JSON.parse(args[1]) : null; - if (level !== null && payload !== null && typeof payload.line === 'string') { - bridge.events.push({ level: level as ConsoleLevel, line: payload.line as string }); - } - return undefined; - }, - ], - ]; - for (const [name, fn] of callbacks) { - const fnHandle = shim.newFunction(name, hostFn(fn)); - shim.setProp(shim.global, name, fnHandle); - fnHandle.dispose(); - } - const outcome = await vm.evalCode(buildGuestLibrarySource(version)); - assert.equal(outcome.kind, 'value', `library v${version} install failed: ${JSON.stringify(outcome)}`); -} - -function value(outcome: Awaited>): unknown { - assert.equal(outcome.kind, 'value', `expected value outcome, got ${JSON.stringify(outcome)}`); - return outcome.value; -} - -function pending(outcome: Awaited>): void { - assert.equal(outcome.kind, 'pending', `expected pending outcome, got ${JSON.stringify(outcome)}`); -} - -// ──────────────────────────────────────────────────────────────────────── -// Installation, version marker, deleted vocabulary -// ──────────────────────────────────────────────────────────────────────── - -test('install: the doc-mandated globals exist; phase() and the budget surface are deleted', async () => { - const { vm } = await createGuest(); - const out = value( - await vm.evalCode(`({ - agent: typeof agent, checkpoint: typeof checkpoint, - answer: typeof checkpoint.answer, - console: typeof console, log: typeof console.log, - parallel: typeof parallel, pipeline: typeof pipeline, verify: typeof verify, - judgePanel: typeof judgePanel, gate: typeof gate, retry: typeof retry, - loopUntilDry: typeof loopUntilDry, - sleep: typeof sleep, workspace: typeof workspace, agents: typeof agents, - reset: typeof reset, underscore: typeof _, - phase: typeof phase, budget: typeof budget, - hostBudget: typeof globalThis.__host_budget, - marker: globalThis[Symbol.for(${JSON.stringify(GUEST_SURFACE_KEY)})] !== undefined, - markerVersion: globalThis[Symbol.for(${JSON.stringify(GUEST_SURFACE_KEY)})].version, - })`), - ); - assert.deepEqual(out, { - agent: 'function', - checkpoint: 'function', - answer: 'function', - console: 'object', - log: 'function', - parallel: 'function', - sleep: 'function', - workspace: 'function', - agents: 'function', - reset: 'function', - underscore: 'undefined', - pipeline: 'function', - verify: 'function', - judgePanel: 'function', - gate: 'function', - retry: 'function', - loopUntilDry: 'function', - phase: 'undefined', - budget: 'undefined', - hostBudget: 'undefined', - marker: true, - markerVersion: GUEST_LIBRARY_VERSION, - }); - // The version marker global is a non-writable, non-enumerable data property. - const marker = value(await vm.evalCode(`(() => { - const d = Object.getOwnPropertyDescriptor(globalThis, ${JSON.stringify(GUEST_VERSION_GLOBAL)}); - return { value: d.value, writable: d.writable, enumerable: d.enumerable, configurable: d.configurable }; - })()`)); - assert.deepEqual(marker, { - value: GUEST_LIBRARY_VERSION, - writable: false, - enumerable: false, - configurable: false, - }); - vm.dispose(); -}); - -test('install is idempotent: re-evaluating the library never wipes state or counters', async () => { - const { vm } = await createGuest(); - value(await vm.evalCode('agent("pi/x", "first"); "done"')); - // The library guards itself: re-evaluation is a no-op. - const outcome = await vm.evalCode(buildGuestLibrarySource()); - assert.equal(outcome.kind, 'value'); - value(await vm.evalCode('agent("pi/x", "second"); "done"')); - assert.equal(value(await vm.evalCode('typeof agent')), 'function'); - // The call-id counter kept counting across the re-evaluation. - const stats = readGuestSurface(vm)!; - assert.equal(stats.stats().callSeq, 2); - assert.equal(stats.stats().pendingCalls, 2); - // installGuestBridge over an installed workspace is a no-op too. - await installGuestBridge(vm, mockBridge().handlers); - assert.equal(value(await vm.evalCode('1 + 1')), 2); - vm.dispose(); -}); - -test('a fresh VM without the bridge has no guest library (surface absent)', async () => { - using vm = await ReplVm.create(); - assert.equal(readGuestSurface(vm), undefined); - assert.deepEqual(readRealmSlot(vm, 'agent'), { kind: 'absent' }); - // The host functions are not installed either. - assert.deepEqual(readRealmSlot(vm, HOST_AGENT), { kind: 'absent' }); -}); - -// ──────────────────────────────────────────────────────────────────────── -// agent() and the live handle -// ──────────────────────────────────────────────────────────────────────── - -test('agent() round trip: the mocked host receives modelSpec + task and its result resolves in-eval', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push({ resolveWith: 'research done' }); - assert.equal(value(await vm.evalCode('await agent("pi/deepseek-v4-flash-max", "research X")')), 'research done'); - assert.equal(bridge.agentCalls.length, 1); - assert.equal(bridge.agentCalls[0].callId, 'c1'); - assert.equal(bridge.agentCalls[0].modelSpec, 'pi/deepseek-v4-flash-max'); - assert.equal(bridge.agentCalls[0].task, 'research X'); - assert.equal(bridge.agentCalls[0].optionsJson, null); - vm.dispose(); -}); - -test('agent() options cross the bridge as JSON (schema, cwd, configOptions, mode)', async () => { - const { vm, bridge } = await createGuest(); - const options = { - schema: { type: 'object', required: ['x'] }, - cwd: '/tmp', - configOptions: { thinkingLevel: 'high' }, - mode: 'read-only', - }; - bridge.script.push({ resolveWith: { x: 1 } }); - const result = value(await vm.evalCode(`await agent("pi/default", "p", ${JSON.stringify(options)})`)); - assert.deepEqual(result, { x: 1 }); - const parsed = JSON.parse(bridge.agentCalls[0].optionsJson!); - assert.deepEqual(parsed, options); - vm.dispose(); -}); - -test('agent() preserves unknown option keys with non-JSON-representable values for host validation', async () => { - const { vm, bridge } = await createGuest(); - const rejection = { - rejectWith: { - message: 'agent options: unknown option "bogus" (valid options: schema, cwd, configOptions, mode)', - code: 'SCRIPT_VALIDATION_ERROR', - recoverable: false, - replBackend: 'pi', - }, - }; - bridge.script.push(rejection, rejection, rejection, rejection); - const values = ['undefined', 'function () {}', 'Symbol("s")', '10n']; - for (const optionValue of values) { - const message = value( - await vm.evalCode( - `await agent("pi/x", "task", { bogus: ${optionValue} }).then(() => "accepted", (err) => err.code + "|" + err.message)`, - ), - ); - assert.equal( - message, - 'SCRIPT_VALIDATION_ERROR|agent options: unknown option "bogus" (valid options: schema, cwd, configOptions, mode)', - ); - } - assert.equal(bridge.agentCalls.length, 4); - assert.deepEqual( - bridge.agentCalls.map((call) => call.optionsJson), - ['{"bogus":null}', '{"bogus":null}', '{"bogus":null}', '{"bogus":null}'], - 'every present unknown key survives the JSON bridge regardless of its value', - ); - assert.equal(readGuestSurface(vm)!.stats().pendingCalls, 0, 'the synchronous host refusal settled the registry'); - vm.dispose(); -}); - -test('agent() omits undefined known options while still dispatching the call', async () => { - const { vm, bridge } = await createGuest(); - const options = [ - '{ schema: undefined }', - '{ cwd: undefined }', - '{ configOptions: undefined }', - '{ mode: undefined }', - '{ cwd: "/tmp", schema: undefined }', - '{ configOptions: { thinkingLevel: undefined } }', - ]; - bridge.script.push(...options.map(() => ({ resolveWith: 'accepted' }))); - for (const option of options) { - assert.equal( - value(await vm.evalCode(`await agent("pi/x", "task", ${option})`)), - 'accepted', - `${option} dispatches`, - ); - } - assert.deepEqual( - bridge.agentCalls.map((call) => call.optionsJson), - ['{}', '{}', '{}', '{}', '{"cwd":"/tmp"}', '{"configOptions":{}}'], - 'known undefined values retain ordinary JSON omission semantics at every depth', - ); - vm.dispose(); -}); - -test('handle options omit undefined promptMeta but preserve undefined unknown keys for host validation', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push( - { resolveWith: 'initial result' }, - { resolveWith: 'queued result' }, - { - rejectWith: { - message: 'steer options: unknown option "bogus"', - code: 'SCRIPT_VALIDATION_ERROR', - recoverable: false, - replBackend: 'pi', - }, - }, - ); - const result = value( - await vm.evalCode(` - const h = agent("pi/x", "task"); - const accepted = await h.queue("next", { promptMeta: undefined }); - const rejected = await h.steer("redirect", { bogus: undefined }) - .then(() => "accepted", (err) => err.code + "|" + err.message); - ({ accepted, rejected }) - `), - ); - assert.deepEqual(result, { - accepted: 'queued result', - rejected: 'SCRIPT_VALIDATION_ERROR|steer options: unknown option "bogus"', - }); - assert.deepEqual( - [...bridge.queueCalls, ...bridge.steerCalls].map((call) => call.payloadJson === null ? null : JSON.parse(call.payloadJson)), - [ - { prompt: 'next', options: {} }, - { prompt: 'redirect', options: { bogus: null } }, - ], - 'known undefined steer options are omitted while unknown keys survive the JSON bridge', - ); - vm.dispose(); -}); - -test('agent() validation: non-string modelSpec/task reject with a TypeError', async () => { - const { vm } = await createGuest(); - const e1 = await vm.evalCode('await agent(42, "task").then(() => "no", (err) => err.message)'); - assert.match(value(e1), /model spec string/); - const e2 = await vm.evalCode('await agent("pi/x", 42).then(() => "no", (err) => err.message)'); - assert.match(value(e2), /task string/); - vm.dispose(); -}); - -test('agent() rejections normalize to Errors carrying code/recoverable', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push({ rejectWith: { message: 'cap hit', code: 'X', recoverable: false } }); - const e = await vm.evalCode('await agent("pi/x", "p").then(() => "no", (err) => ({ name: err.name, message: err.message, code: err.code, recoverable: err.recoverable }))'); - assert.deepEqual(value(e), { name: 'Error', message: 'cap hit', code: 'X', recoverable: false }); - vm.dispose(); -}); - -test('a host handler that throws synchronously rejects the call (documented refusal path)', async () => { - const { vm } = await createGuest(); - const bridge = mockBridge(); - // Replace the agent handler with a throwing one — the shim turns the - // throw into a guest error, which issueCall converts into a rejection. - // Re-installing callbacks over the live workspace keeps the library. - bridge.handlers.agent = () => { - throw new Error('refused at dispatch'); - }; - await installGuestBridge(vm, bridge.handlers); // no-op (already installed) — so register directly - registerGuestHostCallbacks(vm, bridge.handlers); - const e = await vm.evalCode('await agent("pi/x", "p").then(() => "no", (err) => err.message)'); - assert.equal(value(e), 'refused at dispatch'); - vm.dispose(); -}); - -test('agent and queued-turn handles expose exactly their distinct queue/steer/cancel surfaces', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push( - { resolveWith: 'result' }, - { resolveWith: 'queued answer' }, - { resolveWith: 'injected' }, - { resolveWith: 'idle' }, - { resolveWith: 'cancelled' }, - ); - const out = value( - await vm.evalCode(` - const h = agent("pi/x", "p"); - const q = h.queue("more", { promptMeta: { label: "x" } }); - const queueIdBeforeSettlement = q.id; - const agentNames = Object.getOwnPropertyNames(h).sort(); - const queueNames = Object.getOwnPropertyNames(q).sort(); - const queueShape = { - id: q.id, - queue: typeof q.queue, - steer: typeof q.steer, - followUp: typeof q.followUp, - cancel: typeof q.cancel, - }; - const [queued, steered, queueCancelled, sessionCancelled] = [ - await q, - await h.steer("urgent"), - await q.cancel(), - await h.cancel(), - ]; - const result = await h; - ({ - agentIsPromise: h instanceof Promise, - queueIsPromise: q instanceof Promise, - result, - queued, - steered, - queueCancelled, - sessionCancelled, - agentId: h.id, - queueIdBeforeSettlement, - agentNames, - queueNames, - queueShape, - agentFollowUp: typeof h.followUp, - enumerableAgentKeys: Object.keys(h), - enumerableQueueKeys: Object.keys(q), - }) - `), - ); - assert.deepEqual(out, { - agentIsPromise: true, - queueIsPromise: true, - result: 'result', - queued: 'queued answer', - steered: 'injected', - queueCancelled: 'idle', - sessionCancelled: 'cancelled', - agentId: 'c1', - queueIdBeforeSettlement: 'c2', - agentNames: ['cancel', 'id', 'queue', 'steer'], - queueNames: ['cancel', 'id'], - queueShape: { - id: 'c2', - queue: 'undefined', - steer: 'undefined', - followUp: 'undefined', - cancel: 'function', - }, - agentFollowUp: 'undefined', - enumerableAgentKeys: [], - enumerableQueueKeys: [], - }); - assert.deepEqual(bridge.queueCalls.map(({ callId, sessionId, payloadJson }) => ({ callId, sessionId, payload: JSON.parse(payloadJson!) })), [ - { callId: 'c2', sessionId: 'c1', payload: { prompt: 'more', options: { promptMeta: { label: 'x' } } } }, - ]); - assert.deepEqual(bridge.steerCalls.map(({ callId, sessionId, payloadJson }) => ({ callId, sessionId, payload: JSON.parse(payloadJson!) })), [ - { callId: 'c3', sessionId: 'c1', payload: { prompt: 'urgent' } }, - ]); - assert.deepEqual(bridge.queueCancelCalls.map(({ callId, queueCallId }) => ({ callId, queueCallId })), [ - { callId: 'c4', queueCallId: 'c2' }, - ]); - assert.deepEqual(bridge.sessionCancelCalls.map(({ callId, sessionId }) => ({ callId, sessionId })), [ - { callId: 'c5', sessionId: 'c1' }, - ]); - vm.dispose(); -}); - -test('queue() exposes its minted id synchronously before the queued turn settles', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push({ resolveWith: 'founding answer' }, {}); - const issued = value( - await vm.evalCode(` - const h = agent("pi/x", "founding"); - const q = h.queue("later"); - ({ id: q.id, - kind: globalThis[Symbol.for(${JSON.stringify(GUEST_SURFACE_KEY)})].pending().find((entry) => entry.id === q.id).kind }) - `), - ); - assert.deepEqual(issued, { id: 'c2', kind: 'queue' }); - assert.equal(bridge.queueCalls.length, 1); - bridge.queueCalls[0].call.resolve('queued answer'); - vm.drainJobs(); - assert.equal(value(await vm.evalCode('await q')), 'queued answer'); - vm.dispose(); -}); - -test('missing followUp is an ordinary guest TypeError and does not alias queue', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push({ resolveWith: 'x' }); - const out = value( - await vm.evalCode(` - const h = agent("pi/x", "p"); - const bad = await (async () => { try { return await h.followUp("next"); } catch (err) { return err.name; } })(); - const badCancel = await (async () => { try { h.cancel("nope"); return "no-throw"; } catch (e) { return e.name; } })(); - ({ bad, badCancel }) - `), - ); - assert.deepEqual(out, { bad: 'TypeError', badCancel: 'no-throw' }); - assert.equal(bridge.queueCalls.length, 0, 'missing followUp never dispatches through queue'); - assert.equal(bridge.steerCalls.length, 0, 'missing followUp never dispatches through steer'); - vm.dispose(); -}); - -test('started-not-awaited handles: settlement arrives through a later standalone drain', async () => { - const { vm, bridge } = await createGuest(); - // No scripted resolution: the call parks in the mock. - const first = await vm.evalCode('const research = agent("pi/x", "research Y"); "started"'); - assert.equal(value(first), 'started'); - assert.equal(bridge.agentCalls.length, 1); - // Nothing has settled yet: awaiting the still-pending handle suspends the eval. - const still = await vm.evalCode('await research'); - pending(still); - // The host settles the call, then the drain fires the continuation. - bridge.agentCalls[0].call.resolve({ findings: [1, 2] }); - vm.drainJobs(); - assert.deepEqual(value(await vm.evalCode('await research')), { findings: [1, 2] }); - vm.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// checkpoint() / checkpoint.answer() -// ──────────────────────────────────────────────────────────────────────── - -test('checkpoint question → answer flow across evals', async () => { - const { vm } = await createGuest(); - // Ask a question (the eval suspends on it). - const asked = await vm.evalCode('const q = checkpoint("proceed?"); "asked"'); - assert.equal(value(asked), 'asked'); - assert.equal(vm.drainJobs(), 0); - // The orchestrator delivers the answer in a later eval. - const answered = value( - await vm.evalCode('checkpoint.answer("c1", { yes: true, note: "go" }); "delivered"'), - ); - assert.equal(answered, 'delivered'); - // The checkpoint promise resolved with the answer during that eval's drain. - assert.deepEqual(value(await vm.evalCode('await q')), { yes: true, note: 'go' }); - vm.dispose(); -}); - -test('checkpoint.answer returns false for unknown or already-answered ids', async () => { - const { vm } = await createGuest(); - value(await vm.evalCode('const q = checkpoint("q"); "asked"')); - assert.equal(value(await vm.evalCode('checkpoint.answer("c99", 1)')), false); - assert.equal(value(await vm.evalCode('checkpoint.answer("c1", 1); checkpoint.answer("c1", 2)')), false); - vm.dispose(); -}); - -test('checkpoint.answer with a non-JSON value throws a TypeError (synchronously)', async () => { - const { vm } = await createGuest(); - value(await vm.evalCode('const q = checkpoint("q"); "asked"')); - const e = value(await vm.evalCode(` - (() => { try { checkpoint.answer("c1", (() => { const o = {}; o.self = o; return o; })()); return "no-throw"; } catch (err) { return err.name; } })() - `)); - assert.equal(e, 'TypeError'); - vm.dispose(); -}); - -test('checkpoint options cross the bridge as JSON', async () => { - const { vm, bridge } = await createGuest(); - value(await vm.evalCode('checkpoint("q", { choices: ["a", "b"] }); "asked"')); - const surface = readGuestSurface(vm)!; - const pendingList = surface.pending(); - assert.equal(pendingList.length, 1); - assert.equal(pendingList[0].kind, 'checkpoint'); - assert.equal(pendingList[0].detail, 'q'); - assert.equal(pendingList[0].optionsJson, '{"choices":["a","b"]}'); - vm.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Combinators over a mocked agent() -// ──────────────────────────────────────────────────────────────────────── - -test('parallel: runs thunks concurrently, resolves in input order, recoverable failures become null slots', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push({ resolveWith: 'a' }); - bridge.script.push({ rejectWith: { message: 'boom' } }); // recoverable (no flag) - bridge.script.push({ resolveWith: 'c' }); - const out = value(await vm.evalCode('await parallel([() => agent("pi/x", "a"), () => agent("pi/x", "b"), () => agent("pi/x", "c")])')); - assert.deepEqual(out, ['a', null, 'c']); - // The swallowed failure was reported through console.warn (the bridge). - assert.ok(bridge.events.some((e) => e.level === 'warn' && e.line.includes('parallel[1] failed: boom'))); - vm.dispose(); -}); - -test('parallel: non-recoverable failures (recoverable: false) halt the whole parallel', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push({ rejectWith: { message: 'halt', recoverable: false } }); - const e = await vm.evalCode('await parallel([() => agent("pi/x", "a"), () => agent("pi/x", "b")]).then(() => "no", (err) => err.message)'); - assert.equal(value(e), 'halt'); - vm.dispose(); -}); - -test('parallel: validates its input (functions, not promises)', async () => { - const { vm } = await createGuest(); - assert.equal( - value(await vm.evalCode('await parallel([agent("pi/x", "a")]).then(() => "no", (err) => err.name)')), - 'TypeError', - ); - assert.equal(value(await vm.evalCode('await parallel("nope").then(() => "no", (err) => err.name)')), 'TypeError'); - vm.dispose(); -}); - -test('pipeline: stages run sequentially per item, concurrently across items', async () => { - const { vm } = await createGuest(); - const out = value( - await vm.evalCode(` - await pipeline( - [1, 2, 3], - (prev, original, index) => prev * 10 + original + index, - async (prev) => prev + 1, - ) - `), - ); - // item 1: (1*10+1+0)+1 = 12; item 2: (2*10+2+1)+1 = 24; item 3: (3*10+3+2)+1 = 36 - assert.deepEqual(out, [12, 24, 36]); - vm.dispose(); -}); - -test('pipeline: recoverable per-item failures yield null; non-recoverable halt', async () => { - const { vm } = await createGuest(); - const out = value( - await vm.evalCode(` - await pipeline( - [1, 2, 3], - (prev) => { if (prev === 2) throw { message: 'skip me' }; return prev; }, - ) - `), - ); - assert.deepEqual(out, [1, null, 3]); - const e = await vm.evalCode(` - await pipeline([1], () => { throw { message: 'halt', recoverable: false }; }).then(() => "no", (err) => err.message) - `); - assert.equal(value(e), 'halt'); - vm.dispose(); -}); - -test('retry: bounded attempts, early stop on until(); no until accepts the first result', async () => { - const { vm } = await createGuest(); - // Review regression: without `until` the guest ran EVERY attempt — the - // repository DSL (workflow.ts: `if (!opts.until || opts.until(last)) - // return last`) accepts the FIRST result when no predicate is supplied. - const out = value( - await vm.evalCode(` - let tries = 0; - const last = await retry( - () => { tries++; return tries < 5 ? 'not yet' : 'ok'; }, - { attempts: 3 }, - ); - ({ last, tries }) - `), - ); - assert.deepEqual(out, { last: 'not yet', tries: 1 }); - const out2 = value( - await vm.evalCode(` - let tries2 = 0; - const last2 = await retry( - () => { tries2++; return tries2 === 2 ? 'good' : 'bad'; }, - { attempts: 5, until: (r) => r === 'good' }, - ); - ({ last: last2, tries: tries2 }) - `), - ); - assert.deepEqual(out2, { last: 'good', tries: 2 }); - vm.dispose(); -}); - -test('gate: validator feedback loops into the next attempt; verdict shapes are honored', async () => { - const { vm } = await createGuest(); - const out = value( - await vm.evalCode(` - const history = []; - const result = await gate( - (feedback, attempt) => { history.push({ feedback, attempt }); return 'draft ' + attempt; }, - (value) => value === 'draft 2' ? { ok: true, feedback: 'pass' } : { ok: false, feedback: 'needs work' }, - { attempts: 4 }, - ); - ({ result, history }) - `), - ); - assert.deepEqual(out, { - result: { ok: true, value: 'draft 2', verdict: { ok: true, feedback: 'pass' }, attempts: 3 }, - history: [ - { feedback: undefined, attempt: 0 }, - { feedback: 'needs work', attempt: 1 }, - { feedback: 'needs work', attempt: 2 }, - ], - }); - // Boolean verdicts work too; exhausted attempts report ok: false. - const out2 = value( - await vm.evalCode(` - const result2 = await gate( - () => 'x', - () => false, - { attempts: 2 }, - ); - result2 - `), - ); - assert.deepEqual(out2, { ok: false, value: 'x', verdict: false, attempts: 2 }); - vm.dispose(); -}); - -test('loopUntilDry: dedupes by key, stops after consecutiveEmpty empty rounds, honors maxRounds', async () => { - const { vm } = await createGuest(); - const out = value( - await vm.evalCode(` - let round = 0; - const items = await loopUntilDry({ - round: () => { - round++; - if (round === 1) return [{ id: 1 }, { id: 1 }, { id: 2 }]; - if (round === 2) return [{ id: 3 }]; - return []; - }, - key: (x) => 'k' + x.id, - consecutiveEmpty: 2, - maxRounds: 10, - }); - ({ items, round }) - `), - ); - assert.deepEqual(out, { items: [{ id: 1 }, { id: 2 }, { id: 3 }], round: 4 }); - // maxRounds caps the loop even when never dry. - const out2 = value( - await vm.evalCode(` - let n2 = 0; - const items2 = await loopUntilDry({ - round: () => { n2++; return [{ n: n2 }]; }, - key: (x) => 'n' + x.n, - maxRounds: 3, - }); - ({ count: items2.length, n: n2 }) - `), - ); - assert.deepEqual(out2, { count: 3, n: 3 }); - vm.dispose(); -}); - -test('loopUntilDry: the default key degrades safely for circular items', async () => { - const { vm } = await createGuest(); - const out = value( - await vm.evalCode(` - const circle = { name: 'c' }; circle.self = circle; - const items = await loopUntilDry({ - round: async (r) => r === 0 ? [circle] : [], - consecutiveEmpty: 1, - }); - items.length - `), - ); - assert.equal(out, 1); - vm.dispose(); -}); - -test('verify: reviewers vote; passes when the real-share meets threshold', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push({ resolveWith: { real: true, reason: 'yes' } }); - bridge.script.push({ resolveWith: { real: false, reason: 'no' } }); - bridge.script.push({ resolveWith: { real: true, reason: 'yes2' } }); - const out = value(await vm.evalCode('await verify("claim", { reviewers: 3, threshold: 0.5 })')); - assert.deepEqual(out, { - real: true, - realCount: 2, - total: 3, - votes: [ - { real: true, reason: 'yes' }, - { real: false, reason: 'no' }, - { real: true, reason: 'yes2' }, - ], - }); - // Reviewers were spawned as schema-carrying agent calls, all routed - // through the HOST's configured default backend id (served by - // '__host_default_backend' — a real registered segment; the v1 - // reserved 'default' sentinel that bypassed registry validation is - // deleted). The DSL options are exactly { reviewers, threshold, lens } - // — there is no per-call model option (an invented opts.model was - // removed in review; dsl.d.ts's verify lets reviewers inherit the - // run's default model). - assert.equal(bridge.agentCalls.length, 3); - for (const call of bridge.agentCalls) { - const options = JSON.parse(call.optionsJson!); - assert.equal(options.schema.type, 'object'); - assert.ok(call.task.includes('claim')); - } - assert.ok(bridge.agentCalls.every((c) => c.modelSpec === 'claude'), 'reviewers route through the host default backend id'); - vm.dispose(); -}); - -test('verify: recoverably-failed reviewers are dropped from the vote', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push({ resolveWith: { real: true } }); - bridge.script.push({ rejectWith: { message: 'worker died' } }); - bridge.script.push({ resolveWith: { real: true } }); - const out = value(await vm.evalCode('await verify("claim", { reviewers: 3 })')); - assert.equal(out.real, true); - assert.equal(out.total, 2); - vm.dispose(); -}); - -test('judgePanel: highest mean score wins; stable tie-break by index', async () => { - const { vm, bridge } = await createGuest(); - // Candidate 1 scores 0.4/0.6 → 0.5; candidate 2 scores 0.8/0.6 → 0.7; - // candidate 3 ties candidate 1 at 0.5 → index 2 loses the tie to 0. - bridge.script.push( - { resolveWith: { score: 0.4, reason: 'r' } }, - { resolveWith: { score: 0.6, reason: 'r' } }, - { resolveWith: { score: 0.8, reason: 'r' } }, - { resolveWith: { score: 0.6, reason: 'r' } }, - { resolveWith: { score: 0.5, reason: 'r' } }, - { resolveWith: { score: 0.5, reason: 'r' } }, - ); - const out = value( - await vm.evalCode('await judgePanel(["cand-a", "cand-b", "cand-c"], { judges: 2, rubric: "quality" })'), - ); - assert.equal(out.index, 1); - assert.equal(out.attempt, 'cand-b'); - assert.ok(Math.abs(out.score - 0.7) < 1e-9); - assert.equal(out.judgments.length, 2); - // Judge prompts carried the rubric, and the graders all routed through - // the host's configured default backend id (no opts.model in the DSL's - // { judges, rubric } — the reserved 'default' sentinel is deleted). - assert.ok(bridge.agentCalls.some((c) => c.task.includes('quality'))); - assert.ok(bridge.agentCalls.every((c) => c.modelSpec === 'claude'), 'graders route through the host default backend id'); - vm.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// The console bridge and $N freezing -// ──────────────────────────────────────────────────────────────────────── - -test('§4.4: console.log renders ONE joined line per call — args joined with a single space; direct strings print whole', async () => { - const { vm, bridge } = await createGuest(); - value(await vm.evalCode('console.log("a", "b", "c"); "done"')); - assert.equal(bridge.events.length, 1); - assert.equal(bridge.events[0].level, 'log'); - assert.equal(bridge.events[0].line, 'a b c'); - // A directly logged long string prints WHOLE — no upper bound (the - // Python posture). The length EXCEEDS the deleted 49 488-char - // emission budget: reintroducing the cap would clip this string, so - // the assertion is a real over-threshold probe. - const long = 'x'.repeat(60_000); - value(await vm.evalCode(`console.log(${JSON.stringify(long)}); "done"`)); - assert.equal(bridge.events[1].line, long); - vm.dispose(); -}); - -test('§4.4: objects/arrays render to depth 2; deeper levels render as {…}/[…]', async () => { - const { vm, bridge } = await createGuest(); - value( - await vm.evalCode(` - console.log({ a: 1, nested: { b: { c: 2 } }, arr: [1, [2, [3]]] }); - "done" - `), - ); - assert.equal(bridge.events[0].line, '{a: 1, nested: {b: {…}}, arr: [1, […]]}'); - // Depth 2 means levels 0 and 1 expand; the level-2 values collapse. - value(await vm.evalCode('console.log([[1, 2], { x: { y: "deep" } }]); "done"')); - assert.equal(bridge.events[1].line, "[[1, 2], {x: {…}}]"); - vm.dispose(); -}); - -test('§4.4: collections render their first 20 entries per level, then … +N more', async () => { - const { vm, bridge } = await createGuest(); - value( - await vm.evalCode(` - console.log(Array.from({ length: 25 }, (_, i) => i)); - const o = {}; for (let i = 0; i < 23; i++) o['k' + i] = i; - console.log(o); - "done" - `), - ); - assert.equal(bridge.events[0].line, '[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, … +5 more]'); - assert.match(bridge.events[1].line, /^\{k0: 0, k1: 1, /); - assert.ok(bridge.events[1].line.endsWith('… +3 more}')); - vm.dispose(); -}); - -test('§4.4: nested strings (inside a collection) render head-limited at 200 chars, quoted', async () => { - const { vm, bridge } = await createGuest(); - const long = 'y'.repeat(500); - value(await vm.evalCode(`console.log({ long: ${JSON.stringify(long)}, short: 'hi', list: [${JSON.stringify(long)}] }); "done"`)); - const line = bridge.events[0].line; - assert.ok(line.includes(`long: '${'y'.repeat(200)}…'`), line); - assert.ok(line.includes("short: 'hi'"), line); - assert.ok(line.includes(`['${'y'.repeat(200)}…'`), line); - const belowLimitEmoji = '😀'.repeat(150); - value(await vm.evalCode(`console.log({ emoji: ${JSON.stringify(belowLimitEmoji)} }); "done"`)); - assert.equal( - bridge.events[1].line, - `{emoji: '${belowLimitEmoji}'}`, - '150 Unicode characters are below the 200-character bound even though they occupy 300 UTF-16 units', - ); - const aboveLimitEmoji = '😀'.repeat(250); - value(await vm.evalCode(`console.log({ emoji: ${JSON.stringify(aboveLimitEmoji)} }); "done"`)); - assert.equal(bridge.events[2].line, `{emoji: '${'😀'.repeat(200)}…'}`); - vm.dispose(); -}); - -test('§4.4: primitives, brands and hostile values render predictably; console.* NEVER throws', async () => { - const { vm, bridge } = await createGuest(); - value( - await vm.evalCode(` - console.log(undefined, null, true, 42, -0, NaN, Infinity, 123n, Symbol('s')); - console.log(new Date(0), /ab+c/gi, new Map(), new Set(), new WeakMap(), new WeakSet(), new ArrayBuffer(8), new Error('boom')); - console.warn("warned"); console.error("errored"); console.info("infoed"); console.debug("debugged"); - "done" - `), - ); - assert.equal(bridge.events[0].line, 'undefined null true 42 -0 NaN Infinity 123n Symbol'); - assert.equal(bridge.events[1].line, 'Date RegExp Map Set WeakMap WeakSet ArrayBuffer Error: boom'); - assert.deepEqual(bridge.events.slice(2).map((e) => [e.level, e.line]), [ - ['warn', 'warned'], - ['error', 'errored'], - ['info', 'infoed'], - ['debug', 'debugged'], - ]); - // A revoked proxy degrades to a marker — console never throws. - const out = value( - await vm.evalCode(` - const { proxy, revoke } = Proxy.revocable({}, {}); - revoke(); - let threw = false; - try { console.log(proxy, { ok: 1 }); } catch (e) { threw = true; } - threw - `), - ); - assert.equal(out, false); - vm.dispose(); -}); - -test('§4.4: cycles and shared refs collapse to {…}/[…] instead of recursing forever', async () => { - const { vm, bridge } = await createGuest(); - value( - await vm.evalCode(` - const o = { name: 'ring' }; o.self = o; - const a = [1]; a.push(a); - console.log(o, a); - "done" - `), - ); - assert.equal(bridge.events[0].line, "{name: 'ring', self: {…}} [1, […]]"); - vm.dispose(); -}); - -test('console.* and pipeline are immune to Array.prototype.slice / Function.prototype.call pollution (captured intrinsics)', async () => { - // Review regression: console.* gathered its arguments through - // Array.prototype.slice at call time, so replacing that method with a - // throwing function made console.log throw — contradicting the bridge - // contract (console.* NEVER throws). The library captures the - // slice/call intrinsic pair at installation; pipeline() had the same - // exposure for its stage list. - const { vm, bridge } = await createGuest(); - value(await vm.evalCode(` - Array.prototype.slice = () => { throw new Error('sabotaged slice'); }; - Function.prototype.call = () => { throw new Error('sabotaged call'); }; - "polluted" - `)); - // console.log still bridges its one joined line. - value(await vm.evalCode('console.log("a", 42, { k: 1 }); "done"')); - assert.equal(bridge.events.length, 1); - assert.equal(bridge.events[0].line, 'a 42 {k: 1}'); - // pipeline still gathers its stage list under the same pollution. - const out = value(await vm.evalCode('await pipeline([1, 2], (x) => x * 10, (x) => x + 1)')); - assert.deepEqual(out, [11, 21]); - vm.dispose(); -}); - -test('sleep(ms) is a guest helper settled by a host-side timer (the VM itself stays timer-free)', async () => { - const { vm } = await createGuest(); - // The eval suspends on the sleep; the host timer settles it; a later - // drain resumes the continuation with the elapsed wall clock. - const started = await vm.evalCode('await sleep(30); 42'); - assert.equal(started.kind, 'pending'); - await new Promise((resolve) => setTimeout(resolve, 60)); - vm.drainJobs(); - assert.equal(value(await vm.evalCode('typeof sleep')), 'function'); - // A second sleep round-trips through the bridge too. - const again = await vm.evalCode('await sleep(5); "slept"'); - assert.equal(again.kind, 'pending'); - await new Promise((resolve) => setTimeout(resolve, 20)); - vm.drainJobs(); - assert.equal(value(await vm.evalCode('"still alive"')), 'still alive'); - vm.dispose(); -}); - -test('sleep validates its argument synchronously', async () => { - const { vm } = await createGuest(); - assert.equal( - value(await vm.evalCode('await sleep(-1).then(() => "no", (err) => err.name)')), - 'TypeError', - ); - assert.equal( - value(await vm.evalCode('await sleep("x").then(() => "no", (err) => err.name)')), - 'TypeError', - ); - vm.dispose(); -}); - -test('workspace()/agents() round-trip the host JSON into plain sliceable values; reset() returns nothing meaningful', async () => { - const vm = await ReplVm.create(); - const bridge = mockBridge(); - bridge.handlers.workspace = () => JSON.stringify({ bindings: [{ name: 'x', type: 'number', sizeBytes: 8, provenance: 'eval 1', task: null }], inFlight: ['c1'], checkpoints: [{ id: 'c2', question: 'why?' }], diagnostics: { reconcile: null, drainError: null, childrenClosed: false } }); - bridge.handlers.agents = () => JSON.stringify([{ callId: 'c1', modelSpec: 'pi/x', task: 'do it', state: 'running', supportsSteering: true, queuedTurns: 2 }]); - await installGuestBridge(vm, bridge.handlers); - const out = value( - await vm.evalCode(` - const w = workspace(); - const a = agents(); - ({ - binding: w.bindings[0].name, - inFlight: w.inFlight[0], - question: w.checkpoints[0].question, - drained: w.diagnostics.childrenClosed, - agent: a[0].callId, - queuedTurns: a[0].queuedTurns, - slice: a.filter((x) => x.state === 'running').length, - resetReturn: reset(), - }) - `), - ); - assert.deepEqual(out, { - binding: 'x', - inFlight: 'c1', - question: 'why?', - drained: false, - agent: 'c1', - queuedTurns: 2, - slice: 1, - resetReturn: undefined, - }); - vm.dispose(); -}); - -test('rejected registry calls carry replCallId on their Errors (§4.6 attribution)', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push({ rejectWith: { message: 'boom', replBackend: 'pi' } }); - const out = value( - await vm.evalCode('await agent("pi/x", "task").then(() => "no", (err) => ({ id: err.replCallId, backend: err.replBackend, message: err.message }))'), - ); - assert.deepEqual(out, { id: 'c1', backend: 'pi', message: 'boom' }); - vm.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// The reconciliation surface -// ──────────────────────────────────────────────────────────────────────── - -test('surface.pending() lists parked calls oldest-first with verbatim details', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push({}); // park - value(await vm.evalCode('agent("pi/deepseek-v4-flash-max", "first"); "ok"')); - value(await vm.evalCode('agent("codex/gpt-5.6-sol", "second", { mode: "read-only" }); "ok"')); - value(await vm.evalCode('checkpoint("question?"); "ok"')); - const surface = readGuestSurface(vm)!; - assert.equal(surface.version, GUEST_LIBRARY_VERSION); - const pendingList = surface.pending(); - assert.deepEqual( - pendingList.map((e) => ({ id: e.id, kind: e.kind, detail: e.detail, optionsJson: e.optionsJson, sessionId: e.sessionId, modelSpec: e.modelSpec })), - [ - { id: 'c1', kind: 'agent', detail: 'first', optionsJson: null, sessionId: 'c1', modelSpec: 'pi/deepseek-v4-flash-max' }, - { id: 'c2', kind: 'agent', detail: 'second', optionsJson: '{"mode":"read-only"}', sessionId: 'c2', modelSpec: 'codex/gpt-5.6-sol' }, - { id: 'c3', kind: 'checkpoint', detail: 'question?', optionsJson: null, sessionId: 'c3', modelSpec: null }, - ], - ); - vm.dispose(); -}); - -test('surface.settle() settles parked calls (the reconciliation route); first settlement wins', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push({}); // park c1 - value(await vm.evalCode('const p = agent("pi/x", "work"); "ok"')); - const surface = readGuestSurface(vm)!; - assert.equal(surface.settle('c1', 'resolve', { done: true }), true); - vm.drainJobs(); - assert.deepEqual(value(await vm.evalCode('await p')), { done: true }); - // Second settlement of the same id is a no-op; unknown ids are false. - assert.equal(surface.settle('c1', 'resolve', 'again'), false); - assert.equal(surface.settle('c99', 'resolve', 'x'), false); - // Rejections through the surface normalize into Errors guest-side. - bridge.script.push({}); - value(await vm.evalCode('const q = agent("pi/x", "w2"); "ok"')); - assert.equal(surface.settle('c2', 'reject', { message: 'gone', recoverable: true }), true); - vm.drainJobs(); - const msg = value(await vm.evalCode('await q.then(() => "no", (err) => err.message)')); - assert.equal(msg, 'gone'); - vm.dispose(); -}); - -test('surface.stats() reports the counters; settlement empties the registry', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push({}); - bridge.script.push({}); - value(await vm.evalCode('agent("pi/x", "a"); agent("pi/x", "b"); "ok"')); - const surface = readGuestSurface(vm)!; - assert.deepEqual(surface.stats(), { - version: GUEST_LIBRARY_VERSION, - callSeq: 2, - pendingCalls: 2, - }); - surface.settle('c1', 'resolve', 1); - assert.equal(surface.stats().pendingCalls, 1); - vm.dispose(); -}); - -test('the surface survives Map.prototype pollution (captured intrinsics)', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push({}); - value(await vm.evalCode('agent("pi/x", "a"); "ok"')); - value( - await vm.evalCode(` - let traps = 0; - Map.prototype.set = function () { traps++; }; - Map.prototype.forEach = function () { traps++; }; - Object.defineProperty(Map.prototype, 'size', { get() { traps++; return 0; } }); - "polluted" - `), - ); - const surface = readGuestSurface(vm)!; - const pendingList = surface.pending(); - assert.equal(pendingList.length, 1); - assert.equal(pendingList[0].id, 'c1'); - assert.equal(surface.stats().pendingCalls, 1); - assert.equal(surface.settle('c1', 'resolve', 'ok'), true); - assert.equal(value(await vm.evalCode('traps')), 0, 'no polluted Map method ran'); - vm.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Snapshot travel (the evolution discipline: the library travels inside -// snapshots; the host re-registers callbacks by name after restore) -// ──────────────────────────────────────────────────────────────────────── - -test('snapshot/restore: state, pending registry and version marker travel; callbacks re-register by name', async () => { - const { vm, bridge } = await createGuest(); - // Park one agent call and one checkpoint; hold state. - bridge.script.push({}); // park c1 - value(await vm.evalCode('const findings = [1, 2, 3]; const research = agent("pi/deepseek-v4-flash-max", "deep dive"); "ok"')); - value(await vm.evalCode('const q = checkpoint("still there?"); "ok"')); - const snapshot = (getVmShim(vm) as QuickJS).snapshot(); - const surfaceBefore = readGuestSurface(vm)!; - assert.equal(surfaceBefore.pending().length, 2); - vm.dispose(); - - // Restore into a fresh instance; re-register the host callbacks by name. - const restored = await ReplVm.restore(snapshot); - const restoredBridge = mockBridge(); - // The restored workspace's parked calls are settled through the - // reconciliation surface (the live deferreds died with the old instance). - registerGuestHostCallbacks(restored, restoredBridge.handlers); - const surface = readGuestSurface(restored)!; - assert.equal(surface.version, GUEST_LIBRARY_VERSION, 'resident version survives'); - const pendingList = surface.pending(); - assert.deepEqual(pendingList.map((e) => e.kind), ['agent', 'checkpoint']); - assert.equal(pendingList[0].detail, 'deep dive'); - assert.equal(pendingList[0].modelSpec, 'pi/deepseek-v4-flash-max', 'model spec survives for re-issue'); - // State survived. - assert.deepEqual(value(await restored.evalCode('findings')), [1, 2, 3]); - // Settle the parked calls (three-way reconciliation: completed while - // down → settle from the store; here the mock settles both). - assert.equal(surface.settle('c1', 'resolve', { report: 'done' }), true); - assert.equal(surface.settle('c2', 'resolve', 'yes still here'), true); - restored.drainJobs(); - assert.deepEqual(value(await restored.evalCode('await research')), { report: 'done' }); - assert.equal(value(await restored.evalCode('await q')), 'yes still here'); - // The library is NOT re-evaluated on restore (idempotence guard); the - // guest globals still work and new calls mint fresh ids. - assert.equal(value(await restored.evalCode('typeof agent')), 'function'); - restoredBridge.script.push({ resolveWith: 'after' }); - assert.equal(value(await restored.evalCode('await agent("pi/x", "next")')), 'after'); - restored.dispose(); -}); - -test('a VM restored from a snapshot keeps working without the guest library re-injected', async () => { - // Covered by the travel test above; this pins the no-op guard once more - // on the restored workspace: installGuestBridge must not re-evaluate. - const { vm } = await createGuest(); - value(await vm.evalCode('globalThis.counter = 7; "ok"')); - const snapshot = (getVmShim(vm) as QuickJS).snapshot(); - vm.dispose(); - const restored = await ReplVm.restore(snapshot); - const bridge = mockBridge(); - await installGuestBridge(restored, bridge.handlers); // no-op - assert.equal(value(await restored.evalCode('counter + 1')), 8); - restored.dispose(); -}); - -test('a host serves a workspace whose resident library is OLDER than the one it ships (evolution discipline)', async () => { - // The doc's rule: the library carries a version marker and travels - // inside snapshots, and a host must serve a restored workspace whose - // resident library is older than the version it currently injects — the - // resident version stays authoritative (never re-inject over a - // workspace) and the host re-registers its callbacks by name against - // whatever version it finds. Simulate the older host: a fresh VM with - // the library built at version 0.0.1. - const vm = await ReplVm.create(); - const bridge = mockBridge(); - await installGuestLibraryAtVersion(vm, '0.0.1', bridge); - - // The surface reports the RESIDENT version, not the host's shipped one. - const surface = readGuestSurface(vm)!; - assert.equal(surface.version, '0.0.1'); - assert.equal(value(await vm.evalCode(GUEST_VERSION_GLOBAL)), '0.0.1'); - assert.deepEqual(surface.stats(), { - version: '0.0.1', - callSeq: 0, - pendingCalls: 0, - }); - - // The old library's surface is fully usable and host calls work against - // it (the host-callback surface is backward compatible): agent parks, - // console events bridge. - value(await vm.evalCode('agent("pi/x", "work"); "started"')); - value(await vm.evalCode('console.log({ a: 1 }); "done"')); - assert.equal(surface.pending().length, 1); - assert.equal(surface.pending()[0].modelSpec, 'pi/x'); - assert.equal(bridge.events.length, 1); - assert.deepEqual(bridge.events[0].line, '{a: 1}'); - - // The current host's install path over the old library is a no-op: the - // resident (older) copy stays authoritative. - await installGuestBridge(vm, mockBridge().handlers); - assert.equal(readGuestSurface(vm)!.version, '0.0.1', 'the resident version stays authoritative'); - vm.dispose(); -}); - -test('GuestLibraryInstallError carries trap-free info (the install-failure surface)', async () => { - // The shipped library source is static and covered by every install in - // this suite; the error class is the surface an install failure would - // surface through. Constructible with EvalErrorInfo, exactly like the - // eval-failure report. - const err = new GuestLibraryInstallError({ - name: 'SyntaxError', - message: 'boom', - interrupted: false, - outOfMemory: false, - }); - assert.equal(err.name, 'GuestLibraryInstallError'); - assert.match(err.message, /SyntaxError: boom/); - assert.equal(err.info.outOfMemory, false); -}); - -test('surface.settle validates its outcome argument (host-side pre-validation avoids the throw path)', async () => { - const { vm } = await createGuest(); - const surface = readGuestSurface(vm)!; - // Host-side validation happens before the guest call: an invalid outcome - // is a TypeError from the facade (the guest's own TypeError is the same). - assert.throws(() => surface.settle('c1', 'bogus' as 'resolve', 1), TypeError); - vm.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Steering reconciliation (a pending steer remains correlated for durable -// interruption on restore; the host must never replay its transient payload) -// ──────────────────────────────────────────────────────────────────────── - -test('a pending steer is snapshot-reconcilable: the registry entry records both ids and settle works by registry id', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push({}); // park the founding agent call (c1) - bridge.script.push({}); // park the steer (c2) - value(await vm.evalCode('const h = agent("pi/x", "work"); "ok"')); - value(await vm.evalCode('const steered = h.steer("go faster"); "ok"')); - - // The live channel: the host saw the operation's own id AND the session. - assert.equal(bridge.steerCalls.length, 1); - assert.equal(bridge.steerCalls[0].callId, 'c2'); - assert.equal(bridge.steerCalls[0].sessionId, 'c1'); - - // The manifest: the pending entry omits nothing the host needs to - // correlate, settle, or re-issue the steer after a restore. - const surface = readGuestSurface(vm)!; - const pendingList = surface.pending(); - assert.deepEqual( - pendingList.map((e) => ({ id: e.id, kind: e.kind, detail: e.detail, sessionId: e.sessionId, modelSpec: e.modelSpec })), - [ - { id: 'c1', kind: 'agent', detail: 'work', sessionId: 'c1', modelSpec: 'pi/x' }, - { id: 'c2', kind: 'steer', detail: 'go faster', sessionId: 'c1', modelSpec: null }, - ], - ); - // The steer's optionsJson carries the verbatim payload (re-issue needs it). - assert.deepEqual(JSON.parse(pendingList[1].optionsJson!), { prompt: 'go faster' }); - - // Settlement through the reconciliation route works by the registry id. - assert.equal(surface.settle('c2', 'resolve', 'injected'), true); - vm.drainJobs(); - assert.equal(value(await vm.evalCode('await steered')), 'injected'); - // The founding call is untouched by the steer settlement. - assert.equal(surface.settle('c1', 'resolve', 'done'), true); - vm.drainJobs(); - assert.equal(value(await vm.evalCode('await h')), 'done'); - vm.dispose(); -}); - -test('a pending steer survives snapshot/restore with correlation for durable refusal, not replay', async () => { - const { vm, bridge } = await createGuest(); - bridge.script.push({}); // park c1 (founding agent) - bridge.script.push({}); // park c2 (steer) - value(await vm.evalCode('const h = agent("pi/x", "work"); "ok"')); - value(await vm.evalCode('const steered = h.steer("go faster"); "ok"')); - const snapshot = (getVmShim(vm) as QuickJS).snapshot(); - vm.dispose(); - - const restored = await ReplVm.restore(snapshot); - const restoredBridge = mockBridge(); - registerGuestHostCallbacks(restored, restoredBridge.handlers); - const surface = readGuestSurface(restored)!; - const pendingList = surface.pending(); - assert.equal(pendingList.length, 2); - // The steer entry names its session so restore can refuse it durably - // without replaying the transient steering payload. - const steerEntry = pendingList.find((e) => e.kind === 'steer')!; - assert.equal(steerEntry.id, 'c2'); - assert.equal(steerEntry.sessionId, 'c1'); - assert.equal(steerEntry.detail, 'go faster'); - // Reconcile rejects by the registry id; registering callbacks above - // emitted no new steering request. - assert.equal(restoredBridge.steerCalls.length, 0); - assert.equal(surface.settle('c2', 'reject', { - message: 'steering was interrupted by restart', - code: 'AGENT_EXECUTION_ERROR', - recoverable: true, - details: { reason: 'steering_interrupted' }, - }), true); - assert.equal(surface.settle('c1', 'resolve', 'done'), true); - restored.drainJobs(); - assert.deepEqual( - value(await restored.evalCode('await steered.then(() => null, (err) => ({ code: err.code, reason: err.details.reason }))')), - { code: 'AGENT_EXECUTION_ERROR', reason: 'steering_interrupted' }, - ); - assert.equal(value(await restored.evalCode('await h')), 'done'); - restored.dispose(); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Handle hygiene (the review discipline: a long-lived VM must not -// accumulate guest memory from settled host calls) -// ──────────────────────────────────────────────────────────────────────── - -test('round 7: __replAwaitIterable over a SYNC iterable preserves AsyncFromSyncIterator value unwrapping — `for await (const x of [Promise.resolve(1), 2])` yields the RESOLVED values `[1, 2]`, never the promise objects (the reviewer\'s repro: the result wrapper resolved with the RAW iterator result, and because the wrapper is an ASYNC iterable the machinery used the value as-is — the promise object leaked through)', async () => { - const { vm } = await createGuest(); - try { - // Each eval body is BLOCK-scoped: the realm is shared across evals, - // and a top-level `const` would redeclare on the next eval. The - // top-level for-await keeps the script's completion a promise the - // engine awaits (like `await agent(...)` in the round-trip test). - const out = value(await vm.evalCode(`{ - const got = []; - for await (const x of __replAwaitIterable([Promise.resolve(1), 2], 't1')) got.push(x); - JSON.stringify(got); - }`)); - assert.equal(out, '[1,2]', 'sync-iterable values are awaited and unwrapped'); - // The unwrap is faithful even when the value settles later (a - // thenable): the result promise waits for the value's settlement. - const later = value(await vm.evalCode(`{ - const got = []; - let n = 0; - const iter = __replAwaitIterable({ - [Symbol.iterator]: () => ({ - next: () => (n++ === 0 ? { value: Promise.resolve(7), done: false } : { done: true }), - return: () => ({ value: undefined, done: true }), - }), - }, 't1'); - for await (const x of iter) got.push(x); - JSON.stringify(got); - }`)); - assert.equal(later, '[7]'); - } finally { - vm.dispose(); - } -}); - -test('round 7: __replAwaitIterable ACQUISITION failures propagate exactly once — an observable/throwing `Symbol.asyncIterator` getter runs a SINGLE time and the loop reports its ORIGINAL error (the reviewer\'s repro: the old degrade-to-unwrapped made the for-await machinery acquire the iterable a second time, so the getter ran twice and could report `boom2` instead of native `boom1`)', async () => { - const { vm } = await createGuest(); - try { - const out = value(await vm.evalCode(`{ - let n = 0; - const obj = { - get [Symbol.asyncIterator]() { n++; if (n === 1) throw new Error('boom1'); throw new Error('boom2'); }, - }; - let err = null; - try { for await (const x of __replAwaitIterable(obj, 't1')) {} } - catch (e) { err = e.message; } - JSON.stringify([err, n]); - }`)); - assert.equal(out, JSON.stringify(['boom1', 1]), 'the acquisition error propagates, the getter runs once'); - // A present-but-not-callable @@asyncIterator is a TypeError (GetMethod - // semantics), never a silent fallback to @@iterator. - const nonCallable = value(await vm.evalCode(`{ - const obj = { [Symbol.asyncIterator]: 42, [Symbol.iterator]: () => ({ next: () => ({ done: true }) }) }; - let err = null; - try { for await (const x of __replAwaitIterable(obj, 't1')) {} } - catch (e) { err = e.name + ':' + e.message; } - err; - }`)); - assert.ok(String(nonCallable).startsWith('TypeError:'), 'non-callable @@asyncIterator is a TypeError'); - } finally { - vm.dispose(); - } -}); - -test('round 7: the await/iterable instrumentation runs on CAPTURED pristine Promise intrinsics — replacing `Promise.prototype.then`, overwriting `Promise.resolve`, or shadowing `Promise` lexically cannot change its semantics (the reviewer\'s repro: replacing `Promise.prototype.then` made the instrumented `await 40` return `99`; the native evaluation returned `40`) and the continuation lease is still set', async () => { - // Each sabotage case runs in its OWN VM: the mutations are - // irreversible guest-side (the originals survive only in the - // library's captured intrinsics), so one realm cannot host two cases. - // Replaced prototype: the guest-visible then is gone, the - // instrumentation still mirrors natively and sets the lease. - { - const { vm } = await createGuest(); - try { - const mutated = value(await vm.evalCode(`{ - Promise.prototype.then = function () { return 99; }; - const out = await __replAwait(Promise.resolve(40), 't1'); - JSON.stringify([out, __replLease]); - }`)); - assert.equal(mutated, JSON.stringify([40, 't1']), 'replaced Promise.prototype.then cannot change the mirror or skip the lease'); - } finally { - vm.dispose(); - } - } - // Overwritten static: the value is minted BEFORE the sabotage (the - // guest's own later `Promise.resolve` call is guest semantics — the - // instrumentation's INTERNAL adoption must keep using the captured - // original), the mirror still resolves natively and the lease is - // still set. - { - const { vm } = await createGuest(); - try { - const overwritten = value(await vm.evalCode(`{ - const p = Promise.resolve(40); - Promise.resolve = function () { return 99; }; - const out = await __replAwait(p, 't1'); - JSON.stringify([out, __replLease]); - }`)); - assert.equal(overwritten, JSON.stringify([40, 't1']), 'overwritten Promise.resolve cannot change the mirror or skip the lease'); - } finally { - vm.dispose(); - } - } - // Shadowed: a block-level lexical Promise (a legitimate user - // program) — the mirror still works and the lease is still set. - { - const { vm } = await createGuest(); - try { - const shadowed = value(await vm.evalCode(`{ - let out, lease; - { - const Promise = { resolve: (v) => v }; - out = await __replAwait(Promise.resolve(40), 't1'); - lease = __replLease; - } - JSON.stringify([out, lease]); - }`)); - assert.equal(shadowed, JSON.stringify([40, 't1']), 'a lexical Promise shadow cannot change the mirror or skip the lease'); - } finally { - vm.dispose(); - } - } - // The iterable wrap is equally isolated: a replaced prototype must - // not break a for-await over a sync iterable. - { - const { vm } = await createGuest(); - try { - const iterated = value(await vm.evalCode(`{ - const p = Promise.resolve(1); - Promise.prototype.then = function () { return 99; }; - const got = []; - for await (const x of __replAwaitIterable([p], 't1')) got.push(x); - JSON.stringify(got); - }`)); - assert.equal(iterated, '[1]', 'the iterable wrap mirrors natively under a replaced prototype'); - } finally { - vm.dispose(); - } - } -}); - -test('round 7: __replAwaitIterable over an ASYNC iterable passes result objects through untouched (its value is used as-is — native async iteration semantics; the promise VALUES of async generators are NOT awaited by for-await)', async () => { - const { vm } = await createGuest(); - try { - const out = value(await vm.evalCode(` - const iterable = { - [Symbol.asyncIterator]: () => { - let n = 0; - return { - next: () => Promise.resolve({ value: ++n, done: n > 2 }), - }; - }, - }; - const got = []; - for await (const x of __replAwaitIterable(iterable, 't1')) got.push(x); - JSON.stringify(got); - `)); - assert.equal(out, '[1,2]', 'async-iterator result objects pass through untouched'); - } finally { - vm.dispose(); - } -}); - -test('5,000 sequential resolved agent calls leave a 2 MiB VM healthy (no handle leak)', async () => { - // Review regression: GuestCall never disposed its deferred promise - // handle or the handles returned by marshalValue, so every settled call - // pinned its promise and the marshalled value — a 2 MiB VM failed after - // roughly 5,000 sequential resolved agent calls. The bridge now releases - // both (the promise handle once the trampoline has dupped it, the value - // handle right after settlement), so memory stays flat. - const vm = await ReplVm.create({ memoryLimit: 2 * 1024 * 1024 }); - const bridge = mockBridge(); - await installGuestBridge(vm, bridge.handlers); - for (let i = 0; i < 5000; i++) { - bridge.script.push({ resolveWith: { i } }); - const out = await vm.evalCode(`await agent("pi/x", "task ${i}")`); - assert.equal(out.kind, 'value'); - if (out.kind === 'value') assert.equal((out.value as { i: number }).i, i); - } - // The VM is fully healthy afterwards — fresh work still completes. - bridge.script.push({ resolveWith: 'after' }); - assert.equal(value(await vm.evalCode('await agent("pi/x", "after")')), 'after'); - assert.equal(value(await vm.evalCode('1 + 1')), 2); - vm.dispose(); -}); - -test('unsettled parked calls do not leak either (promise handles are released after return)', async () => { - const vm = await ReplVm.create({ memoryLimit: 12 * 1024 * 1024 }); - const bridge = mockBridge(); - await installGuestBridge(vm, bridge.handlers); - // 5,000 parked calls (never settled): each returned promise handle must - // be released once the guest holds its own reference. - // - // Memory-limit note (review round): parked registry entries are LIVE for - // the VM's lifetime (deleted only on settlement), so 5,000 parked calls - // have an honest footprint of ~2.09 MB — 99.9% of a 2 MiB limit, a - // knife-edge where any library-source evolution (even comment growth) - // tipped the GC/malloc interplay into a hard failure at ~725 calls. The - // limit is 12 MiB here: the honest footprint (library source + live - // registry entries) grows with the 0.5.0 four-callback queue/control - // surface. This remains a deterministic high-volume health check; - // the separate 30,000-refusal test below is the tighter leak detector. - for (let i = 0; i < 5000; i++) { - const out = await vm.evalCode(`agent("pi/x", "task ${i}"); "started"`); - assert.equal(out.kind, 'value'); - } - assert.equal(value(await vm.evalCode('1 + 1')), 2); - vm.dispose(); -}); - -test('30,000 synchronous host refusals leave a 2 MiB VM healthy (throwing handlers dispose every deferred part)', async () => { - // Review regression: when a handler threw, the shim converted the throw - // into a guest error — but the GuestCall's raw promise and both - // resolving functions were never disposed (`releaseToRealm` was - // bypassed on the throwing path), so every refusal leaked ~490 bytes of - // guest memory (promise + 2 resolvers + heap boxes). After 30,000 - // rejected calls the 2 MiB VM was saturated and the next NORMAL agent - // call failed with `Error: null`. Every throwing-handler path (agent, - // queue, steer, both cancellation callbacks, and checkpoint question - // mode) now disposes all owned parts before re-throwing. - // - // The regression is pinned TWO ways, both deterministic: the guest - // runtime's own memory usage after the refusals (qjs_compute_memory_ - // usage — ~190–250 KB on the fixed code at any refusal volume; 1.6 MB - // at 3,000 refusals and the 2 MiB cap at 9,000+ on the broken code), - // and the behavioral probe (a normal agent call still completes). - const vm = await ReplVm.create({ memoryLimit: 2 * 1024 * 1024 }); - const bridge = mockBridge(); - bridge.handlers.agent = () => { - throw new Error('refused at dispatch'); - }; - bridge.handlers.queue = () => { - throw new Error('refused at dispatch'); - }; - bridge.handlers.steer = () => { - throw new Error('refused at dispatch'); - }; - bridge.handlers.cancelSession = () => { - throw new Error('refused at dispatch'); - }; - bridge.handlers.cancelQueue = () => { - throw new Error('refused at dispatch'); - }; - bridge.handlers.checkpoint = () => { - throw new Error('refused at dispatch'); - }; - await installGuestBridge(vm, bridge.handlers); - // 30,000 refused calls across all six host-callback kinds (50 evals - // × 100 iterations × 6 calls), every promise rejection handled in-eval. - for (let i = 0; i < 50; i++) { - const out = await vm.evalCode(` - (async () => { - for (let k = 0; k < 100; k++) { - const h = agent("pi/x", "t" + k); - const queued = h.queue("later"); - const s = h.steer("go"); - const sc = h.cancel(); - const qc = queued.cancel(); - const q = checkpoint("q?"); - await Promise.all([h, queued, s, sc, qc, q].map((p) => p.then(() => "no", (e) => e.message))); - } - })() - `); - assert.equal(out.kind, 'value'); - } - // The guest runtime's memory stayed flat: far below the 2 MiB limit - // (the broken code saturates the cap — qjs_memory_usage() reads - // 2,092,244 there; the fixed code holds ~190–250 KB). - const usageBytes = vmMemoryUsage(vm); - assert.ok( - usageBytes < 1024 * 1024, - `guest memory after 30,000 refusals must stay below 1 MiB, got ${usageBytes} bytes`, - ); - // A NORMAL agent call still works after the mass refusals (the review's - // exact failure mode) — swap in working handlers via the - // re-registration path, and the VM is fully healthy. - const working = mockBridge(); - working.script.push({ resolveWith: 'after' }); - registerGuestHostCallbacks(vm, working.handlers); - assert.equal(value(await vm.evalCode('await agent("pi/x", "after")')), 'after'); - assert.equal(value(await vm.evalCode('1 + 1')), 2); - vm.dispose(); -}); - -/** - * Guest runtime memory usage in bytes (mallocSize from the runtime's - * memory-usage statistics). Read through the shim's getMemoryUsage(), - * which allocates the COMPLETE 26-int64 (208-byte) JSMemoryUsage - * structure before qjs_compute_memory_usage writes into it, reads every - * field back, and frees it — a raw 4-byte buffer would let the C write - * 208 bytes past its end, corrupting adjacent WASM memory and - * invalidating the very measurement it feeds (review regression: the - * corruption made the deterministic refusal-memory assertion read - * garbage). - */ -function vmMemoryUsage(vm: ReplVm): number { - return (getVmShim(vm) as QuickJS).getMemoryUsage().mallocSize; -} diff --git a/packages/repl-engine/test/preview.test.ts b/packages/repl-engine/test/preview.test.ts deleted file mode 100644 index 16c42c66..00000000 --- a/packages/repl-engine/test/preview.test.ts +++ /dev/null @@ -1,371 +0,0 @@ -/** - * Previewer tests for the eval-plane redesign: the §4.4 depth-limited - * completion repr (renderCompletionLine — direct strings whole, objects/ - * arrays to depth 2, 20 entries per level, nested strings head-limited at - * 200 chars, NO byte ceiling), the RETAINED metadata-formatting token - * rules (§7: stringDescription/shortString/headTailDescription, - * formatNumber/formatByteSize, the manifest seam), and the trap-freedom - * discipline around the completion rendering (never execute guest - * getters — the R69 rule). - * - * The old `$N`-line previewer surface (renderRefLine/renderGlobalLine/ - * renderCollapsed over `$N` slots) and the EMISSION_STRING_MAX_CHARS - * emission budget are DELETED features — their tests are deleted with - * them. - */ - -import assert from 'node:assert/strict'; -import { test } from 'node:test'; - -import { - ReplVm, - Workspace, - escapeString, - formatByteSize, - formatNumber, - headTailDescription, - installGuestBridge, - inspectGlobal, - isCanonicalIndex, - manifestBinding, - shortString, - stringDescription, - MAX_COLLAPSED_CHARS, - REPR_MAX_DEPTH, - REPR_MAX_ENTRIES, - REPR_NESTED_STRING_CHARS, -} from '../src/index.js'; -import { renderCompletionLine } from '../src/preview.js'; -import { getVmShim } from '../src/vm.js'; -import type { JSValueHandle, QuickJS } from 'quickjs-wasi'; - -// ──────────────────────────────────────────────────────────────────────── -// Host-side repr probes: eval a source expression, read the LIVE -// completion handle (exactly what the broker's render does), and run the -// completion repr over it. -// ──────────────────────────────────────────────────────────────────────── - -/** Eval a script in a bare VM, returning the live completion handle for a - * RESOLVED eval (owned by the caller). */ -function describeOutcome(outcome: unknown): string { - try { - return JSON.stringify(outcome); - } catch { - return String(outcome); - } -} - -function evalCompletion(vm: ReplVm, source: string): JSValueHandle { - const { outcome, completion } = vm.evalCodeWithCompletion(source); - assert.equal(outcome.kind, 'value', `expected value, got ${describeOutcome(outcome)}`); - assert.ok(completion !== undefined); - return completion as JSValueHandle; -} - -function reprOf(vm: ReplVm, source: string): string { - const handle = evalCompletion(vm, source); - try { - return renderCompletionLine(handle); - } finally { - handle.dispose(); - } -} - -async function createVm(): Promise { - const vm = await ReplVm.create(); - await installGuestBridge(vm, { - agent: () => undefined, - checkpoint: () => undefined, - queue: () => undefined, - steer: () => undefined, - cancelSession: () => undefined, - cancelQueue: () => undefined, - console: () => undefined, - sleep: () => undefined, - workspace: () => '{}', - agents: () => '[]', - reset: () => undefined, - defaultBackend: () => undefined, - }); - return vm; -} - -// ──────────────────────────────────────────────────────────────────────── -// The §4.4 completion repr (the rules, whole direct strings included) -// ──────────────────────────────────────────────────────────────────────── - -test('§4.4 completion repr: a string completion value prints WHOLE with no upper bound', async () => { - using vm = await createVm(); - assert.equal(reprOf(vm, '"hi"'), 'hi'); - const short = 'a'.repeat(200); - assert.equal(reprOf(vm, JSON.stringify(short)), short); - // NO byte ceiling: the Python posture — the whole string is the result. - const long = 'x'.repeat(50_000); - assert.equal(reprOf(vm, JSON.stringify(long)), long); - const multiline = 'line1\nline2\n' + 'y'.repeat(5000); - assert.equal(reprOf(vm, JSON.stringify(multiline)), multiline); -}); - -test('§4.4 completion repr: primitives', async () => { - using vm = await createVm(); - assert.equal(reprOf(vm, 'undefined'), 'undefined'); - assert.equal(reprOf(vm, 'null'), 'null'); - assert.equal(reprOf(vm, 'true'), 'true'); - assert.equal(reprOf(vm, 'false'), 'false'); - assert.equal(reprOf(vm, '42'), '42'); - assert.equal(reprOf(vm, '-0'), '-0'); - assert.equal(reprOf(vm, 'NaN'), 'NaN'); - assert.equal(reprOf(vm, 'Infinity'), 'Infinity'); - assert.equal(reprOf(vm, '123n'), '123n'); -}); - -test('§4.4 completion repr: objects/arrays to depth 2; deeper levels render as {…}/[…]', async () => { - using vm = await createVm(); - assert.equal(reprOf(vm, '({ a: 1, b: [1, 2] })'), '{a: 1, b: [1, 2]}'); - assert.equal(reprOf(vm, '({ a: { b: { c: 2 } } })'), '{a: {b: {…}}}'); - assert.equal(reprOf(vm, '[[1, 2], [3, [4]]]'), '[[1, 2], [3, […]]]'); - // Level-0 entries expand; level-2 values collapse. - assert.equal(reprOf(vm, '({ deep: { deeper: { deepest: [1] } }, arr: [0, { x: { y: 1 } }] })'), '{deep: {deeper: {…}}, arr: [0, {…}]}'); -}); - -test('§4.4 completion repr: 20 entries per level, then … +N more', async () => { - using vm = await createVm(); - const arr = reprOf(vm, 'Array.from({ length: 22 }, (_, i) => i)'); - assert.equal(arr, '[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, … +2 more]'); - const obj = reprOf(vm, '(function () { const o = {}; for (let i = 0; i < 25; i++) o["k" + i] = i; return o; })()'); - assert.ok(obj.endsWith('… +5 more}'), obj); - assert.ok(obj.split(', ').length === 21, obj); -}); - -test('§4.4 completion repr: nested strings head-limited at 200 chars, quoted', async () => { - using vm = await createVm(); - const long = 'y'.repeat(500); - const rendered = reprOf(vm, `({ long: ${JSON.stringify(long)}, short: 'hi', list: [${JSON.stringify(long)}] })`); - assert.ok(rendered.includes(`long: '${'y'.repeat(200)}…'`), rendered); - assert.ok(rendered.includes("short: 'hi'"), rendered); - assert.ok(rendered.includes(`['${'y'.repeat(200)}…'`), rendered); -}); - -test('§4.4 completion repr: functions, symbols, branded objects and errors render as predictable leaves', async () => { - using vm = await createVm(); - assert.equal(reprOf(vm, '(function named() {})'), 'ƒ named()'); - assert.equal(reprOf(vm, 'Symbol()'), 'Symbol'); - assert.equal(reprOf(vm, 'new Date(0)'), 'Date'); - assert.equal(reprOf(vm, '/ab+c/gi'), 'RegExp'); - assert.equal(reprOf(vm, 'new Map()'), 'Map'); - assert.equal(reprOf(vm, 'new Set()'), 'Set'); - assert.equal(reprOf(vm, 'new WeakMap()'), 'WeakMap'); - assert.equal(reprOf(vm, 'new ArrayBuffer(8)'), 'ArrayBuffer(8)'); - assert.equal(reprOf(vm, 'new Uint8Array(4)'), 'Uint8Array'); - assert.equal(reprOf(vm, 'new Error("boom")'), 'Error: boom'); -}); - -test('§4.4 completion repr: cycles and shared refs collapse to {…}/[…]', async () => { - using vm = await createVm(); - const ring = reprOf(vm, '(function () { const o = { name: "ring" }; o.self = o; return o; })()'); - assert.equal(ring, "{name: 'ring', self: {…}}"); - const arr = reprOf(vm, '(function () { const a = [1]; a.push(a); return a; })()'); - assert.equal(arr, '[1, […]]'); -}); - -test('§4.4 completion repr: the repr constants carry the bible numbers', () => { - assert.equal(REPR_MAX_DEPTH, 2); - assert.equal(REPR_MAX_ENTRIES, 20); - assert.equal(REPR_NESTED_STRING_CHARS, 200); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Trap-freedom around the completion rendering (the R69 discipline: the -// renderer never executes guest getters) -// ──────────────────────────────────────────────────────────────────────── - -test('trap-freedom: hostile getters on the completion value never fire while rendering', async () => { - using vm = await createVm(); - const out = reprOf( - vm, - `(function () { - let traps = 0; - globalThis.__traps = () => traps; - return { - get hostile() { traps++; return 'x'; }, - nested: { get also() { traps++; return 'y'; } }, - safe: 'ok', - }; - })()`, - ); - assert.equal(out, "{hostile: (…), nested: {also: (…)}, safe: 'ok'}", 'accessors render as (…) — the getters never fired'); - // The traps counter can be observed through a global counter instead. - const counter = await createVm(); - const rendered = reprOf( - counter, - `(function () { - globalThis.traps = 0; - return { - get hostile() { globalThis.traps++; return 'x'; }, - safe: 'ok', - }; - })()`, - ); - assert.equal(rendered, "{hostile: (…), safe: 'ok'}"); - const read = await counter.evalCode('globalThis.traps'); - assert.equal(read.kind, 'value'); - assert.equal((read as { value: unknown }).value, 0, 'no getter ran'); -}); - -test('trap-freedom: Object.prototype.value pollution cannot hijack the completion repr', async () => { - // Review regression lineage (R69): the engine's completion read takes - // the value own-property-descriptor-wise; a polluted Object.prototype - // getter must never FIRE and its forged value must never leak into a - // rendered result. (The engine's pinned quirk — the completion - // wrapper's [[Set]] silently no-ops under the pollution — may change - // WHICH value renders, but the forged "polluted" string never does.) - using vm = await createVm(); - assert.equal(reprOf(vm, '42'), '42'); - const prep = await vm.evalCode('Object.defineProperty(Object.prototype, "value", { get() { return "polluted"; } }); "done"'); - assert.equal(prep.kind, 'value'); - const rendered = reprOf(vm, '42'); - assert.ok(!rendered.includes('polluted'), `the forged value never leaks (got ${rendered})`); - const obj = reprOf(vm, '({ a: 1 })'); - assert.ok(!obj.includes('polluted'), `the forged value never leaks (got ${obj})`); -}); - -test('trap-freedom: proxy traps never fire while rendering the completion repr', async () => { - using vm = await createVm(); - const out = reprOf( - vm, - `(function () { - globalThis.pTraps = 0; - const p = new Proxy({ a: 1 }, { - get() { globalThis.pTraps++; return 7; }, - ownKeys() { globalThis.pTraps++; return ['a']; }, - }); - return p; - })()`, - ); - assert.equal(out, 'Proxy(Object)'); - const read = await vm.evalCode('globalThis.pTraps'); - assert.equal(read.kind, 'value'); - assert.equal((read as { value: unknown }).value, 0, 'no proxy trap ran'); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Retained metadata formatting (§7: the internal 200-char previews the -// engine still uses — manifest tokens, checkpoint-question previews) -// ──────────────────────────────────────────────────────────────────────── - -test('retained token rules: formatNumber and formatByteSize', () => { - assert.equal(formatNumber(0), '0'); - assert.equal(formatNumber(-0), '-0'); - assert.equal(formatNumber(0.1), '0.1'); - assert.equal(formatNumber(1e21), '1e+21'); - assert.equal(formatNumber(1e-7), '1e-7'); - assert.equal(formatNumber(123.456), '123.456'); - assert.equal(formatNumber(NaN), 'NaN'); - assert.equal(formatNumber(Infinity), 'Infinity'); - assert.equal(formatNumber(-Infinity), '-Infinity'); - assert.equal(formatNumber(1 / 3), String(1 / 3)); - assert.equal(formatByteSize(3), '3B'); - assert.equal(formatByteSize(999), '999B'); - assert.equal(formatByteSize(1000), '1kB'); - assert.equal(formatByteSize(48000), '48kB'); - assert.equal(formatByteSize(999_999), '1MB'); -}); - -test('retained token rules: stringDescription/shortString/headTailDescription/escapeString/isCanonicalIndex', () => { - assert.equal(stringDescription('ok'), '"ok"'); - assert.equal(stringDescription('x'.repeat(200)).length, 202); - const longDesc = stringDescription('y'.repeat(201)); - assert.ok(longDesc.startsWith(`"${'y'.repeat(120)}"`)); - assert.ok(longDesc.endsWith(`"${'y'.repeat(40)}"`)); - assert.ok(longDesc.includes('[41 chars elided]')); - assert.equal(shortString('v'.repeat(40)), `"${'v'.repeat(40)}"`); - const longShort = shortString('z'.repeat(41)); - assert.equal(longShort, `"${'z'.repeat(24)}…[9 chars elided]…${'z'.repeat(8)}"`); - assert.equal(headTailDescription('short', 120), 'short'); - const elided = headTailDescription('e'.repeat(200), 120); - assert.ok(elided.startsWith('e'.repeat(72))); - assert.ok(elided.endsWith('e'.repeat(24))); - assert.ok(elided.includes('[104 chars elided]')); - assert.equal(escapeString('a"b\\c\nd\te\rf\u0001g'), 'a\\"b\\\\c\\nd\\te\\rf\\u0001g'); - assert.equal(isCanonicalIndex('0'), true); - assert.equal(isCanonicalIndex('01'), false); - assert.equal(isCanonicalIndex('-1'), false); - assert.equal(isCanonicalIndex('4294967295'), false); -}); - -test('retained seam: inspectGlobal and manifestBinding still serve the workspace manifest (metadata, never content)', async () => { - using vm = await createVm(); - const prep = await vm.evalCode('var userValue = { a: 1 }; "done"'); - assert.equal(prep.kind, 'value'); - const meta = inspectGlobal(vm, 'userValue'); - assert.equal(meta.kind, 'data'); - assert.ok(meta.label.length > 0); - assert.ok(meta.sizeBytes > 0); - const binding = manifestBinding(vm, 'userValue'); - assert.ok(binding !== null); - assert.match(binding.token, /B/); - assert.equal(binding.handleCallId, null); - // The manifest seam stays trap-free: an accessor binding reads as - // absent/sabotage, never fired. - await vm.evalCode('Object.defineProperty(globalThis, "getterThing", { get() { return 1; } }); "ok"'); - assert.equal(inspectGlobal(vm, 'getterThing').kind, 'accessor'); -}); - -test('MAX_COLLAPSED_CHARS is retained (the collapsed-preview backstop constant)', () => { - assert.equal(typeof MAX_COLLAPSED_CHARS, 'number'); - assert.ok(MAX_COLLAPSED_CHARS > 0); -}); - -// ──────────────────────────────────────────────────────────────────────── -// The workspace manifest remains functional (the retained metadata seam -// the workspace()/status surface builds on) -// ──────────────────────────────────────────────────────────────────────── - -test('the workspace manifest lists user bindings; the deleted $N refs never appear', async () => { - const ws = await Workspace.create('/tmp/repl-preview-project'); - try { - await ws.eval('const findings = [1, 2, 3]; "done"'); - const manifest = ws.manifest(); - const names = manifest.bindings.map((b) => b.name); - assert.ok(names.includes('findings')); - assert.ok(!names.some((n) => /^\$\d+$/.test(n)), 'no $N refs in the manifest'); - assert.deepEqual(manifest.logs, { first: null, last: null, count: 0 }); - } finally { - ws.dispose(); - } -}); - -// The trap-free read path is exercised through the workspace eval below -// (a completion repr over a live handle from a real workspace). -test('the completion repr round-trips through a real workspace eval', async () => { - const ws = await Workspace.create('/tmp/repl-preview-project-2'); - try { - const { outcome, completion } = ws.evalWithCompletion('({ answer: 42, nested: { deep: true } })'); - assert.equal(outcome.kind, 'value'); - const handle = completion as JSValueHandle; - try { - assert.equal(renderCompletionLine(handle), '{answer: 42, nested: {deep: true}}'); - } finally { - handle.dispose(); - } - } finally { - ws.dispose(); - } -}); - -// The raw shim import is kept referenced for the trap-free handle probes. -void getVmShim; -void (undefined as unknown as QuickJS); - -test('§7: the $N capture previewer surface is DELETED from the public exports (renderRefLine / renderGlobalLine / renderPreviewLine / previewGlobal)', async () => { - const index = await import('../src/index.js'); - assert.equal('renderRefLine' in index, false, 'renderRefLine must be deleted'); - assert.equal('renderGlobalLine' in index, false, 'renderGlobalLine must be deleted'); - assert.equal('renderPreviewLine' in index, false, 'renderPreviewLine must be deleted'); - assert.equal('previewGlobal' in index, false, 'previewGlobal must be deleted'); - assert.equal('Workspace' in index, true, 'the retained seams stay exported'); - // The retained metadata-formatting tokens and the manifest seam stay. - assert.equal('inspectGlobal' in index, true); - assert.equal('stringDescription' in index, true); - assert.equal('renderCollapsed' in index, true); -}); diff --git a/packages/repl-engine/test/public-types.test.ts b/packages/repl-engine/test/public-types.test.ts deleted file mode 100644 index b8804db6..00000000 --- a/packages/repl-engine/test/public-types.test.ts +++ /dev/null @@ -1,45 +0,0 @@ -/** - * Published-type-graph check: the `dist` declarations must be usable by a - * consumer with the repository's non-DOM lib and `skipLibCheck: false`, - * with no ambient `@types` and no unpublished source declarations. - * - * Regression (review): the public options referenced `BufferSource` / - * `WebAssembly.Module`, declared only in `src/wasm-ambient.d.ts` — a - * source-only ambient that TypeScript does not emit, while the published - * package ships `dist` only. A consumer check failed with seven - * missing-type errors across `dist/vm.d.ts` and `dist/workspace.d.ts`. - * - * The fixture (`test/fixtures/types-consumer/`) imports the built - * `dist/index.js` declarations and exercises the whole public surface; - * this test compiles it and fails if any declaration is missing. - */ - -import assert from 'node:assert/strict'; -import { execFileSync } from 'node:child_process'; -import { existsSync } from 'node:fs'; -import { createRequire } from 'node:module'; -import { dirname, join } from 'node:path'; -import { fileURLToPath } from 'node:url'; -import { test } from 'node:test'; - -const here = dirname(fileURLToPath(import.meta.url)); -const packageRoot = join(here, '..'); -const tsc = createRequire(import.meta.url).resolve('typescript/bin/tsc'); -const consumerDir = join(here, 'fixtures', 'types-consumer'); - -test('published type graph is self-contained for a non-DOM consumer (skipLibCheck: false)', () => { - // The consumer imports the published declarations (dist), so build the - // package first — with --force so the check never depends on incremental - // build state being correct (a stale dist that looks newer than src would - // otherwise be checked as-is). - execFileSync(process.execPath, [tsc, '-b', '--force', join(packageRoot, 'tsconfig.json')], { - stdio: 'pipe', - }); - assert.ok( - existsSync(join(packageRoot, 'dist', 'index.d.ts')), - 'build must produce dist/index.d.ts', - ); - // The fixture's tsconfig: ES2022 lib (no DOM), skipLibCheck: false, - // types: [] (no @types/node) — nothing ambient may be relied on. - execFileSync(process.execPath, [tsc, '-p', consumerDir], { stdio: 'pipe' }); -}); diff --git a/packages/repl-engine/test/repl-store.test.ts b/packages/repl-engine/test/repl-store.test.ts deleted file mode 100644 index 74382ee1..00000000 --- a/packages/repl-engine/test/repl-store.test.ts +++ /dev/null @@ -1,450 +0,0 @@ -/** - * Per-project store tests (phase D): the `repl/` subdirectory under - * `workflowHomeDir()/projects//` holding the enveloped snapshot and - * the call store. Pins: - * - * - the layout (the workflow store-layout helpers' key, verbatim), - * - the atomic write mechanics (snapshot file replaced; a stale tmp - * from a crashed write does not corrupt the next one), - * - the load path with the wasm-hash-mismatch REFUSAL naming BOTH - * hashes (never a restore into garbage), - * - the version-bump refusal through the store, - * - corrupted/truncated snapshot handling: loud single-shot failure, - * the store stays usable (no crash-loop), - * - the snapshot-write cadence + debounce (one write per drain burst; - * `debounceBursts: false` writes per boundary), - * - the call store + snapshot coexisting in the same `repl/` directory, - * - `reset()` teardown. - */ - -import assert from 'node:assert/strict'; -import { existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from 'node:fs'; -import { tmpdir } from 'node:os'; -import { join } from 'node:path'; -import { test } from 'node:test'; -import { gzipSync } from 'node:zlib'; - -import { - CALL_STORE_FILENAME, - GUEST_LIBRARY_VERSION, - REPL_STORE_SUBDIR, - SNAPSHOT_FILENAME, - SNAPSHOT_FORMAT_VERSION, - ReplWorkspaceStore, - SnapshotEnvelopeError, - Workspace, - loadShippedWasm, - wasmSha256Of, -} from '../src/index.js'; -import { workflowHomeDir, workflowProjectPaths } from '@automatalabs/workflows'; - -const PROJECT = '/tmp/repl-store-project'; - -/** `assert.throws` returns undefined at runtime — capture the error. */ -function captureThrows(fn: () => unknown): Error { - try { - fn(); - } catch (error) { - return error as Error; - } - assert.fail('expected the call to throw'); -} - -/** A scratch persistence root for the store (workflowHomeDir override). */ -function root(): string { - return mkdtempSync(join(tmpdir(), 'repl-store-root-')); -} - -async function setup() { - const dir = root(); - const module = await loadShippedWasm(); - const store = ReplWorkspaceStore.open(PROJECT, { persistenceRoot: dir }); - return { dir, module, store }; -} - -function teardown(dir: string): void { - rmSync(dir, { recursive: true, force: true }); -} - -// ──────────────────────────────────────────────────────────────────────── -// Layout -// ──────────────────────────────────────────────────────────────────────── - -test('layout: the repl/ store sits next to the workflow state under workflowHomeDir()/projects//', () => { - const dir = root(); - const store = ReplWorkspaceStore.open(PROJECT, { persistenceRoot: dir }); - const paths = workflowProjectPaths(PROJECT, { persistenceRoot: dir }); - assert.equal(store.replDir, join(paths.rootDir, REPL_STORE_SUBDIR)); - assert.equal(store.snapshotPath, join(paths.rootDir, REPL_STORE_SUBDIR, SNAPSHOT_FILENAME)); - assert.equal(store.callStorePath, join(paths.rootDir, REPL_STORE_SUBDIR, CALL_STORE_FILENAME)); - // The workflow engine's own store directory is the sibling (the repl - // dir is "next to the workflow state", never inside it). - assert.ok(store.replDir.startsWith(join(workflowHomeDir({ persistenceRoot: dir }), 'projects')), store.replDir); - assert.ok(store.replDir.includes(`projects/${paths.key}/repl`), store.replDir); - assert.equal(store.hasSnapshot(), false, 'a fresh store has no snapshot'); - teardown(dir); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Write + load round trip; hash-mismatch refusal -// ──────────────────────────────────────────────────────────────────────── - -test('write/load round trip: the enveloped snapshot restores the workspace; the call store coexists in the same repl/ dir', async () => { - const { dir, module, store } = await setup(); - const ws = await Workspace.create(PROJECT, { wasm: module }); - await ws.eval('const durable = { n: 41 }; globalThis.tag = "before-crash";'); - store.writeSnapshot(ws.snapshot(), module); - assert.equal(store.hasSnapshot(), true); - assert.equal(store.stats().snapshotWrites, 1); - - // The call store lives beside the snapshot. - const calls = store.callStore(); - calls.recordDispatched({ - callId: 'c1', - kind: 'agent', - detail: 'task', - optionsJson: null, - modelSpec: 'pi/x', - backendId: null, - foundingCallId: null, - admittedAtMs: 1, - admissionSequence: 1, - dispatchedAtMs: 1, - reissues: 0, - completion: null, - sessionId: null, - queuedAtMs: null, - handoffAtMs: null, - cancelledAtMs: null, - }); - calls.recordCompleted('c1', { outcome: 'resolve', value: 'done', completedAtMs: 2 }); - store.close(); - // Reopening the store replays the call log and reads the snapshot. - const reopened = ReplWorkspaceStore.open(PROJECT, { persistenceRoot: dir }); - assert.equal(reopened.callStore().lookup('c1')!.completion!.value, 'done'); - const loaded = reopened.loadSnapshot(module); - assert.equal(loaded.wasmSha256, wasmSha256Of(module)); - assert.equal(loaded.formatVersion, SNAPSHOT_FORMAT_VERSION); - const ws2 = await Workspace.restore(PROJECT, loaded.snapshot, { wasm: module }); - const outcome = await ws2.eval('durable.n + 1 + "/" + tag'); - assert.equal(outcome.kind, 'value'); - assert.equal(outcome.value, '42/before-crash'); - ws.dispose(); - ws2.dispose(); - reopened.close(); - teardown(dir); -}); - -test('hash-mismatch refusal: a snapshot recorded by another binary refuses LOUDLY naming both hashes', async () => { - const { dir, module, store } = await setup(); - const ws = await Workspace.create(PROJECT, { wasm: module }); - await ws.eval('globalThis.x = 1;'); - store.writeSnapshot(ws.snapshot(), module); - ws.dispose(); - - // A different binary: the same shipped wasm with one byte flipped. The - // restore must refuse before instantiating anything. - const resolved = import.meta.resolve('quickjs-wasi/quickjs.wasm'); - const { readFile } = await import('node:fs/promises'); - const bytes = new Uint8Array(await readFile(new URL(resolved))); - bytes[1024] ^= 0xff; - const foreign = bytes; - - const error = captureThrows(() => store.loadSnapshot(foreign)); - assert.ok(error instanceof SnapshotEnvelopeError, error.message); - assert.equal((error as SnapshotEnvelopeError).code, 'WASM_HASH_MISMATCH'); - // NAMES BOTH HASHES — never a silent restore into garbage. - const recordedHash = wasmSha256Of(module); - const runningHash = wasmSha256Of(foreign); - assert.ok(error.message.includes(recordedHash), `names the recorded hash: ${error.message}`); - assert.ok(error.message.includes(runningHash), `names the running hash: ${error.message}`); - assert.equal(error.recorded, recordedHash); - assert.equal(error.expected, runningHash); - teardown(dir); -}); - -test('hash-mismatch refusal PRECEDES payload interpretation: a foreign-binary payload that cannot be deserialized refuses as WASM_HASH_MISMATCH, never CORRUPT_PAYLOAD', async () => { - // The phase-D review regression: the payload used to be gunzipped and - // passed through `QuickJS.deserializeSnapshot()` BEFORE the running wasm - // hash was compared — so a snapshot recorded by an incompatible binary - // (whose raw memory layout is garbage to this binary) failed as - // CORRUPT_PAYLOAD without naming the hashes. The identity check now - // lives between the header parse and the payload decode. - const { dir, module, store } = await setup(); - const runningHash = wasmSha256Of(module); - // A FOREIGN recorded hash (valid hex, not the running binary's) over a - // payload that is a VALID gzip stream but NOT a serialized snapshot (a - // foreign binary's memory image would look exactly like this to this - // binary: gunzip succeeds, deserialization would fail). - const foreignHash = 'f'.repeat(64); - const gz = gzipSync(Buffer.from('not a quickjs snapshot payload')); - const header = Buffer.from( - JSON.stringify({ - format: 'repl-snapshot', - formatVersion: SNAPSHOT_FORMAT_VERSION, - wasmSha256: foreignHash, - createdAtMs: Date.now(), - }) + '\n', - 'utf8', - ); - writeFileSync(store.snapshotPath, Buffer.concat([header, gz])); - - const error = captureThrows(() => store.loadSnapshot(module)); - assert.ok(error instanceof SnapshotEnvelopeError, error.message); - assert.equal((error as SnapshotEnvelopeError).code, 'WASM_HASH_MISMATCH', `names the hashes instead of the payload: ${error.message}`); - assert.ok(error.message.includes(foreignHash), `names the recorded hash: ${error.message}`); - assert.ok(error.message.includes(runningHash), `names the running hash: ${error.message}`); - assert.equal(error.recorded, foreignHash); - assert.equal(error.expected, runningHash); - teardown(dir); -}); - -test('version-bump refusal through the store: an upgraded format version refuses naming both versions', async () => { - const { dir, module, store } = await setup(); - const ws = await Workspace.create(PROJECT, { wasm: module }); - await ws.eval('globalThis.x = 1;'); - store.writeSnapshot(ws.snapshot(), module); - ws.dispose(); - // Forge a bumped envelope over the file (a future format release). - const raw = readFileSync(store.snapshotPath); - const nl = raw.indexOf(0x0a); - const header = JSON.parse(raw.subarray(0, nl).toString('utf8')); - writeFileSync(store.snapshotPath, Buffer.concat([ - Buffer.from(JSON.stringify({ ...header, formatVersion: SNAPSHOT_FORMAT_VERSION + 1 }) + '\n'), - raw.subarray(nl + 1), - ])); - const error = captureThrows(() => store.loadSnapshot(module)); - assert.ok(error instanceof SnapshotEnvelopeError, error.message); - assert.equal((error as SnapshotEnvelopeError).code, 'VERSION_MISMATCH'); - assert.ok(error.message.includes(String(SNAPSHOT_FORMAT_VERSION + 1)), error.message); - assert.ok(error.message.includes(String(SNAPSHOT_FORMAT_VERSION)), error.message); - assert.ok(error.message.includes(store.snapshotPath), error.message); - teardown(dir); -}); - -test('format 3 / guest 0.5: a format-2 snapshot is refused before old guest state can be restored or executed', async () => { - assert.equal(SNAPSHOT_FORMAT_VERSION, 3); - assert.equal(GUEST_LIBRARY_VERSION, '0.5.0'); - const { dir, module, store } = await setup(); - const ws = await Workspace.create(PROJECT, { wasm: module }); - await ws.eval(` - globalThis.oldGuestExecutionSentinel = "must never be observed by a new workspace"; - globalThis.followUp = () => { throw new Error("old guest followUp executed"); }; - `); - store.writeSnapshot(ws.snapshot(), module); - ws.dispose(); - - const raw = readFileSync(store.snapshotPath); - const nl = raw.indexOf(0x0a); - const header = JSON.parse(raw.subarray(0, nl).toString('utf8')); - writeFileSync(store.snapshotPath, Buffer.concat([ - Buffer.from(JSON.stringify({ ...header, formatVersion: 2 }) + '\n'), - raw.subarray(nl + 1), - ])); - - const error = captureThrows(() => store.loadSnapshot(module)); - assert.ok(error instanceof SnapshotEnvelopeError, error.message); - assert.equal((error as SnapshotEnvelopeError).code, 'VERSION_MISMATCH'); - assert.equal(error.recorded, '2'); - assert.equal(error.expected, '3'); - assert.match(error.message, /format version 2/); - // No decoded snapshot is returned, so Workspace.restore — the only path - // that can register callbacks or resume guest jobs — is never reachable. - assert.equal(store.hasSnapshot(), true, 'the incompatible bytes remain available for the refusal/rename-aside path'); - teardown(dir); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Corrupted / truncated handling -// ──────────────────────────────────────────────────────────────────────── - -test('corrupted/truncated snapshot: loud single-shot failure, no crash-loop, the store stays usable', async () => { - const { dir, module, store } = await setup(); - const ws = await Workspace.create(PROJECT, { wasm: module }); - await ws.eval('globalThis.x = 1;'); - store.writeSnapshot(ws.snapshot(), module); - - // Garbage over the file: loud refusal naming the file. - writeFileSync(store.snapshotPath, 'this is not a snapshot file at all'); - let error = captureThrows(() => store.loadSnapshot(module)); - assert.ok(error instanceof SnapshotEnvelopeError, error.message); - - // Truncated envelope (the gzip body cut): loud refusal, same code. - store.writeSnapshot(ws.snapshot(), module); - const valid = readFileSync(store.snapshotPath); - writeFileSync(store.snapshotPath, valid.subarray(0, Math.floor(valid.length / 2))); - error = captureThrows(() => store.loadSnapshot(module)); - assert.ok(error instanceof SnapshotEnvelopeError, error.message); - assert.equal((error as SnapshotEnvelopeError).code, 'CORRUPT_PAYLOAD'); - - // No crash-loop: the store is immediately usable — a fresh write - // replaces the bad file and the load succeeds again. - store.writeSnapshot(ws.snapshot(), module); - assert.equal(store.hasSnapshot(), true); - const loaded = store.loadSnapshot(module); - const ws2 = await Workspace.restore(PROJECT, loaded.snapshot, { wasm: module }); - const outcome = await ws2.eval('x + 1'); - assert.equal(outcome.kind, 'value'); - assert.equal(outcome.value, 2); - ws.dispose(); - ws2.dispose(); - teardown(dir); -}); - -test('a snapshot write that fails leaves the previous snapshot untouched and removes the tmp file', async () => { - const { dir, module, store } = await setup(); - const ws = await Workspace.create(PROJECT, { wasm: module }); - await ws.eval('globalThis.x = 1;'); - store.writeSnapshot(ws.snapshot(), module); - - // Make the atomic replace fail: rename over a DIRECTORY at the target - // path is impossible, so the write throws loudly. - rmSync(store.snapshotPath); - mkdirSync(store.snapshotPath); - assert.throws(() => store.writeSnapshot(ws.snapshot(), module), /EISDIR|ENOTDIR|EEXIST|ENOTEMPTY|rename/); - // The tmp file was cleaned up; the target directory is still there - // (the failure happened before any destructive step). - const { readdirSync } = await import('node:fs'); - const entries = readdirSync(store.replDir); - assert.ok(!entries.some((e) => e.endsWith('.tmp')), `no tmp file left behind: ${entries.join(', ')}`); - - // The store stays usable: remove the blocking directory and write again. - rmSync(store.snapshotPath, { recursive: true }); - store.writeSnapshot(ws.snapshot(), module); - const reloaded = store.loadSnapshot(module); - const ws2 = await Workspace.restore(PROJECT, reloaded.snapshot, { wasm: module }); - assert.equal((await ws2.eval('x + 1')).value, 2, 'the store is fully usable after the failed write'); - ws2.dispose(); - ws.dispose(); - teardown(dir); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Cadence, debounce, and teardown -// ──────────────────────────────────────────────────────────────────────── - -test('snapshotWriter: one atomic write per drain burst (the doc\'s debounce), boundaries coalesced', async () => { - const { dir, module, store } = await setup(); - const ws = await Workspace.create(PROJECT, { wasm: module }); - const sink = store.snapshotWriter(ws, module); - - // A burst of boundaries coalesces into ONE write at the flush. - sink.boundary('settlement'); - sink.boundary('eval'); - assert.equal(store.stats().snapshotWrites, 0, 'debounced: nothing written inside the burst'); - sink.flush(); - assert.equal(store.stats().snapshotWrites, 1, 'one write for the whole burst'); - // An empty flush writes nothing. - sink.flush(); - assert.equal(store.stats().snapshotWrites, 1); - - // The written snapshot reflects the LIVE workspace state at flush time. - await ws.eval('globalThis.burstState = "yes";'); - sink.boundary('eval'); - await ws.eval('globalThis.burstState = "mutated-after-boundary";'); - sink.flush(); - const loaded = store.loadSnapshot(module); - const ws2 = await Workspace.restore(PROJECT, loaded.snapshot, { wasm: module }); - const burstOutcome = await ws2.eval('burstState'); - assert.equal(burstOutcome.kind, 'value'); - assert.equal(burstOutcome.value, 'mutated-after-boundary', 'the snapshot carries the state at FLUSH time, not at boundary time'); - ws.dispose(); - ws2.dispose(); - teardown(dir); -}); - -test('debounceBursts: false writes synchronously at every boundary', async () => { - const dir = root(); - const module = await loadShippedWasm(); - const store = ReplWorkspaceStore.open(PROJECT, { - persistenceRoot: dir, - snapshotWrite: { debounceBursts: false }, - }); - const ws = await Workspace.create(PROJECT, { wasm: module }); - const sink = store.snapshotWriter(ws, module); - sink.boundary('eval'); - assert.equal(store.stats().snapshotWrites, 1, 'the boundary wrote immediately'); - sink.boundary('settlement'); - sink.flush(); - assert.equal(store.stats().snapshotWrites, 2, 'every boundary writes when the debounce is off'); - ws.dispose(); - teardown(dir); -}); - -test('THE REVIEW REGRESSION: a failed flush RETAINS the dirty boundary — the next flush retries the SAME state, never a silent drop (phase-D review round 6: the boundary used to clear before the write)', async () => { - const dir = root(); - const module = await loadShippedWasm(); - const store = ReplWorkspaceStore.open(PROJECT, { persistenceRoot: dir }); - const ws = await Workspace.create(PROJECT, { wasm: module }); - await ws.eval('globalThis.pending = "must survive";'); - const sink = store.snapshotWriter(ws, module); - sink.boundary('eval'); - // Sabotage the atomic write: a DIRECTORY at the tmp path makes the - // write's open fail (EISDIR) — the previous snapshot file, if any, - // is untouched. - mkdirSync(`${store.snapshotPath}.tmp`); - const error = captureThrows(() => sink.flush()); - assert.ok(error instanceof Error, 'the failed write throws loudly'); - assert.equal(store.stats().snapshotWrites, 0, 'the failed write is not counted'); - assert.equal(existsSync(store.snapshotPath), false, 'no partial snapshot file'); - // The boundary is STILL DIRTY: after the obstruction is removed, the - // next flush writes the same state (the failed boundary is retained - // for retry — a kill after the failure loses nothing that was - // acknowledged, and the next drain burst persists it). - rmSync(`${store.snapshotPath}.tmp`, { recursive: true, force: true }); - sink.flush(); - assert.equal(store.stats().snapshotWrites, 1, 'the retained boundary wrote on the retry'); - const loaded = store.loadSnapshot(module); - const ws2 = await Workspace.restore(PROJECT, loaded.snapshot, { wasm: module }); - const outcome = await ws2.eval('globalThis.pending'); - assert.equal(outcome.kind, 'value'); - assert.equal(outcome.value, 'must survive', 'the retried snapshot carries the SAME state the failed flush was asked to persist'); - ws.dispose(); - ws2.dispose(); - teardown(dir); -}); - -test('reset tears the repl/ directory down (the reset() guest function\'s engine-side) — §6.1 [C]13: a renamed-aside refused snapshot is NEVER deleted', async () => { - const { dir, module, store } = await setup(); - const ws = await Workspace.create(PROJECT, { wasm: module }); - store.writeSnapshot(ws.snapshot(), module); - store.callStore().recordDispatched({ - callId: 'c1', - kind: 'checkpoint', - detail: 'question?', - optionsJson: null, - modelSpec: null, - backendId: null, - foundingCallId: null, - admittedAtMs: 1, - admissionSequence: 1, - dispatchedAtMs: 1, - reissues: 0, - completion: null, - sessionId: null, - queuedAtMs: null, - handoffAtMs: null, - cancelledAtMs: null, - }); - assert.equal(store.hasSnapshot(), true); - // A refused snapshot that auto-reset renamed aside survives the wipe - // (the §6.1 data-safety guarantee — auto-reset is never silent data - // destruction, and neither is a later reset()). - const refusedAside = `${store.snapshotPath}.refused-1720000000000`; - writeFileSync(refusedAside, 'refused bytes'); - store.reset(); - assert.equal(store.hasSnapshot(), false, 'the snapshot is gone'); - assert.equal(store.stats().snapshotWrites, 0, 'the counters reset'); - assert.equal(existsSync(store.replDir), true, 'the repl/ directory itself stays'); - assert.deepEqual( - [...readdirSync(store.replDir)], - ['snapshot.bin.refused-1720000000000'], - 'every store file was dropped — EXCEPT the renamed-aside refused snapshot', - ); - // The store is usable again from scratch. - store.writeSnapshot(ws.snapshot(), module); - assert.equal(store.hasSnapshot(), true); - assert.equal(store.callStore().lookup('c1'), undefined, 'the call log was dropped with the directory contents'); - ws.dispose(); - teardown(dir); -}); diff --git a/packages/repl-engine/test/restore.test.ts b/packages/repl-engine/test/restore.test.ts deleted file mode 100644 index af55525a..00000000 --- a/packages/repl-engine/test/restore.test.ts +++ /dev/null @@ -1,2681 +0,0 @@ -/** - * Restore-path tests (phase D): the full restore flow — restore the VM - * from the enveloped snapshot, re-register the host callbacks by name, - * read the in-VM pending-call registry, and reconcile each outstanding - * call three ways (mock backends, per the phase deliverable): - * - * - completed while down → settle from the call store, - * - still resumable at the backend → re-attach via loadSession - * (capability-gated; a custom backend without the capability degrades - * through the same gate, surfaced guest-visibly), - * - lost → re-issue under the same call id (reissues counter bumped, - * the existing guest promise settles exactly once). - * - * Plus: pending checkpoints re-surface (answerable across the restore), - * in-flight steers resolve the honest `failed`, reconcile idempotence, - * the snapshot cadence (a boundary after each eval and after each - * settlement drain that changed VM state — nothing for a drain that - * changed nothing), and the end-to-end debounce (one atomic write per - * drain burst through the per-project store). - */ - -import assert from 'node:assert/strict'; -import { mkdtempSync, readFileSync, rmSync } from 'node:fs'; -import { tmpdir } from 'node:os'; -import { join } from 'node:path'; -import { fileURLToPath } from 'node:url'; -import { test } from 'node:test'; - -import { AcpAgentRunner, LoadedTurnFailedError, LoadedTurnStillRunningError } from '@automatalabs/acp-agents'; - -import { - Broker, - JsonlCallStore, - ReplWorkspaceStore, - Workspace, - loadShippedWasm, - type BrokerLoadSessionOptions, - type BrokerOpenSessionOptions, - type BrokerPromptOptions, - type BrokerRunner, - type BrokerSession, - type BrokerTurn, - type CallStore, - type ReplEvalResult, - type SnapshotSink, -} from '../src/index.js'; - -const PROJECT = '/tmp/repl-restore-project'; - -/** The acp-agents integration fixture: a REAL ACP agent server (SDK-side) - * that speaks real ACP over stdio — the "actual acp-agents adapter" the - * phase-D review demands the re-attach arm be tested through. */ -const FAKE_AGENT_FIXTURE = fileURLToPath( - new URL('../../acp-agents/test/fixtures/fake-acp-agent.mjs', import.meta.url), -); - -/** The fake-agent spawn/env keys (see acp-agents' test helpers). */ -const FAKE_ENV_KEYS = [ - 'AGENTPRISM_CLAUDE_ACP_CMD', - 'AGENTPRISM_CLAUDE_ACP_ARGS', - 'AGENTPRISM_FAKE_LOG', - 'AGENTPRISM_FAKE_SCENARIO', - 'AGENTPRISM_DEFAULT_BACKEND', -] as const; - -function clearFakeEnv(): void { - for (const key of FAKE_ENV_KEYS) delete process.env[key]; -} - -/** Point the claude built-in's spawn at the fake ACP agent (the acp-agents - * integration pattern) and script its scenario. */ -function configureFakeAgent(scenario: unknown, logPath: string): void { - clearFakeEnv(); - process.env.AGENTPRISM_CLAUDE_ACP_CMD = process.execPath; - process.env.AGENTPRISM_CLAUDE_ACP_ARGS = FAKE_AGENT_FIXTURE; - process.env.AGENTPRISM_DEFAULT_BACKEND = 'claude'; - process.env.AGENTPRISM_FAKE_SCENARIO = JSON.stringify(scenario); - process.env.AGENTPRISM_FAKE_LOG = logPath; -} - -interface WireLogEntry { - method: string; - pid?: number; - params?: { sessionId?: string }; -} - -/** The fake agent's request log (one JSON line per observed ACP request). */ -function readWireLog(path: string): WireLogEntry[] { - const content = readFileSync(path, 'utf8').trim(); - if (!content) return []; - return content - .split('\n') - .filter(Boolean) - .map((line) => JSON.parse(line) as WireLogEntry); -} - -/** Poll until `predicate` holds (the fake agent answers asynchronously). */ -async function waitFor(predicate: () => boolean, timeoutMs = 5000): Promise { - const start = Date.now(); - while (!predicate()) { - if (Date.now() - start > timeoutMs) throw new Error('waitFor: condition not met in time'); - await new Promise((resolve) => setTimeout(resolve, 10)); - } -} -async function tick(): Promise { - await new Promise((resolve) => setTimeout(resolve, 0)); -} - -/** A fake held-open ACP session (phase-C shape plus the phase-D re-attach - * seam: `awaitCurrentTurn`, mirroring the REAL acp-agents adapter's - * semantics — it resolves with a scripted loaded-turn outcome when one is - * set (the replay made the completed-while-down turn observable), and - * PARKS otherwise (the still-running-at-load case: the real adapter keeps - * the loaded session attached and waits for the turn's authoritative - * completion — reconcile arms the call on the seam and returns; the test - * resolves or rejects the parked seam to drive the outcome). */ -class FakeSession implements BrokerSession { - readonly sessionId: string; - initializeMeta: Readonly> | undefined; - readonly prompts: Array<{ content: string; resolve: (turn: BrokerTurn) => void; reject: (error: unknown) => void }> = []; - readonly steers: Array<{ content: string; resolve: (outcome: unknown) => void; reject: (error: unknown) => void }> = []; - /** The re-attach seam's parked loaded-turn completions (only used when - * no scripted outcome is set). */ - readonly loadedTurns: Array<{ resolve: (turn: BrokerTurn) => void; reject: (error: unknown) => void }> = []; - releases = 0; - /** The session-level loaded-turn terminal state (the real adapter's - * `loadedTurnEndedState` — the `_session/loaded_turn/ended` - * notification a seam-less backend pushes anyway). Null until - * `fireLoadedTurnEnded`. */ - endedState: { stopReason?: string; error?: { name: string; message: string } } | null = null; - private readonly endedWatchers = new Set<() => void>(); - /** The `released()` watch (the real adapter's release promise). */ - private releasedWatchers: Array<() => void> = []; - private releasedFlag = false; - stopReason = 'end_turn'; - readonly completedTexts: string[] = []; - /** The seam's scripted loaded-turn outcome (the real adapter reads it - * from the session/load replay + stream settling). Null parks the seam - * (still running at load). */ - loadedTurnTextValue: string | null = null; - /** A hung cancel (the drain-bound regression: the post-deadline cancel - * await must not block disconnect past the bound). */ - hangCancel = false; - /** A hung release (same regression for the release phase). */ - hangRelease = false; - /** Cancel invocations (the mid-drain-abort regression: an aborted - * drain must never cancel anything). */ - cancelCalls = 0; - - constructor(readonly openedWith: BrokerOpenSessionOptions | BrokerLoadSessionOptions) { - this.sessionId = `fake-session-${FakeSession.nextId++}`; - this.initializeMeta = { steering: { supported: true } }; - } - - static nextId = 0; - - prompt(content: string, opts: BrokerPromptOptions = {}): Promise { - return new Promise((resolve, reject) => { - this.prompts.push({ content, resolve, reject }); - // The handoff acknowledgment (the phase-C seam order). - opts.onHandoff?.(); - }); - } - - steer(content: string): Promise { - return new Promise((resolve, reject) => { - this.steers.push({ content, resolve, reject }); - }); - } - - awaitCurrentTurn(): Promise { - if (this.loadedTurnTextValue !== null) { - // The loaded session's founding turn observably completed (its final - // message is in the replay) — resolve immediately, like the real - // adapter. - return Promise.resolve({ stopReason: this.stopReason, text: this.loadedTurnTextValue }); - } - return new Promise((resolve, reject) => { - this.loadedTurns.push({ resolve, reject }); - }); - } - - cancel(): Promise { - this.cancelCalls++; - if (this.hangCancel) return new Promise(() => {}); - for (const pending of this.prompts.splice(0)) { - pending.resolve({ stopReason: 'cancelled', text: '' }); - } - return Promise.resolve(); - } - - loadedTurnEndedState(): { stopReason?: string; error?: { name: string; message: string } } | null { - return this.endedState; - } - - subscribeLoadedTurnEnded(listener: () => void): () => void { - if (this.endedState !== null) { - queueMicrotask(listener); - return () => {}; - } - this.endedWatchers.add(listener); - return () => { - this.endedWatchers.delete(listener); - }; - } - - /** Drive the session-level ended notification (the test's handle for - * the non-re-armable wait's observability surface). */ - fireLoadedTurnEnded(state: { stopReason?: string; error?: { name: string; message: string } }, text?: string): void { - this.endedState = state; - if (text !== undefined) this.completedTexts.push(text); - for (const watcher of [...this.endedWatchers]) watcher(); - } - - released(): Promise { - if (this.releasedFlag) return Promise.resolve(); - return new Promise((resolve) => { - if (this.releasedFlag) { - resolve(); - return; - } - this.releasedWatchers.push(resolve); - }); - } - - release(): Promise { - this.releases++; - if (this.hangRelease) return new Promise(() => {}); - this.releasedFlag = true; - for (const watcher of this.releasedWatchers.splice(0)) watcher(); - return Promise.resolve(); - } - - currentTurnText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ''; - } - - finalMessageText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ''; - } - - rawStructuredOutput(): unknown { - return undefined; - } - - completeTurn(text: string): void { - const pending = this.prompts.shift(); - assert.ok(pending, 'a prompt turn must be in flight'); - this.completedTexts.push(text); - pending.resolve({ stopReason: this.stopReason, text }); - } - - completeSteer(outcome: string): void { - const pending = this.steers.shift(); - assert.ok(pending, 'a steer wire call must be in flight'); - pending.resolve({ outcome }); - } -} - -/** A fake runner with the phase-D loadSession seam. */ -class FakeRunner implements BrokerRunner { - readonly sessions: FakeSession[] = []; - readonly openedWith: BrokerOpenSessionOptions[] = []; - readonly loadedWith: BrokerLoadSessionOptions[] = []; - supportsSteering = true; - /** The re-attach capability gate (acp-agents' supportsLoadSession). */ - supportsLoadSession = true; - /** When true, loadSession returns sessions WITHOUT the awaitCurrentTurn - * seam (a third-party adapter whose loaded-turn completion is - * unobservable — the broker degrades to re-issue through the same - * honest gate). */ - seamless = false; - /** The scripted loaded-turn outcome for loadSession-created sessions - * (the real adapter resolves the seam from the session/load replay). - * Null parks the seam (the still-running-at-load case). */ - loadedTurnText: string | null = null; - failNextOpens = 0; - /** The next N loadSession calls reject (each one once). */ - failNextLoads = 0; - /** LoadSession calls at ordinal >= failLoadsFrom reject (1-based). */ - failLoadsFrom = Infinity; - - listBackends(): string[] { - return ['claude', 'codex', 'opencode', 'pi']; - } - - defaultBackendId(): string { - return 'claude'; - } - - async openSession(opts: BrokerOpenSessionOptions): Promise { - if (this.failNextOpens > 0) { - this.failNextOpens--; - throw new Error('spawn failed'); - } - const session = new FakeSession(opts); - session.initializeMeta = this.supportsSteering ? { steering: { supported: true } } : {}; - this.sessions.push(session); - this.openedWith.push(opts); - return session; - } - - async loadSession(opts: BrokerLoadSessionOptions): Promise { - this.loadedWith.push(opts); - const ordinal = this.loadedWith.length; - if (!this.supportsLoadSession) { - // The acp-agents capability gate (capabilities.ts): a backend - // that omits session/load rejects BEFORE any wire request. - throw new Error('backend does not advertise session/load (loadSession capability gate)'); - } - if (this.failNextLoads > 0) { - this.failNextLoads--; - throw new Error('session not found at the backend'); - } - if (ordinal >= this.failLoadsFrom) { - throw new Error('session not found at the backend'); - } - const session = new FakeSession(opts); - session.initializeMeta = this.supportsSteering ? { steering: { supported: true } } : {}; - session.loadedTurnTextValue = this.loadedTurnText; - if (this.seamless) { - // Shadow the prototype method with an own undefined property — the - // broker's optional-seam probe sees a seam-less adapter. - Object.defineProperty(session, 'awaitCurrentTurn', { value: undefined, configurable: true }); - } - this.sessions.push(session); - return session; - } - - async dispose(): Promise {} - - last(): FakeSession { - assert.ok(this.sessions.length > 0, 'a session must have been opened'); - return this.sessions[this.sessions.length - 1]; - } -} - -async function setup(options: { - store?: CallStore; - snapshotSink?: SnapshotSink; - runner?: FakeRunner; - maxConcurrentAgents?: number; - interruptHandler?: () => boolean; - evalTimeoutMs?: number; -} = {}) { - const ws = await Workspace.create(PROJECT); - const runner = options.runner ?? new FakeRunner(); - const broker = await Broker.attach(ws, { - runner, - store: options.store, - snapshotSink: options.snapshotSink, - maxConcurrentAgents: options.maxConcurrentAgents, - interruptHandler: options.interruptHandler, - evalTimeoutMs: options.evalTimeoutMs, - }); - return { ws, broker, runner }; -} - -function output(r: ReplEvalResult): string[] { - return r.output; -} - -/** §6.2: the re-attach / re-issue / refusal / lost-steer surfacing lines - * demote to workspace().diagnostics.reconcileNotes — read them through - * the guest introspection surface. */ -async function reconcileNotesOf(broker: Broker): Promise> { - const r = await broker.eval('JSON.stringify(workspace().diagnostics.reconcileNotes)'); - assert.equal(r.kind, 'value', 'the diagnostics read resolved'); - return JSON.parse(r.result ?? '[]') as Array<{ level: string; line: string; atMs: number }>; -} - -/** Dispatch one agent call and wait until its session is open. */ -async function dispatchAgent(broker: Broker, runner: FakeRunner, code = 'const p = agent("pi/x", "task"); "started"') { - const r = await broker.eval(code); - assert.equal(r.result, 'started', JSON.stringify(r)); - await tick(); - assert.equal(runner.sessions.length, 1); - assert.equal(runner.last().prompts.length, 1, 'the initial turn is in flight'); -} - -/** Crash: dispose the broker and workspace (the store file survives). */ -async function crash(ws: Workspace, broker: Broker): Promise { - await broker.dispose(); - ws.dispose(); -} - -// ──────────────────────────────────────────────────────────────────────── -// The three reconciliation arms -// ──────────────────────────────────────────────────────────────────────── - -test('restore with all three arms: settle-from-store, re-attach, re-issue (mock backends)', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-arms-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - - const r = await broker.eval( - 'const a = agent("pi/a", "task A"); const b = agent("pi/b", "task B"); const c = agent("pi/c", "task C"); "started"', - ); - assert.equal(r.result, 'started'); - await tick(); - assert.equal(runner.sessions.length, 3); - const [sessionA, sessionB, sessionC] = runner.sessions; - - // c1's worker completes, but the process crashes BEFORE the pump - // settles it: the completion is recorded in the store (the pump's - // record step) while the guest registry still holds c1 pending (the - // settle step never ran) — the store arm's exact crash window. - sessionA.completeTurn('result A'); - await tick(); - broker.store().recordCompleted('c1', { outcome: 'resolve', value: 'result A', completedAtMs: Date.now() }); - assert.deepEqual(broker.pendingCalls().map((e) => e.id), ['c1', 'c2', 'c3'], 'all three calls are pending in the registry'); - - // The re-attach keys were recorded in the store the moment each - // session opened (BEFORE any prompt). - assert.equal(broker.store().lookup('c2')!.sessionId, sessionB.sessionId); - assert.equal(broker.store().lookup('c3')!.sessionId, sessionC.sessionId); - - // Snapshot the LIVE VM (c2, c3 pending in the registry), then crash. - const snapshot = ws.snapshot(); - await crash(ws, broker); - - // Restore: fresh workspace over the snapshot, fresh broker + runner - // over the same store. - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - // c2's loaded turn completed at the backend while we were down — its - // final message is in the replayed transcript, so the seam observes it - // DURING reconcile (the real adapter's semantics). - runner2.loadedTurnText = 'result B (loaded)'; - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - // The SECOND load (c3's) fails — the session was lost at the backend. - runner2.failLoadsFrom = 2; - const report = await broker2.reconcile(); - assert.deepEqual(report.settledFromStore, ['c1']); - assert.deepEqual(report.reattached, ['c2']); - assert.deepEqual(report.reissued, ['c3']); - assert.deepEqual(report.failedLost, []); - assert.deepEqual(report.requeuedCheckpoints, []); - assert.deepEqual(report.leftPending, [], 'the full three-way reconcile leaves nothing pending'); - - // Guest-visible surfacing (§6.2): the re-attach info line and the - // re-issue warn line leave the eval result surface entirely — they - // are retained under workspace().diagnostics.reconcileNotes with the - // reconcile summary (only the [C]14 LOSS notice may ride an eval's - // output). - const probe = await broker2.eval('"probe"'); - assert.deepEqual(output(probe), [], 'ordinary reconciliation never rides the eval output'); - const notes = await reconcileNotesOf(broker2); - assert.ok( - notes.some((n) => n.level === 'info' && n.line.includes('c2') && n.line.includes('re-attached')), - JSON.stringify(notes), - ); - assert.ok( - notes.some( - (n) => n.level === 'warn' && n.line.includes('c3') && n.line.includes('re-issued') && n.line.includes('session not found'), - ), - JSON.stringify(notes), - ); - assert.equal((await broker2.eval('await a')).result, 'result A', 'the store arm settled c1 exactly once'); - - // The re-attach went to the RECORDED backend session with the founding - // routing (model spec + cwd recovered from the registry entry). - assert.equal(runner2.loadedWith.length, 2, 'two load attempts: c2 re-attaches, c3 is lost'); - assert.equal(runner2.loadedWith[0].sessionId, sessionB.sessionId); - assert.equal(runner2.loadedWith[0].model, 'pi/b'); - assert.equal(runner2.loadedWith[0].cwd, PROJECT); - assert.equal(runner2.sessions[0].openedWith.runId, 'c2', 'the loaded session is addressed by the founding call id'); - - // The re-issue opened a FRESH session under the SAME call id and the - // store bumped the reissues counter (a re-attach is not a re-issue). - assert.equal(runner2.sessions.length, 2); - assert.equal(runner2.sessions[1].openedWith.runId, 'c3'); - assert.equal(broker2.store().lookup('c3')!.reissues, 1); - assert.equal(broker2.store().lookup('c2')!.reissues, 0); - - // The re-attached call's loaded turn was observed during reconcile; the - // next pump delivers it through the same record → settle → consume path - // — the guest promise resolves exactly once, and the outcome is durable. - await broker2.pump(); - assert.equal((await broker2.eval('await b')).result, 'result B (loaded)'); - assert.equal(broker2.store().lookup('c2')!.completion!.value, 'result B (loaded)'); - - // The re-issued call's fresh turn completes → the SAME guest promise - // resolves (never a duplicate), and the store's session id now points - // at the re-issue's new session (a later restore re-attaches THAT one). - runner2.sessions[1].completeTurn('result C (re-issued)'); - await tick(); - await broker2.pump(); - assert.equal((await broker2.eval('await c')).result, 'result C (re-issued)'); - assert.equal(broker2.store().lookup('c3')!.sessionId, runner2.sessions[1].sessionId); - - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('a custom backend without the loadSession capability degrades through the same gate — re-issue, surfaced guest-visibly', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-cap-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await dispatchAgent(broker, runner); - const snapshot = ws.snapshot(); - await crash(ws, broker); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - runner2.supportsLoadSession = false; // the custom backend omits the capability - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - const report = await broker2.reconcile(); - assert.deepEqual(report.reattached, [], 'no re-attach — the backend lacks the capability'); - assert.deepEqual(report.reissued, ['c1'], 're-issue is the honest fallback through the same gate'); - assert.equal(runner2.loadedWith.length, 1, 'the load WAS attempted — the runner enforces the gate'); - assert.equal(broker2.store().lookup('c1')!.reissues, 1); - - // The re-issued call completes normally; the guest sees the result, - // and the capability-degradation line demotes to diagnostics (§6.2). - const probe = await broker2.eval('"probe"'); - assert.deepEqual(output(probe), [], 'no reconcile line leaks into the eval output'); - const notes = await reconcileNotesOf(broker2); - assert.ok( - notes.some((n) => n.line.includes('c1') && n.line.includes('loadSession') && n.line.includes('re-issued')), - JSON.stringify(notes), - ); - runner2.sessions[0].completeTurn('custom backend result'); - await tick(); - await broker2.pump(); - assert.equal((await broker2.eval('await p')).result, 'custom backend result'); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('a session lost at the backend (loadSession fails) degrades to re-issue, surfaced guest-visibly', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-lost-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await dispatchAgent(broker, runner); - const snapshot = ws.snapshot(); - await crash(ws, broker); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - runner2.failNextLoads = 1; - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - const report = await broker2.reconcile(); - assert.deepEqual(report.reissued, ['c1']); - const probe = await broker2.eval('"probe"'); - assert.deepEqual(output(probe), [], 'no reconcile line leaks into the eval output'); - const notes = await reconcileNotesOf(broker2); - assert.ok( - notes.some((n) => n.line.includes('not resumable') && n.line.includes('session not found')), - JSON.stringify(notes), - ); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('reconcile is idempotent: a repeated reconcile never re-attaches or re-issues twice', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-idem-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await dispatchAgent(broker, runner); - const snapshot = ws.snapshot(); - await crash(ws, broker); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - runner2.loadedTurnText = 'loaded turn'; - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - const first = await broker2.reconcile(); - assert.deepEqual(first.reattached, ['c1']); - assert.equal(runner2.loadedWith.length, 1); - const second = await broker2.reconcile(); - assert.deepEqual(second.reattached, ['c1'], 'already-tracked calls report as re-attached'); - assert.equal(runner2.loadedWith.length, 1, 'no second loadSession'); - assert.equal(broker2.store().lookup('c1')!.reissues, 0, 'no second dispatch'); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('re-issues respect the concurrency cap: an over-cap re-issue QUEUES in dispatch order for the next free slot — never a rejection', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-capq-')); - const storePath = join(dir, 'calls.jsonl'); - const kinds: Array<'eval' | 'settlement'> = []; - const sink: SnapshotSink = { boundary: (kind) => kinds.push(kind), flush: () => {} }; - const runner = new FakeRunner(); - const { ws, broker } = await setup({ - store: JsonlCallStore.open(storePath), - runner, - maxConcurrentAgents: 3, - }); - await broker.eval('const p1 = agent("pi/x", "t1"); const p2 = agent("pi/y", "t2"); const p3 = agent("pi/z", "t3"); "started"'); - await tick(); - assert.equal(runner.sessions.length, 3); - const snapshot = ws.snapshot(); - await crash(ws, broker); - - // All three calls are lost at the backend. The RESTORED broker runs a - // TIGHTER cap (server configuration can change between processes): two - // re-issues fit, the third QUEUES — it stays PENDING in the guest - // registry (never a ConcurrencyLimitError rejection) and dispatches in - // dispatch order the moment a slot frees (§4.1 queue-above-cap). - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - runner2.failNextLoads = 3; - const broker2 = await Broker.attach(ws2, { - runner: runner2, - store: JsonlCallStore.open(storePath), - maxConcurrentAgents: 2, - snapshotSink: sink, - }); - const report = await broker2.reconcile(); - assert.deepEqual(report.reissued, ['c1', 'c2', 'c3'], 'the over-cap re-issue QUEUED (reported re-issued, never failed-lost)'); - assert.deepEqual(report.failedLost, []); - assert.deepEqual(kinds, [], 'queueing settles nothing — no settlement boundary'); - const pending = await broker2.eval('"probe"'); - assert.ok(pending.pending.includes('c3'), 'the queued re-issue stays PENDING in the guest registry'); - // The store records no completion for c3 — it was never rejected. - assert.equal(broker2.store().lookup('c3')!.completion, null); - // A slot frees (c1's re-issue settles) and the queued re-issue - // dispatches IN ORDER for the free slot. - runner2.sessions[0].completeTurn('done-1'); - await tick(); - await broker2.pump(); - await tick(); - assert.equal(runner2.sessions.length, 3, 'the queued re-issue opened a fresh session once a slot freed'); - assert.equal(broker2.store().lookup('c3')!.reissues, 1); - // c3's guest promise is still the same one — it settles from the - // re-issued turn's answer. - runner2.sessions[2].completeTurn('done-3'); - await tick(); - await broker2.pump(); - const c3 = await broker2.eval('await p3'); - assert.equal(c3.result, 'done-3'); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Checkpoints and steers across a restore -// ──────────────────────────────────────────────────────────────────────── - -test('a pending checkpoint re-surfaces across the restore: listed again, answerable through the surface', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-cp-')); - const storePath = join(dir, 'calls.jsonl'); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath) }); - await broker.eval('const q = checkpoint("What color?"); "raised"'); - const snapshot = ws.snapshot(); - await crash(ws, broker); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const broker2 = await Broker.attach(ws2, { runner: new FakeRunner(), store: JsonlCallStore.open(storePath) }); - const report = await broker2.reconcile(); - assert.deepEqual(report.requeuedCheckpoints, ['c1']); - assert.deepEqual(report.leftPending, []); - - // The question is listed again in the next tool result. - const listed = await broker2.eval('"probe"'); - assert.deepEqual(listed.checkpoints.map((c) => c.id), ['c1']); - assert.equal(listed.checkpoints[0].question, 'What color?'); - - // The answer settles the RESTORED checkpoint (no live GuestCall — - // through the reconciliation surface), exactly once. - const answered = await broker2.eval('checkpoint.answer("c1", "blue"); "delivered"'); - assert.equal(answered.result, 'delivered'); - assert.equal((await broker2.eval('await q')).result, 'blue'); - assert.equal(broker2.store().lookup('c1')!.completion!.value, 'blue'); - // A second answer reports false (first-wins). - assert.equal((await broker2.eval('checkpoint.answer("c1", "again")')).result, 'false'); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('a steer whose wire call died with the process rejects steering_interrupted and is never replayed', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-steer-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await dispatchAgent(broker, runner, 'const pi = agent("pi/x", "task"); "started"'); - // The steering-extension backend: the injected steer's wire call is in - // flight at the crash (its outcome never resolves in this process). - const steered = await broker.eval('const o = await pi.steer("go deeper"); "outcome:" + o'); - assert.equal(steered.result, undefined, 'the injected steer is in flight'); - const snapshot = ws.snapshot(); - await crash(ws, broker); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - runner2.loadedTurnText = 'loaded turn'; - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - const report = await broker2.reconcile(); - assert.deepEqual(report.reattached, ['c1'], 'the founding call re-attaches'); - assert.deepEqual(report.failedLost, ['c2'], 'the in-flight steer rejects without replay'); - const probe = await broker2.eval('"probe"'); - assert.ok(output(probe).some((line) => line.includes('steering_interrupted') || line.includes('interrupted by restart')), output(probe).join('\n')); - const notes = await reconcileNotesOf(broker2); - assert.ok( - notes.some((n) => n.level === 'warn' && n.line.includes('c2') && n.line.includes('not replayed')), - JSON.stringify(notes), - ); - // The interrupted rejection is durable: a second restore settles it - // from the store and never re-injects it. - const interrupted = broker2.store().lookup('c2')!.completion!.value as { details?: { reason?: string } }; - assert.equal(interrupted.details?.reason, 'steering_interrupted'); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -// ──────────────────────────────────────────────────────────────────────── -// The state-changing-boundary cadence -// ──────────────────────────────────────────────────────────────────────── - -test('cadence: a boundary fires after each eval and after each settlement drain that changed VM state — never for an empty drain', async () => { - const kinds: Array<'eval' | 'settlement'> = []; - const sink: SnapshotSink = { - boundary: (kind) => kinds.push(kind), - flush: () => {}, - }; - const runner = new FakeRunner(); - const { ws, broker } = await setup({ snapshotSink: sink, runner }); - - // A plain eval: one eval boundary. - await broker.eval('6 * 7'); - assert.deepEqual(kinds.splice(0), ['eval']); - - // A dispatch eval (suspends): one eval boundary, no settlement. - await broker.eval('const p = agent("pi/x", "task"); "started"'); - await tick(); - assert.deepEqual(kinds.splice(0), ['eval']); - - // A settlement drain that changed VM state: one settlement boundary. - runner.last().completeTurn('done'); - await tick(); - await broker.pump(); - assert.deepEqual(kinds.splice(0), ['settlement']); - - // A pump with nothing ready: drains nothing, fires nothing. - await broker.pump(); - assert.deepEqual(kinds.splice(0), []); - - // An eval whose pump settles a completed call AND runs the eval: both - // boundaries fire, in order (the pump's first). - await broker.eval('const p2 = agent("pi/x", "task2"); "started"'); - assert.deepEqual(kinds.splice(0), ['eval'], 'the dispatch eval fired its own boundary'); - await tick(); - runner.last().completeTurn('second'); - await tick(); - const resumed = await broker.eval('const got = await p2; "resolved:" + got'); - assert.equal(resumed.result, 'resolved:second'); - assert.deepEqual(kinds.splice(0), ['settlement', 'eval']); - - // A reconcile that settles from the store fires the settlement - // boundary (its drain changed VM state). - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-cadence-')); - const storePath = join(dir, 'calls.jsonl'); - const runner2 = new FakeRunner(); - const { ws: ws2, broker: broker2 } = await setup({ snapshotSink: sink, store: JsonlCallStore.open(storePath), runner: runner2 }); - await broker2.eval('const q = agent("pi/x", "t"); "started"'); - assert.deepEqual(kinds.splice(0), ['eval'], 'the dispatch eval fired its own boundary'); - await tick(); - const snapshot = ws2.snapshot(); - await crash(ws2, broker2); - const ws3 = await Workspace.restore(PROJECT, snapshot); - const broker3 = await Broker.attach(ws3, { - runner: new FakeRunner(), - store: JsonlCallStore.open(storePath), - snapshotSink: sink, - }); - broker3.store().recordCompleted('c1', { outcome: 'resolve', value: 'while down', completedAtMs: Date.now() }); - await broker3.reconcile(); - assert.deepEqual(kinds.splice(0), ['settlement'], 'the reconcile drain fired its boundary'); - assert.equal((await broker3.eval('await q')).result, 'while down'); - await broker3.dispose(); - ws3.dispose(); - rmSync(dir, { recursive: true, force: true }); - await ws.dispose(); -}); - -test('debounce end to end: one eval that pumps settlements and runs the eval writes ONE atomic snapshot through the store', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-debounce-')); - const storePath = join(dir, 'calls.jsonl'); - const module = await loadShippedWasm(); - const store = ReplWorkspaceStore.open(PROJECT, { persistenceRoot: dir }); - const ws = await Workspace.create(PROJECT, { wasm: module }); - const runner = new FakeRunner(); - const broker = await Broker.attach(ws, { - runner, - store: JsonlCallStore.open(storePath), - snapshotSink: store.snapshotWriter(ws, module), - }); - - // Eval 1: a dispatch. One write (the eval boundary). - await broker.eval('const p = agent("pi/x", "task"); "started"'); - await tick(); - assert.equal(store.stats().snapshotWrites, 1, 'the eval boundary wrote once'); - - // The worker completes; the NEXT eval both pumps the settlement and - // runs the eval — one drain burst, ONE write (the doc's debounce). - runner.last().completeTurn('result'); - await tick(); - const r = await broker.eval('const got = await p; "resolved:" + got'); - assert.equal(r.result, 'resolved:result'); - assert.equal(store.stats().snapshotWrites, 2, 'the settlement + eval boundaries of one burst coalesced into one write'); - - // A standalone settlement drain writes once (its own burst): the - // dispatch eval wrote its own boundary (3), the pump's drain its own - // (4). - await broker.eval('const q = agent("pi/y", "t2"); "started"'); - await tick(); - runner.last().completeTurn('second'); - await tick(); - await broker.pump(); - assert.equal(store.stats().snapshotWrites, 4, 'the standalone settlement drain wrote once'); - - // The disk snapshot is loadable and restores (the state survives). - const loaded = store.loadSnapshot(module); - const ws2 = await Workspace.restore(PROJECT, loaded.snapshot, { wasm: module }); - assert.equal((await ws2.eval('"state survived"')).kind, 'value'); - ws.dispose(); - ws2.dispose(); - await broker.dispose(); - store.close(); - rmSync(dir, { recursive: true, force: true }); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Reconcile-time refusals and drain failures participate in the cadence -// ──────────────────────────────────────────────────────────────────────── - -test('cadence: a reconcile-time invalid-options refusal settles the guest and fires the settlement boundary', async () => { - const kinds: Array<'eval' | 'settlement'> = []; - const sink: SnapshotSink = { boundary: (kind) => kinds.push(kind), flush: () => {} }; - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-badopts-')); - const storePath = join(dir, 'calls.jsonl'); - - // A foreign-style registry entry with an invalid options bag: park the - // call with the PARKING bridge (no broker validation at dispatch), so - // the snapshot carries a pending agent call whose options the restored - // broker refuses at reconcile time. - const ws = await Workspace.create(PROJECT); - await ws.eval('const p = agent("pi/x", "task", { bogus: 1 }); "started"'); - assert.equal(ws.surface()!.pending().length, 1); - const snapshot = ws.snapshot(); - ws.dispose(); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const broker2 = await Broker.attach(ws2, { - runner: new FakeRunner(), - store: JsonlCallStore.open(storePath), - snapshotSink: sink, - }); - const report = await broker2.reconcile(); - assert.deepEqual(report.failedLost, ['c1'], 'the invalid options refused the re-issue'); - assert.deepEqual(kinds, ['settlement'], 'the refusal settled the guest — its drain fired the boundary (review: refusals used to skip the changed-VM settlement boundary)'); - const probe = await broker2.eval('"probe"'); - assert.deepEqual(output(probe), [], 'the refusal line demotes to diagnostics — never an eval output line'); - const notes = await reconcileNotesOf(broker2); - assert.ok( - notes.some((n) => n.level === 'warn' && n.line.includes('c1') && n.line.includes('invalid options')), - JSON.stringify(notes), - ); - // The refusal is durable and the guest call rejected (never re-issued). - assert.equal(broker2.store().lookup('c1')!.completion!.outcome, 'reject'); - assert.equal(broker2.store().lookup('c1')!.reissues, 0); - const rejected = await broker2.eval('await p.catch((e) => e.message)'); - assert.ok(String(rejected.result).includes('unknown option'), String(rejected.result)); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('cadence: a changed-VM settlement drain that FAILS still fires the settlement boundary, and the reconcile drain failure DEMOTES to workspace().diagnostics (reconcile and pump)', async () => { - // The reconcile arm: the store-arm settlement changed the VM, the drain - // runs the snapshot-carried continuation, and the continuation runs - // away — interrupted by the broker-level handler. The boundary must - // still fire: the settlements landed and the operation-end flush needs - // the dirty boundary to persist them (review regression: an interrupted - // drain used to skip the boundary entirely). The DrainJobError itself - // DEMOTES (§6.2): the settlements landed and will persist — nothing - // was lost — so the reconcile RESOLVES with its report (the first - // touch never fails outside the eval result contract), the failure is - // retained under workspace().diagnostics.drainError, and it never - // rides the next eval's output surface. - const kinds: Array<'eval' | 'settlement'> = []; - const sink: SnapshotSink = { boundary: (kind) => kinds.push(kind), flush: () => {} }; - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-drainfail-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await broker.eval('const p = agent("pi/x", "t"); await p; let i = 0; while (true) i++; "unreachable"'); - await tick(); - const snapshot = ws.snapshot(); - await crash(ws, broker); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const broker2 = await Broker.attach(ws2, { - runner: new FakeRunner(), - store: JsonlCallStore.open(storePath), - snapshotSink: sink, - interruptHandler: () => true, - }); - broker2.store().recordCompleted('c1', { outcome: 'resolve', value: 'while down', completedAtMs: Date.now() }); - const report = await broker2.reconcile(); - assert.deepEqual(report.failedLost, [], 'the store arm settled the call — nothing was lost'); - assert.deepEqual(kinds, ['settlement'], 'the settlement boundary fired despite the failed drain'); - // The settlement itself is durable (the store write precedes the guest - // settle) — a fresh restore settles it from the store arm. - assert.equal(broker2.store().lookup('c1')!.completion!.value, 'while down'); - // §6.2: the interrupted reconcile drain is RETAINED under - // workspace().diagnostics.drainError — never a reconcile rejection, - // never an eval output line. - const diag = await broker2.eval( - 'workspace().diagnostics.drainError === null ? "null" : workspace().diagnostics.drainError.name + ":" + workspace().diagnostics.drainError.message', - ); - assert.ok(String(diag.result).startsWith('InternalError:'), String(diag.result)); - const probe = await broker2.eval('"probe"'); - assert.deepEqual(output(probe), [], 'the reconcile drain failure never rides the eval output surface'); - await broker2.dispose(); - ws2.dispose(); - - // The pump's changed-VM drain failure fires its boundary too (the same - // requirement on the standalone settlement path, pinned here). The - // broker-level interrupt handler applies to the eval as well, so the - // interrupted eval fires its own 'eval' boundary first. - const kinds2: Array<'eval' | 'settlement'> = []; - const sink2: SnapshotSink = { boundary: (kind) => kinds2.push(kind), flush: () => {} }; - const { ws: ws3, broker: broker3, runner: runner3 } = await setup({ - snapshotSink: sink2, - runner: new FakeRunner(), - interruptHandler: () => true, - }); - await broker3.eval('const q = agent("pi/x", "t2"); q.then(() => { let j = 0; while (true) j++; }); "started"'); - await tick(); - runner3.last().completeTurn('final'); - await tick(); - await assert.rejects( - () => broker3.pump(), - (error: unknown) => (error as Error).name === 'DrainJobError', - ); - assert.deepEqual(kinds2, ['eval', 'settlement'], 'the pump\'s settlement boundary fired despite the failed drain'); - await broker3.dispose(); - ws3.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -// ──────────────────────────────────────────────────────────────────────── -// The re-attach arm through the REAL acp-agents adapter -// (a real AcpAgentRunner + InteractiveSession over the fake ACP agent) -// ──────────────────────────────────────────────────────────────────────── - -test('restore through the REAL acp-agents adapter: a completed-while-down call re-attaches and settles from the loaded session\'s replay (no re-issue)', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-real-')); - const storePath = join(dir, 'calls.jsonl'); - // The founding turn's completion is persisted at the backend (the fake - // replays the user prompt + the turn's final message on session/load, - // exactly like a real agent replays its stored conversation). - configureFakeAgent( - { - loadSessionSupport: true, - turns: [{ text: 'initial' }], - loadSession: { - // The _session/loaded_turn extension's authoritative answer: the - // founding turn completed while down — the replay's trailing - // assistant message is its FINAL message. - loadedTurn: { status: 'completed' }, - replay: [ - { role: 'user', text: 'task' }, - { role: 'assistant', text: 'result B (loaded)' }, - ], - }, - }, - join(dir, 'log1.jsonl'), - ); - const runner = new AcpAgentRunner(); - const ws = await Workspace.create(PROJECT); - const broker = await Broker.attach(ws, { runner, store: JsonlCallStore.open(storePath) }); - try { - // Dispatch one call; the founding turn is in flight when we crash. - const r = await broker.eval('const p = agent("claude/x", "task"); "started"'); - assert.equal(r.result, 'started'); - assert.deepEqual(broker.pendingCalls().map((e) => e.id), ['c1']); - // The re-attach key: the REAL backend session id recorded at open - // (the real runner's openSession is async — spawn + initialize). - await waitFor(() => broker.store().lookup('c1')!.sessionId !== null); - const recordedId = broker.store().lookup('c1')!.sessionId!; - assert.ok(recordedId.startsWith('fake-session-'), recordedId); - // Snapshot the live VM and crash before any pump settles the call. - const snapshot = ws.snapshot(); - await broker.dispose(); - ws.dispose(); - - // Restore with a FRESH real runner over the same store; the backend - // serves session/load from its persisted transcript. - configureFakeAgent( - { - loadSessionSupport: true, - turns: [{ text: 'initial' }], - loadSession: { - loadedTurn: { status: 'completed' }, - replay: [ - { role: 'user', text: 'task' }, - { role: 'assistant', text: 'result B (loaded)' }, - ], - }, - }, - join(dir, 'log2.jsonl'), - ); - const runner2 = new AcpAgentRunner(); - const ws2 = await Workspace.restore(PROJECT, snapshot); - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - try { - const report = await broker2.reconcile(); - if (JSON.stringify(report.reattached) !== JSON.stringify(['c1'])) { - throw new Error('REPORTPROBE ' + JSON.stringify(report)); - } - assert.deepEqual(report.reissued, []); - assert.deepEqual(report.failedLost, []); - // The wire evidence: the restored process LOADED the recorded - // session id and never opened a fresh session (no re-issue). - const entries = readWireLog(join(dir, 'log2.jsonl')); - const pids = [...new Set(entries.filter((e) => e.method === '__start').map((e) => e.pid))]; - assert.equal(pids.length, 1, 'exactly one backend process'); - const byPid = entries.filter((e) => e.pid === pids[0]); - assert.ok( - byPid.some((e) => e.method === 'loadSession' && e.params?.sessionId === recordedId), - JSON.stringify(byPid), - ); - assert.ok(!byPid.some((e) => e.method === 'newSession'), 'no fresh session — the call was NOT re-issued'); - // The re-attached call settles with the loaded turn's real outcome, - // exactly once, and the store is authoritative. The seam's stream- - // settled wait is bounded by the settle grace, so the settlement - // lands a moment after reconcile — poll for it like a live call. - let settled: string | undefined; - for (let attempt = 0; attempt < 100; attempt++) { - const got = await broker2.eval('await p.catch((e) => "ERR:" + e.message)'); - if (got.result !== undefined) { - settled = got.result; - break; - } - await new Promise((resolve) => setTimeout(resolve, 25)); - } - assert.equal(settled, 'result B (loaded)'); - assert.equal(broker2.store().lookup('c1')!.completion!.value, 'result B (loaded)'); - assert.equal(broker2.store().lookup('c1')!.reissues, 0, 're-attachment is not a re-issue'); - } finally { - await broker2.dispose(); - ws2.dispose(); - await runner2.dispose(); - } - } finally { - await runner.dispose(); - clearFakeEnv(); - rmSync(dir, { recursive: true, force: true }); - } -}); - -test('restore through the REAL acp-agents adapter: a founding turn still in flight at the backend is KEPT ATTACHED and settles from the authoritative _session/loaded_turn/ended notification — partial output is never settled from a quiet gap, and the still-running turn is never re-issued (duplicated work)', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-real-run-')); - const storePath = join(dir, 'calls.jsonl'); - const prevMax = process.env.AGENTPRISM_ACP_LOADED_TURN_MAX_WAIT_MS; - process.env.AGENTPRISM_ACP_LOADED_TURN_MAX_WAIT_MS = '2000'; - try { - // The replay ends at the founding turn's user message, and the backend - // CONTINUES streaming live chunks AFTER the session/load response — - // the turn is still running at the backend when we reconnect (the - // extension's query answers `running`). The seam keeps the loaded - // session attached (phase-D review: this case used to be released and - // re-issued immediately, risking duplicated work) and NEVER settles - // from a quiet gap: the turn's completion is the AUTHORITATIVE - // `_session/loaded_turn/ended` notification (phase-D review round 3: - // the quiet-grace heuristic and the blind re-issue were both rejected - // — a restored transcript ending in an assistant partial used to be - // durably settled as a completed-while-down turn when the next live - // chunk arrived later, and a still-running backend turn used to be - // re-issued). The call settles with the turn's REAL accumulated text - // at the notification — never the partial, never a duplicate issue. - configureFakeAgent( - { - loadSessionSupport: true, - turns: [{ text: 'fresh result' }], - loadSession: { - loadedTurn: { status: 'running' }, - replay: [{ role: 'user', text: 'task' }], - continue: [ - { afterMs: 50, update: { sessionUpdate: 'agent_message_chunk', messageId: 'm-live', content: { type: 'text', text: 'live ' } } }, - { afterMs: 120, update: { sessionUpdate: 'agent_message_chunk', messageId: 'm-partial', content: { type: 'text', text: 'partial' } } }, - ], - turnEnded: { afterMs: 250, stopReason: 'end_turn' }, - }, - }, - join(dir, 'log1.jsonl'), - ); - const runner = new AcpAgentRunner(); - const ws = await Workspace.create(PROJECT); - const broker = await Broker.attach(ws, { runner, store: JsonlCallStore.open(storePath) }); - try { - await broker.eval('const p = agent("claude/x", "task"); "started"'); - await waitFor(() => broker.store().lookup('c1')!.sessionId !== null); - const recordedId = broker.store().lookup('c1')!.sessionId!; - const snapshot = ws.snapshot(); - await broker.dispose(); - ws.dispose(); - - configureFakeAgent( - { - loadSessionSupport: true, - turns: [{ text: 'fresh result' }], - loadSession: { - loadedTurn: { status: 'running' }, - replay: [{ role: 'user', text: 'task' }], - continue: [ - { afterMs: 50, update: { sessionUpdate: 'agent_message_chunk', messageId: 'm-live', content: { type: 'text', text: 'live ' } } }, - { afterMs: 120, update: { sessionUpdate: 'agent_message_chunk', messageId: 'm-partial', content: { type: 'text', text: 'partial' } } }, - ], - turnEnded: { afterMs: 250, stopReason: 'end_turn' }, - }, - }, - join(dir, 'log2.jsonl'), - ); - const runner2 = new AcpAgentRunner(); - const ws2 = await Workspace.restore(PROJECT, snapshot); - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - try { - // Reconcile returns IMMEDIATELY (it arms the call on the seam - // instead of blocking on the still-running turn) and reports the - // call as re-attached — never re-issued while the turn may still - // be running. - const report = await broker2.reconcile(); - assert.deepEqual(report.reattached, ['c1'], 'the still-running turn stays attached'); - assert.deepEqual(report.reissued, [], 'no re-issue while the seam waits'); - assert.deepEqual(report.failedLost, []); - // Wire evidence: the restored process LOADED the recorded session - // and has not opened a fresh session — and never will (the live - // turn settles from its authoritative terminal notification). - const entries = readWireLog(join(dir, 'log2.jsonl')); - assert.ok( - entries.some((e) => e.method === 'loadSession' && e.params?.sessionId === recordedId), - JSON.stringify(entries), - ); - // The seam's authoritative query fires asynchronously (reconcile - // arms the task and returns) — wait for it on the wire. - await waitFor(() => - readWireLog(join(dir, 'log2.jsonl')).some( - (e) => e.method === 'extensionRequest' && e.extensionMethod === '_session/loaded_turn/query', - ), - ); - // The turn's real accumulated text settles the call exactly once — - // from the authoritative ended notification (the round-3 - // regression: the old seam settled the pause as completion, - // durably recording "live partial" as the call's outcome BEFORE - // the notification, or re-issued the still-running turn). The - // restored result applies the §5 [C]12 message-boundary fold - // like the live fold — the two continuation chunks carry - // DISTINCT messageIds, so they join with "\n\n"; same-message - // deltas (no boundary) would concatenate verbatim (review - // finding: the restored path glued them before the broker - // recorded the result). - let result: string | undefined; - for (let attempt = 0; attempt < 100; attempt++) { - await broker2.pump(); - const got = await broker2.eval('await p.catch((e) => "ERR:" + e.message)'); - if (got.result !== undefined) { - result = got.result; - break; - } - await new Promise((resolve) => setTimeout(resolve, 20)); - } - assert.equal(result, 'live \n\npartial', 'the authoritative completion is the turn\'s REAL accumulated text, folded with the §5 chunk joiner'); - assert.equal(broker2.store().lookup('c1')!.completion!.value, 'live \n\npartial'); - assert.equal(broker2.store().lookup('c1')!.reissues, 0, 'the still-running turn was never re-issued'); - assert.equal(broker2.store().lookup('c1')!.sessionId, recordedId, 'the call settled on the SAME backend session'); - const after = readWireLog(join(dir, 'log2.jsonl')); - assert.ok( - !after.some((e) => e.method === 'newSession'), - 'no fresh session was ever opened — the still-running turn was kept attached, never duplicated: ' + JSON.stringify(after), - ); - } finally { - await broker2.dispose(); - ws2.dispose(); - await runner2.dispose(); - } - } finally { - await broker.dispose(); - ws.dispose(); - await runner.dispose(); - } - } finally { - if (prevMax === undefined) delete process.env.AGENTPRISM_ACP_LOADED_TURN_MAX_WAIT_MS; - else process.env.AGENTPRISM_ACP_LOADED_TURN_MAX_WAIT_MS = prevMax; - clearFakeEnv(); - rmSync(dir, { recursive: true, force: true }); - } -}); - -test('restore through the REAL acp-agents adapter: a founding turn that ended without a terminal message (interrupted while down) degrades to re-issue IMMEDIATELY — the extension answer makes nothing-is-running authoritative', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-real-expire-')); - const storePath = join(dir, 'calls.jsonl'); - try { - // The replay ends at the founding turn's user message and the backend - // answers the extension's query `interrupted`: no turn is running at - // the backend, and the founding turn ended without a terminal - // assistant message while the host was down. Its outcome is not - // observable — but nothing is running, so the seam rejects with the - // SAFE-RE-ISSUE class immediately (no max-wait backstop needed) and - // the broker re-issues under the same id, surfaced guest-visibly. - configureFakeAgent( - { - loadSessionSupport: true, - turns: [{ text: 'fresh result' }], - loadSession: { - loadedTurn: { status: 'interrupted' }, - replay: [{ role: 'user', text: 'task' }], - }, - }, - join(dir, 'log1.jsonl'), - ); - const runner = new AcpAgentRunner(); - const ws = await Workspace.create(PROJECT); - const broker = await Broker.attach(ws, { runner, store: JsonlCallStore.open(storePath) }); - try { - await broker.eval('const p = agent("claude/x", "task"); "started"'); - await waitFor(() => broker.store().lookup('c1')!.sessionId !== null); - const recordedId = broker.store().lookup('c1')!.sessionId!; - const snapshot = ws.snapshot(); - await broker.dispose(); - ws.dispose(); - - configureFakeAgent( - { - loadSessionSupport: true, - turns: [{ text: 'fresh result' }], - loadSession: { - loadedTurn: { status: 'interrupted' }, - replay: [{ role: 'user', text: 'task' }], - }, - }, - join(dir, 'log2.jsonl'), - ); - const runner2 = new AcpAgentRunner(); - const ws2 = await Workspace.restore(PROJECT, snapshot); - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - try { - const report = await broker2.reconcile(); - assert.deepEqual(report.reattached, ['c1'], 'the call is armed on the loaded session first'); - assert.deepEqual(report.reissued, [], 'the in-task degradation is not part of the reconcile report'); - // The in-task re-issue opens a fresh session immediately (the - // `interrupted` answer is authoritative — no max-wait). - await waitFor(() => - readWireLog(join(dir, 'log2.jsonl')).some((e) => e.method === 'newSession'), - ); - const entries = readWireLog(join(dir, 'log2.jsonl')); - assert.ok( - entries.some((e) => e.method === 'loadSession' && e.params?.sessionId === recordedId), - JSON.stringify(entries), - ); - assert.ok( - entries.some((e) => e.method === 'extensionRequest' && e.extensionMethod === '_session/loaded_turn/query'), - JSON.stringify(entries), - ); - assert.ok(entries.some((e) => e.method === 'newSession'), 'the re-issue opened a fresh session'); - // The degradation demotes to diagnostics (§6.2), naming the condition. - const probe = await broker2.eval('"probe"'); - assert.deepEqual(output(probe), [], 'no reconcile line leaks into the eval output'); - const notes = await reconcileNotesOf(broker2); - assert.ok( - notes.some( - (n) => - n.line.includes('c1') && - n.line.includes('ended without a terminal assistant message') && - n.line.includes('re-issued'), - ), - JSON.stringify(notes), - ); - // The re-issued call's fresh turn completes and settles the SAME - // guest promise exactly once. - await waitFor(() => broker2.store().lookup('c1')!.sessionId !== recordedId); - let result: string | undefined; - for (let attempt = 0; attempt < 100; attempt++) { - await broker2.pump(); - const got = await broker2.eval('await p.catch((e) => "ERR:" + e.message)'); - if (got.result !== undefined) { - result = got.result; - break; - } - await new Promise((resolve) => setTimeout(resolve, 20)); - } - assert.equal(result, 'fresh result'); - assert.equal(broker2.store().lookup('c1')!.reissues, 1); - } finally { - await broker2.dispose(); - ws2.dispose(); - await runner2.dispose(); - } - } finally { - await broker.dispose(); - ws.dispose(); - await runner.dispose(); - } - } finally { - clearFakeEnv(); - rmSync(dir, { recursive: true, force: true }); - } -}); - -test('restore through the REAL acp-agents adapter WITHOUT the _session/loaded_turn extension: the observation path classifies a completed-while-down turn from the replay (trailing assistant message, no live continuation) and settles from the loaded session — never a re-issue (phase-F review round 2: the seam-less built-ins\' terminal state is authoritative under the connection-death contract — their ACP servers terminate in-flight turns when the client connection closes, and the replay holds only completed messages)', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-real-seamless-completed-')); - const storePath = join(dir, 'calls.jsonl'); - const prevObserve = process.env.AGENTPRISM_ACP_LOADED_TURN_OBSERVE_MS; - process.env.AGENTPRISM_ACP_LOADED_TURN_OBSERVE_MS = '120'; - try { - configureFakeAgent( - { - loadSessionSupport: true, - turns: [{ text: 'initial' }], - loadSession: { - // NO `loadedTurn` / `turnEnded` / `loadedTurnQueryError`: the - // backend does NOT advertise the extension — the seam-less - // observation path classifies the founding turn. - replay: [ - { role: 'user', text: 'task' }, - { role: 'assistant', text: 'result B (loaded)' }, - ], - }, - }, - join(dir, 'log1.jsonl'), - ); - const runner = new AcpAgentRunner(); - const ws = await Workspace.create(PROJECT); - const broker = await Broker.attach(ws, { runner, store: JsonlCallStore.open(storePath) }); - try { - await broker.eval('const p = agent("claude/x", "task"); "started"'); - await waitFor(() => broker.store().lookup('c1')!.sessionId !== null); - const recordedId = broker.store().lookup('c1')!.sessionId!; - const snapshot = ws.snapshot(); - await broker.dispose(); - ws.dispose(); - - configureFakeAgent( - { - loadSessionSupport: true, - turns: [{ text: 'initial' }], - loadSession: { - replay: [ - { role: 'user', text: 'task' }, - { role: 'assistant', text: 'result B (loaded)' }, - ], - }, - }, - join(dir, 'log2.jsonl'), - ); - const runner2 = new AcpAgentRunner(); - const ws2 = await Workspace.restore(PROJECT, snapshot); - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - try { - const report = await broker2.reconcile(); - assert.deepEqual(report.reattached, ['c1'], 'the call re-attached through the real adapter'); - assert.deepEqual(report.reissued, []); - // The observation path never asks the extension query on the wire - // (there is nothing to ask — the backend did not advertise it). - await new Promise((resolve) => setTimeout(resolve, 50)); - assert.ok( - !readWireLog(join(dir, 'log2.jsonl')).some( - (e) => e.method === 'extensionRequest' && e.extensionMethod === '_session/loaded_turn/query', - ), - 'no extension query — the observation path classifies from the replay + stream', - ); - // The completed-while-down classification settles the call with - // the loaded turn's real outcome (after the observation window), - // exactly once — never a re-issue. - let settled: string | undefined; - for (let attempt = 0; attempt < 100; attempt++) { - const got = await broker2.eval('await p.catch((e) => "ERR:" + e.message)'); - if (got.result !== undefined) { - settled = got.result; - break; - } - await new Promise((resolve) => setTimeout(resolve, 20)); - } - assert.equal(settled, 'result B (loaded)'); - assert.equal(broker2.store().lookup('c1')!.completion!.value, 'result B (loaded)'); - assert.equal(broker2.store().lookup('c1')!.reissues, 0, 're-attachment is not a re-issue'); - assert.equal(broker2.store().lookup('c1')!.sessionId, recordedId, 'settled on the SAME loaded session'); - const entries = readWireLog(join(dir, 'log2.jsonl')); - assert.ok(!entries.some((e) => e.method === 'newSession'), 'no fresh session — the call was NOT re-issued'); - } finally { - await broker2.dispose(); - ws2.dispose(); - await runner2.dispose(); - } - } finally { - await broker.dispose(); - ws.dispose(); - await runner.dispose(); - } - } finally { - if (prevObserve === undefined) delete process.env.AGENTPRISM_ACP_LOADED_TURN_OBSERVE_MS; - else process.env.AGENTPRISM_ACP_LOADED_TURN_OBSERVE_MS = prevObserve; - clearFakeEnv(); - rmSync(dir, { recursive: true, force: true }); - } -}); - -test('restore through the REAL acp-agents adapter WITHOUT the extension: a replay that ends without a terminal assistant message (the turn died mid-way while down) is the INTERRUPTED classification — nothing is running at the backend (the connection-death contract), so the in-task degradation re-issues under the same id, surfaced guest-visibly, and no duplication is possible', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-real-seamless-interrupted-')); - const storePath = join(dir, 'calls.jsonl'); - const prevObserve = process.env.AGENTPRISM_ACP_LOADED_TURN_OBSERVE_MS; - process.env.AGENTPRISM_ACP_LOADED_TURN_OBSERVE_MS = '120'; - try { - configureFakeAgent( - { - loadSessionSupport: true, - turns: [{ text: 'fresh result' }], - loadSession: { - replay: [{ role: 'user', text: 'task' }], - }, - }, - join(dir, 'log1.jsonl'), - ); - const runner = new AcpAgentRunner(); - const ws = await Workspace.create(PROJECT); - const broker = await Broker.attach(ws, { runner, store: JsonlCallStore.open(storePath) }); - try { - await broker.eval('const p = agent("claude/x", "task"); "started"'); - await waitFor(() => broker.store().lookup('c1')!.sessionId !== null); - const recordedId = broker.store().lookup('c1')!.sessionId!; - const snapshot = ws.snapshot(); - await broker.dispose(); - ws.dispose(); - - configureFakeAgent( - { - loadSessionSupport: true, - turns: [{ text: 'fresh result' }], - loadSession: { - replay: [{ role: 'user', text: 'task' }], - }, - }, - join(dir, 'log2.jsonl'), - ); - const runner2 = new AcpAgentRunner(); - const ws2 = await Workspace.restore(PROJECT, snapshot); - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - try { - const report = await broker2.reconcile(); - assert.deepEqual(report.reattached, ['c1'], 'the call is armed on the loaded session first'); - assert.deepEqual(report.reissued, [], 'the in-task degradation is not part of the reconcile report'); - // The interrupted classification (nothing running) degrades to a - // re-issue after the observation window — surfaced guest-visibly. - await waitFor(() => - readWireLog(join(dir, 'log2.jsonl')).some((e) => e.method === 'newSession'), - ); - assert.ok( - !readWireLog(join(dir, 'log2.jsonl')).some( - (e) => e.method === 'extensionRequest' && e.extensionMethod === '_session/loaded_turn/query', - ), - 'no extension query — the observation path classifies from the replay + stream', - ); - const probe = await broker2.eval('"probe"'); - assert.deepEqual(output(probe), [], 'no reconcile line leaks into the eval output'); - const notes = await reconcileNotesOf(broker2); - assert.ok( - notes.some( - (n) => - n.line.includes('c1') && - n.line.includes('without a terminal assistant message') && - n.line.includes('re-issue'), - ), - JSON.stringify(notes), - ); - // The re-issued call's fresh turn completes and settles the SAME - // guest promise exactly once. - await waitFor(() => broker2.store().lookup('c1')!.sessionId !== recordedId); - let result: string | undefined; - for (let attempt = 0; attempt < 100; attempt++) { - await broker2.pump(); - const got = await broker2.eval('await p.catch((e) => "ERR:" + e.message)'); - if (got.result !== undefined) { - result = got.result; - break; - } - await new Promise((resolve) => setTimeout(resolve, 20)); - } - assert.equal(result, 'fresh result'); - assert.equal(broker2.store().lookup('c1')!.reissues, 1); - } finally { - await broker2.dispose(); - ws2.dispose(); - await runner2.dispose(); - } - } finally { - await broker.dispose(); - ws.dispose(); - await runner.dispose(); - } - } finally { - if (prevObserve === undefined) delete process.env.AGENTPRISM_ACP_LOADED_TURN_OBSERVE_MS; - else process.env.AGENTPRISM_ACP_LOADED_TURN_OBSERVE_MS = prevObserve; - clearFakeEnv(); - rmSync(dir, { recursive: true, force: true }); - } -}); - -test('restore through the REAL acp-agents adapter WITHOUT the extension: live continuation within the observation window classifies the founding turn STILL RUNNING — the loaded session stays attached and the call is never re-issued, across the re-armable bound (phase-F review round 2: a possibly-running call is never duplicated)', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-real-seamless-running-')); - const storePath = join(dir, 'calls.jsonl'); - const prevObserve = process.env.AGENTPRISM_ACP_LOADED_TURN_OBSERVE_MS; - const prevMax = process.env.AGENTPRISM_ACP_LOADED_TURN_MAX_WAIT_MS; - process.env.AGENTPRISM_ACP_LOADED_TURN_OBSERVE_MS = '120'; - process.env.AGENTPRISM_ACP_LOADED_TURN_MAX_WAIT_MS = '200'; - try { - configureFakeAgent( - { - loadSessionSupport: true, - turns: [{ text: 'fresh result' }], - loadSession: { - // The replay ends at an assistant PARTIAL, and the backend - // CONTINUES streaming live chunks after the session/load - // response — the turn is still running at the backend when we - // reconnect. WITHOUT the extension, the observation path sees - // the live continuation within its window and classifies the - // turn STILL RUNNING: the seam keeps the loaded session - // attached (never settles the quiet gap, never re-issues), and - // the re-armable bound keeps the wait live. - replay: [ - { role: 'user', text: 'task' }, - { role: 'assistant', text: 'partial ' }, - ], - continue: [ - { afterMs: 40, update: { sessionUpdate: 'agent_message_chunk', content: { type: 'text', text: 'live ' } } }, - { afterMs: 90, update: { sessionUpdate: 'agent_message_chunk', content: { type: 'text', text: 'continuation' } } }, - ], - }, - }, - join(dir, 'log1.jsonl'), - ); - const runner = new AcpAgentRunner(); - const ws = await Workspace.create(PROJECT); - const broker = await Broker.attach(ws, { runner, store: JsonlCallStore.open(storePath) }); - try { - await broker.eval('const p = agent("claude/x", "task"); "started"'); - await waitFor(() => broker.store().lookup('c1')!.sessionId !== null); - const recordedId = broker.store().lookup('c1')!.sessionId!; - const snapshot = ws.snapshot(); - await broker.dispose(); - ws.dispose(); - - configureFakeAgent( - { - loadSessionSupport: true, - turns: [{ text: 'fresh result' }], - loadSession: { - replay: [ - { role: 'user', text: 'task' }, - { role: 'assistant', text: 'partial ' }, - ], - continue: [ - { afterMs: 40, update: { sessionUpdate: 'agent_message_chunk', content: { type: 'text', text: 'live ' } } }, - { afterMs: 90, update: { sessionUpdate: 'agent_message_chunk', content: { type: 'text', text: 'continuation' } } }, - ], - }, - }, - join(dir, 'log2.jsonl'), - ); - const runner2 = new AcpAgentRunner(); - const ws2 = await Workspace.restore(PROJECT, snapshot); - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - try { - const report = await broker2.reconcile(); - assert.deepEqual(report.reattached, ['c1'], 'the still-running call is re-attached and KEPT attached'); - assert.deepEqual(report.reissued, [], 'never a re-issue while the loaded session is attached'); - assert.deepEqual(report.failedLost, []); - // Across the observation window AND two max-wait re-arm cycles: - // the call stays pending on the SAME loaded session — no fresh - // session is ever opened, and the partial is never settled. - await new Promise((resolve) => setTimeout(resolve, 700)); - assert.deepEqual(broker2.pendingCalls().map((e) => e.id), ['c1'], 'the call is still pending — the partial was never settled'); - const entries = readWireLog(join(dir, 'log2.jsonl')); - assert.ok( - !entries.some((e) => e.method === 'newSession'), - 'no fresh session across the re-arm cycles — a possibly-running call is never duplicated: ' + JSON.stringify(entries), - ); - assert.equal( - entries.filter((e) => e.method === 'loadSession' && e.params?.sessionId === recordedId).length, - 1, - 'the recorded session was loaded exactly once — the re-arms ride the SAME attached session', - ); - assert.ok( - !readWireLog(join(dir, 'log2.jsonl')).some( - (e) => e.method === 'extensionRequest' && e.extensionMethod === '_session/loaded_turn/query', - ), - 'no extension query — the observation path classifies from the replay + stream', - ); - } finally { - await broker2.dispose(); - ws2.dispose(); - await runner2.dispose(); - } - } finally { - await broker.dispose(); - ws.dispose(); - await runner.dispose(); - } - } finally { - if (prevObserve === undefined) delete process.env.AGENTPRISM_ACP_LOADED_TURN_OBSERVE_MS; - else process.env.AGENTPRISM_ACP_LOADED_TURN_OBSERVE_MS = prevObserve; - if (prevMax === undefined) delete process.env.AGENTPRISM_ACP_LOADED_TURN_MAX_WAIT_MS; - else process.env.AGENTPRISM_ACP_LOADED_TURN_MAX_WAIT_MS = prevMax; - clearFakeEnv(); - rmSync(dir, { recursive: true, force: true }); - } -}); - -test('a still-running-at-load call stays attached: reconcile arms it on the parked seam and the call settles from the turn\'s later completion (never a re-issue)', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-still-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await dispatchAgent(broker, runner); - const recordedId = broker.store().lookup('c1')!.sessionId!; - const snapshot = ws.snapshot(); - await crash(ws, broker); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - // No scripted loaded-turn outcome: the seam parks — the founding turn is - // still running at the backend (live chunks keep streaming after the - // load response). - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - const report = await broker2.reconcile(); - assert.deepEqual(report.reattached, ['c1'], 'the still-running call is re-attached and KEPT attached'); - assert.deepEqual(report.reissued, [], 'never a re-issue while the loaded session is attached'); - assert.deepEqual(report.failedLost, []); - assert.equal(runner2.loadedWith.length, 1); - assert.equal( - runner2.loadedWith[0].sessionId, - recordedId, - 'the load was addressed at the RECORDED backend session (the fake mints a fresh id per load, the real adapter keeps the loaded id)', - ); - // The call is still pending — the seam observes the turn's completion. - assert.deepEqual(broker2.pendingCalls().map((e) => e.id), ['c1']); - // The backend finishes the turn: the parked seam resolves with the real - // accumulated text, the pump delivers it, and the guest settles exactly - // once — no duplicate dispatch ever happened. - const parked = runner2.sessions[0].loadedTurns.shift(); - assert.ok(parked, 'the seam is parked on the loaded session'); - parked.resolve({ stopReason: 'end_turn', text: 'completed live' }); - await tick(); - await broker2.pump(); - assert.equal((await broker2.eval('await p')).result, 'completed live'); - assert.equal(broker2.store().lookup('c1')!.completion!.value, 'completed live'); - assert.equal(broker2.store().lookup('c1')!.reissues, 0, 'kept attached — no re-issue'); - assert.equal(broker2.store().lookup('c1')!.sessionId, recordedId); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('a restored turn failure rejects with its resolved backend attribution', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-turn-failed-backend-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await dispatchAgent(broker, runner); - const snapshot = ws.snapshot(); - await crash(ws, broker); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - const broker2 = await Broker.attach(ws2, { - runner: runner2, - store: JsonlCallStore.open(storePath), - }); - await broker2.reconcile(); - const parked = runner2.sessions[0].loadedTurns.shift(); - assert.ok(parked, 'the restored founding turn is being observed'); - parked.reject(new LoadedTurnFailedError('restored turn failed at the backend')); - await tick(); - await broker2.pump(); - - const record = broker2.store().lookup('c1')!; - assert.equal(record.completion!.outcome, 'reject'); - assert.equal((record.completion!.value as { replBackend?: string }).replBackend, 'pi'); - assert.equal((await broker2.eval('await p.catch((e) => e.replBackend + "/" + e.replCallId)')).result, 'pi/c1'); - assert.equal(record.reissues, 0, 'a definitive backend failure is never re-issued'); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('a seam rejection degrades to re-issue inside the task: the loaded session is released and the fresh turn settles the SAME guest promise', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-seamreject-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await dispatchAgent(broker, runner); - const recordedId = broker.store().lookup('c1')!.sessionId!; - const snapshot = ws.snapshot(); - await crash(ws, broker); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - const report = await broker2.reconcile(); - assert.deepEqual(report.reattached, ['c1'], 'armed on the loaded session first'); - assert.deepEqual(report.reissued, [], 'the in-task degradation is not part of the reconcile report'); - // The seam rejects — the founding turn's outcome is genuinely - // unobservable (e.g. the stream settled without a terminal assistant - // message within the max-wait bound). - const parked = runner2.sessions[0].loadedTurns.shift(); - assert.ok(parked, 'the seam is parked on the loaded session'); - parked.reject(new Error('the loaded session\'s founding turn never reached a terminal assistant message')); - await tick(); - await tick(); - // The loaded session was released and a FRESH session opened for the - // re-issue; the store bumped the reissues counter. - assert.equal(runner2.sessions[0].releases, 1, 'the loaded session was released'); - assert.equal(runner2.sessions.length, 2, 'a fresh session opened for the re-issue'); - assert.equal(broker2.store().lookup('c1')!.reissues, 1); - assert.equal(broker2.store().lookup('c1')!.sessionId, runner2.sessions[1].sessionId, 'the re-issue\'s session is the new attach key'); - // The degradation demotes to diagnostics (§6.2), naming the reason. - const probe = await broker2.eval('"probe"'); - assert.deepEqual(output(probe), [], 'no reconcile line leaks into the eval output'); - const notes = await reconcileNotesOf(broker2); - assert.ok( - notes.some((n) => n.line.includes('c1') && n.line.includes('re-issued') && n.line.includes('released')), - JSON.stringify(notes), - ); - // The re-issued call's fresh turn completes and settles the SAME guest - // promise exactly once. - runner2.sessions[1].completeTurn('fresh result'); - await tick(); - await broker2.pump(); - assert.equal((await broker2.eval('await p')).result, 'fresh result'); - assert.equal(broker2.store().lookup('c1')!.completion!.value, 'fresh result'); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('a safe loaded-turn reissue whose release parks past the disconnect bound is HELD — no reissue recorded, no fresh child opens after the broker reported drained (late-resolving-release regression)', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-laterelease-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await dispatchAgent(broker, runner); - const snapshot = ws.snapshot(); - await crash(ws, broker); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - const report = await broker2.reconcile(); - assert.deepEqual(report.reattached, ['c1'], 'armed on the loaded session first'); - - // The seam rejects with a SAFE re-issue class error, and the loaded - // session's release PARKS (a hung backend release): the re-issue is - // now blocked inside `reissueReattached`'s awaited release. - const loaded = runner2.sessions[0]; - const parkedSeam = loaded.loadedTurns.shift(); - assert.ok(parkedSeam, 'the seam is parked on the loaded session'); - // Every release invocation parks on its own resolver — the drain's - // release phase calls `release()` a second time, so the re-issue - // task's await and the drain's bounded release share the parking - // family; all parks are released together later. - const releaseResolvers: Array<() => void> = []; - loaded.release = () => { - loaded.releases++; - return new Promise((resolve) => { - releaseResolvers.push(resolve); - }); - }; - parkedSeam.reject(new Error("the loaded session's founding turn never reached a terminal assistant message")); - await tick(); - await tick(); - assert.equal(runner2.sessions.length, 1, 'no fresh session yet — the re-issue is parked in the release'); - assert.equal(loaded.releases, 1, 'the release was issued and parked'); - - // The disconnect drain runs while the release is parked: the bound - // expires (the call is still pending on the busy entry), the forced - // stop settles the call DURABLY as AGENT_CANCELLED (recorded first, - // settled into the guest), and the drain reports drained. - const started = Date.now(); - const drained = await broker2.drainForDisconnect(120); - assert.equal(drained, false, 'the bound is the honest outcome — the call could not drain'); - assert.ok(Date.now() - started < 2000, 'the drain returned at its bound (never awaited the parked release)'); - assert.ok(broker2.isDrained); - assert.deepEqual(broker2.pendingCalls().map((e) => e.id), [], 'the forced stop settled the call'); - const record = broker2.store().lookup('c1')!; - assert.equal(record.completion!.outcome, 'reject'); - assert.equal((record.completion!.value as { code?: string }).code, 'AGENT_CANCELLED'); - assert.equal((record.completion!.value as { replBackend?: string }).replBackend, 'pi'); - - // The parked release resolves LATE, after the drain reported drained. - // The re-issue must NOT proceed: no reissue recorded, no fresh session, - // no prompt — a fresh child must never open after the broker reported - // drained (phase-D review rejection: the fence was checked only BEFORE - // the awaited release, so a release that parked past the bound resumed - // into a post-drain re-issue that recorded a reissue and opened/prompted - // a new child). - for (const resolve of releaseResolvers) resolve(); - await tick(); - await tick(); - assert.equal(runner2.sessions.length, 1, 'no fresh session opened after the drain'); - assert.ok( - runner2.sessions.every((s) => s.prompts.length === 0), - 'no new child prompted after the drain', - ); - assert.equal(broker2.store().lookup('c1')!.reissues, 0, 'no reissue was recorded'); - assert.equal(record.completion!.outcome, 'reject', 'the call stays as the drain settled it'); - assert.equal((record.completion!.value as { code?: string }).code, 'AGENT_CANCELLED'); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -// ──────────────────────────────────────────────────────────────────────── -// The re-issue branches' refusal cadence (phase-D review round 2) -// ──────────────────────────────────────────────────────────────────────── - -test('cadence: a no-recorded-session re-issue QUEUED by the cap stays pending and dispatches when the slot frees — no refusal, no premature settlement boundary', async () => { - const kinds: Array<'eval' | 'settlement'> = []; - const sink: SnapshotSink = { boundary: (kind) => kinds.push(kind), flush: () => {} }; - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-nosess-')); - const storePath = join(dir, 'calls.jsonl'); - - // Two pending agent calls with NO recorded backend session (parked with - // the parking bridge — the store never saw them; reconcile adopts them, - // the sessionId stays null → the re-issue arm). The restored broker runs - // cap=1: the first re-issue takes the slot, the second QUEUES in - // dispatch order — it stays PENDING (never a ConcurrencyLimitError - // rejection), so nothing settles and no settlement boundary fires at - // reconcile (§4.1 queue-above-cap; review: the old refusal settled the - // guest mid-reconcile). - const ws = await Workspace.create(PROJECT); - await ws.eval('const p1 = agent("pi/x", "t1"); const p2 = agent("pi/y", "t2"); "started"'); - assert.equal(ws.surface()!.pending().length, 2); - const snapshot = ws.snapshot(); - ws.dispose(); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - const broker2 = await Broker.attach(ws2, { - runner: runner2, - store: JsonlCallStore.open(storePath), - maxConcurrentAgents: 1, - snapshotSink: sink, - }); - const report = await broker2.reconcile(); - assert.deepEqual(report.reissued, ['c1', 'c2'], 'the over-cap re-issue QUEUED (reported re-issued)'); - assert.deepEqual(report.failedLost, []); - assert.deepEqual(kinds, [], 'queueing settles nothing — no settlement boundary at reconcile'); - const record = broker2.store().lookup('c2')!; - assert.equal(record.completion, null, 'the queued re-issue was never rejected'); - // The slot frees and the queued re-issue dispatches in order. - runner2.sessions[0].completeTurn('done-1'); - await tick(); - await broker2.pump(); - await tick(); - assert.equal(runner2.sessions.length, 2, 'the queued re-issue opened a fresh session'); - assert.equal(broker2.store().lookup('c2')!.reissues, 1); - runner2.sessions[1].completeTurn('done-2'); - await tick(); - await broker2.pump(); - const p2 = await broker2.eval('await p2'); - assert.equal(p2.result, 'done-2', 'the queued re-issue settled the SAME guest promise'); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('cadence: an adapter without the completion seam degrades through the doc\'s RE-ISSUE fallback — the loaded sessions are released, the calls are re-issued under the same ids (never settled from a quiet gap, never left pending), surfaced guest-visibly', async () => { - const kinds: Array<'eval' | 'settlement'> = []; - const sink: SnapshotSink = { boundary: (kind) => kinds.push(kind), flush: () => {} }; - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-noseam-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await broker.eval('const p1 = agent("pi/x", "t1"); const p2 = agent("pi/y", "t2"); "started"'); - await tick(); - assert.equal(runner.sessions.length, 2); - const snapshot = ws.snapshot(); - await crash(ws, broker); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - runner2.seamless = true; // a third-party adapter: loads fine, no completion seam - const broker2 = await Broker.attach(ws2, { - runner: runner2, - store: JsonlCallStore.open(storePath), - snapshotSink: sink, - }); - const report = await broker2.reconcile(); - assert.deepEqual(report.reattached, [], 'no call stays attached without the seam'); - assert.deepEqual(report.reissued, ['c1', 'c2'], 'both calls degrade to re-issue — the doc\'s honest fallback for a capability-omitting backend'); - assert.deepEqual(report.failedLost, []); - assert.equal(runner2.loadedWith.length, 2, 'both sessions loaded — the seam absence is discovered after a successful load'); - // The degradation demotes to diagnostics (§6.2) — never the eval result surface. - const probe = await broker2.eval('"probe"'); - assert.deepEqual(output(probe), [], 'no reconcile lines leak into the eval output'); - const notes = await reconcileNotesOf(broker2); - assert.ok( - notes.filter((n) => n.line.includes('re-issued')).length >= 2 && - notes.some((n) => n.line.includes('c1') && n.line.includes('not observable') && n.line.includes('re-issued')), - JSON.stringify(notes), - ); - assert.equal(runner2.sessions[0].releases, 1, 'c1\'s seam-less loaded session was released before the re-issue'); - assert.equal(runner2.sessions[2].releases, 1, 'c2\'s seam-less loaded session was released before the re-issue'); - // The re-issue opened a FRESH session per call (the loaded ones are gone): - // [0] c1 loaded, [1] c1 fresh, [2] c2 loaded, [3] c2 fresh. - assert.equal(runner2.sessions.length, 4, '2 loaded + 2 fresh re-issue sessions'); - assert.equal(broker2.store().lookup('c1')!.reissues, 1, 'the reissue was recorded'); - // The calls settle through the fresh dispatch exactly once (phase-F - // review: the old unobservable degradation left them pending until - // interrupt/reset). - assert.deepEqual(broker2.pendingCalls().map((e) => e.id), ['c1', 'c2'], 'the re-issued calls are tracked pending on their fresh turns'); - runner2.sessions[1].completeTurn('fresh outcome 1'); - runner2.sessions[3].completeTurn('fresh outcome 2'); - await tick(); - await broker2.pump(); - assert.equal((await broker2.eval('await p1')).result, 'fresh outcome 1'); - assert.equal((await broker2.eval('await p2')).result, 'fresh outcome 2'); - assert.deepEqual(broker2.pendingCalls().map((e) => e.id), [], 'both continuations settled exactly once'); - assert.equal(broker2.store().lookup('c1')!.completion!.value, 'fresh outcome 1'); - assert.equal(broker2.store().lookup('c2')!.completion!.value, 'fresh outcome 2'); - assert.ok(kinds.includes('settlement'), 'the settlements fired the state-changing boundary: ' + JSON.stringify(kinds)); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -// ──────────────────────────────────────────────────────────────────────── -// The re-attach arm's still-running degradation (phase-D review round 3) -// ──────────────────────────────────────────────────────────────────────── - -test('a RE-ARMABLE still-running seam rejection keeps the call attached and pending: the broker re-arms the seam on the SAME session, never settles a quiet gap, never re-issues — and a later authoritative completion settles the call exactly once', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-rearm-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await dispatchAgent(broker, runner); - const recordedId = broker.store().lookup('c1')!.sessionId!; - const snapshot = ws.snapshot(); - await crash(ws, broker); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - const report = await broker2.reconcile(); - assert.deepEqual(report.reattached, ['c1']); - // The seam rejects with the re-armable still-running class (a `running` - // turn whose terminal notification did not arrive within the max-wait - // bound): the broker retains the line under diagnostics (§6.2), KEEPS - // the loaded session attached, and re-arms the seam — the call is - // never settled from a quiet gap and never re-issued while the - // backend turn may still be running. - const firstSeam = runner2.sessions[0].loadedTurns.shift(); - assert.ok(firstSeam, 'the seam is parked on the loaded session'); - firstSeam.reject(new LoadedTurnStillRunningError('the loaded session\'s founding turn is still running at the backend', true)); - await tick(); - await tick(); - assert.equal(runner2.sessions.length, 1, 'no fresh session — the still-running call was NOT re-issued'); - assert.equal(broker2.store().lookup('c1')!.reissues, 0, 'reissues counter untouched'); - assert.deepEqual(broker2.pendingCalls().map((e) => e.id), ['c1'], 'the call stays pending'); - const probe = await broker2.eval('"probe"'); - assert.deepEqual(output(probe), [], 'no reconcile line leaks into the eval output'); - const notes = await reconcileNotesOf(broker2); - assert.ok( - notes.some((n) => n.line.includes('c1') && n.line.includes('still running')), - JSON.stringify(notes), - ); - // The seam was re-armed on the SAME loaded session: a later - // authoritative completion still settles the call exactly once. - const secondSeam = runner2.sessions[0].loadedTurns.shift(); - assert.ok(secondSeam, 'the seam was re-armed on the still-attached session'); - secondSeam.resolve({ stopReason: 'end_turn', text: 'completed eventually' }); - await tick(); - await broker2.pump(); - assert.equal((await broker2.eval('await p')).result, 'completed eventually'); - assert.equal(broker2.store().lookup('c1')!.completion!.value, 'completed eventually'); - assert.equal(broker2.store().lookup('c1')!.reissues, 0, 'kept attached — no re-issue'); - assert.equal(broker2.store().lookup('c1')!.sessionId, recordedId, 'settled on the SAME backend session'); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('a NON-re-armable still-running seam rejection is NEVER re-invoked — the broker keeps the loaded session ATTACHED and waits for the terminal state from the session-level surfaces: a later ended notification settles the call exactly once, and a cancel settles it as the recoverable AGENT_CANCELLED (phase-F review round 3: the old immediate recursive re-arm spun in an unbounded microtask/warning loop, starving cancellation, drain, and every other task)', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-nonrearm-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await dispatchAgent(broker, runner); - const recordedId = broker.store().lookup('c1')!.sessionId!; - const snapshot = ws.snapshot(); - await crash(ws, broker); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - const report = await broker2.reconcile(); - assert.deepEqual(report.reattached, ['c1']); - const seam = runner2.sessions[0].loadedTurns.shift(); - assert.ok(seam, 'the seam is parked on the loaded session'); - seam.reject( - new LoadedTurnStillRunningError( - 'a third-party seam that can never observe the terminal state', - false, - ), - ); - await tick(); - await tick(); - // Phase-F review round 3: a possibly-running call is NEVER re-issued — - // the loaded session stays attached (no release, no fresh session, no - // reissue record) — and the NON-re-armable seam is NOT re-invoked (an - // immediate recursive re-arm would spin in an unbounded - // microtask/warning loop, starving cancellation, drain, and every - // other task). The call is still pending — partial output is never - // settled — and the broker now waits for the terminal state from the - // remaining authority surfaces below. - assert.equal(runner2.sessions[0].releases, 0, 'the loaded session stays attached'); - assert.equal(runner2.sessions.length, 1, 'no fresh session — no re-issue'); - assert.equal(broker2.store().lookup('c1')!.reissues, 0, 'no reissue was recorded'); - assert.equal(broker2.store().lookup('c1')!.sessionId, recordedId, 'the attach key is unchanged'); - assert.deepEqual(broker2.pendingCalls().map((e) => e.id), ['c1'], 'the call is still pending — partial output is never settled'); - assert.equal(runner2.sessions[0].loadedTurns.length, 0, 'the seam was NOT re-invoked — no loop'); - const probe = await broker2.eval('"probe"'); - assert.deepEqual(output(probe), [], 'no reconcile line leaks into the eval output'); - const notes = await reconcileNotesOf(broker2); - assert.ok( - notes.some((n) => n.line.includes('c1') && n.line.includes('can never observe')), - JSON.stringify(notes), - ); - // The SESSION-LEVEL ended notification settles the call (a seam-less - // backend that pushes `_session/loaded_turn/ended` anyway) with the - // turn's real accumulated text — exactly once. - runner2.sessions[0].fireLoadedTurnEnded({ stopReason: 'end_turn' }, 'the turn eventually completed'); - await tick(); - await broker2.pump(); - const settleProbe = await broker2.eval('await p'); - assert.equal(settleProbe.result, 'the turn eventually completed'); - assert.equal(broker2.store().lookup('c1')!.completion!.value, 'the turn eventually completed'); - assert.deepEqual(broker2.pendingCalls().map((e) => e.id), [], 'the continuation settled exactly once'); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('a NON-re-armable still-running seam rejection settles on a CANCEL as the recoverable AGENT_CANCELLED — the interrupt tool works on a held call (phase-F review round 3: the old re-arm loop never yielded, so cancellation could not reach the held call)', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-nonrearm-cancel-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await dispatchAgent(broker, runner); - const snapshot = ws.snapshot(); - await crash(ws, broker); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - await broker2.reconcile(); - const seam = runner2.sessions[0].loadedTurns.shift(); - assert.ok(seam); - seam.reject( - new LoadedTurnStillRunningError( - 'a third-party seam that can never observe the terminal state', - false, - ), - ); - await tick(); - await tick(); - // The held call is cancelable: the interrupt tool's wire cancel flips - // the entry's cancel flag, the non-re-armable wait observes it, and - // the call settles as the recoverable AGENT_CANCELLED (recorded first, - // delivered by the pump) — never left pending until the drain. - const outcome = await broker2.cancelCall('c1'); - assert.equal(outcome, 'cancelled', 'the interrupt tool reports the cancel'); - await tick(); - await broker2.pump(); - assert.deepEqual(broker2.pendingCalls().map((e) => e.id), [], 'the call settled — not left pending'); - const record = broker2.store().lookup('c1')!; - assert.equal(record.completion!.outcome, 'reject'); - assert.equal((record.completion!.value as { code?: string }).code, 'AGENT_CANCELLED', 'the recoverable cancel code'); - assert.equal((record.completion!.value as { recoverable?: boolean }).recoverable, true); - assert.equal((record.completion!.value as { replBackend?: string }).replBackend, 'pi'); - assert.equal(record.reissues, 0, 'never re-issued'); - assert.equal(runner2.sessions[0].releases, 0, 'the loaded session stays attached (the cancel did not release it)'); - // The guest promise rejected with the recoverable cancellation (a - // later eval reads it; the workspace stays live). - let result: string | undefined; - for (let attempt = 0; attempt < 100; attempt++) { - const got = await broker2.eval('await p.catch((e) => "ERR:" + e.message)'); - if (got.result !== undefined) { - result = got.result; - break; - } - await new Promise((resolve) => setTimeout(resolve, 10)); - } - assert.ok(result?.includes('c1 was cancelled'), `guest-visible settlement: ${result}`); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('a released reattached session is lane-fatal and is never replaced with a blank session', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-nonrearm-release-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await dispatchAgent(broker, runner); - const snapshot = ws.snapshot(); - await crash(ws, broker); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - await broker2.reconcile(); - const seam = runner2.sessions[0].loadedTurns.shift(); - assert.ok(seam); - seam.reject( - new LoadedTurnStillRunningError( - 'a third-party seam that can never observe the terminal state', - false, - ), - ); - await tick(); - await tick(); - assert.equal(runner2.sessions[0].loadedTurns.length, 0, 'the seam was NOT re-invoked — no loop'); - // The loaded session's dedicated process dies: session/process loss is - // lane-fatal. The broker must not continue the conversation on a blank replacement. - await runner2.sessions[0].release(); - await tick(); - await broker2.pump(); - const record = broker2.store().lookup('c1')!; - assert.equal(record.reissues, 0, 'session loss never opens a replacement session'); - assert.equal(runner2.sessions.length, 1, 'no fresh session was opened'); - assert.equal(record.completion!.outcome, 'reject'); - assert.equal((record.completion!.value as { details?: { reason?: string } }).details?.reason, 'session_released'); - assert.deepEqual(broker2.pendingCalls().map((e) => e.id), [], 'the fatal call settled exactly once'); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('a still-running seam rejection during the client-presence drain is STOPPED at the bound — the call settles as the recoverable AGENT_CANCELLED (never a re-issue after the last client disconnected, and never an orphaned pending call)', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-drainhold-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await dispatchAgent(broker, runner); - const snapshot = ws.snapshot(); - await crash(ws, broker); - - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - await broker2.reconcile(); - // The drain starts while the seam is parked (the turn is still running - // at the backend). The seam rejects with the plain released-class error - // mid-drain — and because the broker is draining, the rejection HOLDS - // the call (a re-issue would open a fresh child after the last client - // disconnected). The unobservable turn cannot drain, so the bound is - // the honest outcome: the drain cancels what it caught, then settles - // the still-pending call with the recoverable AGENT_CANCELLED (phase-D - // review round 6: the hold used to leave the call pending forever — - // orphaned by the release phase's bookkeeping clear, uncancelable - // except by reset, because reconcile never runs again on a live - // workspace). - const seam = runner2.sessions[0].loadedTurns.shift(); - assert.ok(seam); - const draining = broker2.drainForDisconnect(400); - await tick(); - seam.reject(new Error('InteractiveSession has been released while awaiting the loaded session\'s founding turn')); - assert.equal(await draining, false, 'the unobservable turn cannot drain — the bound is the honest outcome'); - assert.ok(broker2.isDrained); - assert.deepEqual(broker2.pendingCalls().map((e) => e.id), [], 'the call is NOT orphaned — the bound\'s forced stop settled it'); - const record = broker2.store().lookup('c1')!; - assert.equal(record.completion!.outcome, 'reject'); - assert.equal((record.completion!.value as { code?: string }).code, 'AGENT_CANCELLED', 'the recoverable forced-stop code'); - assert.equal((record.completion!.value as { recoverable?: boolean }).recoverable, true); - assert.equal(record.reissues, 0, 'never re-issued'); - assert.equal(runner2.sessions[0].releases, 1, 'the child closed in the release phase'); - // The guest promise settled with the recoverable error (a later eval - // reads it — the workspace stays live after the drain). - let result: string | undefined; - for (let attempt = 0; attempt < 100; attempt++) { - const got = await broker2.eval('await p.catch((e) => "ERR:" + e.message)'); - if (got.result !== undefined) { - result = got.result; - break; - } - await new Promise((resolve) => setTimeout(resolve, 10)); - } - assert.ok(result?.includes('turn c1 was cancelled'), `guest-visible settlement: ${result}`); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('the drain bound is ABSOLUTE: a hung cancel/release cannot block disconnect past the deadline (the session-eviction TTL is the outer ceiling)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - await broker.eval('const p = agent("pi/x", "task"); "started"'); - await tick(); - // A backend whose cancel AND release hang forever (the worst case the - // review flagged: the post-deadline awaits used to block indefinitely). - const session = runner.sessions[0]; - session.hangCancel = true; - session.hangRelease = true; - const started = Date.now(); - const drained = await broker.drainForDisconnect(100); - const elapsed = Date.now() - started; - assert.equal(drained, false, 'the bound expired with the turn still running'); - // TIGHT ceiling (phase-D review round 7: this used to permit a 100 ms - // drain to take nearly 3 seconds — it did not enforce the required - // ceiling). The bound is absolute: the drain returns at the deadline - // plus timer slop, never a fresh window after it. - assert.ok(elapsed < 500, `the drain returned within the bound, not blocked by the hung backend: ${elapsed} ms`); - assert.ok(broker.isDrained); - await broker.dispose(); - ws.dispose(); -}); - -test('the drain bound is measured from METHOD ENTRY: a drain queued behind a long serialized operation skips straight to the forced stop instead of running a fresh window after the queue wait (phase-D review round 7)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - await broker.eval('const p = agent("pi/x", "task"); "started"'); - await tick(); - // A hung backend: cancel AND release never resolve (the worst case). - const session = runner.sessions[0]; - session.hangCancel = true; - session.hangRelease = true; - // Hold the broker's serialized chain with a guest busy-loop eval for - // ~400 ms (the eval's op runs synchronously, so the drain queued - // behind it cannot start until the loop exits). The drain's bound - // must be measured from METHOD ENTRY — a deadline already past at - // chain acquisition skips straight to the forced stop. The old code - // started the clock inside the serialized closure: the drain then ran - // a fresh ~120 ms window AFTER the 400 ms queue wait (~520 ms total). - const started = Date.now(); - const evalP = broker.eval('const t = Date.now(); while (Date.now() - t < 400) {} "slow"'); - const drainP = broker.drainForDisconnect(120); - await evalP; - const drained = await drainP; - const elapsed = Date.now() - started; - assert.equal(drained, false, 'the turn never completed — the bound is the honest outcome'); - assert.ok(broker.isDrained); - // 400 ms queue wait + a margin: a drain that ran a fresh full window - // after the queue wait would land well past this. - assert.ok(elapsed < 480, `the bound was measured from method entry, not after the queue wait: ${elapsed} ms`); - await broker.dispose(); - ws.dispose(); -}); - -test('the drain bound is ABSOLUTE against the CHAIN WAIT: a YIELDFUL queued operation (a long wait op polling a pending call — async, never blocking the event loop) cannot delay the drain past its deadline — the chain acquisition races the remaining bound (phase-D review round 8)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - await broker.eval('const p = agent("pi/x", "task"); "started"'); - await tick(); - // A hung backend: cancel AND release never resolve (the worst case). - const session = runner.sessions[0]; - session.hangCancel = true; - session.hangRelease = true; - // A YIELDFUL op holds the serialized chain: a wait with a long timeout - // and a pending call loops (pump + sleep + re-poll) without blocking - // the event loop. The round-7 chain wait had NO deadline race — a - // stuck queued op of this kind delayed the drain until ITS OWN - // timeout (indefinitely for an op that never terminates), i.e. the - // 120 ms drain could return ~10 s later. The synchronous busy-loop - // regression above cannot exercise this: a synchronous eval blocks - // the event loop, so no timer can fire until it yields — the - // yieldful case is the one the absolute bound must actually win. - const waiting = broker.waitForCalls(undefined, 10_000); - await tick(); - const started = Date.now(); - const drained = await broker.drainForDisconnect(120); - const elapsed = Date.now() - started; - assert.equal(drained, false, 'the bound expired with the turn still running'); - // TIGHT: the drain returns AT its deadline — the old code returned - // only when the queued op freed the chain (~10 s) and the round-7 - // regression permitted ~480 ms for a 120 ms drain. - assert.ok(elapsed < 400, `the drain returned at its deadline, not after the stuck chain op: ${elapsed} ms`); - assert.ok(broker.isDrained); - // The UNLOCKED forced stop still settled the pending call DURABLY at - // the bound — recorded first, settled into the guest, no pending - // registry entry left (the chain wait can shorten the drain, never - // its settlement discipline). - const record = broker.store().lookup('c1')!; - assert.equal(record.completion!.outcome, 'reject'); - assert.equal((record.completion!.value as { code?: string }).code, 'AGENT_CANCELLED'); - assert.deepEqual( - broker.pendingCalls().map((e) => e.id), - [], - 'the opening call is not left pending in the guest registry', - ); - assert.equal(session.cancelCalls, 1, 'the forced stop issued the cancel'); - assert.equal(session.releases, 1, 'the release phase issued the release (bounded, never awaited past the bound)'); - // The wait op sees the settled call and ends promptly (its next poll - // observes the guest registry empty — no 10 s wait). - await waiting; - await broker.dispose(); - ws.dispose(); -}); - -test('the drain bound is ABSOLUTE against a chain REPLACED mid-wait: an op enqueued precisely as the prior chain releases (in the drain\'s race window) must not become the chain the drain waits behind with no deadline race (review rejection)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - // Two pending calls: c1 drives the FIRST chain holder's release (the - // race fires when its wait completes); c2 keeps the SECOND op — the - // one enqueued DURING the drain's chain wait — alive past the bound - // (a replacement that resolved quickly would let the re-read bug - // pass unnoticed). - await broker.eval('const p1 = agent("pi/x", "task1"); const p2 = agent("pi/x", "task2"); "started"'); - await tick(); - // A hung backend: cancel AND release never resolve (the worst case). - for (const session of runner.sessions) { - session.hangCancel = true; - session.hangRelease = true; - } - // op1 holds the serialized chain waiting on c1 (yieldful — its poll - // loop never blocks the event loop, so the drain's bound timer fires - // normally). - const waiting1 = broker.waitForCalls(['c1'], 10_000); - await tick(); - const started = Date.now(); - // The drain races the chain — op1's — with the remaining bound. - const drainedP = broker.drainForDisconnect(120); - await tick(); - // op2 enqueues WHILE the drain awaits: it chains onto op1's chain and - // REPLACES `this.opChain`. The review-rejected code raced one chain, - // then re-read the mutable field after its race won — the microtasks - // between the chain's release and that re-read let op2's replacement - // win, so the drain queued behind op2 with no deadline race on it (a - // 20 ms drain took 307 ms; here op2 polls c2 for 10 s, so the old - // code returned only at ITS timeout). - const waiting2 = broker.waitForCalls(['c2'], 10_000); - // Release the raced chain: c1's turn completes, op1's wait ends, the - // drain's race fires 'chain' — with op2 already the current chain. - runner.sessions[0].completeTurn('done'); - const drained = await drainedP; - const elapsed = Date.now() - started; - assert.equal(drained, false, 'the bound expired with c2 still running'); - // TIGHT: the drain returns AT its deadline — the old code returned - // only when the replacement op freed the chain (~10 s later). - assert.ok(elapsed < 500, `the drain re-raced the replaced chain instead of waiting behind it: ${elapsed} ms`); - assert.ok(broker.isDrained); - // c1 completed normally (its turn finished before the bound) — the - // unlocked forced stop never touched it. - const record1 = broker.store().lookup('c1')!; - assert.equal(record1.completion!.outcome, 'resolve'); - // c2 — still pending at the bound — was settled DURABLY by the - // unlocked forced stop (recorded first, settled into the guest, no - // pending registry entry left — the chain wait can shorten the - // drain, never its settlement discipline). - const record2 = broker.store().lookup('c2')!; - assert.equal(record2.completion!.outcome, 'reject'); - assert.equal((record2.completion!.value as { code?: string }).code, 'AGENT_CANCELLED'); - assert.deepEqual( - broker.pendingCalls().map((e) => e.id), - [], - 'neither call is left pending in the guest registry', - ); - assert.equal(runner.sessions[0].cancelCalls, 0, 'the completed turn was not cancelled'); - assert.equal(runner.sessions[1].cancelCalls, 1, 'the forced stop issued the cancel for the still-running turn'); - assert.equal(runner.sessions[0].releases, 1, 'the release phase released the completed session (bounded, never awaited past the bound)'); - assert.equal(runner.sessions[1].releases, 1, 'the release phase released the hung session (bounded, never awaited past the bound)'); - // Both wait ops observe their calls settled and end promptly (no 10 s - // wait: op1 saw c1 settle; op2 sees the registry empty on its next - // poll). - await waiting1; - await waiting2; - await broker.dispose(); - ws.dispose(); -}); - -test('Broker.dispose is bounded against the CHAIN WAIT too: a YIELDFUL queued operation cannot delay teardown past the bound — the disposal races the remaining bound like the drain (phase-D review round 8)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - await broker.eval('const p = agent("pi/x", "task"); "started"'); - await tick(); - const session = runner.sessions[0]; - session.hangCancel = true; - session.hangRelease = true; - // A yieldful op holds the chain past the dispose bound (its own - // timeout is far beyond it — the still-pending call keeps it - // polling). The disposal must NOT queue behind it: the absolute - // bound is the outer ceiling for teardown exactly as for the drain - // (a stuck serialized op must not hang daemon shutdown / reset). - // The catch is attached IMMEDIATELY: `dispose` sets `disposed` at - // method entry, so the stuck op dies on its next poll while the test - // is still awaiting the disposal — its rejection must be handled - // from the start, never unhandled. - const waiting = broker.waitForCalls(undefined, 2_000).catch(() => undefined); - await tick(); - const started = Date.now(); - await broker.dispose(120); - const elapsed = Date.now() - started; - assert.ok(elapsed < 400, `dispose returned at its bound, not after the stuck chain op: ${elapsed} ms`); - assert.equal(session.cancelCalls, 1, 'the bounded disposal still issued the cancel'); - assert.equal(session.releases, 1, 'the bounded disposal still issued the release'); - // The stuck wait op's rejection is the chain's own — absorbed by the - // chain bookkeeping; the caught handle settles when the op dies. - await waiting; - ws.dispose(); -}); - -test('the drain bound is ABSOLUTE for the GUEST DRAIN too: a settlement that resumes a runaway continuation near the deadline is interrupted at the remaining bound, never at the eval deadline (phase-D review round 6)', async () => { - const runner = new FakeRunner(); - // The per-eval deadline is far beyond the drain bound: without the - // remaining-bound interrupt, the interrupted-continuation drain would - // run for the whole eval deadline and exceed the session-eviction TTL. - const { ws, broker } = await setup({ runner, evalTimeoutMs: 10_000 }); - // A suspended eval whose continuation runs forever once the call - // settles (the guest drain resumes it — `drainJobs`). - await broker.eval('const p = agent("pi/x", "task"); const r = await p; while (true) {}'); - await tick(); - const started = Date.now(); - const draining = broker.drainForDisconnect(300); - // The settlement lands mid-drain (the backend turn completes): the - // pump delivers it and the guest drain resumes the runaway - // continuation — which must be interrupted at the REMAINING disconnect - // bound, not run to the 10 s eval deadline. - await new Promise((resolve) => setTimeout(resolve, 50)); - runner.last().completeTurn('done'); - const drained = await draining; - const elapsed = Date.now() - started; - assert.equal(drained, true, 'the turn itself drained within the bound; the interrupted continuation is a bounded drain, not a drain failure'); - assert.ok(elapsed < 3000, `the guest drain was bounded by the remaining disconnect bound, not the eval deadline: ${elapsed} ms`); - assert.ok(broker.isDrained); - // §6.2: the interrupted continuation is RETAINED under - // workspace().diagnostics (the settlement itself landed; only its - // continuation was bounded) — the warn line left the eval result - // surface. - const probe = await broker.eval('"probe"'); - assert.ok( - output(probe).every((l) => !l.includes('interrupted at the disconnect bound')), - `the drain failure left the result surface: ${output(probe).join('\n')}`, - ); - const diag = await broker.eval( - 'workspace().diagnostics.drainError === null ? "none" : workspace().diagnostics.drainError.message', - ); - assert.ok( - String(diag.result ?? '').includes('interrupted') || String(diag.result ?? '').includes('Job execution error'), - `the interrupted drain is retained in diagnostics: ${diag.result}`, - ); - await broker.dispose(); - ws.dispose(); -}); - -test('a client reconnecting mid-drain ABORTS the drain: children stay warm — nothing is cancelled, nothing is released, and the next disconnect drains again (phase-D review round 6)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - await broker.eval('const p = agent("pi/x", "task"); "started"'); - await tick(); - // The drain starts with no clients; a client reconnects mid-drain (the - // daemon's presence probe flips to true). - let clientConnected = false; - const draining = broker.drainForDisconnect(2000, () => clientConnected); - await new Promise((resolve) => setTimeout(resolve, 50)); - assert.equal(runner.sessions[0].releases, 0, 'the child is still warm while the drain waits for the in-flight turn'); - clientConnected = true; - assert.equal(await draining, false, 'the drain aborted — it did not run to its release phase'); - assert.equal(broker.isDrained, false, 'the drain latch stays clear — the next disconnect drains again'); - assert.equal(runner.sessions[0].releases, 0, 'the child was NOT released: children stay warm while any client is connected'); - assert.equal(runner.sessions[0].cancelCalls, 0, 'nothing was cancelled'); - // The still-running turn completes normally after the abort and - // settles into the live workspace. - runner.last().completeTurn('warm result'); - await broker.pump(); - const got = await broker.eval('await p'); - assert.equal(got.result, 'warm result'); - await broker.dispose(); - ws.dispose(); -}); - -test('a restore-time loadSession that lands AFTER a bounded dispose is released exactly once — never registered, never re-issued (phase-D review rejection: the parked restore load used to register its session on the disposed broker, leaking it and repopulating liveAgents)', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-late-load-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await dispatchAgent(broker, runner); - const snapshot = ws.snapshot(); - await crash(ws, broker); - - // Restore: fresh workspace over the snapshot, fresh broker + runner - // over the same store. The re-attach loadSession PARKS (never resolves - // on its own — the reviewer's focused probe scenario). - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - let loadCalls = 0; - let resolveLoad: (() => void) | undefined; - const parkedLoad = new Promise((resolve) => { - resolveLoad = resolve; - }); - const originalLoad = runner2.loadSession.bind(runner2); - runner2.loadSession = async (opts) => { - loadCalls++; - await parkedLoad; - return originalLoad(opts); - }; - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - const reconcilePromise = broker2.reconcile(); - await tick(); - assert.equal(loadCalls, 1, 'the re-attach load is in flight (parked)'); - - // A BOUNDED dispose completes while the load is parked (the daemon - // shutdown path): the serialized chain is held by the parked reconcile, - // so the disposal runs unlocked at its deadline — it must return - // within the bound, never after the parked load. - const started = Date.now(); - await broker2.dispose(150); - const elapsed = Date.now() - started; - assert.ok(elapsed < 1500, `dispose was bounded while the reconcile held the chain: ${elapsed} ms`); - assert.equal(broker2.liveAgents().length, 0, 'no live agent after the dispose'); - - // The parked load lands LATER: the child is released exactly once, - // never registered, never re-issued (no fresh openSession, no prompt — - // a disposed broker must never open a child), and the call is never - // settled from a quiet gap. - resolveLoad!(); - const report = await reconcilePromise; - assert.deepEqual(report.reattached, [], 'the call was never re-attached'); - assert.deepEqual(report.reissued, [], 'the call was never re-issued'); - assert.deepEqual(report.failedLost, []); - assert.deepEqual(report.leftPending, ['c1'], 'the call stays pending — the state owning it was torn down'); - assert.equal(runner2.sessions.length, 1, 'only the loaded session exists'); - const loaded = runner2.sessions[0]; - assert.equal(loaded.releases, 1, 'the late-loaded session was released exactly once'); - assert.equal(loaded.prompts.length, 0, 'the late-loaded session never prompted'); - assert.equal(broker2.liveAgents().length, 0, 'no live agent — the session never registered'); - assert.equal(runner2.openedWith.length, 0, 'no re-issue — no fresh session was opened'); - assert.equal(broker2.store().lookup('c1')!.completion, null, 'the call was not settled from a quiet gap'); - ws2.dispose(); -}); - -test('a bounded drain with MULTIPLE pending restored calls settles EVERY outstanding call at the bound — a reconcile parked on a never-resolving first loadSession leaves no pending, uncancelable entry, and the resumed reconcile never initiates subsequent loads after the generation bump (phase-D review rejection)', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-multi-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await broker.eval('const p1 = agent("pi/x", "t1"); const p2 = agent("pi/y", "t2"); const q = checkpoint("Question?"); "started"'); - await tick(); - assert.equal(runner.sessions.length, 2); - const snapshot = ws.snapshot(); - await crash(ws, broker); - - // Restore: the re-attach of the FIRST pending call parks FOREVER, so - // the serialized reconciliation can never reach the second registry - // entry (it registers calls in `openingCalls` only as the loop reaches - // them). - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - let loadCalls = 0; - let resolveLoad!: () => void; - const parkedLoad = new Promise((resolve) => { - resolveLoad = resolve; - }); - const originalLoad = runner2.loadSession.bind(runner2); - runner2.loadSession = async (opts) => { - loadCalls++; - await parkedLoad; - return originalLoad(opts); - }; - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - const reconcilePromise = broker2.reconcile(); - await tick(); - assert.equal(loadCalls, 1, 'reconcile parked on the FIRST call\'s re-attach load'); - assert.deepEqual( - broker2.pendingCalls().map((e) => e.id), - ['c1', 'c2', 'c3'], - 'all three entries (two calls + the checkpoint) are pending in the guest registry', - ); - - // The bounded drain: c1 is covered by the opening-call registry, but - // c2 was never REACHED by the serialized reconcile — the forced stop - // must settle it too (the old code settled only c1, then reported - // drained with c2 pending and uncancelable forever — reconcile never - // runs again on a live workspace). - assert.equal(await broker2.drainForDisconnect(80), false, 'the bound expired with the load parked'); - assert.ok(broker2.isDrained, 'the broker reports drained'); - assert.deepEqual( - broker2.pendingCalls().map((e) => e.id), - ['c3'], - 'EVERY outstanding CALL was settled at the bound — only the checkpoint (which awaits the human\'s answer) stays pending', - ); - for (const id of ['c1', 'c2']) { - const record = broker2.store().lookup(id)!; - assert.equal(record.completion!.outcome, 'reject', `${id} settled durably at the bound`); - assert.equal((record.completion!.value as { code?: string }).code, 'AGENT_CANCELLED', `${id} carries the forced-stop code`); - assert.equal((record.completion!.value as { recoverable?: boolean }).recoverable, true, `${id} is recoverable`); - } - - // The parked load lands AFTER the drain: the resumed reconciliation - // must NOT initiate the SECOND call's load — a fresh child must never - // open and run after the last client disconnected (the generation - // fence used to cover only the parked load itself, so the resumed - // loop initiated subsequent loads for the entries behind it). The - // already-recorded completions settle from the store, first-wins. - resolveLoad(); - const report = await reconcilePromise; - assert.equal(loadCalls, 1, 'no second loadSession after the drain generation bump'); - assert.equal(runner2.openedWith.length, 0, 'no re-issue — no fresh session was opened'); - assert.equal(runner2.sessions.length, 1, 'only the parked-load session exists'); - assert.equal(runner2.sessions[0].releases, 1, 'the late-loaded session was released exactly once'); - assert.equal(runner2.sessions[0].prompts.length, 0, 'the late-loaded session never prompted'); - assert.deepEqual(report.reissued, [], 'nothing was re-issued'); - assert.deepEqual(report.reattached, [], 'nothing was re-attached after the drain'); - assert.deepEqual(report.leftPending, [], 'no call left pending'); - // The bound settlements are exactly-once: the resumed store arm did - // not double-settle (the registry still holds only the checkpoint). - assert.deepEqual(broker2.pendingCalls().map((e) => e.id), ['c3']); - // The checkpoint the parked reconcile never re-surfaced was re-surfaced - // by the bound's pass — it stays ANSWERABLE across the cut-off restore - // (the doc: "answering works across a restore"). - const answered = await broker2.eval('checkpoint.answer("c3", "blue"); "delivered"'); - assert.equal(answered.result, 'delivered'); - assert.equal((await broker2.eval('await q')).result, 'blue'); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('a late-landing restore load whose session.release HANGS cannot hold the reconciliation (or the daemon\'s first touch) — the teardown fence DETACHES the best-effort release (phase-D review rejection: the fence awaited session.release() with no deadline, so a custom backend with a hung release kept reconcile/first-touch pending indefinitely)', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-restore-hungrel-')); - const storePath = join(dir, 'calls.jsonl'); - const runner = new FakeRunner(); - const { ws, broker } = await setup({ store: JsonlCallStore.open(storePath), runner }); - await dispatchAgent(broker, runner); - const snapshot = ws.snapshot(); - await crash(ws, broker); - - // Restore: the re-attach load parks; a bounded dispose completes while - // it is parked (the daemon shutdown path — the disposal runs unlocked - // at its deadline because the parked reconcile holds the chain). - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - let resolveLoad!: () => void; - const parkedLoad = new Promise((resolve) => { - resolveLoad = resolve; - }); - const originalLoad = runner2.loadSession.bind(runner2); - runner2.loadSession = async (opts) => { - await parkedLoad; - const session = await originalLoad(opts); - session.hangRelease = true; // the custom backend's release hangs forever - return session; - }; - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - const reconcilePromise = broker2.reconcile(); - await tick(); - await broker2.dispose(150); - - // The parked load lands LATER, with a HUNG release: the teardown fence - // must DETACH the release (best-effort, catch attached) instead of - // awaiting it — the reconciliation completes promptly and the first - // touch is never left pending on the hung release (the old code - // awaited `session.release()` with no deadline: reconcile stayed - // parked forever). - const started = Date.now(); - resolveLoad(); - const report = await reconcilePromise; - const elapsed = Date.now() - started; - assert.ok(elapsed < 1000, `reconcile completed without awaiting the hung release: ${elapsed} ms`); - assert.equal(runner2.sessions.length, 1, 'only the loaded session exists'); - assert.equal(runner2.sessions[0].releases, 1, 'the release was ISSUED (best-effort, detached)'); - assert.equal(runner2.sessions[0].prompts.length, 0, 'the late-loaded session never prompted'); - assert.deepEqual(report.leftPending, ['c1'], 'the call stays pending — the state owning it was torn down'); - assert.deepEqual(report.reissued, [], 'never re-issued'); - assert.equal(runner2.openedWith.length, 0, 'no fresh session was opened'); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); diff --git a/packages/repl-engine/test/review2.test.ts b/packages/repl-engine/test/review2.test.ts deleted file mode 100644 index 6ea61340..00000000 --- a/packages/repl-engine/test/review2.test.ts +++ /dev/null @@ -1,693 +0,0 @@ -/** - * Phase-D review round 2 regression suite: the reviewer's rejected items, - * pinned at the engine boundary. - * - * 1. Explicit queued turns reattach settled sessions lazily; strict idle - * steering and idle cancellation never reattach. The first-class queue - * acceptance matrix lives in broker.test.ts. - * 2. Backend identity/pool routing is persisted (modelSpec + the - * RESOLVED backendId recorded at session open), so a restore or - * re-issue never re-resolves the model spec against the CURRENT - * default backend and misses the still-resumable original session. - * 3. The client-presence drain: in-flight turns drain to completion - * (each settlement boundary snapshots), then idle children close; - * the spec-owed concrete bound applies (an over-bound turn is - * cancelled — the honest bounded teardown); pending queued turns remain durable. - * 4. The workspace manifest: top-level bindings with structure-only - * tokens, provenance, and live-handle status — metadata, never - * content. - * 5. The per-eval wall-clock deadline: a currently-running runaway eval - * is ALWAYS breakable through the quickjs interrupt handler (the - * armed-signal-only semantics could only break the next execution). - * 6. The per-binding provenance passes (eval / settlement labels, - * sanitized rendering). - */ - -import assert from 'node:assert/strict'; -import { mkdtempSync, rmSync } from 'node:fs'; -import { tmpdir } from 'node:os'; -import { join } from 'node:path'; -import { test } from 'node:test'; - -import { - Broker, - InMemoryCallStore, - JsonlCallStore, - Workspace, - type BrokerLoadSessionOptions, - type BrokerOpenSessionOptions, - type BrokerPromptOptions, - type BrokerRunner, - type BrokerSession, - type BrokerTurn, - type CallStore, - type SnapshotSink, -} from '../src/index.js'; - -const PROJECT = '/tmp/repl-review2-project'; - -async function tick(): Promise { - await new Promise((resolve) => setTimeout(resolve, 0)); -} - -/** Poll until `predicate` holds (the drain tests' async wait). */ -async function waitFor(predicate: () => boolean, timeoutMs = 5000): Promise { - const start = Date.now(); - while (!predicate()) { - if (Date.now() - start > timeoutMs) throw new Error('waitFor: condition not met in time'); - await new Promise((resolve) => setTimeout(resolve, 10)); - } -} - -/** The fake held-open ACP session (see broker.test.ts). */ -class FakeSession implements BrokerSession { - readonly sessionId: string; - initializeMeta: Readonly> | undefined; - readonly prompts: Array<{ content: string; resolve: (turn: BrokerTurn) => void; reject: (error: unknown) => void }> = []; - readonly steers: Array<{ content: string; resolve: (outcome: unknown) => void; reject: (error: unknown) => void }> = []; - releases = 0; - stopReason = 'end_turn'; - readonly completedTexts: string[] = []; - - /** The steering capability advertised at open (the broker captures it - * per session at open time — the test flips this BEFORE dispatching). */ - static supportsSteering = true; - - constructor(readonly openedWith: BrokerOpenSessionOptions | BrokerLoadSessionOptions) { - this.sessionId = `fake-session-${FakeSession.nextId++}`; - this.initializeMeta = FakeSession.supportsSteering ? { steering: { supported: true } } : {}; - } - - static nextId = 0; - - prompt(content: string, opts: BrokerPromptOptions = {}): Promise { - this.texts.push(content); - return new Promise((resolve, reject) => { - this.prompts.push({ content, resolve, reject }); - opts.onHandoff?.(); - }); - } - readonly texts: string[] = []; - - steer(content: string): Promise { - return new Promise((resolve, reject) => { - this.steers.push({ content, resolve, reject }); - }); - } - - awaitCurrentTurn(): Promise { - return new Promise(() => {}); - } - - cancel(): Promise { - for (const pending of this.prompts.splice(0)) { - pending.resolve({ stopReason: 'cancelled', text: '' }); - } - return Promise.resolve(); - } - - release(): Promise { - this.releases++; - return Promise.resolve(); - } - - currentTurnText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ''; - } - - finalMessageText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ''; - } - - rawStructuredOutput(): unknown { - return undefined; - } - - completeTurn(text: string): void { - const pending = this.prompts.shift(); - assert.ok(pending, 'a prompt turn must be in flight'); - this.completedTexts.push(text); - pending.resolve({ stopReason: this.stopReason, text }); - } - - failTurn(error: unknown): void { - const pending = this.prompts.shift(); - assert.ok(pending, 'a prompt turn must be in flight'); - pending.reject(error); - } -} - -/** The fake runner (see broker.test.ts) — sessions carry a `backendId` - * (the routing pin the store records), and `loadSession` is the lazy - * re-attach seam. */ -class FakeRunner implements BrokerRunner { - readonly sessions: FakeSession[] = []; - readonly openedWith: BrokerOpenSessionOptions[] = []; - readonly loadedWith: BrokerLoadSessionOptions[] = []; - /** When set, loadSession rejects (the capability gate / lost session). */ - loadError: Error | null = null; - - listBackends(): string[] { - return ['claude', 'codex', 'opencode', 'pi']; - } - - defaultBackendId(): string { - return 'claude'; - } - - async openSession(opts: BrokerOpenSessionOptions): Promise { - const session = new FakeSession(opts); - session.backendId = 'pi'; - this.sessions.push(session); - this.openedWith.push(opts); - return session; - } - - async loadSession(opts: BrokerLoadSessionOptions): Promise { - if (this.loadError !== null) throw this.loadError; - const session = new FakeSession(opts); - session.backendId = 'pi'; - this.sessions.push(session); - this.loadedWith.push(opts); - return session; - } - - async dispose(): Promise {} - - last(): FakeSession { - assert.ok(this.sessions.length > 0, 'a session must exist'); - return this.sessions[this.sessions.length - 1]; - } -} - -async function setup(options: { runner?: BrokerRunner; store?: CallStore; sink?: SnapshotSink } = {}): Promise<{ - ws: Workspace; - broker: Broker; -}> { - const ws = await Workspace.create(PROJECT); - const broker = await Broker.attach(ws, { - runner: options.runner, - store: options.store, - snapshotSink: options.sink, - evalTimeoutMs: 0, // tests drive interrupts explicitly - }); - return { ws, broker }; -} - -function output(result: { output: string[] }): string[] { - return result.output; -} - -// Queue reattachment, strict idle steering, and failed-reattachment semantics are -// covered by the first-class queue acceptance matrix in broker.test.ts. - -// ── 2. Backend identity/pool routing is persisted ────────────────────── - -test('review 2/2: the resolved backend id is recorded at session open and pins the restore\'s loadSession routing — a changed configured default never misses the original session', async () => { - const dir = mkdtempSync(join(tmpdir(), 'repl-pin-')); - const storePath = `${dir}/calls.jsonl`; - try { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner, store: JsonlCallStore.open(storePath) }); - // The guest's verbatim spec rides admission validation; the RESOLVED - // backend id ("pi" — the fake's own backend) is what the store - // records, distinct from the spec's segment (the deleted 'default' - // sentinel used to be the only way to route off-segment — a real - // registered spec covers the same pin now). - await broker.eval('const p = agent("claude/some-model", "task"); "started"'); - await tick(); - const record = broker.store().lookup('c1')!; - assert.equal(record.modelSpec, 'claude/some-model', 'the verbatim spec is persisted'); - assert.equal(record.backendId, 'pi', 'the resolved backend id is persisted'); - const snapshot = ws.snapshot(); - await broker.dispose(); - ws.dispose(); - void runner; - - // Restore: even though the CURRENT default would route elsewhere, the - // re-attach pins the recorded backend id. - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - const broker2 = await Broker.attach(ws2, { runner: runner2, store: JsonlCallStore.open(storePath) }); - await broker2.reconcile(); - assert.equal(runner2.loadedWith.length, 1, 'the call re-attached'); - assert.equal(runner2.loadedWith[0].model, 'pi', 'routing pinned the ORIGINAL backend id'); - await broker2.dispose(); - ws2.dispose(); - } finally { - rmSync(dir, { recursive: true, force: true }); - } -}); - -test('review 2/2b: a re-issued call re-routes to the ORIGINAL backend (the recorded pin), never the current default', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - await broker.eval('const p = agent("pi/deepseek-v4-flash-max", "task"); "started"'); - await tick(); - const recorded = broker.store().lookup('c1')!.sessionId!; - const snapshot = ws.snapshot(); - await broker.dispose(); - ws.dispose(); - - // A fresh runner whose loadSession FAILS (the session is lost at the - // backend): the fallback re-issue must route by the recorded backend - // id, not by the model spec re-resolved against a changed default. - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - runner2.loadError = new Error('session lost'); - const broker2 = await Broker.attach(ws2, { runner: runner2, store: new InMemoryCallStore() }); - // The in-memory store starts EMPTY — reconcile adopts the registry - // entry but loses the recorded session id (a wiped store): the call - // re-issues through the ordinary dispatch path. - await broker2.reconcile(); - await tick(); - void recorded; - assert.equal(runner2.openedWith.length, 1, 'the lost call was re-issued'); - assert.equal( - runner2.openedWith[0].model, - 'pi/deepseek-v4-flash-max', - 'the re-issue routes by the verbatim recorded spec', - ); - await broker2.dispose(); - ws2.dispose(); -}); - -// ── 3. The client-presence drain ─────────────────────────────────────── - -/** A snapshot sink recording the boundaries (the drain cadence). */ -class BoundSink implements SnapshotSink { - boundaries: Array<'eval' | 'settlement'> = []; - boundary(kind: 'eval' | 'settlement'): void { - this.boundaries.push(kind); - } - flush(): void {} -} - -test('review 2/3: drainForDisconnect drains in-flight turns to completion (settlement boundaries fire), then closes every idle child; a bound-exceeding turn is cancelled (the honest bounded teardown)', async () => { - const runner = new FakeRunner(); - const sink = new BoundSink(); - const { ws, broker } = await setup({ runner, sink }); - await broker.eval('const a = agent("pi/x", "A"); const b = agent("pi/x", "B"); "started"'); - await tick(); - assert.equal(broker.busySessionCount(), 2, 'two turns in flight'); - - // The drain runs WITHOUT the test completing the turns: it must WAIT - // for them (the doc: in-flight turns drain to completion — never a - // cancel of running work). - const draining = broker.drainForDisconnect(60_000); - await tick(); - assert.equal(broker.busySessionCount(), 2, 'the drain waits instead of cancelling'); - runner.sessions[0].completeTurn('A done'); - runner.sessions[1].completeTurn('B done'); - assert.equal(await draining, true, 'both turns drained within the bound'); - assert.ok(broker.isDrained, 'the broker reports drained'); - for (const session of runner.sessions) { - assert.equal(session.releases, 1, 'every child closed after the drain'); - } - // The drain's settlement boundaries fired (the daemon's snapshot sink - // persists each one — a kill mid-drain loses nothing). - assert.ok(sink.boundaries.includes('settlement'), sink.boundaries.join(',')); - // Both results settled into the VM (the continuation can read them). - const got = await broker.eval('await a + "|" + await b'); - assert.equal(got.result, 'A done|B done'); - await broker.dispose(); - ws.dispose(); -}); - -test('review 2/3b: the drain bound is the outer ceiling — an over-bound turn is cancelled and settled as the recoverable AGENT_CANCELLED', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - await broker.eval('const p = agent("pi/x", "task"); "started"'); - await tick(); - assert.equal(await broker.drainForDisconnect(50), false, 'the bound expired with a turn still running'); - assert.ok(broker.isDrained); - assert.equal(runner.sessions[0].releases, 1, 'the child closed even under the bound'); - // The cancelled call rejects recoverably into the guest. - let result: string | undefined; - for (let attempt = 0; attempt < 100; attempt++) { - const got = await broker.eval('await p.catch((e) => "ERR:" + e.message)'); - if (got.result !== undefined) { - result = got.result; - break; - } - await new Promise((resolve) => setTimeout(resolve, 10)); - } - assert.ok(result?.includes('cancelled'), `recoverable cancel: ${result}`); - await broker.dispose(); - ws.dispose(); -}); - -test('review 2/3c-2: a parked open that outlives the drain bound is STOPPED — the late child is closed before it ever prompts, and the call settles as the recoverable AGENT_CANCELLED (nothing runs after the last client disconnected)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - let releaseOpen!: () => void; - const parkedOpen = new Promise((resolve) => { - releaseOpen = resolve; - }); - const originalOpen = runner.openSession.bind(runner); - runner.openSession = async (opts) => { - await parkedOpen; - return originalOpen(opts); - }; - await broker.eval('const p = agent("pi/x", "task"); "started"'); - await tick(); - // The bound expires with the open still parked: the drain returns false - // (the honest bounded teardown) and marks the opening call STOPPED. - assert.equal(await broker.drainForDisconnect(50), false); - assert.ok(broker.isDrained); - // The bound settlement is DURABLE AT THE BOUND (phase-D review round - // 7): the opening call is recorded, guest-settled, drained and - // snapshotted while the open is STILL parked — the broker does not - // report drained with the call pending and uncancelable even though - // the openSession has not resolved (it may never resolve). - assert.deepEqual( - broker.pendingCalls().map((e) => e.id), - [], - 'the opening call is not left pending in the guest registry at the bound', - ); - const boundRecord = broker.store().lookup('c1')!; - assert.equal(boundRecord.completion!.outcome, 'reject'); - assert.equal((boundRecord.completion!.value as { code?: string }).code, 'AGENT_CANCELLED'); - assert.equal((boundRecord.completion!.value as { recoverable?: boolean }).recoverable, true); - assert.equal((boundRecord.completion!.value as { replBackend?: string }).replBackend, 'pi'); - // The parked open lands LATER: the child is closed immediately — it - // never prompts (nothing runs after the last client disconnected) — - // and the late reject is a first-wins no-op against the bound's - // recorded completion. - releaseOpen(); - await waitFor(() => runner.sessions.length === 1); - const session = runner.sessions[0]; - assert.equal(session.releases, 1, 'the stopped child was closed without ever prompting'); - assert.equal(session.prompts.length, 0, 'the stopped call never ran a turn'); - for (let attempt = 0; attempt < 100; attempt++) { - await broker.pump(); - const got = await broker.eval('await p.catch((e) => "ERR:" + e.message)'); - if (got.result !== undefined) { - assert.ok( - got.result.includes('cancelled') || got.result.includes('stopped'), - `the stopped call settles recoverably: ${got.result}`, - ); - break; - } - await new Promise((resolve) => setTimeout(resolve, 10)); - } - const record = broker.store().lookup('c1')!; - assert.equal(record.completion!.outcome, 'reject'); - assert.equal((record.completion!.value as { recoverable?: boolean }).recoverable, true); - assert.equal(record.reissues, 0, 'never re-issued'); - await broker.dispose(); - ws.dispose(); -}); - -test('review 7/3c-3: an openSession that NEVER resolves is settled DURABLY at the bound — recorded, guest-settled, drained and snapshotted, so the drain never reports drained with the call pending and uncancelable', async () => { - const runner = new FakeRunner(); - const boundaries: string[] = []; - const sink: SnapshotSink = { - boundary: (kind) => boundaries.push(kind), - flush: () => undefined, - }; - const { ws, broker } = await setup({ runner, sink }); - // The open is parked FOREVER (a backend that accepts the session - // request but never answers it — the never-resolving openSession). - runner.openSession = async () => new Promise(() => {}); - await broker.eval('const p = agent("pi/x", "task"); "started"'); - await tick(); - // The bound expires with the open still parked. The forced stop must - // settle the call AT THE BOUND — recorded FIRST (durable), settled - // into the guest, one bounded drain + settlement boundary — without - // waiting for the openSession, which NEVER resolves (review round 7: - // the settlement used to be deferred to the landing, leaving the - // broker reporting drained with the call pending and uncancelable). - const before = boundaries.length; - assert.equal( - await broker.drainForDisconnect(50), - false, - 'the never-resolving open cannot drain — the bound is the honest outcome', - ); - assert.ok(broker.isDrained); - const record = broker.store().lookup('c1')!; - assert.equal(record.completion!.outcome, 'reject'); - assert.equal((record.completion!.value as { code?: string }).code, 'AGENT_CANCELLED'); - assert.equal((record.completion!.value as { recoverable?: boolean }).recoverable, true); - assert.equal((record.completion!.value as { replBackend?: string }).replBackend, 'pi'); - assert.deepEqual( - broker.pendingCalls().map((e) => e.id), - [], - 'the opening call is not left pending in the guest registry', - ); - assert.ok( - boundaries.slice(before).includes('settlement'), - `the bound settlement fired a settlement boundary (snapshot): ${boundaries.join(',')}`, - ); - // The guest promise settled with the recoverable error (a later eval - // reads it — the workspace stays live after the drain). - const got = await broker.eval('await p.catch((e) => "ERR:" + e.message)'); - assert.ok( - (got.result ?? '').includes('cancelled by the client-presence drain'), - `guest-visible settlement: ${got.result}`, - ); - // The parked open never lands; the broker is fully teardown-able (the - // parked task cannot block disposal, which is bounded). - await broker.dispose(500); - ws.dispose(); -}); - -// ── 4. The workspace manifest ────────────────────────────────────────── - -test('review 2/4: the workspace manifest lists top-level bindings with structure-only tokens, provenance, and live-handle status — metadata, never content', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - await broker.eval( - 'globalThis.findings = { zekret: "MARKER-STRING".repeat(50), n: 12 }; globalThis.note = "shibboleth"; ' + - 'globalThis.count = 98765; globalThis.research = agent("pi/x", "investigate"); ' + - 'console.log("logged once"); "done"', - ); - await tick(); - // The handle is pending (its turn is in flight): the manifest reports - // the live-handle status from the call store. - let manifest = broker.workspaceManifest(); - const byName = new Map(manifest.bindings.map((b) => [b.name, b])); - assert.ok(byName.has('findings'), [...byName.keys()].join(',')); - assert.ok(byName.get('findings')!.token.startsWith('{2 keys} \u00b7 '), byName.get('findings')!.token); - assert.equal(byName.get('note')!.token, 'string \u00b7 10B'); - assert.equal(byName.get('count')!.token, 'number \u00b7 8B'); - assert.equal(byName.get('count')!.sizeBytes, 8, 'the size is exposed as its own field'); - assert.equal(byName.get('research')!.token, 'agent handle \u00b7 pending \u00b7 call c1 \u00b7 148B'); - assert.equal(byName.get('research')!.provenance, 'eval 1'); - assert.equal(manifest.logs.count, 0, 'the $N capture system is deleted — the logs range is always empty'); - assert.equal(manifest.logs.first, null); - assert.ok(manifest.inFlight.includes('c1'), manifest.inFlight.join(',')); - // The intent-plane hygiene rule: NO fragment of any bound value at ANY - // length appears in the manifest. - const rendered = JSON.stringify(manifest); - for (const leaked of ['MARKER', 'MARK', 'STRING', 'shibboleth', 'shibb', '98765']) { - assert.ok(!rendered.includes(leaked), `value content leaked (${leaked}): ${rendered}`); - } - assert.ok(!rendered.includes('zekret'), 'nested property names never leak'); - // The doc's full provenance surface: "from what task, when" (phase-D - // review round 3: bindings used to carry only the `worker c1`-shaped - // label and an internal timestamp). The handle binding's task is its - // founding agent() call's task, and the attribution wall clock is - // exposed. - assert.equal(byName.get('research')!.task, 'investigate', 'the handle binding carries its founding task'); - assert.equal(typeof byName.get('research')!.provenanceAtMs, 'number'); - assert.ok(byName.get('research')!.provenanceAtMs! > 0, 'the provenance wall clock is real'); - - // Settlement attributes continuation bindings to the worker call. - runner.last().completeTurn('DUG-UP'); - await tick(); - await broker.pump(); - await broker.eval('globalThis.finding = research; "stored"'); - manifest = broker.workspaceManifest(); - const finding = manifest.bindings.find((b) => b.name === 'finding'); - assert.equal(finding?.token, 'agent handle \u00b7 settled \u00b7 call c1 \u00b7 148B'); - // The worker-produced binding carries the worker's TASK text (the "from - // what task" half) and the attribution wall clock (the "when" half). - assert.equal(finding?.task, 'investigate', 'the worker provenance carries its task'); - assert.equal(typeof finding?.provenanceAtMs, 'number'); - assert.ok(finding!.provenanceAtMs! > 0); - assert.ok(!JSON.stringify(manifest).includes('DUG-UP'), 'worker result content never leaks'); - await broker.dispose(); - ws.dispose(); -}); - -test('review 2/4b: the manifest lists GLOBAL LEXICAL bindings — top-level let/const/class, the roadmap\'s canonical `const research = agent(...)` state — with tokens, provenance, and live-handle status (phase-E review rejection: only global-object keys were enumerated, so lexical workspace state was invisible to status)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - // The roadmap's canonical form: `const research = agent(...)` — a - // global LEXICAL binding (top-level let/const/class are NOT - // global-object properties; ECMAScript's global declarative record is - // non-reflectable, and the engine reaches it through the internal - // global-var object — see global-lexical.ts). - await broker.eval( - 'let x = 1; const y = { a: 1 }; class Z {}; var w = 3; globalThis.g = 9; ' + - 'const research = agent("pi/x", "investigate"); "started"', - ); - await tick(); - const manifest = broker.workspaceManifest(); - const byName = new Map(manifest.bindings.map((b) => [b.name, b])); - // EVERY declaration kind is listed: let/const/class (lexical) and - // var/function/globalThis-assignment (object-record). - assert.ok(byName.has('x'), [...byName.keys()].join(',')); - assert.ok(byName.has('y'), [...byName.keys()].join(',')); - assert.ok(byName.has('Z'), [...byName.keys()].join(',')); - assert.ok(byName.has('w'), [...byName.keys()].join(',')); - assert.ok(byName.has('g'), [...byName.keys()].join(',')); - assert.equal(byName.get('x')!.token, 'number \u00b7 8B'); - assert.ok(byName.get('y')!.token.startsWith('{1 key}'), byName.get('y')!.token); - assert.equal(byName.get('Z')!.token, 'function \u00b7 32B'); - assert.equal(byName.get('w')!.token, 'number \u00b7 8B'); - // The roadmap's handle: live-handle status AND the full provenance - // surface (eval label, task, wall clock) — exactly what the reviewer - // required for `const research = agent(...)`. The size travels with - // the handle token and the binding's own sizeBytes field (phase-E - // review rejection: the size surface used to stop at the handle - // marker). - assert.equal(byName.get('research')!.token, 'agent handle \u00b7 pending \u00b7 call c1 \u00b7 148B'); - assert.equal(byName.get('research')!.sizeBytes, 148); - assert.equal(byName.get('research')!.provenance, 'eval 1'); - assert.equal(byName.get('research')!.task, 'investigate'); - assert.equal(typeof byName.get('research')!.provenanceAtMs, 'number'); - assert.ok(byName.get('research')!.provenanceAtMs! > 0); - // The intent-plane hygiene rule holds for lexical bindings too: no - // fragment of any bound value at ANY length appears in the manifest. - const rendered = JSON.stringify(manifest); - assert.ok(!rendered.includes('a: 1'), 'lexical object content never leaks'); - // Settlement + a continuation-created lexical binding: the declaration - // instantiates in ITS eval (top-level let/const exist in TDZ from the - // script's instantiation), but the VALUE the continuation assigns is - // the worker settlement's product — the manifest RE-ATTRIBUTES the - // binding to the worker that produced the current value (phase-E - // review rejection: the lexical entry was recorded on first sight - // only, so the value the worker settlement produced kept the - // declaring eval's label with no task; review2.test.ts used to pin - // that incorrect behavior). - await broker.eval('const finding = await research; "waited"'); - await tick(); - runner.last().completeTurn('DUG-UP'); - await tick(); - await broker.pump(); - const m2 = broker.workspaceManifest(); - const finding = new Map(m2.bindings.map((b) => [b.name, b])).get('finding'); - assert.ok(finding, 'the continuation-created lexical binding is listed'); - assert.equal(finding!.token, 'string \u00b7 6B'); - assert.equal(finding!.sizeBytes, 6, 'the size is exposed as its own field'); - // The doc's full provenance surface for the worker-produced value: - // which subagent produced it (via), from what task (task), when (at). - assert.equal(finding!.provenance, 'worker c1', 'the worker settlement re-attributes the lexical value'); - assert.equal(finding!.task, 'investigate', 'the worker provenance carries its task'); - assert.equal(typeof finding!.provenanceAtMs, 'number'); - assert.ok(finding!.provenanceAtMs! > 0, 'the re-attribution wall clock is real'); - assert.ok(!JSON.stringify(m2).includes('DUG-UP'), 'worker result content never leaks'); - // The re-attribution is STABLE: a later eval that does not touch the - // binding leaves the worker attribution in place. - await broker.eval('1 + 1'); - const findingLater = new Map(broker.workspaceManifest().bindings.map((b) => [b.name, b])).get('finding'); - assert.equal(findingLater!.provenance, 'worker c1', 'the worker attribution survives later evals'); - assert.equal(findingLater!.task, 'investigate'); - // A LEXICAL binding SHADOWS a same-named global-object property for - // identifier resolution, so the manifest lists ONE binding per name — - // the lexical view (what the orchestrator's code sees). A name first - // attributed as a property and later shadowed by a lexical declaration - // is RE-attributed to the eval that created the lexical binding, and - // stays stable afterwards. - await broker.eval('globalThis.n = 1; "p"'); - const n1 = new Map(broker.workspaceManifest().bindings.map((b) => [b.name, b])).get('n'); - assert.equal(n1!.provenance, 'eval 4', 'the property binding is attributed first'); - await broker.eval('let n = 2; "l"'); - const n2 = new Map(broker.workspaceManifest().bindings.map((b) => [b.name, b])).get('n'); - assert.equal(n2!.provenance, 'eval 5', 'the lexical shadow re-attributes to its creating eval'); - assert.equal(n2!.token, 'number \u00b7 8B'); - assert.equal( - broker.workspaceManifest().bindings.filter((b) => b.name === 'n').length, - 1, - 'one binding per name — the lexical view wins', - ); - await broker.eval('1 + 1'); - const n3 = new Map(broker.workspaceManifest().bindings.map((b) => [b.name, b])).get('n'); - assert.equal(n3!.provenance, 'eval 5', 'the lexical attribution is stable across later evals'); - // The restore path: lexical bindings travel inside the snapshot (the - // internal global-var object is part of the VM memory), the re-registered - // bridge leaves them untouched, and the restored workspace's manifest - // lists them WITH their provenance (the registry travels too). - const snapshot = ws.snapshot(); - const restored = await Workspace.restore(PROJECT, snapshot); - try { - const preRestore = ws.manifest(); - const restoredManifest = restored.manifest(); - assert.deepEqual( - restoredManifest.bindings.map((b) => b.name).sort(), - preRestore.bindings.map((b) => b.name).sort(), - 'the restored manifest lists exactly the same bindings (lexical included)', - ); - const restoredByName = new Map(restoredManifest.bindings.map((b) => [b.name, b])); - assert.equal(restoredByName.get('research')!.token, 'agent handle'); - assert.equal(restoredByName.get('research')!.provenance, 'eval 1'); - assert.equal(restoredByName.get('n')!.provenance, 'eval 5'); - assert.equal(restoredByName.get('finding')!.token, 'string \u00b7 6B'); - assert.equal(restoredByName.get('finding')!.provenance, 'worker c1'); - assert.equal(restoredByName.get('finding')!.sizeBytes, 6); - // The restored realm's lexical bindings are live: the workspace keeps - // working with them. - const live = await restored.eval('x + 1'); - assert.equal(live.kind, 'value'); - if (live.kind === 'value') assert.equal(live.value, 2); - } finally { - restored.dispose(); - } - await broker.dispose(); - ws.dispose(); -}); - -// ── 5. The per-eval wall-clock deadline ──────────────────────────────── - -test('review 2/5: the per-eval deadline makes a CURRENTLY running runaway eval breakable through the quickjs interrupt handler; the VM stays usable', async () => { - const runner = new FakeRunner(); - const ws = await Workspace.create(PROJECT); - const broker = await Broker.attach(ws, { runner, evalTimeoutMs: 200 }); - try { - const runaway = await broker.eval('while (true) {}'); - assert.ok( - output(runaway).some((l) => l.includes('interrupted')), - output(runaway).join('\n'), - ); - // The VM stays usable. - const after = await broker.eval('6 * 7'); - assert.equal(after.result, '42'); - } finally { - await broker.dispose(); - ws.dispose(); - } -}); - -// ── 6. The provenance passes ─────────────────────────────────────────── - -test('review 2/6: provenance passes attribute bindings to evals and worker settlements, sanitized at render', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - await broker.eval('globalThis.a = { n: 1 }; globalThis.b = "x".repeat(4)'); - await broker.eval('globalThis.c = 3'); - // In-place mutation does NOT re-attribute; rebinding does. - await broker.eval('globalThis.a.n = 99; globalThis.b = 42'); - let view = ws.provenanceView(); - assert.equal(view.origins.get('a')?.via, 'eval 1'); - assert.equal(view.origins.get('b')?.via, 'eval 3', 'rebinding re-attributes'); - assert.equal(view.origins.get('c')?.via, 'eval 2'); - assert.equal(view.evalSeq, 3); - - // A worker settlement's continuation bindings attribute to the call. - await broker.eval('const p = agent("pi/x", "task").then((r) => { globalThis.finding = r; })'); - await tick(); - runner.last().completeTurn('result text'); - await tick(); - await broker.pump(); - view = ws.provenanceView(); - assert.equal(view.origins.get('finding')?.via, 'worker c1'); - - // A deleted binding drops out of the registry. - await broker.eval('delete globalThis.c'); - view = ws.provenanceView(); - assert.ok(!view.origins.has('c')); - await broker.dispose(); - ws.dispose(); -}); diff --git a/packages/repl-engine/test/review4.test.ts b/packages/repl-engine/test/review4.test.ts deleted file mode 100644 index a82ba02c..00000000 --- a/packages/repl-engine/test/review4.test.ts +++ /dev/null @@ -1,234 +0,0 @@ -/** - * Phase-E review round 4 regression suite: the COMPLETE trap-free - * metadata reads — the two caps the round-3 review left in place: - * - * 1. The pending-call registry read (`surface.pending()`) used to be - * capped at 16 384 elements: 16 400 pending checkpoints returned - * only 16 384 ids plus one `undefined` hole (the `[ArrayTruncated]` - * marker mapping to `undefined` in the broker's id lists). The read - * is now COMPLETE (no array cap) — the registry is the frozen guest - * library's own metadata, bounded by the VM's memory like the - * metadata itself. Pinned here at the engine boundary: the eval's - * `pending` surface, the broker's pending-id reads, and the restore - * path's three-way reconciliation all see the WHOLE registry. - * 2. The provenance registry's `read()` result used to go through the - * generic 256-property object cap: a workspace with 300 bindings - * reported null provenance for bindings 256-299 even though the eval - * created them. The registry read is now complete too — every - * binding's origin (which eval/worker produced the value) is - * preserved in the manifest. - * - * Both reads stay trap-free (own-property-descriptor reads only; the - * metadata is created by the frozen library closures, never by guest - * code) and bounded by the VM's memory like the metadata itself. - */ - -import assert from 'node:assert/strict'; -import { test } from 'node:test'; - -import { - Broker, - Workspace, - type BrokerLoadSessionOptions, - type BrokerOpenSessionOptions, - type BrokerPromptOptions, - type BrokerRunner, - type BrokerSession, - type BrokerTurn, -} from '../src/index.js'; - -const PROJECT = '/tmp/repl-review4-project'; - -/** How many pending checkpoints exceed the round-3 read cap (16 384) - * with margin — the regression's floor. */ -const CHECKPOINTS = 16_500; - -/** The fake held-open ACP session (the same shape as eval-break.test.ts's). */ -class FakeSession implements BrokerSession { - readonly sessionId: string; - initializeMeta: Readonly> | undefined; - readonly prompts: Array<{ content: string; resolve: (turn: BrokerTurn) => void; reject: (error: unknown) => void }> = []; - releases = 0; - cancelCalls = 0; - stopReason = 'end_turn'; - readonly completedTexts: string[] = []; - - constructor(readonly openedWith: BrokerOpenSessionOptions | BrokerLoadSessionOptions) { - this.sessionId = `fake-session-${FakeSession.nextId++}`; - this.initializeMeta = { steering: { supported: true } }; - } - - static nextId = 0; - - prompt(content: string, opts: BrokerPromptOptions = {}): Promise { - return new Promise((resolve, reject) => { - this.prompts.push({ content, resolve, reject }); - opts.onHandoff?.(); - }); - } - - steer(content: string): Promise { - return new Promise((_, reject) => reject(new Error('steer not used in this suite'))); - } - - awaitCurrentTurn(): Promise { - return new Promise(() => {}); - } - - cancel(): Promise { - this.cancelCalls++; - for (const pending of this.prompts.splice(0)) { - pending.resolve({ stopReason: 'cancelled', text: '' }); - } - return Promise.resolve(); - } - - release(): Promise { - this.releases++; - return Promise.resolve(); - } - - currentTurnText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ''; - } - - finalMessageText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ''; - } - - rawStructuredOutput(): unknown { - return undefined; - } - - completeTurn(text: string): void { - const pending = this.prompts.shift(); - assert.ok(pending, 'a prompt turn must be in flight'); - this.completedTexts.push(text); - pending.resolve({ stopReason: this.stopReason, text }); - } -} - -class FakeRunner implements BrokerRunner { - readonly sessions: FakeSession[] = []; - - listBackends(): string[] { - return ['claude', 'codex', 'opencode', 'pi']; - } - - defaultBackendId(): string { - return 'claude'; - } - - async openSession(opts: BrokerOpenSessionOptions): Promise { - const session = new FakeSession(opts); - this.sessions.push(session); - return session; - } - - async loadSession(opts: BrokerLoadSessionOptions): Promise { - const session = new FakeSession(opts); - this.sessions.push(session); - return session; - } - - async dispose(): Promise {} - - last(): FakeSession { - assert.ok(this.sessions.length > 0, 'a session must exist'); - return this.sessions[this.sessions.length - 1]; - } -} - -async function setup(): Promise<{ ws: Workspace; broker: Broker }> { - const ws = await Workspace.create(PROJECT); - const broker = await Broker.attach(ws, { - evalTimeoutMs: 0, - }); - return { ws, broker }; -} - -function pendingIds(result: { pending: string[] }): string[] { - return result.pending; -} - -// ── 1. The pending-call registry read is COMPLETE ────────────────────── - -test('review round 4: the pending-call registry read is COMPLETE — 16 500 parked checkpoints report all 16 500 ids (no 16 384-element truncation, no undefined hole) in the eval result AND through the surface', async () => { - const { ws, broker } = await setup(); - try { - // The former read capped the array at 16 384 elements and appended - // an `[ArrayTruncated]` marker that mapped to `undefined` in the - // broker's id lists: 16 400 checkpoints returned 16 384 ids plus - // one undefined entry. The eval result's pending surface must carry - // EVERY id. - const a = await broker.eval(`for (let i = 0; i < ${CHECKPOINTS}; i++) checkpoint("q" + i); "raised"`); - const pending = pendingIds(a); - assert.equal(pending.length, CHECKPOINTS, `the whole registry is reported: ${pending.length}`); - assert.equal(pending[0], 'c1', 'the first id'); - assert.equal(pending[pending.length - 1], `c${CHECKPOINTS}`, 'the last id'); - for (const id of pending) { - assert.ok(typeof id === 'string' && /^c\d+$/.test(id), `no undefined/truncation hole: ${JSON.stringify(id)}`); - } - // The same complete read serves the broker's other seams (the - // surface read is shared): every pending checkpoint is visible. - assert.equal(broker.pendingCalls().length, CHECKPOINTS, 'the broker sees the whole registry'); - assert.equal(broker.pendingCheckpoints().length, CHECKPOINTS, 'every checkpoint is tracked'); - } finally { - await broker.dispose(); - ws.dispose(); - } -}); - -test('review round 4: the restore path\'s registry read is COMPLETE — after a snapshot/restore, the three-way reconciliation re-surfaces all 16 500 pending checkpoints (no truncation on the reconcile path either)', async () => { - // THIS workspace raises the checkpoints (each setup() is a fresh VM at - // the engine level — there is no disk persistence here), then - // snapshots and restores them. - const { ws, broker } = await setup(); - const a = await broker.eval(`for (let i = 0; i < ${CHECKPOINTS}; i++) checkpoint("q" + i); "raised"`); - assert.equal(pendingIds(a).length, CHECKPOINTS); - const snapshot = ws.snapshot(); - await broker.dispose(); - ws.dispose(); - // A fresh broker over the restored VM: the reconciliation reads the - // in-VM pending-call registry through the same surface read. - const ws2 = await Workspace.restore(PROJECT, snapshot); - const broker2 = await Broker.attach(ws2, { evalTimeoutMs: 0 }); - try { - const report = await broker2.reconcile(); - assert.equal(report.requeuedCheckpoints.length, CHECKPOINTS, 'every pending checkpoint re-surfaced'); - assert.equal(report.leftPending.length, 0, 'nothing was left pending (all checkpoints re-surfaced)'); - const pending = broker2.pendingCalls(); - assert.equal(pending.length, CHECKPOINTS, `the whole registry is reported after restore: ${pending.length}`); - for (const entry of pending) { - assert.ok(typeof entry.id === 'string' && /^c\d+$/.test(entry.id), `no undefined hole: ${JSON.stringify(entry.id)}`); - } - } finally { - await broker2.dispose(); - ws2.dispose(); - } -}); - -// ── 2. The provenance registry read is COMPLETE ──────────────────────── - -test('review round 4: the manifest\'s provenance is COMPLETE — 300 bindings all report their origin (the former 256-property object cap dropped bindings 256-299 to null provenance)', async () => { - const { ws, broker } = await setup(); - try { - // One eval creates 300 top-level bindings (the lexical pass — the - // canonical `const research = agent(...)` state at scale). - const code = Array.from({ length: 300 }, (_, i) => `const b${i} = ${i};`).join('\n') + '\n"created"'; - const a = await broker.eval(code); - assert.equal(a.result, 'created'); - const manifest = broker.workspaceManifest(); - const named = manifest.bindings.filter((binding) => /^b\d+$/.test(binding.name)); - assert.equal(named.length, 300, 'every binding is listed'); - for (const binding of named) { - assert.ok( - binding.provenance !== null && binding.provenance.startsWith('eval '), - `binding ${binding.name} keeps its provenance (the eval created it): ${JSON.stringify(binding.provenance)}`, - ); - } - } finally { - await broker.dispose(); - ws.dispose(); - } -}); diff --git a/packages/repl-engine/test/review5.test.ts b/packages/repl-engine/test/review5.test.ts deleted file mode 100644 index 942d9e56..00000000 --- a/packages/repl-engine/test/review5.test.ts +++ /dev/null @@ -1,574 +0,0 @@ -/** - * Phase-D review round 5 regression suite: the reviewer's rejected items, - * pinned at the engine boundary. - * - * 1. The broker's drain latch must never skip in-flight work on a SECOND - * disconnect: a reconnect's fresh dispatch (or a lazy re-attach) - * clears the latch the moment a child may open, so drain → reconnect → - * parked openSession → disconnect drains (and stops) the open instead - * of returning immediately. - * 2. A lazy re-attach whose `loadSession` lands AFTER the drain deadline - * (or after disposal) is released immediately — it never registers and - * never prompts (the drain/disposal generation fence). - * 3. `cancelCall`'s lazy re-attach runs OUTSIDE the serialized operation - * chain — a hung backend `loadSession` can never hold the chain, so - * `drainForDisconnect` enters promptly and its deadline is effective. - * 4. An `openSession` that lands after `dispose` is released immediately - * (never re-registers, never prompts on the disposed broker). - * 5. Simultaneously ready settlements drain ONE CALL AT A TIME, each - * with its own provenance pass: two independent continuations - * producing separate bindings are attributed to their OWN worker and - * task (`worker c1` / `worker c2`), never a joined batch label. - */ - -import assert from 'node:assert/strict'; -import { mkdtempSync, rmSync } from 'node:fs'; -import { tmpdir } from 'node:os'; -import { join } from 'node:path'; -import { test } from 'node:test'; - -import { - Broker, - InMemoryCallStore, - JsonlCallStore, - Workspace, - type BrokerLoadSessionOptions, - type BrokerOpenSessionOptions, - type BrokerPromptOptions, - type BrokerRunner, - type BrokerSession, - type BrokerTurn, - type CallStore, - type SnapshotBoundaryKind, - type SnapshotSink, -} from '../src/index.js'; - -const PROJECT = '/tmp/repl-review5-project'; - -async function tick(): Promise { - await new Promise((resolve) => setTimeout(resolve, 0)); -} - -/** Poll until `predicate` holds (the async-landing tests' wait). */ -async function waitFor(predicate: () => boolean, timeoutMs = 5000): Promise { - const start = Date.now(); - while (!predicate()) { - if (Date.now() - start > timeoutMs) throw new Error('waitFor: condition not met in time'); - await new Promise((resolve) => setTimeout(resolve, 10)); - } -} - -/** The fake held-open ACP session (see review2.test.ts). */ -class FakeSession implements BrokerSession { - readonly sessionId: string; - initializeMeta: Readonly> | undefined; - readonly prompts: Array<{ content: string; resolve: (turn: BrokerTurn) => void; reject: (error: unknown) => void }> = []; - readonly steers: Array<{ content: string; resolve: (outcome: unknown) => void; reject: (error: unknown) => void }> = []; - releases = 0; - stopReason = 'end_turn'; - readonly completedTexts: string[] = []; - backendId = 'pi'; - - constructor(readonly openedWith: BrokerOpenSessionOptions | BrokerLoadSessionOptions) { - this.sessionId = `fake-session-${FakeSession.nextId++}`; - this.initializeMeta = { steering: { supported: true } }; - } - - static nextId = 0; - - prompt(content: string, opts: BrokerPromptOptions = {}): Promise { - this.texts.push(content); - return new Promise((resolve, reject) => { - this.prompts.push({ content, resolve, reject }); - opts.onHandoff?.(); - }); - } - readonly texts: string[] = []; - - steer(content: string): Promise { - return new Promise((resolve, reject) => { - this.steers.push({ content, resolve, reject }); - }); - } - - awaitCurrentTurn(): Promise { - return new Promise(() => {}); - } - - cancel(): Promise { - for (const pending of this.prompts.splice(0)) { - pending.resolve({ stopReason: 'cancelled', text: '' }); - } - return Promise.resolve(); - } - - release(): Promise { - this.releases++; - return Promise.resolve(); - } - - currentTurnText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ''; - } - - finalMessageText(): string { - return this.completedTexts[this.completedTexts.length - 1] ?? ''; - } - - rawStructuredOutput(): unknown { - return undefined; - } - - completeTurn(text: string): void { - const pending = this.prompts.shift(); - assert.ok(pending, 'a prompt turn must be in flight'); - this.completedTexts.push(text); - pending.resolve({ stopReason: this.stopReason, text }); - } -} - -/** The fake runner (see review2.test.ts). */ -class FakeRunner implements BrokerRunner { - readonly sessions: FakeSession[] = []; - readonly openedWith: BrokerOpenSessionOptions[] = []; - readonly loadedWith: BrokerLoadSessionOptions[] = []; - - listBackends(): string[] { - return ['claude', 'codex', 'opencode', 'pi']; - } - - defaultBackendId(): string { - return 'claude'; - } - - async openSession(opts: BrokerOpenSessionOptions): Promise { - const session = new FakeSession(opts); - session.backendId = 'pi'; - this.sessions.push(session); - this.openedWith.push(opts); - return session; - } - - async loadSession(opts: BrokerLoadSessionOptions): Promise { - const session = new FakeSession(opts); - session.backendId = 'pi'; - this.sessions.push(session); - this.loadedWith.push(opts); - return session; - } - - async dispose(): Promise {} - - last(): FakeSession { - assert.ok(this.sessions.length > 0, 'a session must exist'); - return this.sessions[this.sessions.length - 1]; - } -} - -async function setup(options: { - runner?: BrokerRunner; - maxConcurrentAgents?: number; - store?: CallStore; - sink?: SnapshotSink; -} = {}): Promise<{ - ws: Workspace; - broker: Broker; -}> { - const ws = await Workspace.create(PROJECT); - const broker = await Broker.attach(ws, { - runner: options.runner, - store: options.store ?? new InMemoryCallStore(), - evalTimeoutMs: 0, // tests drive interrupts explicitly - snapshotSink: options.sink, - ...(options.maxConcurrentAgents !== undefined ? { maxConcurrentAgents: options.maxConcurrentAgents } : {}), - }); - return { ws, broker }; -} - -/** Settle a call end to end (complete its turn, pump, drain the latch). */ -async function settleAndDrain(broker: Broker, runner: FakeRunner, boundMs = 5000): Promise { - await broker.eval('const p = agent("pi/x", "task"); "started"'); - await tick(); - runner.last().completeTurn('settled'); - await tick(); - await broker.pump(); - assert.equal(await broker.drainForDisconnect(boundMs), true, 'the settled call drains immediately'); - assert.ok(broker.isDrained); -} - -// ── 1. The drain latch never skips in-flight work on a second disconnect ─ - -test('review 5/1: a second disconnect after a reconnect with a PARKED open drains again — the fresh dispatch cleared the stale drain latch, and the parked open is stopped (never prompts after the last client disconnected)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - await settleAndDrain(broker, runner); - - // Reconnect: the next dispatch's openSession is PARKED (slow backend). - let releaseOpen!: () => void; - const parkedOpen = new Promise((resolve) => { - releaseOpen = resolve; - }); - const originalOpen = runner.openSession.bind(runner); - runner.openSession = async (opts) => { - await parkedOpen; - return originalOpen(opts); - }; - await broker.eval('const q = agent("pi/x", "task2"); "started"'); - await tick(); - // The old code kept the latch set until the open RESOLVED — the second - // disconnect then returned immediately and the child could open and - // prompt after the last client disconnected. - assert.ok(!broker.isDrained, 'the fresh dispatch cleared the stale drain latch'); - assert.equal(await broker.drainForDisconnect(60), false, 'the second disconnect drains (and stops) the parked open'); - assert.ok(broker.isDrained); - // The parked open lands later: the child is closed immediately — it - // never prompts (nothing runs after the last client disconnected). - releaseOpen(); - await waitFor(() => runner.sessions.length === 2); - const late = runner.sessions[1]; - assert.equal(late.releases, 1, 'the stopped child was closed without ever prompting'); - assert.equal(late.prompts.length, 0, 'the stopped call never ran a turn'); - await broker.dispose(); - ws.dispose(); -}); - -// Strict idle steering/cancellation no longer trigger lazy reattachment. -// Queue reattachment and its drain fences are covered in broker.test.ts. - -// ── 4. The disposal fence (late opens never register or prompt) ───────── - -test('review 5/4: an openSession that lands AFTER dispose is released immediately — it never registers and never prompts on the disposed broker', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - // Park the founding session's open. - let releaseOpen!: () => void; - const parkedOpen = new Promise((resolve) => { - releaseOpen = resolve; - }); - const originalOpen = runner.openSession.bind(runner); - runner.openSession = async (opts) => { - await parkedOpen; - return originalOpen(opts); - }; - await broker.eval('const p = agent("pi/x", "task"); "started"'); - await tick(); - assert.equal(runner.sessions.length, 0, 'the open is still parked'); - // Dispose while the open is still in flight (the reset path drives the - // same disposal). - await broker.dispose(); - // The parked open lands LATER: the child is released immediately — it - // never registers and never prompts (the old code cleared - // `openingCalls`/`stoppedOpens` without fencing the unresolved open, - // so the landing re-registered the session and could prompt on the - // disposed broker). - releaseOpen(); - await waitFor(() => runner.sessions.length === 1); - const late = runner.sessions[0]; - assert.equal(late.releases, 1, 'the late child was closed without registering'); - assert.equal(late.prompts.length, 0, 'the late open never prompted'); - ws.dispose(); -}); - -// ── 5. Per-call settlement provenance ────────────────────────────────── - -test('review 5/5: simultaneously ready settlements drain ONE CALL AT A TIME — each continuation binding is attributed to its OWN worker and task (never the joined batch label)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - await broker.eval( - 'const a = agent("pi/x", "task A").then((r) => { globalThis.fromA = r; });' + - 'const b = agent("pi/x", "task B").then((r) => { globalThis.fromB = r; });' + - '"started"', - ); - await tick(); - assert.equal(runner.sessions.length, 2, 'two independent subagents'); - // BOTH turns complete before the pump: both outcomes are ready - // simultaneously. - runner.sessions[0].completeTurn('A-result'); - runner.sessions[1].completeTurn('B-result'); - await tick(); - await broker.pump(); - // The per-value producer attribution: the old batch drain labelled - // every binding changed in the batch with ALL call ids (`worker - // c1+c2`) and joined both tasks — two independent continuations were - // falsely attributed to both workers. - const view = ws.provenanceView(); - assert.equal(view.origins.get('fromA')?.via, 'worker c1', 'the A continuation is attributed to worker c1 alone'); - assert.equal(view.origins.get('fromB')?.via, 'worker c2', 'the B continuation is attributed to worker c2 alone'); - const manifest = broker.workspaceManifest(); - const byName = new Map(manifest.bindings.map((b) => [b.name, b])); - assert.equal(byName.get('fromA')?.provenance, 'worker c1'); - assert.equal(byName.get('fromA')?.task, 'task A', 'the "from what task" half follows the same per-value split'); - assert.equal(byName.get('fromB')?.provenance, 'worker c2'); - assert.equal(byName.get('fromB')?.task, 'task B'); - await broker.dispose(); - ws.dispose(); -}); - -// ── 6. interrupt { id } / handle.cancel() on a still-OPENING call ────── - -test('review 8/6a: cancelCall cancels a call whose openSession is still pending — the decision\'s opening arm fences + settles it DURABLY (recorded AGENT_CANCELLED, guest-settled first-wins, concurrency token released), and the LATE child is closed without ever prompting (the phase-E review rejection: cancelCall ignored openingCalls, returned `none`, and the eventual open resolved into a prompted, supposedly-interrupted call)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner, maxConcurrentAgents: 1 }); - let releaseOpen!: () => void; - const parkedOpen = new Promise((resolve) => { - releaseOpen = resolve; - }); - const originalOpen = runner.openSession.bind(runner); - runner.openSession = async (opts) => { - await parkedOpen; - return originalOpen(opts); - }; - await broker.eval('const p = agent("pi/x", "task"); "started"'); - await tick(); - assert.deepEqual( - broker.pendingCalls().map((e) => e.id), - ['c1'], - 'the opening call is pending while the open is in flight', - ); - // The interrupt lands with the open STILL parked. - assert.equal(await broker.cancelCall('c1'), 'cancelled', 'the opening call reports cancelled'); - // The settlement is DURABLE at the interrupt, not deferred to the - // landing: recorded (AGENT_CANCELLED, recoverable), the guest - // promise already rejected, the registry no longer pending. - const record = broker.store().lookup('c1')!; - assert.equal(record.completion!.outcome, 'reject'); - assert.equal((record.completion!.value as { code?: string }).code, 'AGENT_CANCELLED'); - assert.equal((record.completion!.value as { recoverable?: boolean }).recoverable, true); - assert.equal((record.completion!.value as { replBackend?: string }).replBackend, 'pi'); - assert.deepEqual( - broker.pendingCalls().map((e) => e.id), - [], - 'the cancelled opening call is not left pending', - ); - assert.deepEqual(broker.liveAgents(), [], 'no live session was ever registered'); - const uncaught = await broker.eval('await p'); - const uncaughtLine = uncaught.output.find((line) => line.includes('(call c1')); - assert.ok(uncaughtLine !== undefined, uncaught.output.join('\n')); - assert.ok( - uncaughtLine.includes('(call c1 on backend pi)'), - `the opening-call rejection renders its resolved backend: ${uncaughtLine}`, - ); - const got = await broker.eval('await p.catch((e) => "ERR:" + e.message)'); - assert.ok( - (got.result ?? '').includes('turn c1 was cancelled'), - `guest-visible settlement: ${got.result}`, - ); - // The concurrency token was released: under a cap of ONE, a fresh - // dispatch must not be refused. - runner.openSession = originalOpen; - await broker.eval('const q = agent("pi/x", "again"); "started"'); - await tick(); - assert.equal(runner.sessions.length, 1, 'the fresh dispatch opened (the cancelled call freed its slot)'); - runner.sessions[0].completeTurn('ok'); - await tick(); - await broker.pump(); - // The LATE landing of the cancelled open: the child is closed - // immediately — it never prompts (a supposedly-interrupted call must - // not run a turn) — and the late reject is a first-wins no-op - // against the interrupt's recorded completion. - releaseOpen(); - await waitFor(() => runner.sessions.length === 2); - const session = runner.sessions[1]; - assert.equal(session.releases, 1, 'the stopped child was closed without ever prompting'); - assert.equal(session.prompts.length, 0, 'the supposedly-interrupted call never ran a turn'); - for (let attempt = 0; attempt < 100; attempt++) { - await broker.pump(); - const check = await broker.eval('await p.catch((e) => "ERR:" + e.message)'); - if (check.result !== undefined) { - assert.ok((check.result as string).includes('turn c1 was cancelled'), 'the late landing settled nothing new'); - break; - } - await new Promise((resolve) => setTimeout(resolve, 10)); - } - const after = broker.store().lookup('c1')!; - assert.equal(after.completion!.outcome, 'reject'); - assert.equal((after.completion!.value as { recoverable?: boolean }).recoverable, true); - assert.equal(after.reissues, 0, 'never re-issued'); - await broker.dispose(); - ws.dispose(); -}); - -test('review 8/6b: the guest handle cancel() on a still-OPENING call is the same cancellation as the interrupt tool\'s id path — fenced + settled durably as cancelled, and the steer resolves `cancelled` (the phase-E review rejection: the handle cancel fell through to `failed` — "nothing was steered" — while the eventual open went on to prompt a supposedly-cancelled call)', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner }); - let releaseOpen!: () => void; - const parkedOpen = new Promise((resolve) => { - releaseOpen = resolve; - }); - const originalOpen = runner.openSession.bind(runner); - runner.openSession = async (opts) => { - await parkedOpen; - return originalOpen(opts); - }; - await broker.eval('const pi = agent("pi/x", "task"); "started"'); - await tick(); - const outcome = await broker.eval('await pi.cancel()'); - assert.ok( - String(outcome.result).includes('cancelled'), - `the handle cancel resolves with what actually happened: ${outcome.result}`, - ); - const record = broker.store().lookup('c1')!; - assert.equal(record.completion!.outcome, 'reject'); - assert.equal((record.completion!.value as { code?: string }).code, 'AGENT_CANCELLED'); - assert.equal((record.completion!.value as { recoverable?: boolean }).recoverable, true); - assert.equal((record.completion!.value as { replBackend?: string }).replBackend, 'pi'); - assert.equal((broker.store().lookup('c2')!.completion!.value as string), 'cancelled', 'the steer recorded its outcome'); - assert.deepEqual( - broker.pendingCalls().map((e) => e.id), - [], - 'the cancelled opening call is not left pending', - ); - const got = await broker.eval('await pi.catch((e) => "ERR:" + e.message)'); - assert.ok( - (got.result ?? '').includes('turn c1 was cancelled'), - `guest-visible settlement: ${got.result}`, - ); - // The LATE landing closes the child without prompting. - releaseOpen(); - await waitFor(() => runner.sessions.length === 1); - const session = runner.sessions[0]; - assert.equal(session.releases, 1, 'the stopped child was closed without ever prompting'); - assert.equal(session.prompts.length, 0, 'the supposedly-cancelled call never ran a turn'); - await broker.pump(); - const after = broker.store().lookup('c1')!; - assert.equal(after.completion!.outcome, 'reject'); - assert.equal(after.reissues, 0, 'never re-issued'); - await broker.dispose(); - ws.dispose(); -}); - -// ── 9. The opening-cancel's settlement boundary + provenance, and the -// slot release's queued-delivery kick (phase-E review rejection -// round 9) ──────────────────────────────────────────────────── - -test('review 9/1: cancelling a still-OPENING call is a settlement drain that fires the per-settlement provenance pass AND the state-changing boundary — the manifest immediately attributes the continuation\'s binding to the cancelled worker, and an IMMEDIATE snapshot/restart (no eval or wait in between) restores the settled registry with that provenance intact (the phase-E review rejection: the opening-cancel settled and drained the guest but skipped `provenancePass` and `sink.boundary`, so the manifest missed the settlement\'s provenance and a kill right after the interrupt restored the PRE-settlement snapshot with the call still pending — the round-8 daemon regression masked it by performing another eval and wait before the restart)', async () => { - const runner = new FakeRunner(); - const boundaries: SnapshotBoundaryKind[] = []; - let flushes = 0; - const sink: SnapshotSink = { - boundary(kind) { - boundaries.push(kind); - }, - flush() { - flushes++; - }, - }; - const dir = mkdtempSync(join(tmpdir(), 'repl-review9-')); - const storePath = join(dir, 'calls.jsonl'); - const { ws, broker } = await setup({ runner, store: JsonlCallStore.open(storePath), sink }); - // Park the founding session's open. - let releaseOpen!: () => void; - const parkedOpen = new Promise((resolve) => { - releaseOpen = resolve; - }); - const originalOpen = runner.openSession.bind(runner); - runner.openSession = async (opts) => { - await parkedOpen; - return originalOpen(opts); - }; - // The settlement drain's continuation creates a binding: without the - // per-settlement provenance pass, `wasCancelled` would never be - // attributed to the worker that was cancelled. - await broker.eval('const p = agent("pi/x", "task"); p.catch(() => { globalThis.wasCancelled = true; }); "started"'); - await tick(); - boundaries.length = 0; - assert.equal(await broker.cancelCall('c1'), 'cancelled'); - // THE SINK-BOUNDARY ASSERTION: the interrupt ITSELF fired the - // settlement boundary — no eval or wait in between — and the - // serialized operation flushed the burst (the daemon's writer would - // persist the settled workspace before the interrupt's promise - // resolves). - assert.deepEqual(boundaries, ['settlement'], `exactly the settlement boundary fired: ${boundaries.join(',')}`); - assert.ok(flushes >= 1, 'the operation-end burst flush ran'); - // THE PROVENANCE ASSERTION: the continuation the settlement drain ran - // is attributed to the cancelled worker — not a later eval. - const view = ws.provenanceView(); - assert.equal(view.origins.get('wasCancelled')?.via, 'worker c1', 'the settlement pass attributed the continuation binding'); - const manifest = broker.workspaceManifest(); - const binding = manifest.bindings.find((b) => b.name === 'wasCancelled'); - assert.ok(binding !== undefined, 'the continuation binding is in the manifest'); - assert.equal(binding.provenance, 'worker c1', 'the manifest carries the settlement provenance'); - // THE IMMEDIATE RESTART REGRESSION: snapshot NOW — no eval or wait in - // between — and restore over the same store. The restored VM must - // already carry the settlement (the registry is empty — nothing left - // for the reconcile's store arm) with the provenance intact inside - // the snapshot. - const snapshot = ws.snapshot(); - await broker.dispose(); - ws.dispose(); - const ws2 = await Workspace.restore(PROJECT, snapshot); - const runner2 = new FakeRunner(); - const broker2 = await Broker.attach(ws2, { - runner: runner2, - store: JsonlCallStore.open(storePath), - evalTimeoutMs: 0, - }); - const report = await broker2.reconcile(); - assert.deepEqual(report.settledFromStore, [], 'the snapshot already carries the settlement — the store arm has nothing to settle'); - assert.deepEqual( - broker2.pendingCalls().map((e) => e.id), - [], - 'the restored registry is settled, not pending', - ); - const got = await broker2.eval('await p.catch((e) => "ERR:" + e.message)'); - assert.ok( - (got.result ?? '').includes('turn c1 was cancelled'), - `the guest-visible settlement survives the immediate restart: ${got.result}`, - ); - const view2 = ws2.provenanceView(); - assert.equal(view2.origins.get('wasCancelled')?.via, 'worker c1', 'the provenance traveled INSIDE the snapshot'); - await broker2.dispose(); - ws2.dispose(); - rmSync(dir, { recursive: true, force: true }); -}); - -test('review 9/2: cancelling a still-opening call releases its slot through the global scheduler and starts the oldest eligible queued turn', async () => { - const runner = new FakeRunner(); - const { ws, broker } = await setup({ runner, maxConcurrentAgents: 1 }); - // The founding call opens and settles: its session is IDLE with the - // slot free. - await broker.eval('const pi = agent("pi/x", "task"); "started"'); - await tick(); - runner.last().completeTurn('done'); - await tick(); - await broker.pump(); - // A second call dispatches (holding the only slot) with its open - // PARKED: the cap is exhausted while the first session sits idle. - let releaseOpen!: () => void; - const parkedOpen = new Promise((resolve) => { - releaseOpen = resolve; - }); - const originalOpen = runner.openSession.bind(runner); - runner.openSession = async (opts) => { - await parkedOpen; - return originalOpen(opts); - }; - await broker.eval('const q = agent("pi/x", "second"); "started"'); - await tick(); - assert.equal(runner.sessions.length, 1, 'the second call is still opening'); - // A future public turn on the idle session is admitted but cannot run - // while the only slot is held by the opening call. - const queued = await broker.eval('const future = pi.queue("go deeper"); future.then(o => console.log("outcome", o)); "queued"'); - assert.equal(queued.result, 'queued'); - assert.equal(runner.last().prompts.length, 0, 'no delivery turn can start while the cap is exhausted'); - // Cancel the opening call: its slot frees and the global scheduler - // must start the queued public turn. - assert.equal(await broker.cancelCall('c2'), 'cancelled'); - await tick(); - assert.equal(runner.last().prompts.length, 1, 'the queued turn started'); - assert.equal(runner.last().prompts[0].content, 'go deeper'); - runner.last().completeTurn('deeper answer'); - await tick(); - await broker.pump(); - const outcomeProbe = await broker.eval('"probe"'); - assert.ok(outcomeProbe.output.some((l) => l === 'outcome deeper answer'), outcomeProbe.output.join('\n')); - // The late landing of the cancelled open closes the child without - // prompting. - releaseOpen(); - await waitFor(() => runner.sessions.length === 2); - const late = runner.sessions[1]; - assert.equal(late.releases, 1, 'the stopped child was closed without ever prompting'); - assert.equal(late.prompts.length, 0, 'the supposedly-interrupted call never ran a turn'); - await broker.dispose(); - ws.dispose(); -}); diff --git a/packages/repl-engine/test/snapshot-envelope.test.ts b/packages/repl-engine/test/snapshot-envelope.test.ts deleted file mode 100644 index cf77cdc3..00000000 --- a/packages/repl-engine/test/snapshot-envelope.test.ts +++ /dev/null @@ -1,292 +0,0 @@ -/** - * Snapshot-envelope tests (phase D): the at-rest identity envelope for - * quickjs-wasi snapshots — wasm-binary sha256 + format version + gzip, - * per the roadmap doc's transfer lesson 5. Pins: - * - * - the envelope round trip (serialize → deserialize → restore → state - * intact, gzip actually compressing), - * - `wasmSha256Of` (raw bytes hash directly; a module from - * `loadShippedWasm` hashes to the same value; an unknown module - * refuses), - * - the version-bump refusal (an envelope carrying a newer format - * version refuses loudly naming BOTH versions — never a silent - * restore), - * - the format-name refusal, - * - corrupt/truncated header and payload refusals (loud, single-shot). - * - * The wasm-hash-mismatch refusal (naming both hashes) is pinned in - * `repl-store.test.ts`, where the comparison lives. - */ - -import assert from 'node:assert/strict'; -import { readFile } from 'node:fs/promises'; -import { test } from 'node:test'; - -import { - SNAPSHOT_FORMAT, - SNAPSHOT_FORMAT_VERSION, - SnapshotEnvelopeError, - Workspace, - deserializeSnapshot, - loadShippedWasm, - serializeSnapshot, - wasmSha256Of, - type ReplSnapshot, - type WasmModule, -} from '../src/index.js'; - -const PROJECT = '/tmp/repl-envelope-project'; - -/** `assert.throws` returns undefined at runtime — capture the error. */ -function captureThrows(fn: () => unknown): Error { - try { - fn(); - } catch (error) { - return error as Error; - } - assert.fail('expected the call to throw'); -} - -/** The shipped binary's raw bytes (the identity the envelope records). */ -async function shippedBytes(): Promise { - const resolved = import.meta.resolve('quickjs-wasi/quickjs.wasm'); - return new Uint8Array(await readFile(new URL(resolved))); -} - -/** A tiny but structurally valid snapshot stand-in for the pure - * envelope round trip (the shim's deserialize accepts the header + - * memory layout). The pointers are IN-RANGE integers (strictly inside - * the 4-byte memory, nonzero) — the shape/bounds check the engine's - * decoder applies (phase-D review rejection: the check used to be - * type-only). */ -function tinySnapshot(memory: Uint8Array = new Uint8Array([1, 2, 3, 4])): ReplSnapshot { - return { - memory, - stackPointer: 1, - runtimePtr: 2, - contextPtr: 3, - extensions: [], - }; -} - -// ──────────────────────────────────────────────────────────────────────── -// Envelope round trip -// ──────────────────────────────────────────────────────────────────────── - -test('envelope round trip: serialize → deserialize → restore keeps the workspace state', async () => { - const module = await loadShippedWasm(); - const hash = wasmSha256Of(module); - assert.match(hash, /^[0-9a-f]{64}$/); - - const ws = await Workspace.create(PROJECT, { wasm: module }); - await ws.eval('const findings = ["alpha", "beta"]; globalThis.stage = "live";'); - const raw = ws.snapshot(); - const envelope = serializeSnapshot(raw, hash); - - // The envelope is the header line + a gzip payload (gzip magic 0x1f 0x8b). - assert.equal(Buffer.from(envelope.subarray(0, 1))[0], 0x7b, 'the envelope starts with the JSON header line'); - const nl = envelope.indexOf(0x0a); - assert.ok(nl > 0, 'the header is newline-terminated'); - assert.equal(envelope[nl + 1], 0x1f, 'the payload starts with the gzip magic (0x1f)'); - assert.equal(envelope[nl + 2], 0x8b, 'the payload starts with the gzip magic (0x8b)'); - // Compression is real: a fresh VM's memory is mostly zeros. - assert.ok(envelope.length < raw.memory.byteLength, 'the envelope is smaller than the raw memory'); - - const restored = deserializeSnapshot(envelope); - assert.equal(restored.meta.format, SNAPSHOT_FORMAT); - assert.equal(restored.meta.formatVersion, SNAPSHOT_FORMAT_VERSION); - assert.equal(restored.meta.wasmSha256, hash); - assert.ok(typeof restored.meta.createdAtMs === 'number'); - - // The full restore path: the same wasm module, the deserialized snapshot. - const ws2 = await Workspace.restore(PROJECT, restored.snapshot, { wasm: module }); - const outcome = await ws2.eval('findings.join("+") + "/" + stage'); - assert.equal(outcome.kind, 'value'); - assert.equal(outcome.value, 'alpha+beta/live'); - ws.dispose(); - ws2.dispose(); -}); - -test('wasmSha256Of: raw bytes hash directly; a loadShippedWasm module hashes to the same value; an unknown module refuses', async () => { - const bytes = await shippedBytes(); - const fromBytes = wasmSha256Of(bytes); - assert.match(fromBytes, /^[0-9a-f]{64}$/); - // A view over the same bytes hashes identically (byteOffset respected). - const view = new Uint8Array(bytes.buffer, bytes.byteOffset + 1, bytes.byteLength - 1); - assert.notEqual(wasmSha256Of(view), fromBytes, 'a shifted view hashes differently'); - assert.equal(wasmSha256Of(bytes.slice(1)), wasmSha256Of(view), 'the same bytes hash identically'); - - const module = await loadShippedWasm(); - assert.equal(wasmSha256Of(module), fromBytes, 'the compiled module hashes to its binary'); - assert.equal(wasmSha256Of(new Uint8Array([0, 97, 115, 109])).length, 64); - - // A module the engine did not load cannot be hashed (bytes are not - // recoverable from the compiled form) — loud refusal. - const foreign = (await WebAssembly.compile(bytes)) as unknown as WasmModule; - assert.throws(() => wasmSha256Of(foreign), /was not produced by loadShippedWasm/); -}); - -test('serializeSnapshot rejects a malformed wasm hash (identity must be trustworthy)', () => { - assert.throws(() => serializeSnapshot(tinySnapshot(), 'not-a-hash'), /must be a 64-hex|wasmSha256/); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Version-bump and format refusals -// ──────────────────────────────────────────────────────────────────────── - -test('format 3 refuses the previous format-2 envelope before decoding its guest payload', () => { - assert.equal(SNAPSHOT_FORMAT_VERSION, 3); - const envelope = serializeSnapshot(tinySnapshot(), 'a'.repeat(64)); - const nl = envelope.indexOf(0x0a); - const header = JSON.parse(Buffer.from(envelope.subarray(0, nl)).toString('utf8')); - const old = Buffer.concat([ - Buffer.from(JSON.stringify({ ...header, formatVersion: 2 }) + '\n'), - Buffer.from('old guest bytes must not be decoded or executed'), - ]); - const error = captureThrows(() => deserializeSnapshot(old)); - assert.ok(error instanceof SnapshotEnvelopeError, error.message); - assert.equal(error.code, 'VERSION_MISMATCH'); - assert.ok(error.message.includes('2'), `names the recorded version: ${error.message}`); - assert.ok(error.message.includes(String(SNAPSHOT_FORMAT_VERSION)), `names the supported version: ${error.message}`); - assert.equal(error.recorded, '2'); - assert.equal(error.expected, String(SNAPSHOT_FORMAT_VERSION)); -}); - -test('format-name refusal: an envelope carrying another format refuses naming the format', () => { - const envelope = serializeSnapshot(tinySnapshot(), 'a'.repeat(64)); - const nl = envelope.indexOf(0x0a); - const header = JSON.parse(Buffer.from(envelope.subarray(0, nl)).toString('utf8')); - const other = Buffer.concat([ - Buffer.from(JSON.stringify({ ...header, format: 'harness-snapshot' }) + '\n'), - envelope.subarray(nl + 1), - ]); - const error = captureThrows(() => deserializeSnapshot(other)); - assert.ok(error instanceof SnapshotEnvelopeError, error.message); - assert.equal(error.code, 'FORMAT_MISMATCH'); - assert.ok(error.message.includes('harness-snapshot'), error.message); - assert.ok(error.message.includes(SNAPSHOT_FORMAT), error.message); -}); - -// ──────────────────────────────────────────────────────────────────────── -// Corrupt / truncated handling -// ──────────────────────────────────────────────────────────────────────── - -test('a snapshot whose VM-header pointers are out of bounds refuses as CORRUPT_PAYLOAD at decode — the corrupted in-range-format header never reaches the restore (phase-D review rejection: the shape check used to be type-only, so a valid gzip/QJSS payload with `contextPtr` patched to `0xfffffff0` decoded cleanly and then crashed `Workspace.restore` with `RuntimeError: memory access out of bounds`)', async () => { - const module = await loadShippedWasm(); - const ws = await Workspace.create(PROJECT, { wasm: module }); - const raw = ws.snapshot(); - ws.dispose(); - const hash = wasmSha256Of(module); - const cases: Array<[string, number]> = [ - // The reviewer's repro: a pointer patched far outside the memory. - ['contextPtr', 0xfffffff0], - // Just past the memory end. - ['runtimePtr', raw.memory.byteLength], - // Zeroed (malloc'd offsets are never 0). - ['stackPointer', 0], - // Negative (wraps in the wasm ABI). - ['runtimePtr', -1], - ]; - for (const [field, value] of cases) { - const corrupted = { ...raw, [field]: value }; - const envelope = serializeSnapshot(corrupted, hash); - const error = captureThrows(() => deserializeSnapshot(envelope, { expectedWasmSha256: hash })); - assert.ok(error instanceof SnapshotEnvelopeError, `${field}=${value}: ${error.message}`); - assert.equal(error.code, 'CORRUPT_PAYLOAD'); - assert.ok(error.message.includes('unrecognized shape'), `${field}=${value}: ${error.message}`); - } -}); - -test('a corrupted in-range VM header that PASSES the decode checks refuses at RESTORE as SnapshotRestoreError (RESTORE_CORRUPT), never a raw RuntimeError', async () => { - const module = await loadShippedWasm(); - const hash = wasmSha256Of(module); - const ws = await Workspace.create(PROJECT, { wasm: module }); - await ws.eval('globalThis.x = 1'); - const raw = ws.snapshot(); - ws.dispose(); - // The stack pointer patched to an in-range-but-wrong value (1): the - // envelope is fully valid (same binary hash, proper gzip), the QJSS - // payload parses, and the shape/bounds check passes — yet the wasm - // restore traps (`RuntimeError: memory access out of bounds`) the - // moment the stack is used. This is the corruption class NO at-rest - // check can see; `Workspace.restore` must refuse it as a coded, - // single-shot error naming the underlying failure. - const corrupted = { ...raw, stackPointer: 1 }; - const envelope = serializeSnapshot(corrupted, hash); - const decoded = deserializeSnapshot(envelope, { expectedWasmSha256: hash }); - assert.equal(decoded.snapshot.stackPointer, 1, 'decode accepts the in-range header (the corruption is invisible at rest)'); - let error: unknown; - try { - await Workspace.restore(PROJECT, decoded.snapshot, { wasm: module }); - assert.fail('expected the restore to refuse'); - } catch (caught) { - error = caught; - } - assert.ok(error instanceof SnapshotEnvelopeError, `the refusal is in the envelope family: ${(error as Error).message}`); - assert.equal((error as SnapshotEnvelopeError).code, 'RESTORE_CORRUPT'); - // The trap lands on the FIRST wasm call after the memory copy — the - // host-callback re-registration (`registerGuestHostCallbacks`) — so - // this payload exercises the INITIALIZATION-stage wrap (the partial-VM - // disposal path, the reviewer's exact "callback/provenation - // initialization throws" case). Either stage names itself; both are - // the same coded refusal. - const stage = (error as Error).message; - assert.ok( - stage.includes('restoring the workspace VM from the snapshot failed') || - stage.includes('initializing the restored workspace failed'), - `names the restore stage: ${stage}`, - ); - assert.ok(stage.includes('memory access out of bounds'), `names the cause: ${stage}`); - // Repeatable and stable: a second attempt refuses identically (and the - // failed attempt's partial VM was disposed — a good snapshot still - // restores right after). - try { - await Workspace.restore(PROJECT, decoded.snapshot, { wasm: module }); - assert.fail('expected the second restore to refuse identically'); - } catch (caught) { - assert.ok(caught instanceof SnapshotEnvelopeError && caught.code === 'RESTORE_CORRUPT', String(caught)); - } - const good = await Workspace.restore(PROJECT, raw, { wasm: module }); - const outcome = await good.eval('x'); - assert.equal(outcome.kind, 'value'); - assert.equal(outcome.value, 1, 'an uncorrupted snapshot restores after the refused attempts'); - good.dispose(); -}); - -test('corrupt envelopes refuse loudly, naming the file and the problem (single-shot, no silent pass)', () => { - // No header line at all. - const noHeader = new Uint8Array([1, 2, 3, 4, 5]); - let error = captureThrows(() => deserializeSnapshot(noHeader, { path: '/tmp/x.bin' })); - assert.ok(error instanceof SnapshotEnvelopeError, error.message); - assert.equal(error.code, 'BAD_HEADER'); - assert.ok(error.message.includes('/tmp/x.bin'), error.message); - - // A header that is not JSON. - const badJson = Buffer.from('not json at all\nrest-of-file'); - error = captureThrows(() => deserializeSnapshot(badJson)); - assert.ok(error instanceof SnapshotEnvelopeError, error.message); - - // A header with a missing/invalid wasmSha256. - const badHash = Buffer.from(`${JSON.stringify({ format: SNAPSHOT_FORMAT, formatVersion: SNAPSHOT_FORMAT_VERSION, createdAtMs: 1 })}\npayload`); - error = captureThrows(() => deserializeSnapshot(badHash)); - assert.ok(error instanceof SnapshotEnvelopeError, error.message); - - // A truncated gzip payload (the body cut mid-stream — after the - // header line, so the header itself stays intact). - const envelope = serializeSnapshot(tinySnapshot(), 'a'.repeat(64)); - const headerEnd = envelope.indexOf(0x0a) + 1; - const truncated = envelope.subarray(0, headerEnd + Math.floor((envelope.length - headerEnd) / 2)); - error = captureThrows(() => deserializeSnapshot(truncated)); - assert.ok(error instanceof SnapshotEnvelopeError, error.message); - assert.equal((error as SnapshotEnvelopeError).code, 'CORRUPT_PAYLOAD'); - assert.ok(error.message.includes('corrupt or truncated'), error.message); - - // A payload that gunzips but is not a serialized snapshot. - const garbagePayload = Buffer.concat([ - Buffer.from(`${JSON.stringify({ format: SNAPSHOT_FORMAT, formatVersion: SNAPSHOT_FORMAT_VERSION, wasmSha256: 'a'.repeat(64), createdAtMs: 1 })}\n`), - Buffer.from('this is not a snapshot'), - ]); - error = captureThrows(() => deserializeSnapshot(garbagePayload)); - assert.ok(error instanceof SnapshotEnvelopeError, error.message); - assert.equal((error as SnapshotEnvelopeError).code, 'CORRUPT_PAYLOAD'); -}); diff --git a/packages/repl-engine/test/steering-table.test.ts b/packages/repl-engine/test/steering-table.test.ts deleted file mode 100644 index 43c18183..00000000 --- a/packages/repl-engine/test/steering-table.test.ts +++ /dev/null @@ -1,72 +0,0 @@ -/** - * The generated-documentation GATE for the per-backend steering - * mechanism table (the roadmap doc's spec-owed decision: "the table is - * documentation generated from the capability probes" — implemented as - * a generated artifact, never deferred to a later phase). The gate - * regenerates the document from the LIVE capability probes - * (`ACP_EXTENSION_SUPPORT_MATRIX` in `@automatalabs/acp-agents`) and - * compares it byte-for-byte with the checked-in - * `docs/steering-mechanism-table.md`: a capability-matrix change that - * is not reflected in the documentation fails the suite. - * - * Regenerate with - * `pnpm --filter @automatalabs/repl-engine generate:steering-table`. - */ - -import assert from 'node:assert/strict'; -import { readFile } from 'node:fs/promises'; -import { test } from 'node:test'; -import { fileURLToPath } from 'node:url'; - -import { - generateSteeringMechanismTable, - steeringMechanismRows, -} from '../src/steering-table.js'; - -test('the per-backend steering mechanism table is GENERATED from the capability probes and the checked-in document matches (the doc\'s generated table is implemented and gated, never deferred)', async () => { - const generated = generateSteeringMechanismTable(); - const checkedIn = await readFile( - fileURLToPath(new URL('../docs/steering-mechanism-table.md', import.meta.url)), - 'utf8', - ); - assert.equal( - generated, - checkedIn, - 'docs/steering-mechanism-table.md drifted from the capability probes — run ' + - '`pnpm --filter @automatalabs/repl-engine generate:steering-table` and commit the regenerated document', - ); -}); - -test('the generated table reflects the live probe dispositions (every built-in backend row is derived, and the mechanism follows the disposition)', () => { - const rows = steeringMechanismRows(); - // The built-in backends' `_session/steering` dispositions, straight - // from the probed matrix (protocol-coverage.ts): claude, codex and pi - // advertise the extension; opencode is typed-unsupported. This table - // is documentation only — runtime routing reads raw initialize metadata. - const byBackend = new Map(rows.map((row) => [row.backend, row])); - assert.deepEqual([...byBackend.keys()].sort(), ['claude', 'codex', 'opencode', 'pi']); - assert.equal(byBackend.get('claude')!.advertised, true); - assert.equal(byBackend.get('claude')!.mechanism, 'strict active-turn injection'); - assert.equal(byBackend.get('claude')!.distProbe, 'claude'); - assert.equal(byBackend.get('codex')!.advertised, true); - assert.equal(byBackend.get('codex')!.mechanism, 'strict active-turn injection'); - assert.equal(byBackend.get('codex')!.distProbe, 'codex'); - assert.equal(byBackend.get('pi')!.advertised, true); - assert.equal(byBackend.get('pi')!.mechanism, 'strict active-turn injection'); - assert.equal(byBackend.get('opencode')!.advertised, false); - assert.equal(byBackend.get('opencode')!.disposition, 'not-advertised'); - assert.equal(byBackend.get('opencode')!.mechanism, 'unsupported'); - const doc = generateSteeringMechanismTable(); - assert.ok(doc.includes('custom backend'), 'the raw-metadata custom row is documented'); - assert.ok(doc.includes('handle.queue(prompt)'), 'future turns are documented as explicit queue work'); - assert.ok(doc.includes('no steering wire request'), 'unadvertised steering never falls back to a prompt'); - assert.ok( - !doc.includes('queued-for-next-turn delivery'), - 'the removed queued-steering fallback is absent', - ); - // EXACTLY ONE terminal newline (phase-E review rejection: the - // generator emitted two, so `git diff --check` failed with "new blank - // line at EOF" on the checked-in artifact). - assert.ok(doc.endsWith('\n'), 'the document ends with a newline'); - assert.ok(!doc.endsWith('\n\n'), 'exactly one terminal newline — no trailing blank line'); -}); diff --git a/packages/repl-engine/test/store.test.ts b/packages/repl-engine/test/store.test.ts deleted file mode 100644 index 68d853c3..00000000 --- a/packages/repl-engine/test/store.test.ts +++ /dev/null @@ -1,246 +0,0 @@ -/** - * Call-store tests: the append-only results-by-call-id ledger behind - * exactly-once settlement. Pins the first-wins semantics (dispatch and - * completion idempotence), the JSONL replay, and the crash-torn-tail - * repair discipline (the harness's R55/R81 semantics: kill-at-any-point - * is the normal lifecycle, so a torn tail must repair — never brick the - * session — while newline-terminated corruption stays a hard error). - */ - -import assert from 'node:assert/strict'; -import { existsSync, mkdtempSync, readdirSync, readFileSync, statSync, writeFileSync, rmSync } from 'node:fs'; -import { tmpdir } from 'node:os'; -import { join } from 'node:path'; -import { test } from 'node:test'; - -import { InMemoryCallStore, JsonlCallStore, type CallKind, type CallOutcome, type CallRecord } from '../src/index.js'; - -function record(callId: string, kind: CallKind = 'agent'): CallRecord { - return { - callId, - kind, - detail: `task ${callId}`, - optionsJson: null, - modelSpec: kind === 'agent' ? 'pi/x' : null, - backendId: null, - foundingCallId: kind === 'queue' || kind === 'steer' || kind === 'cancel' ? 'c1' : null, - admittedAtMs: 1, - admissionSequence: Number(callId.slice(1)), - dispatchedAtMs: 1, - reissues: 0, - completion: null, - sessionId: null, - queuedAtMs: null, - handoffAtMs: null, - cancelledAtMs: null, - }; -} - -function outcome(value: unknown, outcomeKind: 'resolve' | 'reject' = 'resolve'): CallOutcome { - return { outcome: outcomeKind, value, completedAtMs: 2 }; -} - -function tmpStore(): { dir: string; path: string } { - const dir = mkdtempSync(join(tmpdir(), 'repl-store-')); - return { dir, path: join(dir, 'calls.jsonl') }; -} - -// ──────────────────────────────────────────────────────────────────────── -// In-memory store -// ──────────────────────────────────────────────────────────────────────── - -test('in-memory store: first-wins dispatch and completion, unknown ids refused', () => { - const store = new InMemoryCallStore(); - store.recordDispatched(record('c1')); - // A re-dispatch of a known id keeps the original record. - store.recordDispatched({ ...record('c1'), detail: 'second dispatch' }); - assert.equal(store.lookup('c1')!.detail, 'task c1'); - // First completion wins; the second reports false and changes nothing. - assert.equal(store.recordCompleted('c1', outcome('first')), true); - assert.equal(store.recordCompleted('c1', outcome('second')), false); - assert.equal(store.lookup('c1')!.completion!.value, 'first'); - // Unknown ids are refused loudly (a dangling completion/re-issue would - // corrupt the replay ledger). - assert.throws(() => store.recordCompleted('cX', outcome('x')), /no record for call cX/); - assert.throws(() => store.recordReissued('cX', 9), /no record for call cX/); - // Re-issues bump the counter on the original record. - store.recordReissued('c1', 9); - assert.equal(store.lookup('c1')!.reissues, 1); - // Dispatch order is preserved in all(). - store.recordDispatched(record('c2', 'checkpoint')); - assert.deepEqual(store.all().map((r) => r.callId), ['c1', 'c2']); -}); - -test('in-memory store: queue admission, handoff, cancellation, and refusal settlement are durable first-wins fields', () => { - const store = new InMemoryCallStore(); - store.recordDispatched({ - ...record('c2', 'queue'), - detail: 'implement the fix', - optionsJson: '{"promptMeta":{"trace":"yes"}}', - }); - store.recordQueued('c2', 10); - store.recordQueued('c2', 11); - store.recordHandoff('c2', 20); - store.recordHandoff('c2', 21); - store.recordCancelled('c2', 30); - store.recordCancelled('c2', 31); - assert.deepEqual(store.lookup('c2'), { - ...record('c2', 'queue'), - detail: 'implement the fix', - optionsJson: '{"promptMeta":{"trace":"yes"}}', - queuedAtMs: 10, - handoffAtMs: 20, - cancelledAtMs: 30, - }); - - store.recordDispatched({ ...record('c3', 'queue'), detail: 'invalid admission' }); - assert.equal(store.recordCompleted('c3', outcome({ - message: 'queue options: unknown option "schema"', - code: 'SCRIPT_VALIDATION_ERROR', - recoverable: false, - }, 'reject')), true); - assert.equal(store.lookup('c3')!.queuedAtMs, null, 'a validation refusal never enters the FIFO'); - assert.equal(store.lookup('c3')!.completion!.outcome, 'reject', 'the refusal is nevertheless durable'); -}); - -// ──────────────────────────────────────────────────────────────────────── -// JSONL store: replay -// ──────────────────────────────────────────────────────────────────────── - -test('jsonl store: appends replay on reopen; first-wins holds across reopens', () => { - const { path } = tmpStore(); - const store = JsonlCallStore.open(path); - store.recordDispatched(record('c1')); - store.recordDispatched(record('c2', 'checkpoint')); - store.recordDispatched(record('c3', 'queue')); - store.recordQueued('c3', 3); - store.recordHandoff('c3', 4); - store.recordCancelled('c3', 5); - store.recordCompleted('c1', outcome('done')); - store.close(); - - const reopened = JsonlCallStore.open(path); - assert.equal(reopened.lookup('c1')!.completion!.value, 'done'); - assert.equal(reopened.lookup('c2')!.kind, 'checkpoint'); - assert.equal(reopened.lookup('c2')!.completion, null); - assert.equal(reopened.lookup('c3')!.kind, 'queue'); - assert.equal(reopened.lookup('c3')!.queuedAtMs, 3); - assert.equal(reopened.lookup('c3')!.handoffAtMs, 4); - assert.equal(reopened.lookup('c3')!.cancelledAtMs, 5); - // A second completion after reopen is refused (first-wins, log unchanged). - assert.equal(reopened.recordCompleted('c1', outcome('second')), false); - assert.equal(reopened.recordCompleted('c2', outcome('answered')), true); - reopened.close(); - - const again = JsonlCallStore.open(path); - assert.equal(again.lookup('c1')!.completion!.value, 'done'); - assert.equal(again.lookup('c2')!.completion!.value, 'answered'); - again.close(); - rmSync(join(path, '..'), { recursive: true, force: true }); -}); - -// ──────────────────────────────────────────────────────────────────────── -// JSONL store: torn-tail repair -// ──────────────────────────────────────────────────────────────────────── - -test('jsonl store: an unterminated unparseable tail is repaired (fragment preserved in a sidecar), records intact', () => { - const { dir, path } = tmpStore(); - const store = JsonlCallStore.open(path); - store.recordDispatched(record('c1')); - store.recordCompleted('c1', outcome('ok')); - store.close(); - // Simulate a crash mid-append: a partial JSON line with no newline. - writeFileSync(path, readFileSync(path).toString() + 'ZZZ-TORN-{"event":"compl'); - // The append-side heal must ALSO apply: opening truncates the torn tail. - const reopened = JsonlCallStore.open(path); - assert.equal(reopened.lookup('c1')!.completion!.value, 'ok'); - // A subsequent append lands cleanly on its own line. - assert.equal(reopened.recordDispatched(record('c2')), undefined); - reopened.close(); - const raw = readFileSync(path, 'utf8'); - assert.ok(raw.endsWith('\n'), 'the repaired log is newline-terminated'); - assert.ok(!raw.includes('ZZZ-TORN-'), 'the torn fragment is gone from the log'); - // The fragment was durably preserved before the truncation. - const sidecars = readdirSync(dir).filter((f) => f.includes('.torn-')); - assert.equal(sidecars.length, 1); - assert.ok(readFileSync(join(dir, sidecars[0]), 'utf8').includes('ZZZ-TORN-')); - rmSync(dir, { recursive: true, force: true }); -}); - -test('jsonl store: an unterminated but complete tail is KEPT with its terminator restored', () => { - const { path } = tmpStore(); - const store = JsonlCallStore.open(path); - store.recordDispatched(record('c1')); - store.close(); - // The crash landed between the record's bytes and its newline: the - // record is complete and must survive (a completion whose result was - // already paid for must not vaporize). - const line = readFileSync(path, 'utf8').trim(); - writeFileSync(path, line); // drop the trailing newline - const reopened = JsonlCallStore.open(path); - assert.equal(reopened.lookup('c1')!.callId, 'c1'); - // The terminator is restored so the next append starts its own line. - assert.equal(reopened.recordDispatched(record('c2')), undefined); - reopened.close(); - const raw = readFileSync(path, 'utf8'); - const lines = raw.trimEnd().split('\n'); - assert.equal(lines.length, 2); - assert.ok(lines.every((l) => l.trim() !== '')); - rmSync(join(path, '..'), { recursive: true, force: true }); -}); - -test('jsonl store: newline-terminated corruption anywhere is a hard error (external damage, not a crash)', () => { - const { path } = tmpStore(); - const store = JsonlCallStore.open(path); - store.recordDispatched(record('c1')); - store.close(); - // A line that HAS its newline was fully written — garbage there means - // external damage, and skipping it would corrupt the replay ledger. - writeFileSync(path, readFileSync(path, 'utf8') + 'not json at all\n'); - assert.throws(() => JsonlCallStore.open(path), /corrupt log line/); - rmSync(join(path, '..'), { recursive: true, force: true }); -}); - -test('jsonl store: a partial append is healed to the acknowledged prefix before the next write', () => { - const { path } = tmpStore(); - const store = JsonlCallStore.open(path); - store.recordDispatched(record('c1')); - store.recordCompleted('c1', outcome('ok')); - // Simulate a failed write's residue: bytes beyond the acknowledged - // prefix with no newline (as if writeSync returned mid-line and the - // caller retried). - const size = statSync(path).size; - writeFileSync(path, readFileSync(path, 'utf8') + '{"event":"completed","callId":"c2","outcome":'); - // The next append heals the residue first: the log stays parseable. - store.recordDispatched(record('c3')); - store.close(); - const reopened = JsonlCallStore.open(path); - assert.equal(reopened.lookup('c1')!.completion!.value, 'ok'); - assert.equal(reopened.lookup('c2'), undefined, 'the partial record was rolled back'); - assert.equal(reopened.lookup('c3')!.callId, 'c3'); - assert.ok(size > 0, 'sanity: the file had content'); - reopened.close(); - rmSync(join(path, '..'), { recursive: true, force: true }); -}); - -test('jsonl store: unknown-id completions refuse to append (the log never carries dangling completions)', () => { - const { path } = tmpStore(); - const store = JsonlCallStore.open(path); - store.recordDispatched(record('c1')); - assert.throws(() => store.recordCompleted('cX', outcome('x')), /no record for call cX/); - store.close(); - const raw = readFileSync(path, 'utf8'); - assert.ok(!raw.includes('cX'), 'nothing was appended'); - rmSync(join(path, '..'), { recursive: true, force: true }); -}); - -test('jsonl store: a missing file opens empty and creates the log on first write', () => { - const { path } = tmpStore(); - assert.ok(!existsSync(path)); - const store = JsonlCallStore.open(path); - assert.equal(store.all().length, 0); - store.recordDispatched(record('c1')); - store.close(); - assert.ok(existsSync(path)); - rmSync(join(path, '..'), { recursive: true, force: true }); -}); diff --git a/packages/repl-engine/test/vm.test.ts b/packages/repl-engine/test/vm.test.ts deleted file mode 100644 index 6144188e..00000000 --- a/packages/repl-engine/test/vm.test.ts +++ /dev/null @@ -1,858 +0,0 @@ -/** - * Engine-level tests: VM instantiation (shipped wasm), eval semantics, - * the job drain, memory limits, and per-eval interrupts. Deterministic - * and credential-free; every case runs against the `quickjs-wasi` npm - * package's shipped `quickjs.wasm` binary (the doc's mapping table: the - * package used as-is, including its binary). - */ - -import assert from 'node:assert/strict'; -import { test } from 'node:test'; - -import { EvalFlags, QuickJS } from 'quickjs-wasi'; - -import { DrainJobError, ReplVm, loadShippedWasm } from '../src/index.js'; -import { getVmShim } from '../src/vm.js'; -import { JSValueHandle, type QuickJS } from 'quickjs-wasi'; - -type VmOptions = NonNullable[0]>; - -async function vm(options?: VmOptions): Promise { - return ReplVm.create(options); -} - -function describe(outcome: unknown): string { - try { - return JSON.stringify(outcome, (_k, v) => (typeof v === 'bigint' ? `${v}n` : v)); - } catch { - return String(outcome); - } -} - -function value(outcome: Awaited>): unknown { - assert.equal(outcome.kind, 'value', `expected value outcome, got ${describe(outcome)}`); - return outcome.value; -} - -function error(outcome: Awaited>): { - name: string; - message: string; - interrupted: boolean; - outOfMemory: boolean; -} { - assert.equal(outcome.kind, 'error', `expected error outcome, got ${describe(outcome)}`); - return outcome.error; -} - -test('loadShippedWasm resolves the npm package binary and compiles it', async () => { - const module = await loadShippedWasm(); - assert.ok(module instanceof WebAssembly.Module); - // Process-wide cache: repeated loads return the same compiled module. - assert.equal(module, await loadShippedWasm()); -}); - -test('eval round-trip: numbers, strings, booleans, undefined, bigint, null', async () => { - using v = await vm(); - assert.equal(value(await v.evalCode('1 + 2')), 3); - assert.equal(value(await v.evalCode('6 * 7')), 42); - assert.equal(value(await v.evalCode('"hello" + " world"')), 'hello world'); - assert.equal(value(await v.evalCode('true && !false')), true); - assert.equal(value(await v.evalCode('undefined')), undefined); - assert.equal(value(await v.evalCode('null')), null); - assert.equal(value(await v.evalCode('10n ** 3n')), 1000n); -}); - -test('eval round-trip: multi-statement scripts complete with the last expression', async () => { - using v = await vm(); - // REPL-critical multi-statement semantics (the harness's pinned shape). - assert.equal(value(await v.evalCode('1; 2; 6 * 7')), 42); -}); - -test('eval round-trip: object and array completions via trap-free reads', async () => { - using v = await vm(); - assert.deepEqual(value(await v.evalCode('({ a: 1, b: "two", c: [1, 2, 3] })')), { - a: 1, - b: 'two', - c: [1, 2, 3], - }); - assert.deepEqual(value(await v.evalCode('[1, "x", true, null]')), [1, 'x', true, null]); -}); - -test('state persists across evals: bindings live in the VM, not in a transcript', async () => { - using v = await vm(); - assert.equal(value(await v.evalCode('let counter = 41; const label = "n"')), undefined); - assert.equal(value(await v.evalCode('counter + 1')), 42); - assert.equal(value(await v.evalCode('`${label}=${counter}`')), 'n=41'); - // var hoists to globalThis, like a REPL. - assert.equal(value(await v.evalCode('var hoisted = 7; hoisted * 6')), 42); - assert.equal(value(await v.evalCode('globalThis.hoisted')), 7); -}); - -test('determinism is intact: Date.now() and Math.random() work natively in the realm (the doc exclusion list — no frozen replacements, no journal)', async () => { - using v = await vm(); - // Date.now(): a REAL timestamp, not a frozen/zeroed replacement — two calls - // in one eval observe real time passing, and the value round-trips. - const stamp = value(await v.evalCode('Date.now()')); - assert.equal(typeof stamp, 'number'); - assert.ok(stamp > 1_500_000_000_000, 'a plausible 2020+ epoch millis value'); - assert.ok(value(await v.evalCode('Date.now()')) >= stamp, 'time never runs backwards'); - // Math.random(): a REAL [0,1) value, not a deterministic replacement — a - // fresh draw differs from the previous one (a fixed-seed replacement would - // make the assert fail; a per-call counter is astronomically unlikely to - // collide once). - const first = value(await v.evalCode('Math.random()')); - const second = value(await v.evalCode('Math.random()')); - assert.equal(typeof first, 'number'); - assert.ok(first >= 0 && first < 1); - assert.ok(second >= 0 && second < 1); - assert.notEqual(first, second); -}); - -test('top-level await is accepted; microtask-only awaits resolve in-eval', async () => { - using v = await vm(); - assert.equal(value(await v.evalCode('await Promise.resolve(42)')), 42); - assert.equal(value(await v.evalCode('const a = await Promise.resolve(6); a * 7')), 42); - assert.equal( - value(await v.evalCode('let x = 0; await Promise.resolve().then(() => { x = 42 }); x')), - 42, - ); -}); - -test('top-level return stays a syntax error (the doc pins this)', async () => { - using v = await vm(); - const e = error(await v.evalCode('return 1')); - assert.equal(e.name, 'SyntaxError'); - assert.match(e.message, /return/); -}); - -test('syntax errors report name and message, VM stays usable', async () => { - using v = await vm(); - const e = error(await v.evalCode('const =')); - assert.equal(e.name, 'SyntaxError'); - assert.ok(e.message.length > 0); - assert.equal(value(await v.evalCode('1 + 1')), 2); -}); - -test('synchronous parse failures never invoke guest getters (trap-free error boundary)', async () => { - using v = await vm(); - // Adversarial regression (review): quickjs-wasi's `evalCode()` wraps a - // synchronous parse failure in a `JSException` whose constructor - // performs guest-visible `[[Get]]` reads of name/message/stack — a - // getter installed on `SyntaxError.prototype.name` executed during - // error reporting, before any host catch. The engine's eval path must - // never construct that exception: descriptor reads only. - assert.equal( - value( - await v.evalCode(` - globalThis.__traps = 0; - for (const key of ['name', 'message', 'stack']) { - Object.defineProperty(SyntaxError.prototype, key, { - configurable: true, - get() { globalThis.__traps++; return 'trapped'; }, - }); - } - 'installed'; - `), - ), - 'installed', - ); - const e = error(await v.evalCode('const =')); - // The real message is an own data property of the SyntaxError instance - // and is still reported; the accessor `name` (and stack) are skipped, - // never invoked, so the name falls back to 'Error'. - assert.equal(e.name, 'Error'); - assert.ok(e.message.length > 0); - assert.equal(value(await v.evalCode('globalThis.__traps')), 0, 'no guest getter ran'); - assert.equal(value(await v.evalCode('1 + 1')), 2); -}); - -test('rejected completions never invoke guest getters on the error', async () => { - using v = await vm(); - value( - await v.evalCode(` - globalThis.__traps = 0; - Object.defineProperty(TypeError.prototype, 'name', { - configurable: true, - get() { globalThis.__traps++; return 'TypeError'; }, - }); - 'installed'; - `), - ); - const e = error(await v.evalCode('const err = new TypeError("boom"); throw err')); - assert.equal(e.message, 'boom'); - assert.equal(e.name, 'Error', 'accessor name is skipped, never invoked'); - assert.equal(value(await v.evalCode('globalThis.__traps')), 0, 'no guest getter ran'); -}); - -test('a thrown proxy is reported trap-free (no descriptor/prototype traps)', async () => { - using v = await vm(); - // Adversarial regression (review): a thrown proxy executed three guest - // traps (one per name/message/stack descriptor read). Every descriptor - // and prototype inspection must be guarded with `isProxy` first, and - // the proxy reports a trap-free marker. - const e = error( - await v.evalCode(` - globalThis.__traps = 0; - const p = new Proxy({}, { - getOwnPropertyDescriptor(t, k) { globalThis.__traps++; return Reflect.getOwnPropertyDescriptor(t, k); }, - getPrototypeOf(t) { globalThis.__traps++; return Reflect.getPrototypeOf(t); }, - get(t, k) { globalThis.__traps++; return Reflect.get(t, k); }, - ownKeys(t) { globalThis.__traps++; return Reflect.ownKeys(t); }, - }); - throw p; - `), - ); - assert.equal(e.name, 'Error'); - assert.equal(e.message, '[Proxy]'); - assert.match(e.stack ?? '', /:\d+:\d+/, 'the proxy throw keeps its submitted-code frame'); - assert.equal(value(await v.evalCode('globalThis.__traps')), 0, 'no proxy trap ran'); -}); - -test('an error whose prototype is a proxy is reported without firing its traps', async () => { - using v = await vm(); - // The error object itself is not a proxy, but its prototype is - // (`Object.setPrototypeOf` works on errors) — reading `name` off that - // prototype would fire the proxy's `getOwnPropertyDescriptor` trap. - const e = error( - await v.evalCode(` - globalThis.__traps = 0; - const proto = new Proxy({ name: 'TypeError' }, { - getOwnPropertyDescriptor(t, k) { globalThis.__traps++; return Reflect.getOwnPropertyDescriptor(t, k); }, - }); - const err = new TypeError('boom'); - Object.setPrototypeOf(err, proto); - throw err; - `), - ); - assert.equal(e.message, 'boom'); - assert.equal(e.name, 'Error', 'proxy-prototype name is never read'); - assert.equal(value(await v.evalCode('globalThis.__traps')), 0, 'no proxy trap ran'); -}); - -test('an eval suspended on an unsettled promise reports pending, with no fabricated value', async () => { - using v = await vm(); - const outcome = await v.evalCode('const gate = new Promise(() => {}); await gate; "never"'); - assert.equal(outcome.kind, 'pending'); - // The VM stays fully usable; the suspended continuation is ordinary state. - assert.equal(value(await v.evalCode('1 + 1')), 2); - // Unawaited started handles also complete as the value being a promise. - assert.equal(value(await v.evalCode('new Promise(() => {})')), '[Promise]'); -}); - -test('thrown values report trap-free error info (name via prototype read)', async () => { - using v = await vm(); - const e = error(await v.evalCode('throw new TypeError("nope")')); - assert.equal(e.name, 'TypeError'); - assert.equal(e.message, 'nope'); - assert.equal(e.interrupted, false); - assert.equal(e.outOfMemory, false); - const e2 = error(await v.evalCode('throw new Error("boom")')); - assert.equal(e2.name, 'Error'); - assert.equal(e2.message, 'boom'); - // Primitive throws surface with their string conversion. - const e3 = error(await v.evalCode('throw "plain string"')); - assert.equal(e3.name, 'Error'); - assert.equal(e3.message, 'plain string'); - assert.match(e3.stack ?? '', /:1:\d+/, 'a thrown string keeps the submitted-code line'); - const e4 = error(await v.evalCode('\nthrow null')); - assert.equal(e4.message, 'null'); - assert.match(e4.stack ?? '', /:2:\d+/, 'a thrown null keeps the submitted-code line'); - const e5 = error(await v.evalCode('\nthrow undefined')); - assert.equal(e5.message, 'undefined'); - assert.match(e5.stack ?? '', /:2:\d+/, 'a thrown undefined keeps the submitted-code line'); -}); - -test('throw-site capture preserves user errors when globalThis is shadowed around a function', async () => { - using topLevel = await vm(); - assert.equal( - value( - await topLevel.evalCode( - 'const globalThis = 7; function f() { throw new Error("T") } try { f() } catch (e) { e.message }', - ), - ), - 'T', - 'a top-level lexical globalThis shadow cannot replace the thrown error', - ); - assert.equal( - value( - await topLevel.evalCode( - 'class C { m() { throw new Error("M") } } try { new C().m() } catch (e) { e.message }', - ), - ), - 'M', - 'a class method keeps the user error after a persistent globalThis shadow', - ); - - using parameter = await vm(); - assert.equal( - value( - await parameter.evalCode( - 'function f(globalThis) { throw new Error("P") } try { f(7) } catch (e) { e.message }', - ), - ), - 'P', - 'a globalThis parameter cannot replace the thrown error', - ); - - using local = await vm(); - assert.equal( - value( - await local.evalCode( - 'function f() { const globalThis = 7; throw new Error("L") } try { f() } catch (e) { e.message }', - ), - ), - 'L', - 'a function-local globalThis binding cannot replace the thrown error', - ); - - using uncaught = await vm(); - const e = error( - await uncaught.evalCode('const globalThis = 7;\nfunction g() { throw new Error("shadowed") }\ng()'), - ); - assert.equal(e.message, 'shadowed', 'the uncaught error itself is preserved'); - assert.match(e.stack ?? '', /:2:\d+/, 'the preserved error keeps its submitted-code frame'); - - using reserved = await vm(); - assert.equal( - value( - await reserved.evalCode( - 'function f(__replCaptureThrownValueV2) { throw new Error("R") } ' + - 'try { f(() => "corrupt") } catch (e) { e.message }', - ), - ), - 'R', - 'a local binding with the reserved helper name makes capture skip the throw instead of changing it', - ); -}); - -test('throw-site capture never reuses a handled primitive throw for an uninstrumented throw', async () => { - using acrossEvals = await vm(); - assert.equal(value(await acrossEvals.evalCode('try { throw "boom" } catch (e) { "handled" }')), 'handled'); - const later = error( - await acrossEvals.evalCode( - 'with ({}) { 1 }\n\n\n\nfunction f() { throw "boom" }\nf();', - ), - ); - assert.equal(later.message, 'boom'); - assert.equal(later.stack, undefined, 'an uninstrumented throw cannot inherit a frame from an earlier eval'); - - using withinEval = await vm(); - const dynamic = error( - await withinEval.evalCode( - 'try { throw "boom" } catch (e) {}\n' + - 'const f = new Function(\'throw "boom"\');\n' + - 'f();', - ), - ); - assert.equal(dynamic.message, 'boom'); - assert.equal(dynamic.stack, undefined, 'a handled throw cannot leak its frame to later dynamic code'); -}); - -test('rejected top-level awaits report the raw thrown value', async () => { - using v = await vm(); - const e = error(await v.evalCode('await Promise.reject(new RangeError("too big"))')); - assert.equal(e.name, 'RangeError'); - assert.equal(e.message, 'too big'); -}); - -test('the completion unwrap is trap-free: Object.prototype.value pollution cannot hijack results', async () => { - using v = await vm(); - // R69's regression: a plain [[Get]] unwrap of the `{ value }` completion - // wrapper lets a guest pollute every eval result. The engine unwraps via - // own-property-descriptor reads; the pollution must not leak through. - assert.equal(value(await v.evalCode('Object.prototype.value = "polluted"')), 'polluted'); - const result = value(await v.evalCode('({ real: 42 })')); - assert.deepEqual(result, { real: 42 }); - assert.equal(value(await v.evalCode('globalThis.value')), 'polluted'); -}); - -test('completion reads never invoke guest getters (trap-free rendering)', async () => { - using v = await vm(); - const result = value( - await v.evalCode( - 'let calls = 0; globalThis.calls = 0; ({ get secret() { globalThis.calls++; return "gotcha" }, plain: 1 })', - ), - ); - assert.deepEqual(result, { plain: 1 }); // accessors are skipped, never invoked - assert.equal(value(await v.evalCode('globalThis.calls')), 0); -}); - -test('job drain: microtasks queued by an eval settle within the drain', async () => { - using v = await vm(); - assert.equal( - value( - await v.evalCode(` - let acc = 0; - for (let i = 0; i < 500; i++) Promise.resolve().then(() => { acc++ }); - "queued" - `), - ), - 'queued', - ); - // Every queued microtask ran inside the eval's drain. - assert.equal(value(await v.evalCode('acc')), 500); -}); - -test('job drain: drainJobs() is the standalone settlement drain', async () => { - using v = await vm(); - // Nothing pending: a drain is a no-op returning 0. - assert.equal(value(await v.evalCode('"idle"')), 'idle'); - assert.equal(v.drainJobs(), 0); - // A suspended top-level await leaves nothing runnable in the queue. - const pending = await v.evalCode('const gate = new Promise(() => {}); await gate; 1'); - assert.equal(pending.kind, 'pending'); - assert.equal(v.drainJobs(), 0); - // And the VM keeps working after both. - assert.equal(value(await v.evalCode('1 + 1')), 2); -}); - -test('job-drain errors never invoke guest getters (trap-free drain boundary)', async () => { - using v = await vm(); - // Adversarial regression: quickjs-wasi's `executePendingJobs()` renders - // a failed job's exception through `toString()`, which executes guest - // code (a getter on `Error.prototype.name` fires while the drain error - // is reported). The engine's drain reads the exception trap-free. - value( - await v.evalCode(` - globalThis.__traps = 0; - Object.defineProperty(Error.prototype, 'name', { - configurable: true, - get() { globalThis.__traps++; return 'Error'; }, - }); - 'installed'; - `), - ); - const e = error( - await v.evalCode('queueMicrotask(() => { throw new Error("job boom") }); "queued"'), - ); - assert.equal(e.message, 'job boom'); - assert.equal(value(await v.evalCode('globalThis.__traps')), 0, 'no guest getter ran'); - // The VM stays usable after the drain failure. - assert.equal(value(await v.evalCode('1 + 1')), 2); -}); - -test('repeated syntax errors do not accumulate guest memory (exception handles are freed)', async () => { - // Adversarial regression (review): the caught `JSException` and its - // owned handle were never disposed, and a 1 MiB VM exhausted after - // ~4,018 syntax errors — even `1 + 1` then returned a null error. The - // exception value must be freed immediately, so the VM is long-lived. - using v = await vm({ memoryLimit: 1024 * 1024 }); - for (let i = 0; i < 20_000; i++) { - const e = error(await v.evalCode('const =')); - assert.equal(e.name, 'SyntaxError'); - } - assert.equal(value(await v.evalCode('1 + 1')), 2); -}); - -test('repeated accessor-valued completions do not accumulate guest memory (accessor handles are freed)', async () => { - // Adversarial regression (review): accessor descriptors own `get`/`set` - // handles that were never disposed, exhausting a 1 MiB VM after ~3,128 - // accessor-valued completions. Both handles must be freed. - using v = await vm({ memoryLimit: 1024 * 1024 }); - for (let i = 0; i < 20_000; i++) { - const out = await v.evalCode('({ get secret() { return "x" } })'); - assert.equal(out.kind, 'value'); - if (out.kind === 'value') assert.deepEqual(out.value, {}, 'accessor is skipped'); - } - assert.equal(value(await v.evalCode('1 + 1')), 2); -}); - -test('resolved evals do not accumulate completion memory (wrapper + discarded handles are freed)', async () => { - // Adversarial regression (review): the resolved completion path - // returned the unwrapped value handle while RETAINING the - // engine-created `{ value }` wrapper (the finally condition never - // disposed it when the completion was kept), and the public evalCode() - // discarded any returned completion handle without disposing it. Every - // Broker.eval (rejectionBridge: true) took both paths, and an - // adversarial 2 MiB VM probe died at eval ~19,346 with `Error: null`; - // 50,000 ordinary evals stayed healthy. Rejection bridging is now - // separate from completion ownership: the wrapper is disposed on the - // unwrap path and evalCode disposes the handle it discards, so the VM - // is long-lived under both entries. - using v = await vm({ memoryLimit: 2 * 1024 * 1024 }); - for (let i = 0; i < 20_000; i++) { - // The broker entry: keepCompletion + bridge, the caller disposes the - // returned completion handle. - const kept = v.evalCodeWithCompletion('({ n: ' + i + ' })', { rejectionBridge: true }); - assert.equal(kept.outcome.kind, 'value'); - if (kept.completion !== undefined) (kept.completion as JSValueHandle).dispose(); - // The public entry: the bridge is armed but the completion handle is - // discarded by evalCode itself — it must dispose it. - const dropped = await v.evalCode('({ m: ' + i + ' })', { rejectionBridge: true }); - assert.equal(dropped.kind, 'value'); - if (dropped.kind === 'value') assert.deepEqual(dropped.value, { m: i }); - } - // A suspended eval with the bridge attached also stays healthy (the - // bridge attaches to the pending completion — no fabricated value). - const pending = await v.evalCode('const gate = new Promise(() => {}); await gate; 1', { rejectionBridge: true }); - assert.equal(pending.kind, 'pending'); - assert.equal(value(await v.evalCode('1 + 1')), 2, 'the VM is healthy after 40,000 resolved evals plus a suspended one'); -}); - -test('memory limit: a per-VM limit turns oversized allocations into out-of-memory errors', async () => { - using v = await vm({ memoryLimit: 1024 * 1024 }); - assert.equal(v.memoryLimit, 1024 * 1024); - const e = error(await v.evalCode("'x'.repeat(64 * 1024 * 1024)")); - assert.equal(e.name, 'InternalError'); - assert.equal(e.message, 'out of memory'); - assert.equal(e.outOfMemory, true); - // The VM remains usable after the failed allocation. - assert.equal(value(await v.evalCode('1 + 1')), 2); -}); - -test('memory limit: independent per-VM limits (engine posture: memoryLimit per VM)', async () => { - using tight = await vm({ memoryLimit: 1024 * 1024 }); - using loose = await vm({ memoryLimit: 256 * 1024 * 1024 }); - const e = error(await tight.evalCode("'y'.repeat(64 * 1024 * 1024)")); - assert.equal(e.outOfMemory, true); - // The same allocation fits in the loose VM. - assert.equal(value(await loose.evalCode("'y'.repeat(64 * 1024 * 1024).length")), 64 * 1024 * 1024); - assert.equal(value(await tight.evalCode('2 + 2')), 4); -}); - -test('per-eval interrupt: a runaway eval is broken with the VM still usable after', async () => { - using v = await vm(); - let checks = 0; - const outcome = await v.evalCode('while (true) {}', { - interruptHandler: () => ++checks > 100, - }); - const e = error(outcome); - assert.equal(e.name, 'InternalError'); - assert.equal(e.message, 'interrupted'); - assert.equal(e.interrupted, true); - // The VM stays usable — this is the "break a runaway eval" contract. - assert.equal(value(await v.evalCode('1 + 1')), 2); - assert.equal(value(await v.evalCode('const after = "alive"; after')), 'alive'); -}); - -test('per-eval interrupt: a runaway microtask loop is broken during the drain', async () => { - using v = await vm(); - // The interrupt budget is instruction-based (quickjs's built-in check - // interval), so a small budget against a tiny loop body keeps the drain - // short while still firing inside a drained job. - let checks = 0; - const outcome = await v.evalCode('(async () => { let i = 0; while (true) { i++; await 0 } })()', { - interruptHandler: () => ++checks > 3, - }); - // The interrupt fired inside a drained job: the drain surfaces it as a - // job error (the harness's pinned "JobError from the drain" shape). - const e = error(outcome); - assert.equal(e.interrupted, true); - // VM still usable, including new async work. - assert.equal(value(await v.evalCode('await Promise.resolve(42)')), 42); -}); - -test('interrupt handlers never leak across evals', async () => { - using v = await vm(); - const e = error( - await v.evalCode('while (true) {}', { - interruptHandler: () => true, - }), - ); - assert.equal(e.interrupted, true); - // The next eval runs without a handler — no stale interrupt fires. - assert.equal(value(await v.evalCode('6 * 7')), 42); -}); - -test('interrupt handlers are per-eval: the same VM serves different handlers', async () => { - using v = await vm(); - let a = 0; - let b = 0; - const e1 = error( - await v.evalCode('while (true) {}', { - interruptHandler: () => ++a > 500, - }), - ); - assert.equal(e1.interrupted, true); - assert.ok(a >= 500); - const e2 = error( - await v.evalCode('while (true) {}', { - interruptHandler: () => ++b > 100, - }), - ); - assert.equal(e2.interrupted, true); - assert.ok(b >= 100 && b < a, 'second handler got its own budget'); -}); - -test('dispose: the VM is torn down and refuses further use', async () => { - const v = await vm(); - assert.equal(value(await v.evalCode('1 + 1')), 2); - v.dispose(); - assert.equal(v.isDisposed, true); - await assert.rejects(v.evalCode('1'), /disposed/); - assert.throws(() => v.drainJobs(), /disposed/); - // Idempotent. - v.dispose(); -}); - -test('the shipped binary round-trips as the public wasm type (loadShippedWasm → create)', async () => { - using v = await vm({ wasm: await loadShippedWasm() }); - assert.equal(value(await v.evalCode('6 * 7')), 42); - assert.equal(v.memoryLimit, ReplVm.DEFAULT_MEMORY_LIMIT); -}); - -test('a failing own-descriptor read never constructs JSException and leaves the VM usable', async () => { - using v = await vm(); - // Adversarial regression (review): `JSValueHandle.getOwnPropertyDescriptor()` - // throws a `JSException` when the C descriptor read fails, and that - // constructor performs guest-visible `[[Get]]` reads of name/message/stack - // on the exception value — a getter on `SyntaxError.prototype.name` would - // execute during error construction, before any host catch. The engine's - // raw descriptor path must take the failed read's exception out of the - // runtime and free it without ever constructing a `JSException`. - value( - await v.evalCode(` - globalThis.__traps = 0; - for (const key of ['name', 'message', 'stack']) { - Object.defineProperty(SyntaxError.prototype, key, { - configurable: true, - get() { globalThis.__traps++; return 'trapped'; }, - }); - } - 'installed'; - `), - ); - // Drive the raw exports directly — the same surface the engine drives — - // and make every descriptor read fail the way the C engine fails under - // an allocation edge: the export returns the exception sentinel and a - // real exception value lands in the runtime slot. Running a real failing - // `qjs_eval` per read keeps the sentinel and the runtime exception - // genuine. The WASM exports object is frozen with non-configurable data - // properties (a Proxy `get` trap cannot override them), so the shim's - // `exports` field (a plain TS-private property) is swapped for an object - // that shadows the descriptor export and delegates everything else to - // the real exports. - const qjs = (v as unknown as { vm: QuickJS }).vm; - const originalExports = qjs._getExports(); - const patched = Object.create(originalExports); - Object.defineProperty(patched, 'qjs_get_own_property_descriptor', { - configurable: true, - writable: true, - value: () => { - const code = qjs._writeString('const ='); - const fn = qjs._writeString(''); - const sentinel = originalExports.qjs_eval(code.ptr, code.len, fn.ptr, EvalFlags.TYPE_GLOBAL); - originalExports.wasm_free(code.ptr); - originalExports.wasm_free(fn.ptr); - return sentinel; - }, - }); - (qjs as unknown as { exports: typeof originalExports }).exports = patched; - try { - const outcome = await v.evalCode('({ a: 1 })'); - // Every descriptor read failed and read as absent; the completion - // renders as an empty object instead of crashing or fabricating data. - assert.equal(outcome.kind, 'value'); - if (outcome.kind === 'value') assert.deepEqual(outcome.value, {}); - } finally { - (qjs as unknown as { exports: typeof originalExports }).exports = originalExports; - } - // No `JSException` was constructed: none of the getters ran. - assert.equal(value(await v.evalCode('globalThis.__traps')), 0, 'no guest getter ran'); - // The failed reads took the runtime exception out each time — no sticky - // exception poisons the VM. - assert.equal(value(await v.evalCode('1 + 1')), 2); -}); - -test('standalone settlement drains arm their own interrupt handler (delayed continuation interruption)', async () => { - using v = await vm(); - // Two runaway continuations, interrupted mid-drain by the per-eval - // handler: the drain stops at the first failed job, so at least one - // runaway continuation stays queued in the VM. Its per-eval handler is - // gone with the eval — a later settlement drain would resume the loop - // with no interrupt protection unless the drain carries its own signal. - let evalChecks = 0; - const outcome = await v.evalCode( - ` - for (let k = 0; k < 2; k++) { - (async () => { let i = 0; while (true) { i++; await 0 } })(); - } - 'queued'; - `, - { interruptHandler: () => ++evalChecks > 3 }, - ); - const e = error(outcome); - assert.equal(e.interrupted, true); - - // Drain the leftovers with a per-drain handler: each failed job throws a - // `DrainJobError` reporting the interrupt; the loop consumes every queued - // continuation and terminates. (Without a handler, this drain would run - // the leftover runaway forever.) - let drainChecks = 0; - let interruptedDrains = 0; - for (;;) { - let n = 0; - try { - n = v.drainJobs({ interruptHandler: () => ++drainChecks > 1 }); - } catch (err) { - assert.ok(err instanceof DrainJobError, 'drain failure is a DrainJobError'); - assert.equal(err.info.interrupted, true); - interruptedDrains++; - continue; - } - if (n === 0) break; - } - assert.ok( - interruptedDrains >= 1, - 'a leftover runaway continuation was interrupted by the standalone drain', - ); - // The per-drain handler is gone too — nothing leaked. - assert.equal(v.drainJobs(), 0); - assert.equal(value(await v.evalCode('1 + 1')), 2); - assert.equal(value(await v.evalCode('await Promise.resolve(42)')), 42); -}); - -test('dispose cannot race an in-flight eval: settled and suspended evals complete first', async () => { - // Review regression: the completion read used to yield through an - // already-settled host promise, so `const p = ws.eval('6*7'); - // ws.dispose(); await p` rejected with `TypeError: Cannot read - // properties of null (reading 'qjs_is_proxy')` once the WASM exports - // were nulled. Eval completion is now synchronous: the eval finishes - // before `dispose` even runs, and both outcomes survive. - const v = await vm(); - const settled = v.evalCode('6 * 7'); - v.dispose(); - const settledOutcome = await settled; - assert.equal(settledOutcome.kind, 'value'); - if (settledOutcome.kind === 'value') assert.equal(settledOutcome.value, 42); - - const v2 = await vm(); - const suspended = v2.evalCode('const gate = new Promise(() => {}); await gate; "never"'); - v2.dispose(); - const suspendedOutcome = await suspended; - assert.equal(suspendedOutcome.kind, 'pending', 'suspended eval reports pending, not a crash'); - - const v3 = await vm(); - const failed = v3.evalCode('const ='); - v3.dispose(); - const failedOutcome = await failed; - assert.equal(failedOutcome.kind, 'error'); - if (failedOutcome.kind === 'error') assert.equal(failedOutcome.error.name, 'SyntaxError'); -}); - -test('concurrent evals never leak interrupt handlers across the batch', async () => { - using v = await vm(); - // Review regression: two overlapping evals restored the interrupt slot - // out of nesting order and left the first handler armed indefinitely; a - // later standalone drain then inherited the unrelated handler. Eval is - // now synchronous, so the calls serialize — this pins that the slot - // save/restore leaves nothing behind under a concurrent call pattern. - const budgets = new Array(32).fill(0); - const results = await Promise.all( - budgets.map((_, k) => - v.evalCode('while (true) {}', { interruptHandler: () => ++budgets[k] > k + 1 }), - ), - ); - for (let k = 0; k < results.length; k++) { - const e = error(results[k]); - assert.equal(e.interrupted, true); - assert.ok(budgets[k] >= k + 1, `eval ${k} was interrupted by its own handler`); - } - // No handler survives the batch: a no-handler eval runs to completion - // and a no-handler drain is clean. - assert.equal(value(await v.evalCode('6 * 7')), 42); - assert.equal(v.drainJobs(), 0); -}); - -test('thrown symbols render the bare brand — the description is not readable trap-free', async () => { - using v = await vm(); - // Review regression: the primitive error-rendering default branch called - // `toNumber()` on symbols, so `throw Symbol('x')` reported message `NaN` - // — a fabricated conversion. The honest rendering is the bare brand - // `Symbol` (FORMAT.md §5.7): the description sits behind - // `qjs_get_symbol_description`, which invokes guest `Symbol.keyFor` — a - // forbidden seam (FORMAT.md §1.1), because a guest that replaces - // `Symbol.keyFor` could forge the classification. The next test pins - // that the seam is never reached. - assert.equal(error(await v.evalCode('throw Symbol("boom")')).message, 'Symbol'); - assert.equal(error(await v.evalCode('throw Symbol()')).message, 'Symbol'); - assert.equal(error(await v.evalCode('throw Symbol("")')).message, 'Symbol'); - assert.equal(error(await v.evalCode('throw Symbol.for("shared")')).message, 'Symbol'); - // Rejected top-level awaits surface the same conversion. - assert.equal(error(await v.evalCode('await Promise.reject(Symbol("rejected"))')).message, 'Symbol'); - // The VM stays usable. - assert.equal(value(await v.evalCode('1 + 1')), 2); -}); - -test('a guest that replaces Symbol.keyFor cannot influence error rendering (no forbidden seam)', async () => { - using v = await vm(); - // The FORMAT.md §1.1 seam: reading a symbol's description calls guest - // `Symbol.keyFor` through the binary. The engine must never reach it — - // a hostile guest swaps it for a trap counter and throws symbols; the - // rendered message must be the bare brand and the counter must stay 0. - value( - await v.evalCode(` - globalThis.__keyForTraps = 0; - Symbol.keyFor = function () { globalThis.__keyForTraps++; return 'FORGED'; }; - try { - Object.defineProperty(Symbol.prototype, 'description', { - configurable: true, - get() { globalThis.__keyForTraps++; return 'FORGED'; }, - }); - } catch (_e) { /* non-configurable on this build — keyFor is the seam anyway */ } - 'installed' - `), - ); - const e = error(await v.evalCode('throw Symbol("secret")')); - assert.equal(e.message, 'Symbol'); - assert.equal(value(await v.evalCode('globalThis.__keyForTraps')), 0, 'Symbol.keyFor/description never ran'); -}); - -test('a value GETTER on Object.prototype empties the completion wrapper — without running guest code (engine quirk, pinned)', async () => { - using v = await vm(); - // Discovered during phase B: quickjs-ng's async-eval completion wrapper - // (`{ value: … }`) comes out EMPTY once `Object.prototype.value` is - // rebound to a GETTER — the engine's own wrapper creation is - // prototype-sensitive for exactly this key. The engine never executes - // the getter (verified with a counter); the completion read degrades - // honestly to `{}` (the trap-free fallback renders the wrapper as-is - // rather than fabricating a value), and the VM stays fully usable. - // The install eval's own completion already degrades (the pollution is - // installed mid-eval, so its wrapper comes out empty) — run it, then - // assert the degradation and the trap-freedom of subsequent evals. - await v.evalCode(` - globalThis.__traps = 0; - Object.defineProperty(Object.prototype, 'value', { - configurable: true, - get() { globalThis.__traps++; return 'polluted'; }, - }); - 'installed'; - `); - const result = value(await v.evalCode('40 + 2')); - assert.deepEqual(result, {}, 'the completion wrapper is empty; nothing fabricated'); - // The getter never ran — read the counter through a descriptor path - // (eval completions read as {} under this pollution, by the quirk above). - const shim = getVmShim(v) as QuickJS; - const e = shim._getExports(); - const globalHandle = shim.global; // cached singleton — do not dispose - const key = shim.newString('__traps'); - let descPtr: number; - try { - descPtr = e.qjs_get_own_property_descriptor(globalHandle.ptr, key.ptr); - } finally { - key.dispose(); - } - assert.notEqual(descPtr, 0); - const desc = new JSValueHandle(shim, descPtr); - try { - const key2 = shim.newString('value'); - let vp: number; - try { - vp = e.qjs_get_prop_value(desc.ptr, key2.ptr); - } finally { - key2.dispose(); - } - const val = new JSValueHandle(shim, vp); - try { - assert.equal(val.isNumber, true); - assert.equal(val.toNumber(), 0, 'no guest getter ran'); - } finally { - val.dispose(); - } - } finally { - desc.dispose(); - } - // The VM stays fully usable (completions still degrade to {} — the - // quirk persists until the pollution is removed). - assert.deepEqual(value(await v.evalCode('1 + 1')), {}); -}); diff --git a/packages/repl-engine/test/workspace.test.ts b/packages/repl-engine/test/workspace.test.ts deleted file mode 100644 index 99f67217..00000000 --- a/packages/repl-engine/test/workspace.test.ts +++ /dev/null @@ -1,596 +0,0 @@ -/** - * Workspace-layer tests: one VM per workspace, the workspace owning the - * VM lifecycle (create, eval, drain, dispose), and the registry's - * project-keyed one-workspace invariant — the seam the `repl` MCP tool - * (a later phase) addresses workspaces through. - */ - -import assert from 'node:assert/strict'; -import { test } from 'node:test'; - -import { Broker, DrainJobError, Workspace, WorkspaceRegistry, loadShippedWasm } from '../src/index.js'; - -async function workspace(options?: Parameters[1]): Promise { - return Workspace.create('/tmp/repl-test-project', options); -} - -test('workspace lifecycle: create → eval → drain → dispose', async () => { - const ws = await workspace(); - assert.equal(ws.isDisposed, false); - assert.equal(ws.projectDir, '/tmp/repl-test-project'); - - const outcome = await ws.eval('6 * 7'); - assert.equal(outcome.kind, 'value'); - if (outcome.kind === 'value') assert.equal(outcome.value, 42); - - assert.equal(ws.drainJobs(), 0); - - ws.dispose(); - assert.equal(ws.isDisposed, true); - // `eval`/`drainJobs` refuse use synchronously after dispose. - assert.throws(() => ws.eval('1'), /disposed/); - assert.throws(() => ws.drainJobs(), /disposed/); - ws.dispose(); // idempotent -}); - -test('the guest bridge is installed at VM creation: DSL globals, console bridge, parked calls, surface', async () => { - // Review rejection: a production Workspace exposed agent/checkpoint/ - // combinators as undefined because creation never installed the bridge. - // The injection happens at creation now — the doc's discipline. - const ws = await workspace(); - // The DSL vocabulary is live from the first eval. - const globals = await ws.eval(`({ - agent: typeof agent, checkpoint: typeof checkpoint, - answer: typeof checkpoint.answer, parallel: typeof parallel, - console: typeof console, marker: typeof globalThis.__REPL_GUEST_VERSION, - phase: typeof phase, budget: typeof budget, - })`); - assert.equal(globals.kind, 'value'); - if (globals.kind === 'value') { - assert.deepEqual(globals.value, { - agent: 'function', - checkpoint: 'function', - answer: 'function', - parallel: 'function', - console: 'object', - marker: 'string', - phase: 'undefined', - budget: 'undefined', - }); - } - - // The console bridge accumulates events on the default parking bridge. - const logged = await ws.eval('console.log({ a: 1 }, "text"); "done"'); - assert.equal(logged.kind, 'value'); - const events = ws.consoleEvents(); - assert.equal(events.length, 1); - // The §4.4 one-line repr (guest-rendered): args joined with one space. - assert.equal(events[0].level, 'log'); - assert.equal(events[0].line, '{a: 1} text'); - - // The default parking bridge parks agent calls (honest no-backend state: - // nothing is fabricated, the calls pend until a later phase attaches - // backends) and the reconciliation surface sees them. - const started = await ws.eval('const research = agent("pi/deepseek-v4-flash-max", "research X"); "started"'); - assert.equal(started.kind, 'value'); - const surface = ws.surface(); - assert.ok(surface !== undefined, 'the guest surface is reachable from the workspace'); - const pending = surface!.pending(); - assert.equal(pending.length, 1); - assert.equal(pending[0].kind, 'agent'); - assert.equal(pending[0].modelSpec, 'pi/deepseek-v4-flash-max'); - assert.equal(pending[0].sessionId, 'c1'); - assert.equal(ws.parkedCalls().size, 1); - // The parked call can be settled through the surface, exactly like the - // post-restore reconciliation route. - assert.equal(surface!.settle('c1', 'resolve', { ok: true }), true); - ws.drainJobs(); - const settled = await ws.eval('await research'); - assert.equal(settled.kind, 'value'); - if (settled.kind === 'value') assert.deepEqual(settled.value, { ok: true }); - // The parked record remains (the surface route settled the GUEST - // registry, not the live deferred); the registry itself is empty — the - // honest "no pending work" signal. - assert.equal(surface!.pending().length, 0); - - // inspectBinding is the manifest seam (name, type, size — never content). - await ws.eval('globalThis.notes = { depth: 3 }; "ok"'); - const meta = ws.inspectBinding('notes'); - assert.equal(meta.kind, 'data'); - assert.equal(meta.label, 'object'); - assert.ok(meta.sizeBytes > 0); - ws.dispose(); -}); - -test('parking bridge: agents() serves the REAL model spec and task of parked agent calls (§4.5 plain-value shape — never fabricated empties)', async () => { - const ws = await workspace(); - await ws.eval('const research = agent("pi/deepseek-v4-flash-max", "research X"); "started"'); - const out = await ws.eval('agents()'); - assert.equal(out.kind, 'value'); - if (out.kind === 'value') { - const agents = out.value as Array<{ callId: string; modelSpec: string; task: string; state: string; supportsSteering: boolean; queuedTurns: number }>; - assert.equal(agents.length, 1); - assert.equal(agents[0].callId, 'c1'); - assert.equal(agents[0].modelSpec, 'pi/deepseek-v4-flash-max', 'the real model spec, never ""'); - assert.equal(agents[0].task, 'research X', 'the real task, never ""'); - assert.equal(agents[0].queuedTurns, 0); - } - ws.dispose(); -}); - -test('workspace-level evals maintain the §4.4 `_` result history — resolved, LATE (settled at the drain), and empty-poll evals', async () => { - const ws = await workspace(); - await ws.eval('40 + 2'); - const first = await ws.eval('_'); - assert.equal(first.kind, 'value'); - if (first.kind === 'value') assert.equal(first.value, 42); - // A suspended eval's completion value becomes `_` once its - // continuation settles at the drain (the parking bridge's sleep timer). - const suspended = await ws.eval('await sleep(10); "late"'); - assert.equal(suspended.kind, 'pending'); - await new Promise((resolve) => setTimeout(resolve, 50)); - ws.drainJobs(); - const second = await ws.eval('_'); - assert.equal(second.kind, 'value'); - if (second.kind === 'value') assert.equal(second.value, 'late'); - // An empty poll (eval "") COMPLETES with undefined — `_` becomes - // undefined: the previous eval's completion value IS undefined (the - // review probe: `42`, then an empty eval, then `_` must read - // undefined, never the stale 42). - await ws.eval('"kept"'); - await ws.eval(''); - const third = await ws.eval('_'); - assert.equal(third.kind, 'value'); - if (third.kind === 'value') assert.equal(third.value, undefined); - ws.dispose(); -}); - -test('parking bridge: reset() in a SUSPENDED eval tears the workspace down after the continuation completes — the workspace stays alive while the eval is in flight', async () => { - const ws = await workspace(); - const out = await ws.eval('reset(); await sleep(30); "finished"'); - assert.equal(out.kind, 'pending'); - assert.equal(ws.isDisposed, false, 'the workspace is ALIVE while the reset eval is suspended'); - await new Promise((resolve) => setTimeout(resolve, 80)); - ws.drainJobs(); - assert.equal(ws.isDisposed, true, 'the teardown ran after the eval completed (the continuation settled at the drain)'); -}); - -test('parking bridge: reset() called after a suspended eval resumes is attributed to that eval and tears down in the completing drain', async () => { - const ws = await workspace(); - const out = await ws.eval('await sleep(30); reset(); 42'); - assert.equal(out.kind, 'pending'); - assert.equal(ws.isDisposed, false, 'the workspace stays alive until the reset-calling eval resumes'); - await new Promise((resolve) => setTimeout(resolve, 80)); - ws.drainJobs(); - assert.equal(ws.isDisposed, true, 'the drain that completed the reset-calling eval performed teardown'); -}); - -test('parking bridge: a completed drain cannot misattribute a later plain reset() to an eval still suspended mid-continuation', async () => { - const ws = await workspace(); - const first = await ws.eval('await sleep(20); await agent("pi/x", "hold"); 1'); - assert.equal(first.kind, 'pending'); - await new Promise((resolve) => setTimeout(resolve, 60)); - ws.drainJobs(); - assert.equal(ws.isDisposed, false, 'the first eval remains suspended on its parked agent'); - - const reset = await ws.eval('reset(); 2'); - assert.equal(reset.kind, 'value'); - if (reset.kind === 'value') assert.equal(reset.value, 2); - assert.equal(ws.isDisposed, true, 'the plain reset belongs to the eval that called it'); -}); - -test('default parking bridge: workspace() checkpoint questions and agents() tasks retain their 200-character metadata previews', async () => { - const ws = await workspace(); - const out = await ws.eval(` - checkpoint("q".repeat(300)); - agent("pi/x", "t".repeat(300)); - const question = workspace().checkpoints[0].question; - const task = agents()[0].task; - ({ question, task }); - `); - assert.equal(out.kind, 'value'); - if (out.kind === 'value') { - const value = out.value as { question: string; task: string }; - assert.ok(value.question.length < 300, 'the raw checkpoint question is not exposed'); - assert.ok(value.question.includes('chars elided'), value.question); - assert.equal(value.task.length, 200, 'the parked agent task uses the engine\'s 200-character preview'); - assert.equal(value.task, `${'t'.repeat(99)}…${'t'.repeat(100)}`); - } - ws.dispose(); -}); - -test('default parking bridge: checkpoint.answer settles the parked checkpoint (first-wins)', async () => { - // Review rejection: the parking bridge's answer mode returned `false` - // for every checkpoint.answer, so the original promise stayed pending - // forever — the data plane could never interrupt the intent plane on - // a default workspace. The bridge now tracks parked checkpoint calls - // separately, parses the answer, and settles the matching call. - const ws = await workspace(); - const asked = await ws.eval('const q = checkpoint("proceed?"); "asked"'); - assert.equal(asked.kind, 'value'); - // The parked checkpoint is visible through the parked-calls surface. - assert.equal(ws.parkedCalls().size, 1); - - // The orchestrator delivers the answer in a later eval; the bridge - // reports delivery truthfully. - const delivered = await ws.eval('checkpoint.answer("c1", { yes: true, note: "go" }); "delivered"'); - assert.equal(delivered.kind, 'value'); - if (delivered.kind === 'value') assert.equal(delivered.value, 'delivered'); - // Delivery consumed the parked record. - assert.equal(ws.parkedCalls().size, 0); - // The checkpoint promise resolved with the ANSWER (not `false`), during - // the delivering eval's own job drain. - const settled = await ws.eval('await q'); - assert.equal(settled.kind, 'value'); - if (settled.kind === 'value') assert.deepEqual(settled.value, { yes: true, note: 'go' }); - - // Unknown and already-answered ids report false and pend nothing new. - const unknown = await ws.eval('checkpoint.answer("c99", 1)'); - assert.equal(unknown.kind, 'value'); - if (unknown.kind === 'value') assert.equal(unknown.value, false); - const again = await ws.eval('checkpoint.answer("c1", 2)'); - assert.equal(again.kind, 'value'); - if (again.kind === 'value') assert.equal(again.value, false); - ws.dispose(); -}); - -test('default parking bridge: checkpoint.answer never settles a parked agent call', async () => { - // The bridge tracks parked CHECKPOINT calls separately from parked - // agent/steer calls: an answer addressed at an agent call's id must - // report false and leave the agent call parked (ids share one space). - const ws = await workspace(); - await ws.eval('const research = agent("pi/deepseek-v4-flash-max", "research X"); const q = checkpoint("q?"); "started"'); - assert.equal(ws.parkedCalls().size, 2); - - // c1 is the AGENT call — answering it must not settle it. - const wrong = await ws.eval('checkpoint.answer("c1", "nope")'); - assert.equal(wrong.kind, 'value'); - if (wrong.kind === 'value') assert.equal(wrong.value, false); - assert.equal(ws.parkedCalls().size, 2, 'the agent call is still parked'); - - // The checkpoint (c2) answers normally, leaving only the agent parked. - const ok = await ws.eval('checkpoint.answer("c2", "yes")'); - assert.equal(ok.kind, 'value'); - if (ok.kind === 'value') assert.equal(ok.value, true); - assert.equal(ws.parkedCalls().size, 1); - const qOutcome = await ws.eval('await q'); - assert.equal(qOutcome.kind, 'value'); - if (qOutcome.kind === 'value') assert.equal(qOutcome.value, 'yes'); - // The agent call is untouched by all of it. - assert.equal(ws.surface()!.pending().length, 1); - assert.equal(ws.surface()!.pending()[0].kind, 'agent'); - ws.dispose(); -}); - -test('custom bridge handlers passed to create override the parking bridge', async () => { - const calls: Array<{ callId: string; modelSpec: string; task: string }> = []; - const ws = await Workspace.create('/tmp/repl-test-custom-bridge', { - handlers: { - agent: (call, callId, modelSpec, task) => { - calls.push({ callId, modelSpec, task }); - call.resolve('custom handled'); - }, - checkpoint: () => undefined, - queue: () => undefined, - steer: () => undefined, - cancelSession: () => undefined, - cancelQueue: () => undefined, - console: () => undefined, - sleep: () => undefined, - workspace: () => '{}', - agents: () => '[]', - reset: () => undefined, - defaultBackend: () => undefined, - }, - }); - const out = await ws.eval('await agent("pi/custom", "do it")'); - assert.equal(out.kind, 'value'); - if (out.kind === 'value') assert.equal(out.value, 'custom handled'); - assert.deepEqual(calls, [{ callId: 'c1', modelSpec: 'pi/custom', task: 'do it' }]); - // Custom handlers own their events: the workspace buffer stays empty. - assert.equal(ws.consoleEvents().length, 0); - ws.dispose(); -}); - -test('workspace state persists across evals (the REPL property)', async () => { - const ws = await workspace(); - await ws.eval('let findings = ["a", "b", "c"]; let notes = { depth: 3 }'); - const outcome = await ws.eval('findings.length + notes.depth'); - assert.equal(outcome.kind, 'value'); - if (outcome.kind === 'value') assert.equal(outcome.value, 6); - ws.dispose(); -}); - -test('workspaces are isolated: one VM per workspace, no cross-workspace state', async () => { - const a = await Workspace.create('/tmp/repl-test-a'); - const b = await Workspace.create('/tmp/repl-test-b'); - await a.eval('let secret = "only-in-a"'); - const bOutcome = await b.eval('typeof secret'); - assert.equal(bOutcome.kind, 'value'); - if (bOutcome.kind === 'value') assert.equal(bOutcome.value, 'undefined'); - a.dispose(); - b.dispose(); -}); - -test('per-workspace memory limits are independent', async () => { - const tight = await Workspace.create('/tmp/repl-test-tight', { memoryLimit: 1024 * 1024 }); - const loose = await Workspace.create('/tmp/repl-test-loose', { memoryLimit: 256 * 1024 * 1024 }); - const tightOutcome = await tight.eval("'z'.repeat(64 * 1024 * 1024)"); - assert.equal(tightOutcome.kind, 'error'); - if (tightOutcome.kind === 'error') assert.equal(tightOutcome.error.outOfMemory, true); - const looseOutcome = await loose.eval("'z'.repeat(64 * 1024 * 1024).length"); - assert.equal(looseOutcome.kind, 'value'); - if (looseOutcome.kind === 'value') assert.equal(looseOutcome.value, 64 * 1024 * 1024); - tight.dispose(); - loose.dispose(); -}); - -test('registry: get-or-create returns the same workspace (and VM) per project dir', async () => { - const registry = new WorkspaceRegistry(); - const first = await registry.get('/tmp/repl-project-1'); - const second = await registry.get('/tmp/repl-project-1'); - assert.equal(first, second, 'one workspace per project directory'); - assert.equal(registry.size, 1); - assert.equal(registry.has('/tmp/repl-project-1'), true); - first.dispose(); - second.dispose(); - registry.disposeAll(); -}); - -test('registry: distinct project dirs get distinct workspaces and VMs', async () => { - const registry = new WorkspaceRegistry(); - const a = await registry.get('/tmp/repl-project-a'); - const b = await registry.get('/tmp/repl-project-b'); - assert.notEqual(a, b); - assert.equal(registry.size, 2); - await a.eval('let marker = "a"'); - const bOutcome = await b.eval('typeof marker'); - assert.equal(bOutcome.kind, 'value'); - if (bOutcome.kind === 'value') assert.equal(bOutcome.value, 'undefined'); - registry.disposeAll(); - assert.equal(registry.size, 0); -}); - -test('registry: dispose drops the workspace; the next get creates a fresh one', async () => { - const registry = new WorkspaceRegistry(); - const first = await registry.get('/tmp/repl-project-3'); - await first.eval('let counter = 41'); - assert.equal(registry.dispose('/tmp/repl-project-3'), true); - assert.equal(registry.dispose('/tmp/repl-project-3'), false, 'second dispose is a miss'); - assert.equal(registry.has('/tmp/repl-project-3'), false); - assert.equal(first.isDisposed, true); - - const fresh = await registry.get('/tmp/repl-project-3'); - assert.notEqual(fresh, first); - const outcome = await fresh.eval('typeof counter'); - assert.equal(outcome.kind, 'value'); - if (outcome.kind === 'value') assert.equal(outcome.value, 'undefined'); - registry.disposeAll(); -}); - -test('registry: concurrent first-touches create exactly one VM', async () => { - // Review regression: two concurrent `get('/same')` calls each ran a full - // `Workspace.create` (two VM instantiations for one project) before the - // loser was disposed. The registry must deduplicate the in-flight - // creation promise, not merely the completed result — the wasm getter - // counts how many creations actually started. - let creations = 0; - const wasm = await loadShippedWasm(); - const registry = new WorkspaceRegistry({ - get wasm() { - creations++; - return wasm; - }, - }); - const [a, b] = await Promise.all([ - registry.get('/tmp/repl-project-race2'), - registry.get('/tmp/repl-project-race2'), - ]); - assert.equal(creations, 1, 'exactly one VM was created for one project'); - assert.equal(a, b); - assert.equal(registry.size, 1); - assert.equal(a.isDisposed, false); - registry.disposeAll(); -}); - -test('registry: dispose during an in-flight create cancels it; a later get creates fresh', async () => { - // `Workspace.create` resolves asynchronously (wasm instantiation), so a - // synchronous `dispose` right after `get` lands mid-creation. The - // registry must not materialize a workspace after dispose: the created - // VM is torn down, the waiting caller's promise rejects, and a later - // `get` starts fresh. - const registry = new WorkspaceRegistry(); - const first = registry.get('/tmp/repl-project-cancel'); - assert.equal(registry.dispose('/tmp/repl-project-cancel'), false, 'no live workspace yet'); - await assert.rejects(first, /creation cancelled by dispose/); - assert.equal(registry.size, 0); - assert.equal(registry.has('/tmp/repl-project-cancel'), false); - - const fresh = await registry.get('/tmp/repl-project-cancel'); - assert.equal(fresh.isDisposed, false); - const outcome = await fresh.eval('6 * 7'); - assert.equal(outcome.kind, 'value'); - if (outcome.kind === 'value') assert.equal(outcome.value, 42); - registry.disposeAll(); -}); - -test('standalone drains accept a per-drain interrupt handler through the workspace', async () => { - // The settlement drain's interrupt signal is a workspace-level concern: - // `Workspace.drainJobs` must forward the per-drain handler (a suspended - // eval's handler is no longer armed once the eval returned). - const ws = await workspace(); - // `> 3` (not `() => true`): the interrupt must fire inside a drained job, - // leaving a runaway continuation queued for the later drain — an - // immediate `true` would abort the script itself, queueing nothing. - let evalChecks = 0; - const outcome = await ws.eval( - ` - for (let k = 0; k < 2; k++) { - (async () => { let i = 0; while (true) { i++; await 0 } })(); - } - 'queued'; - `, - { interruptHandler: () => ++evalChecks > 3 }, - ); - assert.equal(outcome.kind, 'error'); - if (outcome.kind === 'error') assert.equal(outcome.error.interrupted, true); - - let drainChecks = 0; - let interruptedDrains = 0; - for (;;) { - let n = 0; - try { - n = ws.drainJobs({ interruptHandler: () => ++drainChecks > 1 }); - } catch (err) { - assert.ok(err instanceof DrainJobError); - assert.equal(err.info.interrupted, true); - interruptedDrains++; - continue; - } - if (n === 0) break; - } - assert.ok(interruptedDrains >= 1, 'leftover runaway continuation was interrupted'); - ws.dispose(); -}); - -test('registry: default memory limit flows to created workspaces', async () => { - const registry = new WorkspaceRegistry({ memoryLimit: 1024 * 1024 }); - const ws = await registry.get('/tmp/repl-project-limit'); - assert.equal(ws.memoryLimit, 1024 * 1024); - const outcome = await ws.eval("'w'.repeat(64 * 1024 * 1024)"); - assert.equal(outcome.kind, 'error'); - if (outcome.kind === 'error') assert.equal(outcome.error.outOfMemory, true); - registry.disposeAll(); -}); - -test('manifest: user bindings that SHADOW or OVERWRITE baseline globals are enumerated with complete metadata and provenance (phase-E review round 5: the baseline filter removed `const Math = 42` entirely and the provenance registry\'s known-set skip suppressed its origin)', async () => { - // The provenance pass is broker-driven (each eval's maintenance pass - // attributes new/rebound bindings), so the test drives the workspace - // through a broker, exactly like the review suites. - const ws = await Workspace.create('/tmp/repl-shadow-project'); - const broker = await Broker.attach(ws, { evalTimeoutMs: 0 }); - try { - // A LEXICAL shadow of a baseline global: `const Math = 42` — the - // binding the orchestrator's code sees is the user's (identifier - // resolution prefers the lexical binding), so the manifest lists it - // with the lexical value's metadata and the declaring eval's - // provenance. - const r = await broker.eval('const Math = 42; Math'); - assert.equal(r.result, '42'); - let manifest = broker.workspaceManifest(); - let byName = new Map(manifest.bindings.map((b) => [b.name, b])); - const math = byName.get('Math'); - assert.ok(math, `Math is listed: ${[...byName.keys()].join(', ')}`); - assert.equal(math!.token, 'number \u00b7 8B'); - assert.equal(math!.type, 'number'); - assert.equal(math!.sizeBytes, 8); - assert.equal(math!.provenance, 'eval 1', 'the shadowing binding is attributed to its declaring eval'); - assert.ok(typeof math!.provenanceAtMs === 'number' && math!.provenanceAtMs! > 0); - assert.equal(math!.handleCallId, null); - assert.equal(math!.handleStatus, null); - // Exactly one Math binding (the lexical view wins over the global - // property — the same one-binding-per-name rule). - assert.equal(manifest.bindings.filter((b) => b.name === 'Math').length, 1); - // A GLOBAL PROPERTY overwrite of a baseline builtin: `JSON = "x"` - // (sloppy assignment rebinds the global property). The value's type - // token changed from the fresh-realm baseline — the manifest lists - // the overwrite with its provenance. - await broker.eval('JSON = "x"; 1'); - manifest = broker.workspaceManifest(); - byName = new Map(manifest.bindings.map((b) => [b.name, b])); - const json = byName.get('JSON'); - assert.ok(json, `JSON is listed: ${[...byName.keys()].join(', ')}`); - assert.equal(json!.token, 'string \u00b7 1B'); - assert.equal(json!.type, 'string'); - assert.equal(json!.provenance, 'eval 2', 'the overwrite is attributed to its eval'); - // Untouched baseline globals stay hidden (no noise), and the shadow - // survives later evals (a stable attribution). - await broker.eval('1 + 1'); - manifest = broker.workspaceManifest(); - byName = new Map(manifest.bindings.map((b) => [b.name, b])); - assert.ok(byName.has('Math') && !byName.has('Number'), 'shadow listed, untouched builtins still hidden'); - assert.equal(byName.get('Math')!.provenance, 'eval 1', 'the shadow attribution is stable'); - assert.equal(byName.get('JSON')!.provenance, 'eval 2', 'the overwrite attribution is stable'); - // The workspace keeps working: the guest sees the shadowed values. - const live = await broker.eval('Math'); - assert.equal(live.result, '42'); - } finally { - await broker.dispose(); - ws.dispose(); - } -}); - -test('manifest: a top-level LEXICAL `const globalThis = 7` (a legitimate user program) does not blank the provenance pass — a later `var userValue = 42` is enumerated with producer/task/time metadata (phase-E review rejection round 7: the pass read descriptors off the free variable globalThis, which the lexical binding shadows, so every descriptor read hit the NUMBER and the pass\'s catch swallowed the whole attribution — userValue appeared in the manifest with null provenance)', async () => { - const ws = await Workspace.create('/tmp/repl-globalthis-shadow-project'); - const broker = await Broker.attach(ws, { evalTimeoutMs: 0 }); - try { - // `const globalThis = 7` shadows the realm's global object for - // identifier resolution; `var userValue = 42` is a global-object - // property the manifest must attribute to this eval. - const r = await broker.eval('const globalThis = 7; var userValue = 42; userValue'); - assert.equal(r.result, '42'); - const manifest = broker.workspaceManifest(); - const byName = new Map(manifest.bindings.map((b) => [b.name, b])); - const userValue = byName.get('userValue'); - assert.ok(userValue, `userValue is listed: ${[...byName.keys()].join(', ')}`); - assert.equal(userValue!.token, 'number \u00b7 8B'); - assert.equal(userValue!.type, 'number'); - assert.equal(userValue!.provenance, 'eval 1', 'the pass read descriptors off the CAPTURED global object — provenance survives the lexical shadow'); - assert.ok(typeof userValue!.provenanceAtMs === 'number' && userValue!.provenanceAtMs! > 0, 'the attribution carries its timestamp'); - // The workspace keeps working after the shadow (the library's own - // internal references use the captured global too): a fresh eval - // still reaches host functions and the realm globals. - const live = await broker.eval('typeof console.log'); - assert.equal(live.result, 'function', 'the library internals are immune to the globalThis shadow'); - } finally { - await broker.dispose(); - ws.dispose(); - } -}); - -test('manifest: a SAME-TYPE overwrite of a baseline global (`Math = { userOwned: true }`) is enumerated with complete metadata and provenance — the type token cannot see it (both values are objects), the value identity can (phase-E review rejection round 6: the token-only detector missed same-type replacements entirely, leaving them absent from the manifest with no provenance)', async () => { - const ws = await Workspace.create('/tmp/repl-same-type-project'); - const broker = await Broker.attach(ws, { evalTimeoutMs: 0 }); - try { - // `Math = { userOwned: true }` (sloppy assignment rebinds the - // global property): the value's trap-free type token is `object` — - // the same as the pristine baseline Math — so the token-only - // detector saw no change. The registry's baseline-VALUE identity - // (SameValue against the ORIGINAL baseline value, captured when the - // registry was created in the pristine realm) is the detector that - // catches it: the manifest lists the overwrite with its provenance. - const r = await broker.eval('Math = { userOwned: true }; "rebound"'); - assert.equal(r.result, 'rebound'); - let manifest = broker.workspaceManifest(); - let byName = new Map(manifest.bindings.map((b) => [b.name, b])); - const math = byName.get('Math'); - assert.ok(math, `the same-type overwrite of Math is listed: ${[...byName.keys()].join(', ')}`); - assert.equal(math!.type, 'object', 'the overwriting value is reported as an object'); - assert.ok(typeof math!.sizeBytes === 'number' && math!.sizeBytes >= 0); - assert.equal(math!.provenance, 'eval 1', 'the same-type overwrite is attributed to its declaring eval'); - assert.ok(typeof math!.provenanceAtMs === 'number' && math!.provenanceAtMs! > 0); - // Untouched baseline globals stay hidden (no noise). - assert.ok(!byName.has('JSON'), 'an untouched baseline builtin stays hidden'); - // A SECOND same-type rebind re-attributes to its own eval (the - // last-attributed value is the comparison base — a pre-snapshot - // rebind is never re-attributed by a later pass either), and - // IN-PLACE mutation of the rebound value deliberately does NOT - // re-attribute (the binding still refers to the value its recorded - // origin produced — the manifest's documented stance). - await broker.eval('Math = { other: 1 }; 1'); - manifest = broker.workspaceManifest(); - byName = new Map(manifest.bindings.map((b) => [b.name, b])); - assert.equal(byName.get('Math')!.provenance, 'eval 2', 'a second same-type rebind re-attributes to its own eval'); - await broker.eval('Math.other = 2; 1'); - manifest = broker.workspaceManifest(); - byName = new Map(manifest.bindings.map((b) => [b.name, b])); - assert.equal(byName.get('Math')!.provenance, 'eval 2', 'in-place mutation of the rebound value does not re-attribute'); - assert.ok(byName.has('Math'), 'the overwritten binding stays listed'); - // The workspace keeps working: the guest sees the overwritten values. - const live = await broker.eval('Math.userOwned === undefined && Math.other === 2'); - assert.equal(live.result, 'true', 'the guest sees the overwritten binding'); - } finally { - await broker.dispose(); - ws.dispose(); - } -}); diff --git a/packages/repl-engine/tsconfig.json b/packages/repl-engine/tsconfig.json deleted file mode 100644 index 8fb9e2dc..00000000 --- a/packages/repl-engine/tsconfig.json +++ /dev/null @@ -1,9 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "dist" - }, - "include": ["src/**/*"], - "exclude": ["dist"] -} diff --git a/packages/workflows/README.md b/packages/workflows/README.md index 4862dc3d..ed0fa16f 100644 --- a/packages/workflows/README.md +++ b/packages/workflows/README.md @@ -12,7 +12,7 @@ or a registered custom ACP agent — driving the actual subprocess to completion This package is the **canonical SDK** that the stdio MCP server [`@automatalabs/mcp-server`](https://www.npmjs.com/package/@automatalabs/mcp-server) is built on. Its CLI can also delegate to a build-time embedded copy of that server with the `mcp` subcommand, -so an MCP host can expose `workflow`, `workflow_monitor`, and `repl` without a separate package install. The +so an MCP host can expose `workflow` and `workflow_monitor` without a separate package install. The standalone MCP server package remains independently published, while programs embedding the runner continue to use this package's workflow/runner APIs. @@ -879,8 +879,8 @@ hosts (Claude Code `--transport http`, Codex `config.toml` `url`), which can ski entirely. See the [`@automatalabs/mcp-server` README](../mcp-server#the-workflow-daemon) for the daemon's full contract (discovery, project routing, idle shutdown, security posture). -The bundled server exposes `workflow`, the separate `workflow_monitor` view entry on App-capable -hosts, and `repl`; it has no auth tools. `workflow` has the strict +The bundled server exposes `workflow` and the separate `workflow_monitor` view entry on App-capable +hosts; it has no auth tools. `workflow` has the strict config/run/resume/setup-response/status/result/permissions-response/stop lifecycle. Run prepares the script inside the request (structure checks, mocked dry run, routed probes, routing admission) and acknowledges only an admitted run; malformed input and validation failures are tool execution @@ -897,12 +897,6 @@ and terminal notifications use available host capabilities with duplicate suppre activity remains quiet. Panel closure and request timeout never stop an accepted run. Explicit Stop is durable; cold recovery preserves the source, setup receipts, and checkpoint answers. See the [lifecycle reference](../../docs/authoring/agentprism-workflow-authoring/references/run-lifecycle.md). -The `repl` tool is a persistent QuickJS-in-WASM JavaScript REPL, **one VM per `projectDir`** -(the same per-project model as `workflow`), for live, stateful subagent orchestration: workspace -state (bindings, pending subagent calls, checkpoints, logged values) persists in the VM across tool -calls and daemon restarts through the per-project `repl/` store, and drains when the project's last -MCP client disconnects. See [The `repl` tool](../mcp-server#the-repl-tool) for its full contract. - For the source inner loop, build workflows before launching its compiled CLI: ```bash @@ -1084,7 +1078,7 @@ globals documented by the ambient `dsl.d.ts`.) ## Authoring guidance for MCP agents -The MCP server bundled in this package publishes version-matched workflow and REPL Agent Skills +The MCP server bundled in this package publishes the version-matched workflow Agent Skill through the accepted SEP-2640 Skills Extension. A skills-aware host discovers `skill://agentprism-workflow-authoring/SKILL.md`, activates it through its own approval path, and reads supporting references lazily through MCP resources. The concise canonical sources live under @@ -1100,7 +1094,7 @@ reads supporting references lazily through MCP resources. The concise canonical MCP-wired image producer. - **[`@automatalabs/mcp-server`](https://www.npmjs.com/package/@automatalabs/mcp-server)** — the stdio MCP server built on this SDK. It wraps the same engine + ACP backend behind the `workflow` - and `repl` tools, with a separate `workflow_monitor` view (bin: `agentprism-workflow`; no auth tools). Use it when you want + tool, with a separate `workflow_monitor` view (bin: `agentprism-workflow`; no auth tools). Use it when you want the **MCP-tool route** instead of embedding the runner in code. ## License diff --git a/packages/workflows/package.json b/packages/workflows/package.json index 7a2fb41b..e8ffd06a 100644 --- a/packages/workflows/package.json +++ b/packages/workflows/package.json @@ -45,7 +45,6 @@ "prepublishOnly": "pnpm run build" }, "dependencies": { - "@automatalabs/repl-engine": "workspace:*", "@automatalabs/shared-types": "workspace:*", "@automatalabs/workflow-engine": "workspace:*", "@automatalabs/acp-agents": "workspace:*", diff --git a/packages/workflows/test/mcp-server-bundle.test.ts b/packages/workflows/test/mcp-server-bundle.test.ts index 6200b8f8..58482948 100644 --- a/packages/workflows/test/mcp-server-bundle.test.ts +++ b/packages/workflows/test/mcp-server-bundle.test.ts @@ -12,7 +12,7 @@ const WORKFLOWS_DIST_ENTRY = resolve(WORKFLOWS_ROOT, "dist/index.js"); const MCP_SOURCE_ENTRY = resolve(REPOSITORY_ROOT, "packages/mcp-server/src/index.ts"); // The bundle lives under the MCP server's OWN tree so the externalized // `@automatalabs/*` imports resolve exactly like the published package's -// (the mcp-server node_modules links repl-engine, shared-types, and +// (the mcp-server node_modules links shared-types and // workflows — the workflows link is what keeps WORKFLOWS_DIST_ENTRY // load-bearing). const MCP_BUNDLE = resolve(REPOSITORY_ROOT, "packages/mcp-server/dist/mcp-server-bundle-smoke.js"); @@ -50,7 +50,7 @@ function request(id: number, method: string, params?: unknown): string { return `${JSON.stringify({ jsonrpc: "2.0", id, method, ...(params === undefined ? {} : { params }) })}\n`; } -test("the bundled stdio server initializes once and serves workflow/repl plus authoring skills", { timeout: 30_000 }, async () => { +test("the bundled stdio server initializes once and serves the workflow tool plus authoring skills", { timeout: 30_000 }, async () => { const home = mkdtempSync(join(tmpdir(), "automatalabs-workflows-mcp-bundle-")); const child = spawn(process.execPath, [MCP_BUNDLE], { cwd: REPOSITORY_ROOT, @@ -168,7 +168,7 @@ test("the bundled stdio server initializes once and serves workflow/repl plus au ); const toolsResult = toolsList.result as { tools?: Array<{ name?: unknown }> }; - assert.deepEqual(toolsResult.tools?.map((tool) => tool.name).sort(), ["repl", "workflow"]); + assert.deepEqual(toolsResult.tools?.map((tool) => tool.name).sort(), ["workflow"]); const skillsResult = skillsList.result as { skills?: Array<{ uri?: unknown }> }; assert.deepEqual( skillsResult.skills?.map((skill) => skill.uri).sort(), diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index d4c9ca94..1ae8902f 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -130,9 +130,6 @@ importers: packages/mcp-server: dependencies: - '@automatalabs/repl-engine': - specifier: workspace:* - version: link:../repl-engine '@automatalabs/shared-types': specifier: workspace:* version: link:../shared-types @@ -205,27 +202,6 @@ importers: specifier: 0.85.1 version: 0.85.1(@modelcontextprotocol/sdk@1.30.0(zod@4.6.5))(ws@8.21.3)(zod@4.6.5) - packages/repl-engine: - dependencies: - '@automatalabs/acp-agents': - specifier: workspace:* - version: link:../acp-agents - '@automatalabs/shared-types': - specifier: workspace:* - version: link:../shared-types - '@automatalabs/workflows': - specifier: workspace:* - version: link:../workflows - acorn: - specifier: ^8.17.0 - version: 8.17.0 - quickjs-wasi: - specifier: 3.3.1 - version: 3.3.1 - typebox: - specifier: 1.3.2 - version: 1.3.2 - packages/shared-types: dependencies: typebox: @@ -249,9 +225,6 @@ importers: '@automatalabs/acp-agents': specifier: workspace:* version: link:../acp-agents - '@automatalabs/repl-engine': - specifier: workspace:* - version: link:../repl-engine '@automatalabs/shared-types': specifier: workspace:* version: link:../shared-types @@ -2375,9 +2348,6 @@ packages: queue-microtask@1.2.3: resolution: {integrity: sha512-NuaNSa6flKT5JaSYQzJok04JzTL1CA6aGhv5rfLW3PgqA+M2ChpZQnAC8h8i4ZFkBS8X5RqkDBHA7r4hej3K9A==} - quickjs-wasi@3.3.1: - resolution: {integrity: sha512-03RhBUA6hNX4274oa10pWpkXsbOhgqTvCoFPm8Dy3E7jcp+obNHVQGQZXghcNOutJaiOWkyezwdgbANHD1/Gxw==} - range-parser@1.2.1: resolution: {integrity: sha512-Hrgsx+orqoygnmhFbKaHE6c296J+HTAQXoxEF6gNupROmmGJRoyzfG3ccAveqCBrwr/2yxQ5BVd/GTl5agOwSg==} engines: {node: '>= 0.6'} @@ -4919,8 +4889,6 @@ snapshots: queue-microtask@1.2.3: {} - quickjs-wasi@3.3.1: {} - range-parser@1.2.1: {} range-parser@1.3.0: {} diff --git a/tsconfig.json b/tsconfig.json index e79bd6d3..cf3e60a0 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -5,7 +5,6 @@ { "path": "packages/acp-agents" }, { "path": "packages/acp-server" }, { "path": "packages/workflow-engine" }, - { "path": "packages/repl-engine" }, { "path": "packages/mcp-server" }, { "path": "packages/pi-acp" }, { "path": "packages/workflows" },