diff --git a/docs/formats/replay-v1.md b/docs/formats/replay-v1.md new file mode 100644 index 0000000..1aa026e --- /dev/null +++ b/docs/formats/replay-v1.md @@ -0,0 +1,46 @@ +# Replay bundle format v1 + +A replay bundle is a directory containing `manifest.json` and independent scene files. It contains presentation data; it cannot restart a model or execute simulation steps. `microsimulator export-replay CHECKPOINT... --output DIRECTORY` exports an explicitly ordered sequence using CPU checkpoint deserialization. It never imports the original model, invokes controller callbacks, or requires the source run's GPU. + +## Manifest + +The manifest envelope has exactly `format`, `version`, `integrity`, and `recording` fields. Format is `microsimulator-replay`, version is `1`, and `integrity` has `algorithm: "sha256"` and `recording`, the lowercase SHA-256 digest of the RFC 8785 canonical representation of the complete recording object. This detects corruption; it is not proof of publisher authenticity. + +`recording` contains exactly: + +- `export_backend`: backend identity of the CPU used to deserialize the portable checkpoints. This has the same `kind`, `name`, `device`, `device_index`, and `native` fields as a scene backend. +- `frames`: a nonempty ordered array of frame entries. + +Each entry contains exactly: + +| Field | Meaning | +| --- | --- | +| `ordinal` | Zero-based contiguous ordinal equal to the entry's array index | +| `time` | Finite nonnegative recorded simulation time | +| `file` | Safe relative path to a `.scene.json` document | +| `bytes` | Exact positive UTF-8 file byte length | +| `sha256` | Lowercase SHA-256 of the exact scene file bytes, including whitespace | +| `checkpoint_sha256` | Lowercase SHA-256 of the exact source checkpoint bytes consumed | +| `source_backend` | Backend identity recorded by that checkpoint's producer | + +The entry order is authoritative. Paths and checkpoint names are never sorted. Times must be nondecreasing. Equal times remain distinct frames and receive distinct ordinals; they are useful for topology events or separate observations at the same physical time. The exporter does not interpolate, merge, or drop snapshots. Supply checkpoints from the same run when stable cell identity across frames is required; the exporter cannot infer common ancestry from arbitrary checkpoint provenance. + +Paths may use ASCII letters, digits, underscores, hyphens and dots within nonempty segments. Segments start with a letter, digit, underscore or hyphen. Absolute paths, dot segments, backslashes, URI schemes and percent-encoded escapes are rejected. References must be unique. References identify files selected from the bundle folder; the browser never fetches URLs from a manifest. + +Readers verify the recording digest, strict schema, ordinals, timestamps and references before opening a frame. On demand, they verify the file size and exact-file digest, then use the shared scene reader to verify the scene's own digest/schema. The scene time and source backend must match the entry. Scene version 2 or 3 is supported through that reader; current exports use version 3 and preserve channel metadata. + +The scene's backend describes the source run, so the viewer does not present the exporter's CPU as the simulation device. The exporter identity is retained separately. Checkpoint source-backend values are provenance, not a request to allocate that device. Source paths are omitted; `checkpoint_sha256` identifies the source bytes without leaking machine-local paths. Each checkpoint is copied into a temporary snapshot before parsing, so a producer replacing its original path cannot make the parsed bytes differ from the recorded digest. + +## Limits and failure behavior + +Manifests are limited to 16 MiB and 100,000 entries. Each scene is bounded by the scene format's 1 GiB encoded limit. The exporter requires a new destination; it refuses existing files, folders and symlinks. It builds a temporary sibling and publishes the completed bundle only after all frames pass validation. Failures identify the zero-based ordinal and source checkpoint, remove the temporary export, and leave source checkpoints untouched. + +The viewer retains file handles and manifest metadata, but reads scene payloads only on demand. Its LRU cache holds at most three decoded frames and at most 64 MiB of conservative decoded-size accounting units. Objects too large for that budget are displayed without entering the cache. This is not a 64 MiB process-heap limit: the current displayed frame, renderer/GPU buffers, manifest, file handles and one active load/parse can exist outside the cache. Cache accounting includes cell arrays, species, signal levels, boundary values, constraints and labels. + +A single worker performs frame loading/decoding; repeated seeks replace one pending ordinal rather than creating a queue of decodes. Superseded reads are canceled where possible. Both successful and failed stale requests are ignored. Opening a different dataset cancels the previous reader and prevents its completion from changing the view. Failures leave the last successfully displayed frame in place, identify the affected ordinal/file, pause playback and allow seeking to another frame. + +## Playback and presentation + +Playback displays every recorded frame without interpolation. Configurable 1–120 frames/s defines a maximum presentation cadence; slow loading reduces achieved speed instead of skipping frames. Recorded simulation time remains visible independently of playback speed, including in the transport bar at narrow supported window widths. Playback stops at the last frame; pressing Play there restarts from the first when no seek is pending. Pressing Play during a pending seek waits for that requested frame, then starts timed playback from it. After a load failure, Play retries the failed requested frame and resumes only after it loads successfully; another failure leaves playback paused and preserves the last valid view. Manual seeking and previous/next stepping pause automatic playback. + +Opening a recording begins one viewer dataset. Subsequent frames, backward steps and seeks use the shared presentation-update path. Camera pose, reference-grid geometry and index-based channel preferences persist. Selected cells follow stable IDs across slot changes; selection clears if the ID is absent. Missing signal grids temporarily hide the controls, and smaller grids clamp displayed indices while retaining preferences for later compatible frames. Opening another recording or a static scene starts a new dataset. diff --git a/examples/replay_demo.py b/examples/replay_demo.py new file mode 100644 index 0000000..f8ccad9 --- /dev/null +++ b/examples/replay_demo.py @@ -0,0 +1,43 @@ +"""Short deterministic growth/division/removal recording for the replay tutorial.""" + +from microsimulator import ( + CellInit, + ChannelMetadata, + CheckpointBundle, + ControllerStep, + DivisionRequest, + ModelContext, + NativeController, + StepPlan, +) + + +def regulate(step: ControllerStep) -> StepPlan: + if step.completed_steps == 1: + return StepPlan(divisions=(DivisionRequest(step.cells[0].id),)) + if step.completed_steps == 2: + return StepPlan(removals=(step.cells[0].id,)) + return StepPlan() + + +def build(context: ModelContext) -> NativeController: + simulation = context.simulation(species_count=1) + cell = CellInit() + cell.length = 4.0 + cell.growth_rate = 0.5 + cell.species = [0.25] + simulation.add_cell(cell) + return NativeController( + simulation, + model_id="replay-demo", + model_version=1, + rng=context.rng, + regulate=regulate, + channel_metadata=ChannelMetadata(species=("Reporter",)), + ) + + +def resume(context: ModelContext, checkpoint: CheckpointBundle) -> NativeController: + return NativeController.from_checkpoint( + checkpoint, model_id="replay-demo", model_version=1, regulate=regulate + ) diff --git a/python/src/microsimulator/cli.py b/python/src/microsimulator/cli.py index 540473f..b1478f9 100644 --- a/python/src/microsimulator/cli.py +++ b/python/src/microsimulator/cli.py @@ -107,6 +107,12 @@ def _parser() -> argparse.ArgumentParser: ) analysis.add_argument("--overwrite", action="store_true") + replay = commands.add_parser( + "export-replay", help="export explicitly ordered checkpoints for offline replay" + ) + replay.add_argument("checkpoints", nargs="+", type=Path) + replay.add_argument("--output", type=Path, required=True, help="new replay bundle directory") + manifest = commands.add_parser( "run-manifest", help="execute one named job from a data-only run manifest" ) @@ -459,8 +465,7 @@ def _export_analysis(arguments: argparse.Namespace) -> int: if not backend_available(backend, device_index): count = backend_device_count(backend) raise BatchError( - f"backend {backend_name} device {device_index} is unavailable " - f"({count} device(s) found)" + f"backend {backend_name} device {device_index} is unavailable ({count} device(s) found)" ) summary = export_dataset( cast(list[Path], arguments.checkpoints), @@ -511,6 +516,14 @@ def main(argv: Sequence[str] | None = None) -> int: return _view(arguments) if arguments.command == "export-analysis": return _export_analysis(arguments) + if arguments.command == "export-replay": + from .replay import export_replay + + summary = export_replay( + cast(list[Path], arguments.checkpoints), cast(Path, arguments.output) + ) + print(f"wrote {summary.output} frames={summary.frame_count}") + return 0 if arguments.command == "run-manifest": return _run_manifest(arguments) return _run(arguments) diff --git a/python/src/microsimulator/replay.py b/python/src/microsimulator/replay.py new file mode 100644 index 0000000..f8feea3 --- /dev/null +++ b/python/src/microsimulator/replay.py @@ -0,0 +1,146 @@ +"""Data-only replay export from an explicitly ordered checkpoint sequence.""" + +from __future__ import annotations + +import hashlib +import json +import os +import tempfile +from collections.abc import Sequence +from dataclasses import asdict, dataclass, replace +from pathlib import Path +from typing import cast + +import rfc8785 + +from ._core import BackendKind # pyright: ignore[reportMissingModuleSource] +from .checkpoint import MAX_CHECKPOINT_BYTES, CheckpointError, JSONValue, load_checkpoint_bundle +from .scene import MAX_SCENE_BYTES, SceneBackend, SceneBackendKind, capture_scene, dumps_scene + +REPLAY_FORMAT = "microsimulator-replay" +REPLAY_VERSION = 1 +MAX_REPLAY_FRAMES = 100_000 +MAX_REPLAY_MANIFEST_BYTES = 16 * 1024 * 1024 + + +class ReplayExportError(ValueError): + """Raised when an ordered recording cannot be exported without loss.""" + + +@dataclass(frozen=True, slots=True) +class ReplayExportSummary: + output: Path + frame_count: int + + +def export_replay( + checkpoints: Sequence[str | os.PathLike[str]], output: str | os.PathLike[str] +) -> ReplayExportSummary: + """Export exact snapshots on CPU, never loading model source or stepping biology. + + Input order is authoritative. Equal times remain distinct frames; decreasing + time is rejected. The destination must not exist, including an empty folder. + """ + + if not 1 <= len(checkpoints) <= MAX_REPLAY_FRAMES: + raise ReplayExportError(f"expected 1 to {MAX_REPLAY_FRAMES} ordered checkpoints") + destination = Path(output).absolute() + if destination.exists() or destination.is_symlink(): + raise ReplayExportError(f"output already exists: {destination}") + try: + destination.parent.mkdir(parents=True, exist_ok=True) + except OSError as error: + raise ReplayExportError( + f"could not prepare replay destination {destination}: {error}" + ) from error + entries: list[JSONValue] = [] + previous_time = -1.0 + export_backend: JSONValue = None + try: + with tempfile.TemporaryDirectory( + prefix=f".{destination.name}.", dir=destination.parent + ) as temporary: + stage = Path(temporary) + frames_dir = stage / "frames" + frames_dir.mkdir() + snapshot = stage / ".checkpoint.json" + for ordinal, checkpoint_path in enumerate(checkpoints): + try: + # Parse exactly the bytes whose digest is recorded, even if a + # running producer replaces the original checkpoint concurrently. + digest = hashlib.sha256() + size = 0 + with Path(checkpoint_path).open("rb") as source, snapshot.open("wb") as target: + while chunk := source.read(1024 * 1024): + size += len(chunk) + if size > MAX_CHECKPOINT_BYTES: + raise ReplayExportError("checkpoint exceeds its byte limit") + target.write(chunk) + digest.update(chunk) + bundle = load_checkpoint_bundle(snapshot, backend=BackendKind.CPU) + captured = capture_scene( + bundle.simulation, channel_metadata=bundle.channel_metadata + ) + if captured.time < previous_time: + raise ReplayExportError( + f"time {captured.time} precedes previous frame time {previous_time}" + ) + previous_time = captured.time + if export_backend is None: + export_backend = cast(JSONValue, asdict(captured.backend)) + source_backend = bundle.source_backend + # The displayed scene describes its source run, not the CPU + # used solely to deserialize portable state during export. + frame = replace( + captured, + backend=SceneBackend( + kind=cast(SceneBackendKind, source_backend.kind), + name=source_backend.name, + device=source_backend.device, + device_index=source_backend.device_index, + native=source_backend.native, + ), + ) + encoded = dumps_scene(frame).encode("utf-8") + if len(encoded) > MAX_SCENE_BYTES: + raise ReplayExportError("scene exceeds its byte limit") + relative = f"frames/{ordinal:08d}.scene.json" + (stage / relative).write_bytes(encoded) + entries.append( + { + "ordinal": ordinal, + "time": frame.time, + "file": relative, + "bytes": len(encoded), + "sha256": hashlib.sha256(encoded).hexdigest(), + "checkpoint_sha256": digest.hexdigest(), + "source_backend": cast(JSONValue, asdict(source_backend)), + } + ) + except (OSError, ValueError, RuntimeError) as error: + raise ReplayExportError( + f"frame {ordinal} ({checkpoint_path}): {error}" + ) from error + snapshot.unlink() + recording: dict[str, JSONValue] = {"export_backend": export_backend, "frames": entries} + document: dict[str, JSONValue] = { + "format": REPLAY_FORMAT, + "version": REPLAY_VERSION, + "integrity": { + "algorithm": "sha256", + "recording": hashlib.sha256(rfc8785.dumps(recording)).hexdigest(), + }, + "recording": recording, + } + encoded_manifest = ( + json.dumps(document, allow_nan=False, ensure_ascii=False, indent=2) + "\n" + ).encode("utf-8") + if len(encoded_manifest) > MAX_REPLAY_MANIFEST_BYTES: + raise ReplayExportError("replay manifest exceeds the 16 MiB limit") + (stage / "manifest.json").write_bytes(encoded_manifest) + if destination.exists() or destination.is_symlink(): + raise ReplayExportError(f"output already exists: {destination}") + stage.rename(destination) + except (OSError, CheckpointError) as error: + raise ReplayExportError(f"could not export replay to {destination}: {error}") from error + return ReplayExportSummary(destination, len(entries)) diff --git a/python/tests/test_replay.py b/python/tests/test_replay.py new file mode 100644 index 0000000..d49cbaa --- /dev/null +++ b/python/tests/test_replay.py @@ -0,0 +1,195 @@ +from __future__ import annotations + +import hashlib +import json +from pathlib import Path +from typing import Any, cast + +import pytest +import rfc8785 +from microsimulator import ( + CellInit, + ChannelMetadata, + GridShape, + SignalGridSpec, + Simulation, + Vec3, + load_scene, + save_checkpoint, +) +from microsimulator.cli import main +from microsimulator.replay import ReplayExportError, export_replay + + +def lifecycle_checkpoints(directory: Path) -> list[Path]: + """Native growth/division/removal; names deliberately oppose lexical ordering.""" + simulation = Simulation(species_count=1) + cell = CellInit() + cell.length = 2.0 + cell.growth_rate = 0.5 + cell.species = [0.25] + parent = simulation.add_cell(cell) + paths: list[Path] = [] + for ordinal, name in enumerate( + ("z-start.json", "a-growth.json", "m-division.json", "b-removal.json") + ): + if ordinal == 1: + simulation.step(0.2) + elif ordinal == 2: + simulation.divide_equal(parent) + elif ordinal == 3: + simulation.remove_cell(2) + simulation.step(0.1) + path = directory / name + save_checkpoint( + simulation, + path, + channel_metadata=ChannelMetadata(species=("Reporter",)), + provenance={"model": {"path": "/missing/model-that-must-not-be-imported.py"}}, + ) + paths.append(path) + return paths + + +def test_ordered_export_preserves_topology_labels_equal_times_and_input_files( + tmp_path: Path, +) -> None: + paths = lifecycle_checkpoints(tmp_path) + before = [path.read_bytes() for path in paths] + summary = export_replay(paths, tmp_path / "bundle") + assert summary.frame_count == 4 + manifest = json.loads((summary.output / "manifest.json").read_text()) + assert manifest["format"] == "microsimulator-replay" + assert manifest["version"] == 1 + recording = manifest["recording"] + assert ( + hashlib.sha256(rfc8785.dumps(recording)).hexdigest() == manifest["integrity"]["recording"] + ) + frames = [load_scene(summary.output / entry["file"]) for entry in recording["frames"]] + assert [len(frame.cells) for frame in frames] == [1, 1, 2, 1] + assert frames[1].cells[0].length > frames[0].cells[0].length + assert [cell.id for cell in frames[2].cells] == [2, 3] + assert frames[3].cells[0].id == 3 + assert frames[2].time == frames[1].time + for ordinal, (entry, source) in enumerate(zip(recording["frames"], paths, strict=True)): + assert entry["ordinal"] == ordinal + assert entry["checkpoint_sha256"] == hashlib.sha256(source.read_bytes()).hexdigest() + encoded = (summary.output / entry["file"]).read_bytes() + assert entry["bytes"] == len(encoded) + assert entry["sha256"] == hashlib.sha256(encoded).hexdigest() + assert frames[ordinal].channel_metadata.species == ("Reporter",) + assert [path.read_bytes() for path in paths] == before + + +def test_source_backend_preserved_while_exporting_without_original_device(tmp_path: Path) -> None: + paths = lifecycle_checkpoints(tmp_path) + # This is a provenance fixture, not evidence of a CUDA simulation run. + source = cast(dict[str, Any], json.loads(paths[0].read_text())) + source["source_backend"] = { + "kind": "cuda", + "name": "Recorded GPU", + "device": "Unavailable GPU", + "device_index": 7, + "native": True, + } + paths[0].write_text(json.dumps(source)) + result = export_replay(paths[:1], tmp_path / "cpu-export") + manifest = json.loads((result.output / "manifest.json").read_text()) + assert manifest["recording"]["export_backend"]["kind"] == "cpu" + entry = manifest["recording"]["frames"][0] + assert entry["source_backend"] == source["source_backend"] + scene = load_scene(result.output / entry["file"]) + assert scene.backend.kind == "cuda" + assert scene.backend.device_index == 7 + + +def test_failure_is_attributed_to_ordinal_and_leaves_no_partial_bundle(tmp_path: Path) -> None: + paths = lifecycle_checkpoints(tmp_path) + output = tmp_path / "bad-order" + with pytest.raises(ReplayExportError, match=r"frame 1.*precedes previous"): + export_replay([paths[1], paths[0]], output) + assert not output.exists() + assert not list(tmp_path.glob(".bad-order.*")) + paths[2].write_text("invalid") + with pytest.raises(ReplayExportError, match=r"frame 2.*not valid"): + export_replay(paths, output) + assert not output.exists() + with pytest.raises(ReplayExportError, match=r"frame 0.*missing.json"): + export_replay([tmp_path / "missing.json"], output) + output.mkdir() + with pytest.raises(ReplayExportError, match="output already exists"): + export_replay(paths[:1], output) + with pytest.raises(ReplayExportError, match="ordered checkpoints"): + export_replay([], tmp_path / "empty") + + +def test_export_rejects_checkpoint_digest_tampering(tmp_path: Path) -> None: + paths = lifecycle_checkpoints(tmp_path) + document = json.loads(paths[0].read_text()) + document["simulation"]["time"] = 100 + paths[0].write_text(json.dumps(document)) + with pytest.raises(ReplayExportError, match="state digest does not match"): + export_replay(paths, tmp_path / "bad") + + +def test_cli_exports_exact_argument_order_and_reports_errors( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + paths = lifecycle_checkpoints(tmp_path) + output = tmp_path / "cli" + assert main(["export-replay", *(str(path) for path in paths), "--output", str(output)]) == 0 + assert "frames=4" in capsys.readouterr().out + assert ( + main( + ["export-replay", str(paths[1]), str(paths[0]), "--output", str(tmp_path / "backwards")] + ) + == 2 + ) + assert "frame 1" in capsys.readouterr().err + + +def test_signal_grid_changes_are_independent_frames(tmp_path: Path) -> None: + paths: list[Path] = [] + for ordinal, size in enumerate((None, 1, 3)): + simulation = Simulation(species_count=0) + if size is not None: + shape = GridShape() + shape.x, shape.y, shape.z = size, size, size + spec = SignalGridSpec() + spec.signal_count = 2 + spec.shape = shape + spec.spacing = Vec3(1, 1, 1) + spec.diffusion = [0, 0] + spec.advection = [Vec3(), Vec3()] + simulation.configure_signal_grid(spec, [1.0] * (2 * size**3)) + path = tmp_path / f"{ordinal}.json" + save_checkpoint(simulation, path) + paths.append(path) + result = export_replay(paths, tmp_path / "grid") + frames = [ + load_scene(result.output / f"frames/{ordinal:08d}.scene.json") for ordinal in range(3) + ] + assert frames[0].signal_grid is None + assert frames[1].signal_grid is not None and frames[1].signal_grid.shape == (1, 1, 1) + assert frames[2].signal_grid is not None and frames[2].signal_grid.shape == (3, 3, 3) + + +def test_export_bounds_and_parent_io_failures_are_actionable( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + import microsimulator.replay as replay + + paths = lifecycle_checkpoints(tmp_path) + parent = tmp_path / "file-parent" + parent.write_text("preserve") + with pytest.raises(ReplayExportError, match="could not prepare replay destination"): + export_replay(paths, parent / "bundle") + assert parent.read_text() == "preserve" + monkeypatch.setattr(replay, "MAX_REPLAY_MANIFEST_BYTES", 10) + with pytest.raises(ReplayExportError, match="manifest exceeds"): + export_replay(paths, tmp_path / "bounded") + assert not (tmp_path / "bounded").exists() + assert not list(tmp_path.glob(".bounded.*")) + monkeypatch.setattr(replay, "MAX_CHECKPOINT_BYTES", 10) + with pytest.raises(ReplayExportError, match=r"frame 0.*checkpoint exceeds"): + export_replay(paths, tmp_path / "input-limit") diff --git a/viewer/README.md b/viewer/README.md index 77a84b2..42022f5 100644 --- a/viewer/README.md +++ b/viewer/README.md @@ -98,6 +98,26 @@ The ground reference grid is separate from the scientific signal lattice. Its sq `browser/reference-grid.mjs` verifies the reference grid and presentation lifecycle in Chromium against a running Vite server. It uses Playwright (`@playwright/test`) and its installed Chromium; a shared installation can be supplied through `MICROSIMULATOR_PLAYWRIGHT_MODULE` as an absolute module filename. Set `VIEWER_URL` if the server is not on `http://127.0.0.1:4320`, and `EVIDENCE_DIR` to choose the screenshot directory. The test observes renderer transforms through test-only request instrumentation and introduces no production debug interface. +## Replay a recording + +Record periodic checkpoints using the short native growth/division/removal example, then list the checkpoint paths in the order they should play: + +```sh +uv run microsimulator run --model examples/replay_demo.py --backend cpu --seed 17 --steps 5 --dt 0.2 --checkpoint-every 1 --output run/replay.json +uv run microsimulator export-replay run/replay.step-00000001.json run/replay.step-00000002.json run/replay.step-00000003.json run/replay.step-00000004.json run/replay.step-00000005.json --output run/replay-bundle +pnpm --dir viewer dev +``` + +Open the displayed viewer URL, choose **Open recording**, and select the `run/replay-bundle` folder. Select the folder itself, containing `manifest.json` and `frames`, rather than one frame file. The standalone viewer reads the selected local files; no simulation server or source GPU is required. + +Use Play/Pause, Previous/Next, the frame slider and Frames/s. Slider arrow keys seek one recorded frame; the buttons also work with keyboard focus. Manual seeking pauses playback. The transport displays a one-based frame position and recorded simulation time, while the manifest uses zero-based ordinals. Equal-time frames remain individually selectable. Playback stops at the end; Play then restarts from frame one. Opening a static scene ends the recording session. + +Source paths are used exactly in command-line order; avoid relying on shell globs to establish chronological ordering. The final `run/replay.json` duplicates the last periodic state in this example and is intentionally omitted. Decreasing times cause an error. The exporter refuses existing destinations; choose a new bundle directory for another export. Model parameters and source are unnecessary for export, and no callbacks execute during playback. + +The reader loads frames on demand through a bounded three-frame/64 MiB accounting-budget LRU cache; it does not decode the whole recording. Oversized frames are uncached, and renderer/current-load allocations exist outside that cache. See the [replay format](../docs/formats/replay-v1.md) for integrity, provenance, resource bounds and failure behavior. This first implementation imports checkpoint sequences; live recording, video export and timeline-based simulation restart are separate features. + +For browser regression checks, generate native fixtures with `.venv/bin/python viewer/browser/replay-fixtures.py /tmp/replay-fixtures`, run the viewer on port 4326, then run `viewer/browser/replay.mjs` with `REPLAY_FIXTURES=/tmp/replay-fixtures` and `MICROSIMULATOR_PLAYWRIGHT_MODULE` pointing to an installed Playwright module. This uses the existing shared browser harness and adds no production debug API. + ## Concentration color ranges Species coloring and signal slices each offer Automatic and Fixed color ranges. Automatic uses the current frame's species extrema or the selected signal slice's extrema. Constant automatic data uses the midpoint color and a uniform legend; empty automatic data shows “no values” without numerical bounds. Fixed uses the entered minimum and maximum across frames and slices. Values outside that interval use endpoint colors; the underlying concentrations and inspector values remain unchanged. diff --git a/viewer/browser/replay-fixtures.py b/viewer/browser/replay-fixtures.py new file mode 100644 index 0000000..9ac211a --- /dev/null +++ b/viewer/browser/replay-fixtures.py @@ -0,0 +1,66 @@ +"""Generate browser fixtures using the public model, checkpoint and exporter APIs.""" + +from __future__ import annotations + +import argparse +from pathlib import Path + +from microsimulator import ( + BackendKind, + CellInit, + ChannelMetadata, + GridShape, + ModelContext, + SignalGridSpec, + Simulation, + Vec3, + build_model, + run_simulation, + save_checkpoint, +) +from microsimulator.replay import export_replay + +parser = argparse.ArgumentParser() +parser.add_argument("output", type=Path) +root = parser.parse_args().output +root.mkdir(parents=True, exist_ok=True) +model, provenance = build_model("examples/replay_demo.py", ModelContext(BackendKind.CPU, 0, 17)) +summary = run_simulation( + model, + steps=5, + dt=0.2, + output=root / "lifecycle.json", + checkpoint_every=1, + provenance=provenance, +) +export_replay(summary.periodic_checkpoints, root / "lifecycle") +paths = [] +# Data-only compatible scene states exercise unavailable grids and clamping. +# Distinct CPU simulations produce these grid fixtures; no biology claim is made. +for ordinal, size in enumerate((3, None, 1, 3)): + simulation = Simulation(species_count=2) + cell = CellInit() + cell.species = [0.25, 0.75] + if size is not None: + shape = GridShape() + shape.x, shape.y, shape.z = size, size, size + spec = SignalGridSpec() + spec.signal_count = 2 + spec.shape = shape + spec.spacing = Vec3(1, 1, 1) + spec.diffusion = [0, 0] + spec.advection = [Vec3(), Vec3()] + simulation.configure_signal_grid(spec, [0.25] * size**3 + [0.75] * size**3) + simulation.add_cell(cell) + simulation.step(ordinal * 0.2) + path = root / f"grid-{ordinal}.json" + save_checkpoint( + simulation, + path, + channel_metadata=ChannelMetadata( + species=("Green", "Red"), signals=("Nutrient", "Cue") if size else () + ), + ) + paths.append(path) +export_replay(paths, root / "grids") +print(root) diff --git a/viewer/browser/replay.mjs b/viewer/browser/replay.mjs new file mode 100644 index 0000000..35e456c --- /dev/null +++ b/viewer/browser/replay.mjs @@ -0,0 +1,225 @@ +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import { cp, mkdir, readFile, writeFile } from "node:fs/promises"; +import path from "node:path"; +import canonicalize from "canonicalize"; + +const { chromium, expect } = await import( + process.env.MICROSIMULATOR_PLAYWRIGHT_MODULE ?? "@playwright/test" +); +const url = process.env.VIEWER_URL ?? "http://127.0.0.1:4326"; +const evidence = process.env.EVIDENCE_DIR ?? "/tmp/microsimulator-replay"; +const fixtures = process.env.REPLAY_FIXTURES ?? `${evidence}/fixtures-v2`; +await mkdir(evidence, { recursive: true }); +const browser = await chromium.launch({ headless: true }); +try { + const page = await browser.newPage({ + viewport: { width: 1440, height: 960 }, + }); + const errors = []; + page.on("pageerror", (error) => errors.push(error.message)); + await page.route("**/src/colony-viewer.ts*", async (route) => { + const response = await route.fetch(); + const source = await response.text(); + const marker = "this.onSelection = onSelection;"; + assert.equal(source.split(marker).length, 2); + await route.fulfill({ + response, + body: source.replace( + marker, + `${marker}\nglobalThis.__testViewer = this;`, + ), + }); + }); + await page.route("**/src/replay-bundle.ts*", async (route) => { + const response = await route.fetch(); + const source = await response.text(); + const marker = "const bytes = await this.read(file, signal);"; + assert.equal(source.split(marker).length, 2); + await route.fulfill({ + response, + body: source.replace( + marker, + `await globalThis.__delayReplayLoad?.(ordinal);\n${marker}`, + ), + }); + }); + await page.goto(url); + async function open(name, count) { + await page + .locator("#recording-folder") + .setInputFiles(path.resolve(fixtures, name)); + await expect(page.locator("#replay-position")).toContainText( + `1 / ${count}`, + ); + await expect(page.locator("#replay-message")).toHaveText(""); + } + async function seek(ordinal, count) { + await page.locator("#replay-timeline").fill(String(ordinal)); + await page.locator("#replay-timeline").dispatchEvent("input"); + await expect(page.locator("#replay-position")).toContainText( + `${ordinal + 1} / ${count}`, + ); + await expect(page.locator("#replay-message")).toHaveText(""); + } + const snapshot = () => + page.evaluate(() => { + const viewer = globalThis.__testViewer; + viewer.grid.updateMatrixWorld(true); + return { + camera: viewer.camera.position.toArray(), + target: viewer.controls.target.toArray(), + grid: viewer.grid.matrixWorld.elements.slice(), + }; + }); + function assertStationary(actual, expected) { + assert.deepEqual(actual.grid, expected.grid); + for (const key of ["camera", "target"]) + actual[key].forEach((value, index) => { + assert.ok( + Math.abs(value - expected[key][index]) < 1e-9, + `${key}[${index}] remains stationary`, + ); + }); + } + await open("lifecycle", 5); + await expect(page.locator("#time-chip")).toHaveText("t = 0.2"); + await page.evaluate(() => { + const v = globalThis.__testViewer; + v.selectCell(0); + v.controls.enableDamping = false; + v.camera.position.multiplyScalar(2); + v.controls.update(); + }); + const initial = await snapshot(); + await expect(page.locator("#selection-title")).toHaveText("Cell 1"); + await seek(1, 5); + await expect(page.locator("#selection-title")).toHaveText("No cell selected"); + await expect(page.locator("#cell-count")).toHaveText("2"); + await page.evaluate(() => globalThis.__testViewer.selectCell(1)); + await expect(page.locator("#selection-title")).toHaveText("Cell 3"); + await seek(2, 5); + await expect(page.locator("#selection-title")).toHaveText("Cell 3"); + await expect(page.locator("#cell-count")).toHaveText("1"); + await expect(page.locator("#cell-details")).toContainText("Slot0"); + await seek(1, 5); + await expect(page.locator("#cell-details")).toContainText("Slot1"); + assertStationary(await snapshot(), initial); + await page.selectOption("#color-mode", "species"); + await expect(page.locator("#legend-title")).toHaveText("Reporter"); + // Delay an uncached frame and supersede it while its decode worker is occupied. + await page.evaluate(() => { + globalThis.__delayReplayLoad = (ordinal) => + ordinal === 4 + ? new Promise((resolve) => { + globalThis.__releaseReplay = resolve; + }) + : undefined; + }); + await page.locator("#replay-timeline").fill("4"); + await page.locator("#replay-timeline").dispatchEvent("input"); + await expect + .poll(() => page.evaluate(() => typeof globalThis.__releaseReplay)) + .toBe("function"); + await page.locator("#replay-timeline").fill("3"); + await page.locator("#replay-timeline").dispatchEvent("input"); + await page.evaluate(() => { + globalThis.__releaseReplay(); + globalThis.__delayReplayLoad = undefined; + }); + await expect(page.locator("#replay-position")).toContainText("4 / 5"); + await expect(page.locator("#time-chip")).toHaveText("t = 0.8"); + await expect(page.locator("#legend-title")).toHaveText("Reporter"); + await seek(0, 5); + await page.locator("#replay-fps").fill("20"); + await page.locator("#replay-fps").dispatchEvent("change"); + await page.locator("#replay-play").click(); + await expect(page.locator("#replay-position")).toContainText("5 / 5"); + await expect(page.locator("#replay-play")).toHaveText("Play"); + await page.locator("#replay-previous").focus(); + await page.keyboard.press("Enter"); + await expect(page.locator("#replay-position")).toContainText("4 / 5"); + assertStationary(await snapshot(), initial); + await page.screenshot({ path: `${evidence}/lifecycle-replay.png` }); + await open("grids", 4); + await page.selectOption("#color-mode", "species"); + await page.selectOption("#species-channel", "1"); + await page.selectOption("#signal-channel", "1"); + await page.locator("#signal-slice").fill("2"); + await page.locator("#signal-slice").dispatchEvent("input"); + await page.locator("#signal-visible").focus(); + await page.keyboard.press("Space"); + const gridInitial = await snapshot(); + await seek(1, 4); + await expect(page.locator("#signal-section")).toBeHidden(); + await seek(2, 4); + await expect(page.locator("#signal-slice")).toHaveValue("0"); + await seek(3, 4); + await expect(page.locator("#signal-slice")).toHaveValue("2"); + await expect(page.locator("#signal-channel")).toHaveValue("1"); + await expect(page.locator("#species-channel")).toHaveValue("1"); + await expect(page.locator("#signal-visible")).not.toBeChecked(); + await expect(page.locator("#legend-title")).toHaveText("Red"); + assertStationary(await snapshot(), gridInitial); + await page.screenshot({ path: `${evidence}/grid-replay.png` }); + await page.setViewportSize({ width: 880, height: 720 }); + for (const id of [ + "#replay-play", + "#replay-timeline", + "#replay-fps", + "#recording-open", + ]) { + const bounds = await page.locator(id).boundingBox(); + assert.ok( + bounds && bounds.x >= 0 && bounds.x + bounds.width <= 880, + `${id} fits the supported narrow layout`, + ); + } + await expect(page.locator("#replay-position")).toBeVisible(); + await page.screenshot({ path: `${evidence}/narrow-replay.png` }); + await page.setViewportSize({ width: 1440, height: 960 }); + // A damaged frame is reported persistently while retaining the last good frame. + const malformed = `${evidence}/malformed`; + await cp(`${fixtures}/lifecycle`, malformed, { + recursive: true, + force: true, + }); + const manifest = JSON.parse( + await readFile(`${malformed}/manifest.json`, "utf8"), + ); + const broken = Buffer.from("not a scene"); + const entry = manifest.recording.frames[2]; + entry.bytes = broken.byteLength; + entry.sha256 = createHash("sha256").update(broken).digest("hex"); + await writeFile(`${malformed}/${entry.file}`, broken); + manifest.integrity.recording = createHash("sha256") + .update(canonicalize(manifest.recording)) + .digest("hex"); + await writeFile(`${malformed}/manifest.json`, JSON.stringify(manifest)); + await page.locator("#recording-folder").setInputFiles(malformed); + await expect(page.locator("#replay-position")).toContainText("1 / 5"); + await page.locator("#replay-timeline").fill("2"); + await page.locator("#replay-timeline").dispatchEvent("input"); + await expect(page.locator("#replay-message")).toContainText( + "frame 2 (frames/00000002.scene.json)", + ); + await expect(page.locator("#replay-position")).toContainText("1 / 5"); + await seek(1, 5); + // Opening a static scene closes replay; delayed recording work cannot replace it. + await page + .locator("#scene-file") + .setInputFiles(`${fixtures}/grids/frames/00000000.scene.json`); + await expect(page.locator("#replay-transport")).toBeHidden(); + await expect(page.locator("#color-mode")).toHaveValue("cell-type"); + await expect(page.locator("#species-channel")).toHaveValue("0"); + assert.deepEqual(errors, []); + console.log( + JSON.stringify({ + status: "passed", + coverage: + "native exported topology, reverse steps, stable ID selection, time, rapid seeks, playback fps/end, metadata, missing/changing grids, camera/grid/preferences, malformed frame attribution/recovery, keyboard, narrow layout, new dataset reset", + }), + ); +} finally { + await browser.close(); +} diff --git a/viewer/index.html b/viewer/index.html index d216c93..4db8fe5 100644 --- a/viewer/index.html +++ b/viewer/index.html @@ -43,6 +43,20 @@ Open scene + + @@ -225,6 +239,65 @@

Inspect a colony snapshot

+