feat: M6 - WASM depth inference in a module Web Worker

STATUS.md

@@ -4,8 +4,8 @@ Single source of truth for "what's next." One milestone per PR/run. Autopilot:
 pick the one task under **NEXT**, ship it, stop. Do **not** start anything under
 **BLOCKED**.
 
-_Last updated: 2026-07-03 — Milestone 6 (Web Worker WASM inference) queued;
-the human approved the M4-findings candidate on 2026-07-03._
+_Last updated: 2026-07-03 — Milestone 6 (Web Worker WASM inference) done;
+queue empty pending an M7 proposal._
 
 ---
 
@@ -128,6 +128,33 @@ the human approved the M4-findings candidate on 2026-07-03._
   non-tautological (RED with production `src/` reverted, tests kept).
   Pre-change HEAD (rollback) `0c7a26d`.
 
+- **Milestone 6 — Web Worker WASM inference.** On the WASM fallback the depth
+  pipeline now runs in a dedicated **module Web Worker** (`src/depth-worker.js`),
+  so the main thread keeps rendering during a pass; the **WebGPU path is
+  byte-identical** (it already computes off-thread). The split-by-device design
+  also makes the `webInitChain` poison structurally impossible: the worker only
+  ever inits `wasm`, the main thread only ever inits `webgpu`. Facts verified
+  from the installed 4.2.0 source before coding: `env.js` classifies
+  `DedicatedWorkerGlobalScope` as a supported web env (`IS_WEBWORKER_ENV`), and
+  root-exported `RawImage(data, width, height, channels)` is a plain
+  field-assignment constructor — so the bridge posts ImageData pixels
+  (buffer transferred) and the worker wraps them in a `RawImage`; the depth
+  comes back as a plain `{ data, width, height }` (cloned, ~200KB — we don't
+  own the library buffer's lifetime), exactly the subset `applyDepthToCloud`
+  consumes. Both call sites now hand the estimator a **canvas** (the photo
+  handler draws its ImageBitmap to one at native size instead of passing a
+  blob URL, whose in-worker reachability is unverified). `hud.timeEstimator`
+  wraps the bridge proxy, so the M5 readout measures the true round trip;
+  the fake-estimator seam (`__setEstimator`) stays main-thread and never
+  touches the worker — all M3–M5 tests unchanged. Worker init failure rejects
+  the load and clears the estimator cache (retry spawns a fresh worker);
+  a mid-session worker crash rejects in-flight passes (the M4 gen-token catch
+  tears down) and clears the cache too. Smoke test M6-A pins the photo-path
+  canvas input contract (RED pre-M6: it got a blob-URL string). Real-model
+  probe (headless → WASM → worker): a 38.2s pass rendered **558 rAF frames,
+  max gap 1.57s** — pre-M6 the gap was the whole pass. Pre-change HEAD
+  (rollback) `c61d33b`.
+
 **Carry-over facts from M3/M4 (do not re-derive):**
 
 - Depth output contract: `depth` RawImage, Uint8, 1 channel, input W×H, 0–255
@@ -143,26 +170,8 @@ the human approved the M4-findings candidate on 2026-07-03._
 
 ## NEXT (the one actionable task)
 
-- **Milestone 6 — Web Worker WASM inference.** Move depth inference off the
-  main thread into a module Web Worker so machines on the WASM fallback keep
-  the 30fps orbit **during** a pass (today the main thread blocks ~4–11s per
-  WASM pass — M4 finding; WebGPU already computes off-thread and must keep
-  working, worker or not). Human approved this candidate 2026-07-03.
-  Constraints carried from M3–M5:
-  - **Verify before coding, never guess** (CLAUDE.md rule): confirm from the
-    installed transformers.js 4.2.0 source that the pipeline runs in a worker
-    and what input types survive `postMessage` (an `HTMLCanvasElement` does
-    NOT cross the boundary — expect `ImageBitmap`/`ImageData`/raw buffer +
-    `RawImage` on the worker side). If the installed source + a local runtime
-    probe can't settle it, that's a human checkpoint — bail, don't guess.
-  - Keep the probe-once device pick (the `webInitChain` poison quirk lives in
-    the worker now), the drop-never-queue post/consume contract, the M4
-    generation-token teardown semantics, and the M5 inference-time readout
-    (timing must now measure the round trip the user actually experiences).
-  - `window.__app.__setEstimator` and the M4/M5 smoke tests must keep working —
-    the fake-estimator seam should stay on the main thread side of the bridge.
-  - ⚠️ **Effort gate (skill §1–2): this is an inference-decoupling-core task —
-    implement only in a session at known-high effort (`/effort high`).**
+(Nothing queued — propose an M7 candidate with the human before the next
+autopilot run.)
 
 ---
 

src/depth-worker.js

@@ -0,0 +1,64 @@
+// ---------------------------------------------------------------------------
+// Milestone 6: dedicated module worker for WASM depth inference, so the main
+// thread keeps rendering during a pass (on WASM the ONNX session blocks the
+// thread that runs it — measured ~4–11s/pass in the M4 probe).
+//
+// transformers.js 4.2.0 runs here by design: env.js classifies
+// DedicatedWorkerGlobalScope as a supported web env (IS_WEBWORKER_ENV), and
+// RawImage(data, width, height, channels) is a plain field-assignment
+// constructor exported from the package root — both confirmed from the
+// installed source 2026-07-03.
+//
+// Protocol (all messages are plain objects):
+//   main → worker  { type: 'init', model }
+//   worker → main  { type: 'ready' } | { type: 'init-error', error }
+//   main → worker  { type: 'infer', id, data, width, height }
+//                  — RGBA ImageData pixels; the buffer arrives transferred.
+//   worker → main  { type: 'result', id, data, width, height }   (Uint8 depth)
+//                | { type: 'result', id, error }
+//
+// The depth result is structured-cloned, NOT transferred: its typed array
+// comes out of the library's post-processing and we don't own its buffer's
+// lifetime; a ~200KB copy per pass is negligible next to the pass itself.
+//
+// Only device:'wasm' ever initializes in this worker — the WebGPU path stays
+// on the main thread (it already computes off-thread) — so the webInitChain
+// poison quirk (see depth.js) cannot occur here.
+// ---------------------------------------------------------------------------
+
+let estimator = null;
+let RawImageCtor = null;
+
+function errMessage(err) {
+  return err && err.message ? String(err.message) : String(err);
+}
+
+self.onmessage = async (e) => {
+  const msg = e.data;
+  if (msg.type === 'init') {
+    try {
+      const { pipeline, RawImage } = await import('@huggingface/transformers');
+      RawImageCtor = RawImage;
+      estimator = await pipeline('depth-estimation', msg.model, { device: 'wasm' });
+      self.postMessage({ type: 'ready' });
+    } catch (err) {
+      self.postMessage({ type: 'init-error', error: errMessage(err) });
+    }
+    return;
+  }
+  if (msg.type === 'infer') {
+    try {
+      const image = new RawImageCtor(msg.data, msg.width, msg.height, 4);
+      const { depth } = await estimator(image);
+      self.postMessage({
+        type: 'result',
+        id: msg.id,
+        data: depth.data,
+        width: depth.width,
+        height: depth.height,
+      });
+    } catch (err) {
+      self.postMessage({ type: 'result', id: msg.id, error: errMessage(err) });
+    }
+  }
+};

src/depth.js

@@ -35,14 +35,22 @@ export function getSelectedDevice() {
 // the weight download).
 export function getDepthEstimator() {
   if (!estimatorPromise) {
-    estimatorPromise = loadEstimator().catch((err) => {
-      estimatorPromise = null;
+    // Cache resets are guarded by promise identity: only the load that owns
+    // the cache may clear it, so a test-installed fake (or a newer retry)
+    // never gets clobbered by a stale load's failure or worker crash.
+    const p = loadEstimator(() => resetCacheIf(p)).catch((err) => {
+      resetCacheIf(p);
       throw err;
     });
+    estimatorPromise = p;
   }
   return estimatorPromise;
 }
 
+function resetCacheIf(promise) {
+  if (estimatorPromise === promise) estimatorPromise = null;
+}
+
 // Probe WebGPU BEFORE touching transformers.js and pick the device once.
 // Try-webgpu-then-retry-wasm does NOT work on this library version: it chains
 // every web session load through a module-private promise
@@ -63,11 +71,90 @@ async function pickDevice() {
   return 'wasm';
 }
 
-async function loadEstimator() {
-  const { pipeline } = await import('@huggingface/transformers');
+async function loadEstimator(resetCache) {
   const device = await pickDevice();
   selectedDevice = device;
-  return pipeline('depth-estimation', MODEL, { device });
+  if (device === 'webgpu') {
+    // WebGPU already computes off the main thread — keep the direct path.
+    const { pipeline } = await import('@huggingface/transformers');
+    return pipeline('depth-estimation', MODEL, { device });
+  }
+  // Milestone 6: on the WASM fallback the ONNX session blocks whatever thread
+  // runs it (~4–11s/pass, M4 finding) — run it in a dedicated module worker so
+  // orbit stays live during a pass. Splitting by device also keeps the
+  // webInitChain poison quirk structurally impossible: the worker only ever
+  // inits WASM, the main thread only ever inits WebGPU.
+  return createWorkerEstimator(resetCache);
+}
+
+// Spawn the depth worker and resolve to an estimator with the same call shape
+// the direct pipeline has at our call sites: async (canvas) => { depth }.
+// Callers pass a canvas (both call sites in main.js do); a canvas can't cross
+// postMessage, so the bridge reads it back to ImageData here (synchronous,
+// bounded by the M4 512px capture cap on the hot path) and transfers the
+// pixel buffer to the worker. depth comes back as a plain { data, width,
+// height } — exactly the subset of the RawImage contract applyDepthToCloud
+// consumes. hud.timeEstimator wraps the returned function at the call sites,
+// so the M5 readout measures this full round trip.
+function createWorkerEstimator(resetCache) {
+  return new Promise((resolve, reject) => {
+    const worker = new Worker(new URL('./depth-worker.js', import.meta.url), {
+      type: 'module',
+    });
+    const pending = new Map(); // id → { resolve, reject }
+    let nextId = 0;
+    let dead = null;
+
+    // A worker failure (init or crash) kills this estimator for good: reject
+    // everything in flight, terminate, and clear the module cache (identity-
+    // guarded by the caller) so the next upload spawns a fresh worker — the
+    // same retry semantics as a failed load.
+    const fail = (err) => {
+      dead = err;
+      worker.terminate();
+      for (const p of pending.values()) p.reject(err);
+      pending.clear();
+      resetCache();
+      reject(err); // no-op if 'ready' already resolved us
+    };
+
+    worker.addEventListener('error', (e) => {
+      fail(new Error(e.message || 'depth worker crashed'));
+    });
+    worker.addEventListener('messageerror', () => {
+      fail(new Error('depth worker message error'));
+    });
+    worker.addEventListener('message', (e) => {
+      const msg = e.data;
+      if (msg.type === 'ready') {
+        resolve(estimatorFn);
+      } else if (msg.type === 'init-error') {
+        fail(new Error(msg.error));
+      } else if (msg.type === 'result') {
+        const p = pending.get(msg.id);
+        if (!p) return;
+        pending.delete(msg.id);
+        if (msg.error) p.reject(new Error(msg.error));
+        else p.resolve({ depth: { data: msg.data, width: msg.width, height: msg.height } });
+      }
+    });
+
+    worker.postMessage({ type: 'init', model: MODEL });
+
+    function estimatorFn(canvas) {
+      if (dead) return Promise.reject(dead);
+      const ctx = canvas.getContext('2d', { willReadFrequently: true });
+      const img = ctx.getImageData(0, 0, canvas.width, canvas.height);
+      const id = nextId++;
+      return new Promise((res, rej) => {
+        pending.set(id, { resolve: res, reject: rej });
+        worker.postMessage(
+          { type: 'infer', id, data: img.data, width: img.width, height: img.height },
+          [img.data.buffer],
+        );
+      });
+    }
+  });
 }
 
 // Test hook (M4): swap in a fake estimator so the smoke tests can drive the

src/main.js

@@ -261,12 +261,20 @@ photoInput.addEventListener('change', async () => {
     const estimator = await getDepthEstimator();
     hud.noteDevice(getSelectedDevice());
     statusEl.textContent = 'Estimating depth…';
-    const url = URL.createObjectURL(file);
+    // M6: hand the estimator a canvas, not a blob URL — the one input type
+    // valid on BOTH estimator paths (the direct pipeline reads canvases, and
+    // the WASM worker bridge needs pixels it can post across the boundary; a
+    // blob URL's reachability inside a worker is exactly the kind of
+    // unverified assumption this repo forbids). Drawn at the photo's native
+    // size, so depth resolution matches the pre-M6 URL path.
+    const source = document.createElement('canvas');
+    source.width = image.width;
+    source.height = image.height;
+    source.getContext('2d', { willReadFrequently: true }).drawImage(image, 0, 0);
     try {
-      const { depth } = await hud.timeEstimator(estimator)(url);
+      const { depth } = await hud.timeEstimator(estimator)(source);
       applyDepthToCloud(depth, image);
     } finally {
-      URL.revokeObjectURL(url);
       image.close();
     }
     statusEl.textContent = `Done — ${GRID}×${GRID} points from ${file.name}`;

tests/smoke.spec.js

@@ -643,3 +643,68 @@ test('milestone 4: camera-permission failure shows an error state and recovers',
   await expect(page.locator('#photo-input')).toBeEnabled();
   expect(errors, `unexpected page errors:\n${errors.join('\n')}`).toEqual([]);
 });
+
+// Milestone 6: WASM inference moves into a Web Worker so the main thread stays
+// responsive during a pass. The worker itself can't run in CI (it loads the
+// real ~100MB model — the worker-path proof is a local real-model probe, like
+// M3's Z-sign), so the testable main-thread seam is the input contract that
+// feeds the worker bridge: BOTH call sites must hand the estimator a CANVAS.
+// A canvas converts to ImageData for postMessage; the blob-URL string the
+// photo path passed before M6 does not cross the worker boundary usefully.
+// The webcam path already passes a canvas (M4-A pins it); this pins the photo
+// path. Fails on the pre-M6 code, which hands the estimator a string URL.
+test('milestone 6: photo path feeds the estimator a canvas (worker-bridgeable input)', async ({
+  page,
+}) => {
+  const errors = [];
+  page.on('pageerror', (e) => errors.push(`pageerror: ${e}`));
+  page.on('console', (m) => {
+    if (m.type() === 'error') errors.push(`console.error: ${m.text()}`);
+  });
+  await page.goto('/');
+  await page.waitForFunction(() => window.__app?.getPointCount() === 128 * 128);
+
+  // Fake estimator that records what input type it was handed.
+  await page.evaluate(() => {
+    const s = (window.__t6 = { input: null });
+    window.__app.__setEstimator((input) => {
+      s.input = {
+        isCanvas: input instanceof HTMLCanvasElement,
+        type: typeof input,
+        width: input && input.width,
+        height: input && input.height,
+      };
+      return Promise.resolve({
+        depth: { data: new Uint8Array(16), width: 4, height: 4 },
+      });
+    });
+  });
+
+  // A real 3×2 PNG, generated in the page so no hand-rolled magic bytes.
+  const pngB64 = await page.evaluate(() => {
+    const c = document.createElement('canvas');
+    c.width = 3;
+    c.height = 2;
+    const ctx = c.getContext('2d');
+    ctx.fillStyle = '#f00';
+    ctx.fillRect(0, 0, 3, 2);
+    return c.toDataURL('image/png').split(',')[1];
+  });
+  await page.setInputFiles('#photo-input', {
+    name: 'tiny.png',
+    mimeType: 'image/png',
+    buffer: Buffer.from(pngB64, 'base64'),
+  });
+
+  await expect(page.locator('#status')).toContainText(/done/i);
+  const rec = await page.evaluate(() => window.__t6.input);
+  expect(rec, 'estimator was never called').not.toBeNull();
+  expect(
+    rec.isCanvas,
+    `photo path must hand the estimator a canvas, got ${rec.type} (${rec.width}×${rec.height})`,
+  ).toBe(true);
+  // Drawn from the decoded photo bitmap at its native size.
+  expect(rec.width).toBe(3);
+  expect(rec.height).toBe(2);
+  expect(errors, `unexpected page errors:\n${errors.join('\n')}`).toEqual([]);
+});