diff --git a/STATUS.md b/STATUS.md index f5b150a..de629a4 100644 --- a/STATUS.md +++ b/STATUS.md @@ -4,9 +4,8 @@ Single source of truth for "what's next." One milestone per PR/run. Autopilot: pick the one task under **NEXT**, ship it, stop. Do **not** start anything under **BLOCKED**. -_Last updated: 2026-07-08 — Milestone 7 (mobile-responsive UI) done; M8 -model-download progress is NEXT, then M9 STL+PLY export and M10 expanded -depth range. (Queue rebuilt from user direction earlier the same day.)_ +_Last updated: 2026-07-08 — Milestone 8 (model-download progress) done; M9 +STL+PLY export is NEXT, then M10 expanded depth range._ --- @@ -257,6 +256,27 @@ depth range. (Queue rebuilt from user direction earlier the same day.)_ Verified visually on a real build (mobile + desktop screenshots; desktop byte-identical). Pre-change HEAD (rollback) `58d3b9a`. +- **Milestone 8 — Model-download progress.** The first photo/webcam model load + now shows a live `Downloading depth model… NN%` in `#status`. Event shape + confirmed from the installed 4.2.0 source before coding (working rule 1): + `pipeline()` wraps any `progress_callback` in `DefaultProgressCallback` + (`utils/core.js`), which emits aggregate **`progress_total`** events — + `progress` is 0–100 across ALL weight files — alongside the raw per-file + `initiate/download/progress/done` events, so no manual aggregation was + needed. `src/depth.js` filters for `progress_total` in one place + (`emitProgress`) feeding a handler `src/main.js` registers once + (`setProgressHandler`); the WebGPU (main-thread) pipeline gets the callback + directly, the WASM worker relays its raw events over the bridge as a new + `{type:'progress', event}` message (`src/depth-worker.js`). Completion needs + no special-casing: each call site's next status write ('Estimating depth…' / + 'Starting webcam…') ends the readout, and cached loads emit nothing (module + cache) or an instant flash (browser cache re-read). Render loop, decoupling + machinery, and all estimator call sites byte-unchanged. Test seam + `window.__app.__emitProgress` routes fake events through the real filter; + smoke test M8 asserts advancing rounded %, per-file events ignored, and + return to the normal `Done — …` flow. Proven non-tautological (RED with + production `src/` reverted, test kept). Pre-change HEAD (rollback) `c2f0a9f`. + **Carry-over facts from M3/M4 (do not re-derive):** - Depth output contract: `depth` RawImage, Uint8, 1 channel, input W×H, 0–255 @@ -272,24 +292,6 @@ depth range. (Queue rebuilt from user direction earlier the same day.)_ ## NEXT (the one actionable task) -- **Milestone 8 — Model-download progress.** The first photo upload / webcam - start silently downloads the Depth Anything weights (tens of MB) behind a - static status line. Show real progress — a percentage or bar in `#status`. - transformers.js takes a `progress_callback` option at pipeline construction; - **confirm the exact event shape from the installed 4.2.0 source before - coding (working rule 1) — do not assume.** Expect per-file events needing - aggregation into one number. M6 wrinkle: on the WASM path the pipeline is - built inside `src/depth-worker.js`, so progress events must be relayed over - the worker bridge via postMessage; on the WebGPU (main-thread) path they're - direct — both call sites feed the same UI. Weights are cached after first - load, so the bar should appear only while bytes actually move. Smoke: fake - progress events → visible advancing %; completion returns `#status` to the - normal flow. - ---- - -## BLOCKED — do NOT start until the prior milestone has merged - - **Milestone 9 — Export: "Save as STL" + "Save as PLY" buttons.** (Format question resolved with the user 2026-07-08: **both**.) One export module, two buttons in `#controls`, fully client-side (build an ArrayBuffer → Blob → @@ -308,6 +310,10 @@ depth range. (Queue rebuilt from user direction earlier the same day.)_ assert header/counts and a few known vertex values; PLY colors match `aColor`. +--- + +## BLOCKED — do NOT start until the prior milestone has merged + - **Milestone 10 — Expanded depth range** (user: "make the potential depth more expansive"). Displacement is hard-coded — `z = d/255 − 0.5`, a total span of 1.0 world unit — so relief always reads shallow. Add a fourth diff --git a/src/depth-worker.js b/src/depth-worker.js index 455e926..6ae81e0 100644 --- a/src/depth-worker.js +++ b/src/depth-worker.js @@ -11,6 +11,9 @@ // // Protocol (all messages are plain objects): // main → worker { type: 'init', model } +// worker → main { type: 'progress', event } (raw transformers.js +// progress events during the weight download — plain +// clonable objects; the bridge filters for progress_total) // worker → main { type: 'ready' } | { type: 'init-error', error } // main → worker { type: 'infer', id, data, width, height } // — RGBA ImageData pixels; the buffer arrives transferred. @@ -39,7 +42,10 @@ self.onmessage = async (e) => { try { const { pipeline, RawImage } = await import('@huggingface/transformers'); RawImageCtor = RawImage; - estimator = await pipeline('depth-estimation', msg.model, { device: 'wasm' }); + estimator = await pipeline('depth-estimation', msg.model, { + device: 'wasm', + progress_callback: (event) => self.postMessage({ type: 'progress', event }), + }); self.postMessage({ type: 'ready' }); } catch (err) { self.postMessage({ type: 'init-error', error: errMessage(err) }); diff --git a/src/depth.js b/src/depth.js index 3dd58cc..bdb920d 100644 --- a/src/depth.js +++ b/src/depth.js @@ -30,6 +30,31 @@ export function getSelectedDevice() { return selectedDevice; } +// Milestone 8: model-download progress. pipeline() wraps any progress_callback +// in DefaultProgressCallback (utils/core.js, confirmed from the installed +// 4.2.0 source), which emits aggregate `progress_total` events — progress is +// 0–100 across ALL weight files — alongside the raw per-file events. We +// filter for the aggregate here, in one place for both device paths: the +// WebGPU (main-thread) pipeline gets emitProgress directly; the WASM worker +// relays its events over the bridge (see createWorkerEstimator). +let onProgress = null; + +export function setProgressHandler(fn) { + onProgress = fn; +} + +function emitProgress(event) { + if (event && event.status === 'progress_total' && onProgress) { + onProgress(event.progress); + } +} + +// Test seam: CI can't download the real model, so tests feed fake events +// through the same filter the real callbacks use. +export function _emitProgressForTests(event) { + emitProgress(event); +} + // Returns (and caches) the depth-estimation pipeline. A failed load clears // the cache so the next upload can retry (e.g. after a network blip during // the weight download). @@ -77,7 +102,23 @@ async function loadEstimator(resetCache) { if (device === 'webgpu') { // WebGPU already computes off the main thread — keep the direct path. const { pipeline } = await import('@huggingface/transformers'); - return pipeline('depth-estimation', MODEL, { device }); + // Stop forwarding progress once the load settles: pipeline() loads its + // components via Promise.all, so a rejected component (→ 'Depth failed: …' + // in #status) doesn't cancel a sibling download, whose surviving progress + // events would otherwise overwrite the error forever (verifier finding). + // The worker path is immune — fail() terminates the worker. + let settled = false; + const p = pipeline('depth-estimation', MODEL, { + device, + progress_callback: (e) => { + if (!settled) emitProgress(e); + }, + }); + p.then( + () => (settled = true), + () => (settled = true), + ); + return p; } // Milestone 6: on the WASM fallback the ONNX session blocks whatever thread // runs it (~4–11s/pass, M4 finding) — run it in a dedicated module worker so @@ -126,7 +167,9 @@ function createWorkerEstimator(resetCache) { }); worker.addEventListener('message', (e) => { const msg = e.data; - if (msg.type === 'ready') { + if (msg.type === 'progress') { + emitProgress(msg.event); + } else if (msg.type === 'ready') { resolve(estimatorFn); } else if (msg.type === 'init-error') { fail(new Error(msg.error)); diff --git a/src/main.js b/src/main.js index 2de869e..6537745 100644 --- a/src/main.js +++ b/src/main.js @@ -1,6 +1,12 @@ import * as THREE from 'three'; import { OrbitControls } from 'three/addons/controls/OrbitControls.js'; -import { getDepthEstimator, getSelectedDevice, _setEstimatorForTests } from './depth.js'; +import { + getDepthEstimator, + getSelectedDevice, + setProgressHandler, + _setEstimatorForTests, + _emitProgressForTests, +} from './depth.js'; import { initHud } from './hud.js'; // --------------------------------------------------------------------------- @@ -237,6 +243,16 @@ statusEl.id = 'status'; statusEl.textContent = 'Load a photo to see its depth field.'; controlPanel.appendChild(statusEl); +// M8: live model-download progress. depth.js emits the aggregate 0–100 +// percentage (transformers.js `progress_total`) on both device paths; each +// call site's next status write ('Estimating depth…' / 'Starting webcam…') +// naturally ends the readout once the load resolves. Weights are cached +// after the first load, so later loads emit nothing (or an instant flash +// from the browser cache). +setProgressHandler((pct) => { + statusEl.textContent = `Downloading depth model… ${Math.round(pct)}%`; +}); + // =========================================================================== // MILESTONE 5: polish readouts — FPS, per-pass inference time, and a // user-visible warning when depth runs on the WASM fallback. All @@ -517,6 +533,10 @@ window.__app = { // hidden