Estimator overhaul (M-PROGRESS Phase C) + calculation checkpoints (M-CHECKPOINT); v0.7.0

[NCCU-Schultz-Lab]

Two milestones. They landed together because the first one's finding — that
QuantUI's own tests were writing into the user's real performance log — was
found while grounding the second.

M-PROGRESS Phase C — the model was never the problem

Phase C existed to fix a pre-run time estimate that M-EST's tier-4 validation
showed sitting outside its ±25% band. The assumption was that the cost model
needed work. Replaying the recorded history through the estimator said
otherwise:

calc type	n	median |err|	median signed err	within ±25%
frequency	454	52.5%	+6.6%	17%
single_point	1398	53.9%	+8.7%	40%
geometry_opt	300	31.0%	−1.4%	43%
nmr	168	9.7%	+0.7%	87%

Signed error ≈ 0 everywhere — no bias, which is what a wrong cost model looks
like. Just enormous spread. And the same predictor gets NMR to 87%, so the
spread had to be coming from the data.

It was. The history held 448 records for the identical tuple (frequency,
H2O, RHF, 6-31G) with wall times from 0.34 s to 143 s. No cost model fits
ground truth that disagrees with itself by 400×.

Root cause: the *_analysis_history suites drive _do_run end-to-end with
a mocked calculation, and conftest.py isolated QUANTUI_RESULTS_DIR and
QUANTUI_SETTINGS_PATH but never QUANTUI_LOG_DIR. Each test recorded a
fabricated "H2O frequency" whose elapsed_s was really pytest-xdist wall time
under contention. Confirmed directly — running one test file appended 8 records
claiming 56.67 s and 57.47 s. Roughly four fifths of the 2 773 records were
test artifacts.

Second contaminant of the same shape: benchmarks._calibration_worker started
its perf_counter before importing quantui/pyscf, charging every calibration
record with a fresh subprocess's import cost that an in-app run never pays.

⚠️ Worth flagging: the pollution was flattering the score. Excluding the
test-fixture tuples moves median |err| from 37% to 74.5% — 448 identical
records are trivially self-predicting, so they inflated apparent accuracy while
teaching the model nothing about chemistry.

So this half is about what gets measured and how it's labelled:

• conftest._isolate_log_dir — the fix at the source.
• log_calculation gains source/warm/import_s/stages, all additive and
  omitted when unknown, so "untagged" stays distinguishable from "known to be
  an app run".
• estimate_time partitions by source the way it already did by gpu_used,
  with composing confidence downgrades. Self-healing: two tagged records
  switch a calc type's pool over, so the polluted history is superseded rather
  than needing deletion.
• Per-stage wall times collected in _LogCapture at the existing emit_status
  boundaries — no calc module needed new plumbing.
• python -m quantui.estimator_eval — causal replay reporting accuracy next
  to coverage, so a model that stays silent can't look good by refusing to
  answer.

The stage-aware frequency model is deliberately not built: no record
carries stage data yet, and shipping an unvalidatable model is the exact
mistake this phase exists to correct. Tracked as PROG.C6.

M-CHECKPOINT — CHK.1, CHK.2, CHK.3, CHK.5

The design decision that shaped the rest: resume and warm start are
different operations and must not share a key. Resume continues this run, so
its key includes geometry — resuming into a checkpoint from a different
molecule would silently splice two calculations together. Warm start reuses a
converged density as an initial guess, so its key deliberately omits
coordinates: a density from a nearby geometry is a good guess, which is exactly
what a geometry optimization exploits internally.

Checkpoints live in ~/.quantui/checkpoints, not under the result
directory as the roadmap sketched — a result directory is created when a
calculation succeeds, and the runs that need a checkpoint never get there.

The constraint that outranks the feature: a checkpoint must never break a
calculation. Corrupt metadata, a truncated append, a foreign schema version
and an unwritable directory all behave exactly like "no checkpoint".

• CHK.1 — mf.chkfile into the checkpoint dir; compatible earlier density
  as the initial guess. Never reuses the file it's about to overwrite.
• CHK.2 — the ASE trajectory moves out of the TemporaryDirectory that was
  discarding it. Resume reloads the BFGS Hessian as well as the geometry;
  without it BFGS restarts as steepest descent and relearns curvature it had.
• CHK.3 — points banked as they finish, with a self-validating cache (each
  record stores the coordinate value it was computed at). Reusing a point also
  moves the live ASE atoms, since each point relaxes from where the last
  finished.
• CHK.5 — the offer sits on the Calculate tab, where the app can say
  "8 of 20 scan points already computed" rather than asking a bare "Resume?".

CHK.4 (frequency displacements) deferred — freq_ir_workers can run
displacements concurrently, so it needs to be safe under parallel append and
resume a set rather than a prefix. A design task, not a wiring one.

Testing

170 new tests. Both suites mutation-checked against seeded defects. The
checkpoint wiring tests are structural on purpose — the failure mode here is a
checkpoint created but never passed to the calculation, or a control built and
never added to its container, both of which have precedent in this repo.

Full suite: 2321 passed, 25 skipped.

🤖 Generated with Claude Code