Point-source defaults campaign: solved centres, tensor, pairing

Overview

End goal: a full workspace docs update — the point_source guides and modeling scripts present ALL likelihood options with performance/robustness evidence — with the complete evidence base built in autolens_profiling (truth-anchored result JSONs + a results/notes synthesis). Adopts solved-centre demonstrated defaults everywhere, tensor source-plane weighting, and (pending the phase-B discriminator) all-to-all image-plane pairing as the AnalysisPoint default. Follow-on to the #657 series; SUPERSEDES the same-day galaxy/cluster split decision; COORDINATES with autolens_workspace#436 (cluster swap, in flight — phase D reconciles, never duplicates).

Plan

Phase A — library prerequisites (PyAutoLens, small PR, FIRST): log-space rework of the FitPositionsImagePairAll mixture chi-squared (values identical where currently finite, finite where currently -inf at ≳38σ) + a weighting option on free-centre FitPositionsSource ("magnification" default, back-compat; "jacobian" opts into the tensor). wst jax_likelihood literals re-run as the invariance gate; merge + HPCPullPyAuto before phase B.
Phase B — evidence campaign (autolens_profiling, RAL A100s): truth-anchored matrix on the galaxy-scale simple quad AND a ported cluster tier; five experiments — source-plane tensor flavours, image-plane solved-vs-free re-run post-fix, pairing discriminator (missing/extra image), posterior-width honesty, near-caustic stress. JSONs committed to results/searches/** + results/notes/point_source_defaults_campaign.md synthesis.
Phase C — defaults change (PyAutoLens): AnalysisPoint.fit_positions_cls → FitPositionsImagePairAllSolved (expected, pending B); FitPointDataset default reconciled; release notes with ## API Changes heading.
Phase D — workspace docs (autolens_workspace, THE END GOAL): solved-centre demonstrated defaults in point_source modeling scripts; free-centre clearly documented with its use-cases (informative centre priors, linked centres across bands/epochs, centre-as-science, glSNe standard-candle flux caveat); guides pairing doc becomes the full option matrix with phase-B numbers; cluster scripts reconciled with test: mask padding likelihood sanity check #436; lenstool mirror stays free-centre scalar; notebooks regenerated; ship behind the library-first gate.
Explicitly deferred: the time-delay + solved-fluxes free-H0 search arm.

Detailed implementation plan

Affected Repositories

PyAutoLens (primary; phases A + C)
autolens_profiling (phase B)
autolens_workspace (phase D)
autolens_workspace_test (read-only — invariance-gate runs only)

Branch Survey (2026-07-31)

Repository	Current Branch	Dirty?
./PyAutoLens	main	clean — claimed concurrently by potential-correction-validation (#672); scopes disjoint (`autolens/point/fit/` vs pixelization); pre-merge origin/main before PR
./autolens_profiling	main	untracked `dataset/imaging/jwst_lw/` (unrelated)
./autolens_workspace	main	clean — claimed by 3 in-flight tasks incl. #436; phase-D claim deferred until D starts
./autolens_workspace_test	main	clean

Suggested branch: feature/point-source-defaults-campaign
Worktree root: ~/Code/PyAutoLabs-wt/point-source-defaults-campaign/ (PyAutoLens at phase A; worktree_add_repo autolens_profiling at phase B)
Work Classification: Both (phased: library → profiling evidence → library → workspace)

Implementation Steps

Phase A (PyAutoLens):

autolens/point/fit/positions/image/pair_all.py:96-133 — all_permutations_log_likelihoods computes log(sum(exp(log_p))) literally; rework as a shifted logsumexp (max + log(sum(exp(x − max))), xp-threaded so numpy and JAX paths agree). Values identical wherever currently finite; finite where currently -inf; NaN-padded solver rows must drop out exactly as before; the no_image_residual / has_image fallback in chi_squared (:136-188) unchanged.
autolens/point/fit/positions/source/separations.py — weighting class attribute on FitPositionsSource (default "magnification", byte-identical existing behaviour — Lenstool comparisons and literals untouched; "jacobian" computes the quadratic-form chi-squared + matching normalization around the FREE centre, reusing precision_tensor_components_from from fit/solved.py).
Numpy unit tests in test_autolens/point/ (house rule: no JAX in unit tests): old-vs-new equality at moderate σ; finiteness at extreme (≳38σ) mismatch; free-tensor vs solved-tensor consistency when evaluated at the solved centre.
Invariance gate: re-run the autolens_workspace_test jax_likelihood point_source literal scripts — unchanged at truth.
Small PR; after merge, HPCPullPyAuto syncs the RAL mirrors before any phase-B submission.

Phase B (autolens_profiling):
6. Galaxy tier: existing simple quad anchors (results/searches/**, PR#99 methodology — evaluate each flavour AT SIMULATOR TRUTH; compare Nautilus + Prodigy best-logL / recovery / wall against it).
7. Cluster tier: port the autolens_workspace/cluster test case (family-CSV multi-plane system — 2 sources at different redshifts, dPIE members + host halo, point_datasets.csv conventions). Pin cosmology in every solver-chained gradient cell (free cosmology cannot cross the solver custom_jvp boundary — Tracer aux). Absorbs draft/research/autolens_profiling/cluster_gradient_search_benchmark.md (banner in file).
8. Experiments (both tiers): (i) source-plane tensor-solved vs free-centre-tensor (new, A.2) vs scalar flavours; (ii) image-plane solved vs free re-run AFTER A.1 (tests the -inf-plateau hypothesis for the 256-start failure); (iii) pairing discriminator — quad with one image REMOVED and one spurious EXTRA position, PairAllSolved vs PairRepeatSolved posterior bias (Nautilus) — this decides the phase-C default; (iv) posterior WIDTH free vs solved image-plane (profile-vs-marginal honesty — if solved posteriors are materially overconfident, the docs' free-centre section must say so); (v) near-caustic stress test (where tensor source-plane breaks and image-plane must take over → domain-of-validity prose).
9. Execution: hpc/sync push-submit gpu <script> (SLURM gpu-partition array); JAX_ENABLE_X64 set INSIDE scripts/sbatch (sbatch does NOT inherit it — silent float32 poisons the anchors); submit detached (/mnt/ral is NFS-slow); hpc/sync pull JSONs back; device blocks recorded so CPU-laptop vs A100 rows are distinguishable (CPU spot-checks are smoke only, never committed evidence).
10. Synthesis: results/notes/point_source_defaults_campaign.md.

Phase C (PyAutoLens):
11. autolens/point/model/analysis.py:44 fit_positions_cls=FitPositionsImagePairRepeat → FitPositionsImagePairAllSolved (expected, pending step 8.iii); autolens/point/fit/dataset.py default (FitPositionsImagePair, the superseded Hungarian) decided consistently. Release notes with ## API Changes heading (classify_pr matches headings).

Phase D (autolens_workspace):
12. point_source modeling.py scripts → solved-centre demonstrated defaults; free-centre composition as the clearly-documented alternative with use-cases; cluster scripts touched only to reconcile with what #436 lands.
13. The point-source pairing guide → full option matrix with phase-B numbers (per option: params, wall/eval cost, gradient behaviour, robustness to missing/extra images, when to prefer).
14. point_source/fit.py, cluster guides, lenstool mapping updated consistently (lenstool stays free-centre scalar deliberately — its table already notes the solved/tensor siblings); notebooks regenerated; ship_workspace behind the library-first merge gate.

Key Files

autolens/point/fit/positions/image/pair_all.py — A.1 logsumexp rework
autolens/point/fit/positions/source/separations.py + autolens/point/fit/solved.py — A.2 free-centre tensor option
autolens/point/model/analysis.py, autolens/point/fit/dataset.py — phase-C defaults
autolens_profiling/results/searches/**, results/notes/point_source_defaults_campaign.md — phase-B evidence
autolens_workspace/scripts/point_source/** + guides — phase D

Context literals (2026-07-31, #657 + profiling PR#99)

Truth anchors (galaxy-scale simple quad, |mu| at obs = [8.2, 45.9, 366.6, 28.3]): image_plane free +7.20 / solved +7.74; source_plane scalar −33788.4 (the mu=367 image's radial noise mis-mapped), tensor-solved +0.6.
Tensor-vs-solved isolation: solved+scalar still prefers wrong models (truth −2338 vs −848/−610); solved+tensor ranks truth first by >1500 logL — the WEIGHTING is the fix; the solved centre is the orthogonal dimensionality/marginalization win.
Benchmark: nautilus image_plane +9.56 (740s) at truth; prodigy solved +2.37 (982s, 64×300) truth recovered THROUGH the solver; prodigy free −47.7 (3516s, 256×300) basin found but plateaued — the -inf plateau is the prime suspect (A.1).
Constraint: free cosmology cannot cross the solver custom_jvp boundary — pin cosmology in solver-chained cells.

Exit criteria

Phase-A PR merged (literals invariant); phase-B notes doc + JSONs committed; phase-C default change merged with release notes; phase-D workspace PR merged — guides/scripts present all options with evidence-backed defaults (solved centres, tensor source-plane, all-to-all image-plane if phase B confirms) and a clear, use-case-driven free-centre section.

Original Prompt

Click to expand starting prompt

Point-source defaults campaign: tensor/solved defaults, pairing decision, full docs + performance evidence

Human-scoped 2026-07-31 (#657 wrap-up chat; run in a fresh session). End goal: a full
workspace docs update — the point_source guides and modeling scripts present ALL
likelihood options with performance/robustness information — with the complete evidence
base built in autolens_profiling (truth-anchored result JSONs + a results/notes summary
doc; "full information building", not one-off runs).

Target defaults to demonstrate and then adopt:

Source-plane chi-squared: tensor weighting with SOLVED centres
(FitPositionsSourceSolved, weighting="jacobian"), while also demonstrating the
tensor works with FREE centres (requires phase A below — no free-centre tensor exists
in the API yet).
Image-plane chi-squared: SOLVED centres (free centres demonstrably need sampler
muscle: 2026-07-31 benchmark — free-centre Prodigy needed 256 starts vs solved
converging at 64; Nautilus handles both).
Workspace demonstrated defaults move to solved centres everywhere, with the free
option CLEARLY documented (modeling.py scripts + point_source guides), including WHEN
free is required: informative centre priors, linked centres across bands/epochs,
centre-as-science, and the standard-candle flux caveat (PointSolved forces
FitFluxesSolved, whose flat-prior F* discards standardizable-candle priors — glSNe).
Image-plane pairing: expected outcome is all-to-all (FitPositionsImagePairAll*)
as default — smooth in the pairings (gradient-friendly), principled Occam mixture —
pending the phase-B robustness discriminator. This is an AnalysisPoint DEFAULT
CHANGE (currently FitPositionsImagePairRepeat) → release notes ## API Changes.

SUPERSEDES the 2026-07-31 morning split decision (galaxy-scale free / cluster solved)
— the human widened it to solved demonstrated defaults everywhere. COORDINATE, don't
duplicate: the cluster swap is ALREADY IN FLIGHT as its own task
(active/cluster_default_point_solved.md, issue autolens_workspace#436, routed to
start_workspace 2026-07-31) — phase D here covers the galaxy-scale point_source scripts
and the guides; touch cluster scripts only to reconcile with whatever #436 lands (it may
merge first). This prompt also ABSORBS the ideas.md PairAll-logsumexp entry (phase A).
EXPLICITLY DEFERRED: the time-delay + solved-fluxes free-H0 search arm (revisit after
this campaign).

Phase A — library prerequisites (PyAutoLens; small PR, do FIRST)

PairAll log-sum-exp stabilization: FitPositionsImagePairAll.chi_squared
exponentiates before logging → underflows to -inf at ≳38σ worst-image mismatch
(measured 2026-07-31), strangling gradient flow (the -inf plateau is the prime
suspect for the 64-start image-plane failure). Rework in log-space: values identical
wherever currently finite, finite where currently -inf; re-run the wst
jax_likelihood literals (unchanged at truth) as the invariance gate. Prerequisite —
every phase-B result changes if run against the unfixed objective.
Free-centre tensor option: weighting class attribute on FitPositionsSource
(default "magnification" — back-compat, Lenstool comparisons and literals untouched;
"jacobian" opts into the tensor + matching normalization, reusing
precision_tensor_components_from from solved.py). Numpy unit tests per house rule.

Phase B — evidence campaign (autolens_profiling; truth-anchored; A100s on RAL)

Methodology from the 2026-07-31 runs (results/searches/**, PR#99): evaluate each
likelihood flavour AT THE SIMULATOR TRUTH as the anchor; compare Nautilus + Prodigy
best-logL/recovery/wall against it. Write every run's JSON + a
results/notes/point_source_defaults_campaign.md synthesis.

Two dataset tiers — run the full matrix on BOTH:

Galaxy-scale: the existing profiling simple quad (anchors + 2026-07-31 results
already in results/searches/).
Cluster-scale: port the autolens_workspace/cluster test case (the family-CSV
multi-plane system — 2 sources at different redshifts, dPIE members + host halo,
point_datasets.csv conventions; the good test case per the human). Expect MUCH
slower (image-plane forward solve ~0.3 s/call at cluster scale; multi-plane tracer).
This is where the solved-centre dimensionality win compounds (−2 params/source) and
where the defaults matter most. Constraint: pin cosmology in every solver-chained
gradient cell (free cosmology cannot cross the custom_jvp boundary — Tracer aux).
This tier ABSORBS draft/research/autolens_profiling/cluster_gradient_search_benchmark.md.

Execution environment — A100s on RAL (the human's call: the profiling runs that guide
all of this run on GPU):

Drive with the project's hpc/sync CLI: hpc/sync push-submit gpu <script> (SLURM
gpu-partition array, JAX auto-uses the A100), then hpc/sync jobs / tail gpu /
pull. Venv: /mnt/ral/jnightin/PyAuto via its activate.sh (PYAUTO_HPC_BASE).
Phase A must be MERGED and synced to RAL first — refresh the mirrored library
mains with HPCPullPyAuto before submitting anything.
GOTCHA (recorded): sbatch does NOT inherit JAX_ENABLE_X64 — set x64 explicitly
inside the scripts/sbatch or everything runs float32 silently, poisoning the truth
anchors and FD checks. /mnt/ral is NFS-slow — submit detached, don't babysit.
hpc/sync pull the result JSONs back into results/searches/** so the
information-building convention (committed JSONs + notes synthesis) holds; record
device blocks (the JSONs carry device.backend) so CPU-laptop vs A100 rows are
distinguishable. CPU spot-checks on the laptop are fine for smoke, not for the
committed evidence.

Source-plane tensor-solved cells (Nautilus + Prodigy source_plane_solved — fit
dispatch already exists) vs the free-centre TENSOR variant (new, phase A.2) vs the
scalar flavours (already run: scalar truth −33788 vs wrong models −110/−313 — the
bias showcase to reproduce in the notes).
Image-plane solved vs free (extend the existing 2026-07-31 results: solved 64×300
converges +2.37/truth+7.74; free needs 256 starts, plateaus at −47.7/truth+7.20).
Re-run the free-centre arm AFTER phase A.1 to test the plateau hypothesis.
Pairing discriminator (the open decision): simulate a quad with (a) one image
REMOVED from the dataset and (b) one spurious EXTRA position; compare posterior bias
of PairAllSolved vs PairRepeatSolved (Nautilus). This decides all-to-all vs
repeat as default; document whichever caveats fall out.
Posterior WIDTH comparison (profile-vs-marginal honesty): same dataset, Nautilus
free-centre vs solved image-plane — compare mass-parameter error bars. The image-plane
solved variants are plug-in profiles (no marginalization term); if solved posteriors
are materially overconfident, the docs' free-centre section must say so.
Near-caustic stress test (tensor domain of validity): source hugging the caustic
so the linearization A at observed positions degrades; establish where tensor
source-plane itself breaks and image-plane must take over → domain-of-validity prose
for the pairing guide.

Phase C — defaults decision + library change (PyAutoLens)

Informed by phase B: AnalysisPoint fit_positions_cls default →
FitPositionsImagePairAllSolved (expected); decide FitPointDataset default
consistently (currently FitPositionsImagePair — the Hungarian, already superseded).
Release notes with ## API Changes heading (classify_pr matches headings).

Phase D — workspace docs update (autolens_workspace; the END GOAL)

All point_source + cluster modeling.py scripts demonstrate solved-centre defaults;
free-centre composition shown as the clearly-documented alternative WITH its
use-cases (see 3 above).
guides/point_source_pairing.py becomes the full option matrix WITH performance and
robustness numbers from phase B (per-option: params, wall/eval cost, gradient
behaviour, robustness to missing/extra images, when to prefer).
point_source/fit.py, cluster guides, lenstool mapping updated consistently
(lenstool script stays free-centre scalar deliberately — its table already notes the
solved/tensor siblings).
Notebooks regenerated; ship_workspace behind the library-first gate.

Context literals (2026-07-31, all in autolens_profiling results/searches + #657)

Truth anchors (galaxy-scale simple quad, |mu| at obs = [8.2, 45.9, 366.6, 28.3]):
image_plane free +7.20 / solved +7.74; source_plane scalar −33788.4 (the mu=367 image's
radial noise mis-mapped), tensor-solved +0.6.
Tensor-vs-solved isolation: solved+scalar still prefers wrong models (truth −2338 vs
−848/−610); solved+tensor ranks truth first by >1500 logL → the weighting is the
fix; the solved centre is the orthogonal dimensionality/marginalization win.
Benchmark: nautilus image_plane +9.56 (740s) at truth; prodigy solved +2.37 (982s,
64×300) truth recovered THROUGH the solver; prodigy free −47.7 (3516s, 256×300)
basin found but plateaued.
Constraint: free cosmology cannot cross the solver custom_jvp boundary (Tracer aux) —
pin cosmology in solver-chained cells (follow-up filed separately).

Exit criteria

Phase-A PR merged (literals invariant); phase-B notes doc + JSONs committed; phase-C
default change merged with release notes; phase-D workspace PR merged: guides/scripts
present all options with evidence-backed defaults (solved centres, tensor source-plane,
all-to-all image-plane if phase B confirms) and a clear, use-case-driven free-centre
section.

Phase A: Library PR Created

PR: #679 (pending-release, awaiting human merge)

Gate evidence:

Full test_autolens/ suite: 498 passed (worktree, post-merge of origin/main — already up to date with the feat: PointSolver implicit-diff gradients (phase 5 of #657) #677 merge base).
Existing pair_all literal tests pass with their exact == asserts unchanged — value identity in the finite regime, no re-pinning needed.
wst invariance gate: all four scripts/point_source/jax_likelihood/ scripts pass with pinned literals matched exactly (point.py −82.33883111107943 NumPy=JIT; image_plane.py all-solved −82.33883111107943 / repeat-solved −89.71129441664627 NumPy=JIT; source_plane.py eager −94.70750992850252; fluxes_time_delays.py −133.0115150576916 / H0-matched −137.67455435685469 eager=vmap). The source_plane.py/fluxes_time_delays.py jit-path BLOCKER prints are the pre-existing tracked PointSolver-pytree limitation, not a regression.

Heart note: shipped under explicit human authorization with Heart RED from the unrelated overnight release-validation failure (release validation FAILED (stage integrate), nightly v2026.7.31.1.dev69201) — orthogonal to this branch; feature-scoped evidence above is fully green.

Workspace impact: additive + value-preserving — no immediate migration. All workspace consumers of the touched symbols are the point_source/cluster scripts already scheduled as phase D (post-evidence), plus the wst parity scripts validated by the gate. No no_run.yaml exposure.

Next: after this PR merges — HPCPullPyAuto to sync the RAL mirrors, then phase B (evidence campaign on the A100s).

Phase B underway (autolens_profiling)

Phase A merged (#679) and RAL mirrors synced (HPCPullPyAuto — diff confirms #677+#679 on the HPC checkouts). autolens_profiling claimed in the task worktree on feature/point-source-defaults-campaign.

Survey findings that adjust the plan:

The cluster tier is half-ported already: scripts/misc/simulators/cluster.py + dataset/cluster/simple/ (2 dPIE mains + 10 scaling members + NFW host, sources z=1.0/2.0, point_datasets.csv) exist with likelihood_breakdown cells. Phase B adds the missing searches support (dataset_class="cluster" dispatch, factor-graph analysis per the workspace cluster/modeling.py pattern, leaf cells, sweep rows) — cosmology pinned (solver custom_jvp constraint).
Deviation from the prompt: hpc/sync does not exist in autolens_profiling (it lives in the assistant repos). The repo's documented RAL flow is git-based: commit cells + hpc/batch_gpu/submit_* scripts → branch checkout on the HPC clone → sbatch → commit result JSONs back. Following the in-repo convention instead of porting sync. Submit headers already set JAX_ENABLE_X64=True unconditionally (the sbatch-inheritance gotcha is covered).

Build in progress (delegated, spec-reviewed):

Framework: truth.json + truth-anchored truth_log_likelihood/delta_max_ll_vs_truth in every point-source result JSON (the campaign's anchor methodology, new for search cells), posterior_stats (per-param mean/std) for the posterior-width experiment, source_plane_tensor (free-centre tensor, Point-source phase A: PairAll log-sum-exp stabilization + free-centre tensor option #679's weighting="jacobian") and image_plane_repeat_solved model types.
Instrument variants for the discriminator + stress tests: simple_missing (one image dropped), simple_extra (one spurious position), near_caustic (source at 0.95× the tangential caustic).
25 A100 submit scripts + a one-shot submit_all_678_phase_b.sh driver covering: exp 1 source-plane flavours (both tiers), exp 2 image-plane solved-vs-free re-run post-logsumexp, exp 3 pairing discriminator (PairAllSolved vs PairRepeatSolved on missing/extra), exp 4 posterior width (from nautilus posteriors), exp 5 near-caustic domain-of-validity.

Phase B discovery: cluster simulators solve every source to the FINAL plane

The new truth-anchor step caught a real defect on its first cluster run. PointSolver.solve(plane_redshift=None) defaults to the tracer's final plane, and both cluster simulators never pass plane_redshift:

autolens_profiling/scripts/misc/simulators/cluster.py (fixed on this branch)
autolens_workspace/scripts/cluster/simulator.py:482 (NOT yet fixed — see heads-up on autolens_workspace#436)

Consequence: the z=1.0 source's simulated positions are images of a hypothetical z=2.0 source — unphysical for the source they're attributed to. The z=2.0 source is coincidentally correct (final plane).

Evidence chain:

Truth anchor at cluster truth: solved logL = −4,230,422.58 — byte-identical to the committed results/breakdown/cluster/source_plane_breakdown_v2026.7.23.1.json, i.e. this is the pre-existing committed state (feat: point-source likelihood variants design (arXiv:2406.15280) #657 phase 3 compared relative timings, so it went unnoticed).
Per-position: point_0 (z=1.0) back-traces to (−2.79,−5.95), (0.21,0.31), (0.79,7.95) on its own plane — but to (0.3, 0.5) exactly (spread 0.0004″) on the z=2.0 plane. point_1 (z=2.0): perfect.
After passing plane_redshift per source and regenerating: truth anchor = +26.06 (χ²≈0.017/0.016 per system, solved centres exact). The fit-side plane handling was always correct — only simulated datasets are affected, not fits of real data (a2744/smacs0723 unaffected).

Note: the profiling sim's 0.005″ position noise is a deliberate documented choice (5 mas HST centroiding), not part of this bug.

Implications: the committed cluster breakdown/summary artifacts are tainted as accuracy references (timing conclusions stand); phase D must fold the workspace simulator fix + dataset regeneration into the cluster reconcile (or #436 takes it — human's call).

Phase B: 25 A100 jobs queued on RAL

Branch feature/point-source-defaults-campaign (autolens_profiling ef7da54) checked out on the RAL clone (rebuilt as a real https clone — the old checkout was a dead rsync'd worktree pointer). Datasets pre-generated on the login node with the plane_redshift-fixed simulator (stale tainted cluster dataset wiped first); all 25 campaign cells verified PENDING on the gpu partition (search_*, fp64, x64 forced in every submit).

Cells in flight: galaxy tier ×11 (nautilus + prodigy across image/source × free/solved/tensor/repeat-solved), discriminator ×4 (PairAllSolved vs PairRepeatSolved on simple_missing/simple_extra), near-caustic ×3, cluster tier ×7 (factor-graph, cosmology pinned; image-plane cells on 12 h walls). Jobs mail on END/FAIL; next step when they land: pull result JSONs into results/searches/**, write results/notes/point_source_defaults_campaign.md, and take the exp-3 discriminator verdict into phase C.

Phase B checkpoint (end of day)

22/25 result JSONs written on the RAL clone (feature/point-source-defaults-campaign); remaining cluster long-wall cells run overnight. Zero job failures after the shadow-package fix. Early validation: the first four galaxy cells reproduce the truth-anchor literals exactly (image_plane free 7.20 / solved 7.74, source_plane scalar −33788.4, tensor-solved +0.60) with backend=gpu and posterior_stats populated.

First scientific signal (exp 3): both simple_extra arms — image_plane_solved (PairAllSolved) and image_plane_repeat_solved — timed out at their 2 h walls, versus 147 s for the same solved fit on the clean quad. One spurious extra position makes both pairing likelihoods dramatically harder; the missing-image arms completed normally. The timeout is itself robustness evidence, but both cells will be resubmitted with 8 h walls to obtain posteriors before the discriminator verdict is called.

Known caveat to verify at results-pull: the two free-centre galaxy cells recorded total_wall_s=1 — persistent JAX compile cache or a silent PyAutoFit resume of pre-logsumexp search state (identifiers ignore data). If resumed, their search output gets wiped and re-run — exp 2's post-fix comparison depends on it.

Full resume runbook is in PyAutoMind/active.md (this task's entry). Next session: finish/resubmit cells → pull JSONs (laptop-side pull; RAL cannot push) → results/notes synthesis → phase C keyed on the exp-3 verdict.

Phase B checkpoint (2026-08-01): A100 sweep complete at 22/25 cells; the failing MSP cluster cell is root-caused with two fixes in PR.

Bug — multi_start_prodigy/cluster/image_plane_solved crash (job 331843, masked as sacct COMPLETED): two independent defects, both reproduced and fixed locally on CPU:

PyAutoFit#1441 — derived instance attributes (NFWMCRLudlowSph.scale_radius from free mass_at_200) landed in pytree aux data as traced values; the PointSolver custom_jvp rule's inner jax.jvp then received them as stale tracers → UnexpectedTracerError. Fix: flatten promotes any JAX-valued attribute to a dynamic child. Regression script: autofit_workspace_test#81.
fix: sanitize padded rows in the PointSolver implicit-diff rule for reverse mode #685 — with (1) fixed, all gradients were NaN: padded solver rows were zeroed to (0,0) = the cluster profile centres, where the NFW deflection Jacobian is NaN; reverse mode row-sums cotangents, so one NaN padded row poisoned every parameter. Fix: anchor padded rows at a real solved image + sanitize the padded solve inputs. After both fixes: 8/8 random cluster draws give finite value and gradient, FOMs byte-identical to before (forward path untouched).

Once merged + HPCPullPyAuto, the cell resubmits on the A100s.

Timeout cells: the two simple_extra exp-3 arms (nautilus image_plane_solved / image_plane_repeat_solved) resubmitted with 8h walls (jobs 331885/331886, in flight).

Early phase-B evidence (details land in results/notes/): exp-3 missing-image arm is decisive for all-to-all — PairAllSolved recovers truth on simple_missing (Δ=+1.3) while PairRepeatSolved is catastrophically biased (truth logL −183389, Δ=+183402). Tensor source-plane ranks truth highest at both tiers and holds at 0.95× the tangential caustic; image-plane free-centre Prodigy still plateaus below truth post-logsumexp (solved converges), and solved posteriors are not overconfident vs free on einstein_radius (σ 0.038 vs 0.027).

feat!: point-source position fit defaults to FitPositionsImagePairAll…

fix: register Galaxy redshift as no_flatten aux in register_tracer_cl…

Phases C and D are up as PRs:

feat!: point-source position fit defaults to FitPositionsImagePairAllSolved (#678 phase C) #686 (phase C, library): AnalysisPoint + FitPointDataset default → FitPositionsImagePairAllSolved, with ## API Changes migration notes. Also carries the register_tracer_classes Galaxy-redshift aux fix needed by multi-plane jitted solves. Merge gate: the spurious-extra-position exp-3 arms are rerunning at 8h walls (331885/331886) — hold merge until they land; the missing-image arm already rules out repeat pairing (Δ=+1.8×10⁵ mis-ranking).
autolens_workspace#453 (phase D, PENDING RELEASE): all point_source + cluster examples demonstrate ps.PointSolved + the solved all-to-all default; free-centre kept as the documented alternative with use-cases (incl. the glSNe standard-candle flux caveat); guides/point_source_pairing.py now carries the full truth-anchored benchmark evidence; cluster/simulator.py per-source plane_redshift bug fixed (the test: mask padding likelihood sanity check #436 taint) and the simulator validated end-to-end. Merge after feat!: point-source position fit defaults to FitPositionsImagePairAllSolved (#678 phase C) #686 ships in a release.

Remaining before closing the campaign: extra-arm JSONs (in flight), the multi_start_prodigy/cluster/image_plane_solved rerun on the merged gradient fixes (331887), and the notes-doc refresh with those final numbers.

Merge pull request #686 from PyAutoLabs/feature/point-source-defaults…

exp-3 extra-arm verdict — symmetric DNF, gate resolved

The two spurious-extra-position arms both hit the 8h wall and wrote no results:

Job	Arm	Outcome	Ended
331885	extra position, `PairAllSolved`	TIMEOUT 08:00:25	2026-08-01T20:49:46Z
331886	extra position, `PairRepeatSolved` (sibling)	TIMEOUT 08:00:01	2026-08-02T04:49:48Z

No *extra* JSON exists anywhere under results/ — confirmed on RAL.

Verdict

The DNF is the finding, and the important part is that it is symmetric. The old
repeat-pairing convention failed within 24 seconds of the same wall as the new solved
convention. A spurious position imposes an honest ~-1.4e4 likelihood floor on every
model (331885: f_live=0.92, N_eff=12, logZ=-14238), so Nautilus cannot converge
under either pairing.

This experiment therefore characterises contamination, not the default: it cannot
discriminate between PairAllSolved and PairRepeatSolved, and so cannot rule against
the change in #686. The decisive evidence remains the missing-image discriminator
already recorded on that PR — PairAllSolved recovers truth (Δ=+1.3) where
PairRepeatSolved mis-ranks by +1.8e5 logL.

Note on sequencing: both jobs completed 2026-08-02, i.e. before #686 merged
2026-08-03T18:31:35Z. The gate's runs had landed; only this verdict post was outstanding.

Release decision (2026-08-04)

Human-authorized to ship #686 in the release now under validation. Its release notes
carry the required ## API Changes heading for a breaking change.

Still outstanding (campaign bookkeeping, not a release gate)

331887 (msp cluster, COMPLETED 14m16s) and 331888/331889 (clean free-cell, ~3m45s each)
finished on 08-02 and their JSONs have not been pulled or analysed — the msp cluster
end-to-end validation of the merged gradient fixes and the exp-2 refresh both remain to
be read.

Campaign complete — closing

All four phases merged, and the evidence tail is now in:

Phase	PR	Status
A — library prerequisites (logsumexp, free-centre tensor)	PyAutoLens#679	merged 2026-07-31
B — evidence campaign (harness, 23 A100 cells, notes)	autolens_profiling#100	merged 2026-08-04
C — default swap to `FitPositionsImagePairAllSolved`	PyAutoLens#686	merged 2026-08-03
D — workspace docs	autolens_workspace#453 + #468	merged 2026-08-03 / 2026-08-04

Bug fixes the campaign flushed out: PyAutoFit#1441, PyAutoLens#685, autofit_workspace_test#81.

What the evidence settled

The defaults rest on a truth-anchored A100 campaign: every cell evaluates its likelihood flavour at the simulator-truth model and scores the search by delta = max_log_likelihood − truth_log_likelihood, which separates a likelihood defect (a wrong model out-ranking truth) from a search defect (failure to reach the basin).

The all-to-all default rests on the missing-image arm: PairRepeatSolved mis-ranks truth by +1.8×10⁵ (truth logL −183389) while PairAllSolved recovers it at +1.3. On clean data the two are statistically equivalent (+2.85 vs +2.88, 147 s vs 163 s), so robustness rather than performance decides it.

The evidence tail (this last pass)

331887 re-ran the cluster gradient cell that had exposed the two library defects. It now completes end-to-end — 832 s, finite, no NaN — validating both merged fixes through a full search rather than only the 8-draw gradient probe, with a truth anchor (+14.658) identical to its Nautilus twin's.

But it still misses the basin: delta −1723.6, against Nautilus's +16.9 on the same objective. That closes the one question the docs had left open — the solved image-plane likelihood was the last candidate for a cluster-scale gradient-search recommendation, and it fails alongside the source-plane cells (−14.8, −11062). Cluster scale is Nautilus's across every objective tested. Making the gradients finite was a correctness fix; it did not buy convergence. Since a negative delta is an optimizer defect and not a likelihood one, no shipped default is affected — the guides were corrected, nothing was reopened.

331888/331889 replaced two cells that had been silent PyAutoFit resumes (sub-second sampler wall — identifiers ignore the dataset). Both reproduced their prior conclusions within scatter, so the resumes had not misled, but the shipped numbers now rest on searches that actually ran.

Two things recorded rather than glossed

A pre-merge cross-review returned 7 findings, all verified real. The substantive one: three files explained the missing-image discriminator with a wrong mechanism — "nearest-image pairing has no way to leave an unobserved model image unmatched". It can; pair_repeat.py builds an unmatched_model_mask and penalizes through unmatched_model_policy. What actually happens is that the model image which would have paired to the missing observation becomes the nearest neighbour of no observed position, so the default magnification_filter policy classes it a bright extra image and charges its full distance to the nearest other observed position — truth punished for correctly predicting an image the data lacks. The numbers and the default are unchanged; the explanation readers would have learned was wrong, and is now fixed in all three files.

Known evidence gap. The simple_extra (spurious-position) arms are a deliberate non-result: both timed out at 8h walls, symmetrically, so that arm does not discriminate between the pairings and the default never rested on it. The convergence diagnostics recorded for those runs (f_live=0.92, N_eff=12, logZ=−14238) are not reproducible — the surviving output holds only metadata, model.info, .identifier and a 12-line search.log, with no sampler statistics, no checkpoint and no retained stdout. The docs now separate the reproducible part (the TIMEOUTs, the 147/163 s comparison walls) from that recorded interpretation. Re-running one arm with stdout captured is the only way to close it; not planned.

Deferred at scoping and still un-run: the time-delay + solved-fluxes free-H0 search arm.

Post-close hygiene note (2026-08-06): the campaign's evidence-tail work was found unpushed on local worktrees and has now been preserved on origin — autolens_profiling feature/point-source-defaults-campaign (8 commits: phase-B harness, 22/25 A100 evidence cells, 331887 cluster gradient validation, Codex cross-review corrections) and autolens_workspace feature/point-source-campaign-evidence-tail (3 commits: pairing-guide evidence numbers + cluster-example corrections). The merged PyAutoLens feature/point-source-defaults-campaign branch was deleted local+origin. The two pushed branches still need review/merge; untracked nss profiling results in the worktree were left untouched.

Navigation Menu

Repository navigation

Point-source defaults campaign: solved centres, tensor, pairing #678

Overview

Plan

Affected Repositories

Branch Survey (2026-07-31)

Implementation Steps

Key Files

Context literals (2026-07-31, #657 + profiling PR#99)

Exit criteria

Original Prompt

Point-source defaults campaign: tensor/solved defaults, pairing decision, full docs + performance evidence

Phase A — library prerequisites (PyAutoLens; small PR, do FIRST)

Phase B — evidence campaign (autolens_profiling; truth-anchored; A100s on RAL)

Phase C — defaults decision + library change (PyAutoLens)

Phase D — workspace docs update (autolens_workspace; the END GOAL)

Context literals (2026-07-31, all in autolens_profiling results/searches + #657)

Exit criteria

Phase A: Library PR Created

Phase B underway (autolens_profiling)

Phase B discovery: cluster simulators solve every source to the FINAL plane

Phase B: 25 A100 jobs queued on RAL

Phase B checkpoint (end of day)

exp-3 extra-arm verdict — symmetric DNF, gate resolved

Verdict

Release decision (2026-08-04)

Still outstanding (campaign bookkeeping, not a release gate)

Campaign complete — closing

What the evidence settled

The evidence tail (this last pass)

Two things recorded rather than glossed

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Point-source defaults campaign: solved centres, tensor, pairing #678

Description

Overview

Plan

Affected Repositories

Branch Survey (2026-07-31)

Implementation Steps

Key Files

Context literals (2026-07-31, #657 + profiling PR#99)

Exit criteria

Original Prompt

Point-source defaults campaign: tensor/solved defaults, pairing decision, full docs + performance evidence

Phase A — library prerequisites (PyAutoLens; small PR, do FIRST)

Phase B — evidence campaign (autolens_profiling; truth-anchored; A100s on RAL)

Phase C — defaults decision + library change (PyAutoLens)

Phase D — workspace docs update (autolens_workspace; the END GOAL)

Context literals (2026-07-31, all in autolens_profiling results/searches + #657)

Exit criteria

Activity

Jammy2211 commented on Jul 31, 2026

Phase A: Library PR Created

Jammy2211 commented on Jul 31, 2026

Phase B underway (autolens_profiling)

Jammy2211 commented on Jul 31, 2026

Phase B discovery: cluster simulators solve every source to the FINAL plane

Jammy2211 commented on Jul 31, 2026

Phase B: 25 A100 jobs queued on RAL

Jammy2211 commented on Aug 1, 2026

Phase B checkpoint (end of day)

Jammy2211 commented on Aug 1, 2026

Jammy2211 commented on Aug 1, 2026

Jammy2211 commented on Aug 4, 2026

exp-3 extra-arm verdict — symmetric DNF, gate resolved

Verdict

Release decision (2026-08-04)

Still outstanding (campaign bookkeeping, not a release gate)

Jammy2211 commented on Aug 4, 2026

Campaign complete — closing

What the evidence settled

The evidence tail (this last pass)

Two things recorded rather than glossed

Jammy2211 commented on Aug 6, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions