Skip to content

fix: ConstantZeroth regularization is dead code (eye(P) shape bug + missing neighbors_sizes) #448

Closed

Description

@Jammy2211

Overview

al.reg.ConstantZeroth is a public, exported regularization scheme that has never worked through its class API — two independent defects make it dead code presenting as a feature (found in the wsdev#104 reg-logdet investigation, independently confirmed by a second reviewer who ran it, and re-verified against main on 2026-08-17). Fixing it resurrects a genuinely useful well-conditioned alternative: the zeroth-order + lambda_z^2 * I term is exactly what lifts the graph-Laplacian null mode that makes Constant/Adapt fragile (the 1e-8-below-the-noise-floor conditioning collapse). This is Phase 8C of the inference-methods programme (autolens_profiling#134): ConstantZeroth is an ALTERNATIVE regularization model, never a silent AdaptSplit replacement.

Plan

  • Reproduce both failures on clean main FIRST (a two-line call through the class API, then through the util function with correct args).
  • Fix defect 1: constant_zeroth_regularization_matrix_from builds zeroth = xp.eye(P) where P = neighbors.shape[1] is the neighbour count (e.g. 4), not S = mesh pixel count — const + zeroth broadcasts/raises. The zeroth term must be the full S x S scaled identity.
  • Fix defect 2: ConstantZeroth.regularization_matrix_from (class API) omits the required neighbors_sizes argument — TypeError before the shape bug is even reached. Verify the exact call site/signature on main first.
  • Check siblings adapt_split_zeroth.py / brightness_zeroth.py for the same copy-paste pattern (eye(P) / missing arg); fix if present.
  • Unit tests (numpy-only per repo policy): matrix shape (S, S); positive-definiteness (smallest eigenvalue >= lambda_z^2, i.e. NO null mode — the whole point of the zeroth-order term); class-API path returns without raising; zeroth term reduces to Constant's matrix + lambda_z^2*I where the schemes overlap.
  • Run the full suite pytest test_autoarray/.
  • Do NOT bundle with the reg-logdet log-det change or the Adapt double-square probe — independent, self-contained defect. Do NOT change Constant/Adapt/AdaptSplit numerics.
Detailed notes

Affected Repositories

  • PyAutoArray (primary, only)

Key Files

  • autoarray/inversion/regularization/constant_zeroth.py — both defects (shape bug ~lines 68-72; missing-arg at the class API ~lines 121-129 as of the 2026-07 read; re-verify line numbers on main).
  • autoarray/inversion/regularization/abstract.py / regularization_util.py — reference for the correct call signatures.
  • Possible siblings: adapt_split_zeroth.py, brightness_zeroth.py.

Suggested branch: feature/constant-zeroth-repair

Coordination

PyAutoArray also carries the version-stamp-sync-guards branch (touches only autoarray/__init__.py + files/release.sh) and this work is file-disjoint. This issue is prepared for a REMOTE (cloud) session: clone PyAutoLabs/PyAutoArray, branch from main, PR when green. Merge is a human act.

Science classification

Target-changing relative to AdaptSplit (a different regularization model) — this fix makes the scheme WORK, it does not recommend it. Record the null-mode-lift verification (spectrum before/after) in the PR so autolens_profiling Phase 8C can cite it.

Original Prompt

Click to expand starting prompt

See PyAutoMind/active/constant_zeroth_broken_dead_code.md (filed 2026-07-17 during the reg-logdet investigation; re-verified against main 2026-08-17 — both defects still present).

Activity

  1. Jammy2211 commented on Aug 18, 2026

    @Jammy2211
    CollaboratorAuthor

    Reproduced both defects on clean main (commit 7d1906e, 2026-08-18, numpy path, Python 3.12) before touching any code.

    Defect 2 — class API omits neighbors_sizes (fails first, masking defect 1):

    reg = aa.reg.ConstantZeroth(coefficient_neighbor=1.0, coefficient_zeroth=2.0)
    reg.regularization_matrix_from(linear_obj=mapper)  # 3x3 rectangular MockMapper
    Traceback (most recent call last):
      File "repro_448.py", line 31, in <module>
        reg.regularization_matrix_from(linear_obj=mapper)
      File "autoarray/inversion/regularization/constant_zeroth.py", line 121, in regularization_matrix_from
        return constant_zeroth_regularization_matrix_from(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    TypeError: constant_zeroth_regularization_matrix_from() missing 1 required positional argument: 'neighbors_sizes'
    

    Defect 1 — eye(P) shape bug in the util function (called directly with correct args; neighbors.shape = (9, 4), so S=9, P=4):

    constant_zeroth_regularization_matrix_from(
        coefficient=1.0, coefficient_zeroth=2.0,
        neighbors=np.asarray(mapper.neighbors),
        neighbors_sizes=np.asarray(mapper.neighbors.sizes),
    )
    Traceback (most recent call last):
      File "repro_448.py", line 45, in <module>
        mat = constant_zeroth_regularization_matrix_from(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "autoarray/inversion/regularization/constant_zeroth.py", line 73, in constant_zeroth_regularization_matrix_from
        return const + zeroth
               ~~~~~~^~~~~~~~
    ValueError: operands could not be broadcast together with shapes (9,9) (4,4)
    

    Sibling check: adapt_split_zeroth.py and brightness_zeroth.py do not carry the copy-paste pattern — their zeroth terms are built via xp.diag over per-pixel weights (brightness_zeroth_regularization_matrix_from), no eye(P) and no dropped argument. zeroth.py uses xp.eye(pixels) with pixels = linear_obj.params, which is the correct semantics constant_zeroth.py should have mirrored.

    Proceeding with the fix on feature/constant-zeroth-repair.


    Generated by Claude Code

  2. Jammy2211 commented on Aug 18, 2026

    @Jammy2211
    CollaboratorAuthor

    Repair complete — PR #449 is up from feature/constant-zeroth-repair (commit 143ab43), closing this issue on merge.

    Done, in the issue's order:

    • Both failures reproduced on clean main first (tracebacks in the comment above).
    • Defect 1 fixed: zeroth term is now the full S x S scaled identity (xp.eye(S) * coefficient_zeroth**2), matching zeroth.py's semantics.
    • Defect 2 fixed: class API now threads neighbors_sizes=linear_obj.neighbors.sizes, mirroring Constant.
    • Siblings checked: adapt_split_zeroth.py / brightness_zeroth.py are clean of the pattern (diag-over-weights zeroth terms) — no changes there.
    • Tests added (numpy-only): shape (S, S) with S != P, no null mode (min eig >= lambda_z^2), reduction to Constant + lambda_z^2 * I, class-API smoke. All 4 pass.
    • Full pytest test_autoarray/: 988 passed, 59 skipped; 3 pre-existing environment failures (pynufft vs modern scipy's removed pinv2), identical on clean main, unrelated.

    Spectrum verification (9-pixel rectangular fixture, lambda_n = lambda_z = 1): Constant min eigenvalue 1e-8 (null mode on the jitter floor, condition number 6e8) → ConstantZeroth min eigenvalue 1.0 = lambda_z^2 (condition number 7). Exact element-wise + lambda_z^2 I shift relative to Constant. Full table in the PR body for Phase 8C to cite.

    Constraints honored: autoarray/__init__.py untouched, Constant/Adapt/AdaptSplit numerics untouched, nothing bundled, LF endings.

    Remaining: merge only — which stays with @Jammy2211.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions