Skip to content

feat: Convolver convolve_over_sample_size + Imaging plumbing (phase 2a) #354

New issue
New issue
Closed
Closed
feat: Convolver convolve_over_sample_size + Imaging plumbing (phase 2a)#354

Description

@Jammy2211
Jammy2211
opened on Jul 8, 2026

Overview

Phase 2a of oversampled PSF convolution: the PyAutoArray core API. Implements the design approved in #353 (PyAutoMind/feature/autoarray/oversampling_design.md) verbatim — Convolver.convolve_over_sample_size, the fine-mask machinery, Imaging plumbing and the adaptive guards, with unit tests pinned to the phase-1 numerical ground truth. Behaviour at convolve_over_sample_size=1 is byte-identical to today.

Run mode: --auto at effective level supervised (plan-to-issue, ship sign-off parks with a batched question). Phase split: 2a (this issue) → 2b inversion wiring → 2c PyAutoGalaxy consumer (queued, blocked by an unrelated PyAutoGalaxy worktree claim) → phases 3–4 (workspace, docs).

Plan

  • Add mask_2d_upscaled_from(mask, s) and the cached permutation between autoarray's per-pixel sub-block ordering and the fine mask's row-major slim ordering.
  • Extend Convolver with convolve_over_sample_size: int = 1: fine-resolution kernel semantics, fine ConvolverState from state_from, oversampled scatter → existing convolve → s×s mean bin → slim in all four convolution methods, loud shape/pixel-scale validation.
  • Extend Imaging/GridsDataset with convolve_over_sample_size_lp / convolve_over_sample_size_pixelization (int, default 1), the equality rule vs over_sample_size_*, blurring grid at over_sample_size=s, psf_setup_state fine state, ctor/from_fits/apply_* pass-through, sparse_operator + adaptive guards.
  • Add binned: bool = True pass-through to the over_sample decorator (autoarray side only).
  • Unit tests (numpy-only): s=1 strict parity, s=2 ground-truth reference values, guard raises; full suite green.
Detailed implementation plan

Affected Repositories

  • PyAutoArray (primary, only repo edited)

Branch Survey

Repository Current Branch Dirty?
./PyAutoArray main clean

Suggested branch: feature/psf-oversample-core
Worktree root: ~/Code/PyAutoLabs-wt/psf-oversample-core/

Implementation Steps

  1. Fine-mask utils — autoarray/mask/mask_2d_util.py: mask_2d_upscaled_from(mask_2d, over_sample_size) (each unmasked pixel → s×s unmasked block, pixel scale ps/s, same origin); permutation builder sub_slim_to_fine_slim_from(mask, s) mapping per-pixel sub-block slim indices to fine-mask row-major slim indices (and its inverse). Unit tests: round-trip permutation, upscaled-mask geometry, s=1 identity.
  2. ConvolverState fine geometry — autoarray/operators/convolver.py: when built for s>1, construct from (kernel_fine, mask_fine); cache sub_slim_to_fine_slim, the blurring-region permutation, and bin/reshape indices alongside the existing FFT precomputes.
  3. Convolver API — convolve_over_sample_size: int = 1 ctor param (validate plain int ≥ 1, TypeError otherwise); state_from(mask) derives the fine mask when s>1 and checks kernel.pixel_scales ≈ mask.pixel_scales / s (KernelException on mismatch); convolved_image_from, convolved_mapping_matrix_from and both _via_real_space_* variants accept over-sampled slim inputs when s>1 (length n_unmasked·s², sub-block order), raise on binned-length input, return image-resolution slim via mean bin-down. s=1 paths untouched.
  4. Imaging / GridsDataset — autoarray/dataset/imaging/dataset.py + dataset/grids.py: new ctor ints (default 1) stored + passed through from_fits / apply_mask / apply_over_sampling; equality rule (s>1 requires matching over_sample_size_* uniform int equal to it, DatasetException otherwise, checked via OverSampler.sub_is_uniform); blurring grid gets over_sample_size=s when s>1; psf_setup_state builds the fine state; raise if sparse_operator is not None and convolve_over_sample_size_pixelization > 1.
  5. Decorator — autoarray/operators/over_sampling/decorator.py: binned: bool = True kwarg; binned=False returns sub-gridded values in sub-block order (the s>1 Convolver input format). No callers change in this PR.
  6. Tests — test_autoarray/: mirror the ground-truth scene (11×11, ps=1, r=3.5 circular mask, Gaussian source σ=1.2 @ (0.3,−0.4), Gaussian PSF σ=0.8, kernel radius 2.0"): s=1 parity vs existing path; s=2 == design §7 reference values (sum 2.796562184524787, slim0 3.726289901353439e-02, slim17 2.025075336159483e-01, slim36 1.090767109119494e-02); guard raises; decorator binned=False ordering.

Key Files

  • autoarray/operators/convolver.py — ConvolverState + Convolver (design §§1–2)
  • autoarray/mask/mask_2d_util.py — upscale + permutation utils
  • autoarray/dataset/imaging/dataset.py, autoarray/dataset/grids.py — plumbing + guards (design §3, §6)
  • autoarray/operators/over_sampling/decorator.py — binned kwarg (design §5)
  • PyAutoMind/feature/autoarray/oversampling_design.md — the approved design (do not re-design)
  • PyAutoMind/feature/autoarray/oversampling_ground_truth.py — reference numbers

Acceptance

  • Full PyAutoArray pytest suite green; every pre-existing test unmodified.
  • s=2 unit tests reproduce the ground-truth numbers to ≤1e-12.
  • PR body carries ## API Changes for phases 2b/2c/3.

Original Prompt

Click to expand starting prompt

See PyAutoMind/issued/oversampling_phase_2a_convolver_dataset.md (phase 2a split of oversampling_phase_2_core_api.md, itself phase 2 of oversampling.md; design approved in #353).

Activity

  1. Jammy2211 commented on Jul 8, 2026

    @Jammy2211
    Jammy2211
    on Jul 8, 2026
    CollaboratorAuthor

    Phase 2a implemented — parking for ship sign-off

    The core API is implemented, committed locally on feature/psf-oversample-core (PyAutoArray, commit 4eceda06, not pushed), and fully verified. Per the supervised --auto contract the commit/push/PR step is the gated action, so this is the batched sign-off question.

    Verification

    Leg Result
    Tests (shipped repo) PyAutoArray 857 passed, 0 failed (includes 13 new tests)
    Tests (downstream, API kwarg added) PyAutoGalaxy 933 passed, PyAutoLens 331 passed against the branch via worktree PYTHONPATH
    Ground truth s=2 reference values reproduced to ≤1e-12 through both the raw Convolver and the full Imaging.apply_mask flow; s=1 parity exact
    Review Surface: 9 files, +1363/−277, python-source flag. Verdict: CLEAN — additive API, all four scoping guards fire in tests, apply_mask state-rebuild edge fixed during implementation
    Smoke Not run (change is additive behind convolve_over_sample_size=1 defaults; can run the curated subsets before push on request)
    Heart YELLOW (same chronic reason set as this morning: stale workspace validation, 55 stale parked scripts, assistant pin, no install verification, old PRs). No reason set was acknowledged at this launch, so an unattended ship is not permitted regardless

    API Changes (drafted PR section)

    • Convolver gains convolve_over_sample_size: int = 1 (kernel supplied at the fine resolution when >1), kernel_shape_image_resolution property. convolved_image_from / convolved_image_via_real_space_np_from gain an optional keyword mask= (needed because over-sampled inputs don't carry the mask; all existing call sites pass keywords, verified in PyAutoGalaxy/PyAutoLens). When s>1, convolution methods expect over-sampled (sub-block ordered) inputs and raise KernelException on binned input; the JAX oversampled path always uses the FFT formalism.
    • ConvolverState gains an optional blurring_mask override; FFT frame is sized to keep an explicit blurring region.
    • Imaging gains convolve_over_sample_size_lp / convolve_over_sample_size_pixelization (int, default 1) with guards: over_sample_size must be uniform and equal when >1; differing lp/pixelization sizes raise (single kernel); sparse_operator + oversampled pixelization raises; apply_sparse_operator raises on an oversampled PSF. from_fits gains the two sizes + psf_pixel_scales. apply_mask / apply_over_sampling pass through.
    • GridsDataset.blurring uses the kernel's image-resolution footprint and evaluates at the fine resolution when oversampled.
    • over_sample decorator gains keyword-only binned: bool = True; binned=False returns sub-block-ordered values.
    • New utils: mask_2d_upscaled_from, sub_slim_to_fine_slim_from.
    • Not in this PR (per the approved phasing): inversion wiring (2b — an s>1 Convolver currently raises loudly if an inversion hands it a binned mapping matrix), PyAutoGalaxy operate/image.py consumer switch (2c, also blocked by the kaplinghat PyAutoGalaxy claim), workspace examples (3), docs (4).

    ⏸ The question

    Approve pushing this commit and opening the pending-release PR on PyAutoLabs/PyAutoArray?

    • "Ship it" → push feature/psf-oversample-core, open the PR with the body above (pending-release, library-first gate), file phase 2b's prompt for issue, update Mind state.
    • "Run smoke first" → run the curated workspace smoke subsets against the branch, append results here, then ship if clean.
    • Amendments → reply with changes; they're applied before push.

    Also note: the Heart YELLOW reasons above are the same chronic set you acknowledged for profiling-polish-design today — if you want that acknowledgement to cover this task too, say so and it's recorded in active.md (it never carries over implicitly).

    Task psf-oversample-core parked awaiting-input in PyAutoMind/active.md; the worktree holds the local commit.

    🤖 Generated with Claude Code

  2. Jammy2211 commented on Jul 8, 2026

    @Jammy2211
    Jammy2211
    on Jul 8, 2026
    CollaboratorAuthor

    🚢 Shipped (human-approved at the sign-off checkpoint): PR #355 (pending-release, branch feature/psf-oversample-core).

    Workspace impact: none for existing scripts — the API is additive behind convolve_over_sample_size=1 defaults, and PyAutoGalaxy (933) / PyAutoLens (331) suites pass against the branch unchanged. No migration follow-up needed; new workspace demos arrive in phase 3 after 2b/2c per the approved phasing.

    Next in the series: phase 2b (inversion wiring) is filed as its own issue and starts once this PR merges. Phase 2c (PyAutoGalaxy consumer) was queued behind the kaplinghat-sidm-cored-nfw PyAutoGalaxy claim, which has since been released — it now waits only on this merge.

    Merge and issue-close stay human acts — happy to do both on your word.

    🤖 Generated with Claude Code

  3. Jammy2211
    added a commit that references this issue on Jul 8, 2026

    feat: oversampled PSF convolution core API (convolve_over_sample_size) (

    279a5a0

  4. Jammy2211 commented on Jul 8, 2026

    @Jammy2211
    Jammy2211
    on Jul 8, 2026
    CollaboratorAuthor

    ✅ Merged as PR #355 (squash). Phase 2a complete — convolve_over_sample_size core API is on main. Phase 2b (inversion wiring) continues in #356.

    🤖 Generated with Claude Code

  5. Jammy2211
    closed this as completedon Jul 8, 2026
  6. Jammy2211
    added a commit that references this issue on Jul 8, 2026

    fix: oversampled fine state when the blurring mask is padded (#354 fo…

    12d0dce
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions