Skip to content

refactor: separate PYAUTOFIT_TEST_MODE into distinct PYAUTO_* env vars #1193

Closed

Description

@Jammy2211

Overview

PYAUTOFIT_TEST_MODE has become a catch-all that controls sampler speedup, visualization, fit I/O, and validation checks. This task separates it into distinct, purpose-specific environment variables under a unified PYAUTO_* prefix. Also renames PYAUTOARRAY_OUTPUT_MODE → PYAUTO_OUTPUT_MODE and PYAUTO_WORKSPACE_SMALL_DATASETS → PYAUTO_SMALL_DATASETS.

Plan

  • Add new env var accessor functions to PyAutoConf test_mode.py (skip_fit_output, skip_visualization, skip_checks)
  • Rename PYAUTOFIT_TEST_MODE → PYAUTO_TEST_MODE (sampler-only, levels 0-3)
  • Update every call site in PyAutoFit, PyAutoArray, PyAutoGalaxy, and PyAutoLens to use the correct new function
  • Rename PYAUTOARRAY_OUTPUT_MODE → PYAUTO_OUTPUT_MODE and PYAUTO_WORKSPACE_SMALL_DATASETS → PYAUTO_SMALL_DATASETS
  • Update all workspace env_vars.yaml files and shell scripts to set the new variables explicitly
  • Update CLAUDE.md documentation across repos
Detailed implementation plan

Affected Repositories

  • PyAutoConf (primary — defines env var functions)
  • PyAutoFit (most call sites)
  • PyAutoArray (hilbert checks, output_mode rename, small_datasets rename)
  • PyAutoGalaxy (small_datasets rename)
  • PyAutoLens (position checks, visualization)
  • autofit_workspace (env_vars.yaml, run_scripts.sh)
  • autogalaxy_workspace (env_vars.yaml, run_all_scripts.sh)
  • autolens_workspace (env_vars.yaml, run_all_scripts.sh)

Work Classification

Library (then workspace follow-up)

Branch Survey

Repository Current Branch Dirty?
PyAutoConf main clean
PyAutoFit main clean
PyAutoArray main clean
PyAutoGalaxy main clean
PyAutoLens main clean
autofit_workspace main dirty (datasets)
autogalaxy_workspace main dirty (datasets)
autolens_workspace main dirty (datasets)

Suggested branch: feature/test-mode-separate
Worktree root: ~/Code/PyAutoLabs-wt/test-mode-separate/

New Environment Variables

New env var Replaces Type What it controls
PYAUTO_TEST_MODE PYAUTOFIT_TEST_MODE int 0-3 Sampler speedup only
PYAUTO_SKIP_FIT_OUTPUT (was in test_mode) bool Pre/post-fit I/O, VRAM, result text, likelihood check
PYAUTO_SKIP_VISUALIZATION (was in test_mode) bool Fit visualization and plotting
PYAUTO_SKIP_CHECKS (was in test_mode) bool Mesh validation, position resampling, weight thresholds
PYAUTO_OUTPUT_MODE PYAUTOARRAY_OUTPUT_MODE bool Save all plots as numbered PNGs
PYAUTO_SMALL_DATASETS PYAUTO_WORKSPACE_SMALL_DATASETS bool Cap grids to 15x15

Implementation Steps

Step 1: PyAutoConf — extend test_mode.py

Add new functions alongside existing test_mode_level() / is_test_mode():

  • skip_fit_output() → reads PYAUTO_SKIP_FIT_OUTPUT
  • skip_visualization() → reads PYAUTO_SKIP_VISUALIZATION
  • skip_checks() → reads PYAUTO_SKIP_CHECKS

Rename env var from PYAUTOFIT_TEST_MODE to PYAUTO_TEST_MODE.

Export all new functions from autoconf/__init__.py.

Step 2: PyAutoFit — update call sites

Sampler speedup (keep test_mode_level() / is_test_mode()):

  • non_linear/initializer.py:65 — is_test_mode()
  • non_linear/search/abstract_search.py:668-674 — test_mode_level() >= 2
  • non_linear/search/abstract_search.py:829-874 — _fit_bypass_test_mode()
  • non_linear/search/mcmc/emcee/search.py:270-284 — is_test_mode()
  • non_linear/search/mcmc/zeus/search.py:289-304 — is_test_mode()
  • non_linear/search/mcmc/auto_correlations.py:52-53 — is_test_mode()

Change to skip_fit_output():

  • non_linear/search/abstract_search.py:500-518 — pre-fit output
  • non_linear/search/abstract_search.py:530-537 — post-fit output
  • non_linear/analysis/analysis.py:331-334 — print_vram_use
  • non_linear/fitness.py:459-461 — check_likelihood_function
  • text/text_util.py:60-61 — model_text_from

Change to skip_visualization():

  • non_linear/analysis/visualize.py:44-45 — should_visualize
  • non_linear/plot/plot_util.py:16-18 — skip_in_test_mode decorator

Change to skip_checks():

  • non_linear/samples/samples.py:382-383 — weight threshold

Step 3: PyAutoArray — update call sites + renames

Change to skip_checks():

  • inversion/mesh/image_mesh/hilbert.py:334-335 — check_adapt_pixelisation
  • inversion/mesh/image_mesh/hilbert.py:401-402 — check_adapt_background_pixels

Rename PYAUTOARRAY_OUTPUT_MODE → PYAUTO_OUTPUT_MODE:

  • plot/output.py:196,225
  • plot/utils.py:308

Rename PYAUTO_WORKSPACE_SMALL_DATASETS → PYAUTO_SMALL_DATASETS:

  • structures/grids/uniform_2d.py:174,493
  • mask/mask_2d.py:363
  • operators/over_sampling/over_sample_util.py:272
  • util/dataset_util.py:20

Step 4: PyAutoGalaxy — renames

Rename PYAUTO_WORKSPACE_SMALL_DATASETS → PYAUTO_SMALL_DATASETS:

  • autogalaxy/analysis/model_util.py:93

Step 5: PyAutoLens — update call sites

Change to skip_checks():

  • autolens/analysis/result.py:307-308 — position resampling
  • autolens/analysis/analysis/dataset.py:113-114 — inversion position exception

Change to skip_visualization():

  • autolens/quantity/model/visualizer.py:43-44

Step 6: Workspace env_vars.yaml and shell scripts

Update all env_vars.yaml files to set all new vars explicitly:

PYAUTO_TEST_MODE: "2"
PYAUTO_SKIP_FIT_OUTPUT: "1"
PYAUTO_SKIP_VISUALIZATION: "1"
PYAUTO_SKIP_CHECKS: "1"
PYAUTO_SMALL_DATASETS: "1"

Update run_scripts.sh / run_all_scripts.sh to export all new vars.

Update any unset: overrides that referenced old names.

Step 7: Update CLAUDE.md and documentation

Update env var references in CLAUDE.md across all repos.

Key Files

  • PyAutoConf/autoconf/test_mode.py — central definition
  • PyAutoConf/autoconf/__init__.py — exports
  • PyAutoFit/autofit/non_linear/search/abstract_search.py — most call sites
  • PyAutoArray/autoarray/plot/output.py — output_mode rename
  • All workspace config/build/env_vars.yaml files

Original Prompt

Click to expand starting prompt

PYAUTOFIT_TEST_MODE has become a bit of a catch all term for things I dont want to run when testing,
it incldues disable some visualization and posiition resmapling in autolens as opposed to just
making the sampler run faster.

Furthermore, there is also PYAUTOARRAY_OUTPUT_MODE which outputs images to hard disk, which si use.

Can you look at all PYAUTOFIT_TEST_MODE clauses and work out which are not doing a sampler speed up.
Can you then suggest names for us to use for them all.

I also want us to move to environment variables just being PYAUTO, so PYAUTO_TEST_MODE, PYAUTO_OUTPUT_MODE,
we will also rename PYAUTO_WORKSPACE_SMALL_DATASETS to PYAUTO_SMALL_DATASETS.

Activity

  1. Jammy2211 commented on Apr 12, 2026

    @Jammy2211
    CollaboratorAuthor

    Library PRs Created — 2026-04-12

    PRs

    Status

    Workspace changes needed. Next: /start_workspace

    API Changes

    • PYAUTOFIT_TEST_MODE → PYAUTO_TEST_MODE
    • PYAUTOARRAY_OUTPUT_MODE → PYAUTO_OUTPUT_MODE
    • PYAUTO_WORKSPACE_SMALL_DATASETS → PYAUTO_SMALL_DATASETS
    • New vars: PYAUTO_SKIP_FIT_OUTPUT, PYAUTO_SKIP_VISUALIZATION, PYAUTO_SKIP_CHECKS
    • All workspace env_vars.yaml, shell scripts, and 2 Python scripts need updating
  2. Jammy2211 commented on Apr 12, 2026

    @Jammy2211
    CollaboratorAuthor

    Shipped — 2026-04-12

    Library PRs

    Workspace PRs

    Summary

    Separated the catch-all PYAUTOFIT_TEST_MODE into purpose-specific environment variables: PYAUTO_TEST_MODE (sampler), PYAUTO_SKIP_FIT_OUTPUT, PYAUTO_SKIP_VISUALIZATION, PYAUTO_SKIP_CHECKS. Also renamed PYAUTOARRAY_OUTPUT_MODE → PYAUTO_OUTPUT_MODE and PYAUTO_WORKSPACE_SMALL_DATASETS → PYAUTO_SMALL_DATASETS. Updated all 8 repos (5 libraries + 3 workspaces).

    Session Notes

    • The categorisation was: sampler speedup (6 call sites stay as test_mode), fit I/O (5 sites → skip_fit_output), visualization (3 sites → skip_visualization), validation checks (5 sites → skip_checks).
    • Workspace env_vars.yaml now sets all new skip vars explicitly rather than having test_mode imply them.
    • Pre-existing smoke test failure in autofit_workspace/scripts/cookbooks/samples.py (bypass mode generates 2 samples but script indexes [4]) — unrelated to this change.
  3. Jammy2211 commented on May 28, 2026

    @Jammy2211
    CollaboratorAuthor

    Shipped via #1195. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions