[Jammy2211]

Summary

The Sibson (DelaunayNN) numba kernel was always parallel=True. Without TBB, numba's threading layer is GNU OpenMP, and its fork handler kills any forked child that launches a parallel region after the parent has used the pool. PyAutoFit evaluates the likelihood once in the parent before forking its search workers, so every NumPy-path DelaunayNN fit with more than one core failed with Terminating: fork() called from a process already using GNU OpenMP, this is unsafe. followed by a Nautilus pool TimeoutError. This was seen on RAL, job 414150, the likelihood-lifecycle phase 4 validation leg.

Fix: numba parallel threads never run inside a Python multiprocessing worker. A new predicate, numba_util.parallel_kernel_allowed(), is False in any multiprocessing child (fork, spawn or forkserver) and in a pid other than the one that first called it. In that case the Sibson dispatch uses a parallel=False compilation of the same body, which gives identical results, and the interferometer direct_conv route falls back to its existing serial or symmetric kernels. A single-process main process keeps the parallel kernel and numba's default threading layer, so profiled single-process performance is unchanged. This also stops the oversubscription where every worker started one numba thread per node core.

Closes #676

API Changes

• Added autoarray.numba_util.parallel_kernel_allowed() and autoarray.numba_util.function_copy() (internal helpers).
• sibson.sibson_numba_kernel() gains parallel: bool = True; the default behaviour is unchanged.
• general.yaml numba: parallel comment: single-process benchmarking only, unsafe under multiprocessing searches. numba_util.jit logs a one-time warning if it is True inside a worker.
• No user-facing API change; results are bit-identical between the serial and parallel builds.
  See full details below.

Test Plan

• New test_autoarray/test_numba_fork_safety.py (8 tests): a subprocess under NUMBA_THREADING_LAYER=omp runs the Sibson dispatch in the parent, then in fork and spawn Pool(2) children, and the outputs must equal the parent's. It fails before the fix (4× "Terminating: fork()…", TimeoutError) and passes after. A control with the predicate forced True reproduces the termination. Serial output is array_equal to parallel. There are unit tests of the predicate and of the worker route for _numba_parallel().
• Full pytest test_autoarray -x: 2295 passed, 4 xfailed, 0 failed (at head 56db9c0).
• Laptop check (numba 0.68.1): with the parent on omp, fork and spawn children running the serial build never initialise a threading layer.
• RAL: rerun the autolens_inference imaging/delaunay_nn/hst numba_cpu_sparse validation leg (414150's submit) on a checkout carrying this fix; it should pass source_pix[1].

Heart at ship: STALE, development pass. Reason: "release validation stale: source moved since rehearsal (PyAutoNerves)".

Full API Changes (for automation & release notes)

Removed

• None

Added

• autoarray.numba_util.parallel_kernel_allowed() -> bool: whether this process may launch a numba prange kernel (False in multiprocessing children and forked pids).
• autoarray.numba_util.function_copy(func, name): a renamed copy of a plain function, so one body can be numba-compiled twice and both builds are cached on disk (numba's cache index key ignores compile flags).
• autoarray.inversion.mesh.interpolator._sibson_numba._sibson_from_tables_serial: the parallel=False compilation of the Sibson body.

Changed

• sibson.sibson_numba_kernel(parallel=True): returns the parallel or serial build.
• _sibson_mappings_weights_numba uses the serial build in search workers.
• interferometer_numba.sparse._numba_parallel() is False in search workers.
• The Sibson kernel's cache file name changes (_sibson_from_tables body), so it recompiles once.

Migration

• None.

Generated by the PyAutoLabs agent workflow.

🤖 Generated with Claude Code