diff --git a/notebooks/chapter_1_introduction/tutorial_0_visualization.ipynb b/notebooks/chapter_1_introduction/tutorial_0_visualization.ipynb index 6f5c37a..366cb27 100644 --- a/notebooks/chapter_1_introduction/tutorial_0_visualization.ipynb +++ b/notebooks/chapter_1_introduction/tutorial_0_visualization.ipynb @@ -275,13 +275,8 @@ "`subplot_shape_to_figsize_factor` controls subplot sizes. The per-call options above are enough for this lesson.\n", "\n", "**\ud83e\udd16 Ask AI:** Why can logarithmic colour scaling make faint structure easier to see without increasing its\n", - "signal-to-noise ratio?" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "signal-to-noise ratio?\n", + "\n", "__Overlays__\n", "\n", "Overlays place reference curves or positions on an image. The `lines=` argument draws curves, while\n", diff --git a/notebooks/chapter_1_introduction/tutorial_2_ray_tracing.ipynb b/notebooks/chapter_1_introduction/tutorial_2_ray_tracing.ipynb index 514a1e4..2980e3a 100644 --- a/notebooks/chapter_1_introduction/tutorial_2_ray_tracing.ipynb +++ b/notebooks/chapter_1_introduction/tutorial_2_ray_tracing.ipynb @@ -1049,13 +1049,8 @@ "physical units when needed.\n", "\n", "Tutorial 3 explains the cosmological distances behind this\n", - "conversion, as part of its optional mathematical discussion." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "conversion, as part of its optional mathematical discussion.\n", + "\n", "__More Complexity__\n", "\n", "Our tracer currently contains one spherical lens and one source.\n", diff --git a/notebooks/chapter_1_introduction/tutorial_3_more_ray_tracing.ipynb b/notebooks/chapter_1_introduction/tutorial_3_more_ray_tracing.ipynb index c47d917..6f2038d 100644 --- a/notebooks/chapter_1_introduction/tutorial_3_more_ray_tracing.ipynb +++ b/notebooks/chapter_1_introduction/tutorial_3_more_ray_tracing.ipynb @@ -260,13 +260,8 @@ "expressed in arcseconds. They are the displacement we subtract\n", "from an image-plane position to obtain its source-plane position,\n", "as we did in tutorial 2. We will continue to call them\n", - "\"deflection angles\" throughout this tutorial." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "\"deflection angles\" throughout this tutorial.\n", + "\n", "__Convergence__\n", "\n", "Imagine adding up the lens galaxy's mass along our line of sight.\n", @@ -348,13 +343,8 @@ "\n", "This is an average over the enclosed area, rather than the\n", "convergence at its edge. We will return to this connection\n", - "when we use the Einstein radius to measure the lens's mass." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "when we use the Einstein radius to measure the lens's mass.\n", + "\n", "__Deflection Angles__\n", "\n", "The convergence describes the projected mass distribution.\n", @@ -417,13 +407,8 @@ "source": [ "Each component changes sign across the lens centre. Together,\n", "they describe the displacement we subtract from each\n", - "image-plane coordinate when tracing it to the source plane." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "image-plane coordinate when tracing it to the source plane.\n", + "\n", "__The Lensing Potential__\n", "\n", "We can describe the same lens using a single scalar field:\n", @@ -485,13 +470,8 @@ "source": [ "For this spherical lens, the potential has circular contours.\n", "Its slope points radially outwards, matching the direction\n", - "of the deflection vectors used in our lens equation." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "of the deflection vectors used in our lens equation.\n", + "\n", "We can check $\\boldsymbol{\\alpha} = \\nabla\\psi$ directly.\n", "\n", "First, we calculate the deflections using the mass profile's\n", @@ -1037,13 +1017,8 @@ "The recovered angular radius should be close to the\n", "1.6 arcseconds specified in our spherical mass profile.\n", "Small differences arise from locating the critical curve\n", - "on a finite grid." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "on a finite grid.\n", + "\n", "Two simple mass distributions have useful expressions\n", "for the Einstein radius.\n", "\n", @@ -1075,13 +1050,8 @@ "They show how the Einstein radius connects the lens's mass\n", "or velocity dispersion to the observer\u2013lens\u2013source geometry.\n", "Our `IsothermalSph` profile specifies this angular scale\n", - "directly through its `einstein_radius` parameter." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "directly through its `einstein_radius` parameter.\n", + "\n", "For a circular lens, the mean convergence inside the Einstein\n", "radius equals one. Its enclosed projected mass is therefore\n", "\n", @@ -1125,13 +1095,8 @@ "For non-circular lenses, the effective Einstein radius remains\n", "a useful summary of the lensing scale, but the circular\n", "enclosed-mass relation is not generally exact. The detailed\n", - "mass model and surrounding matter still matter." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "mass model and surrounding matter still matter.\n", + "\n", "__Time Delays__\n", "\n", "Different images of the same source follow different paths\n", @@ -1206,13 +1171,8 @@ "same brightness variation can appear in different images\n", "at different times. Measuring those delays provides\n", "information about cosmological distances as well as\n", - "the lens's mass distribution." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "the lens's mass distribution.\n", + "\n", "__Wrap Up__\n", "\n", "The quantities in this tutorial describe different aspects\n", diff --git a/notebooks/chapter_1_introduction/tutorial_4_point_sources.ipynb b/notebooks/chapter_1_introduction/tutorial_4_point_sources.ipynb index c3c1fe6..954abfe 100644 --- a/notebooks/chapter_1_introduction/tutorial_4_point_sources.ipynb +++ b/notebooks/chapter_1_introduction/tutorial_4_point_sources.ipynb @@ -187,13 +187,8 @@ "between the appearances of the same variation in different images.\n", "\n", "A quasar or supernova can also sit inside a visible host galaxy. Such a lens may therefore contain both\n", - "unresolved point images and extended arcs." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "unresolved point images and extended arcs.\n", + "\n", "__Point Source Tracer__\n", "\n", "PyAutoLens represents a point source using the `Point` profile in `al.ps`. This profile specifies its\n", @@ -228,13 +223,8 @@ "contains several background sources.\n", "\n", "This tracer now contains the lens mass and the point-source position. Next, we will find the multiple image\n", - "positions that this combination predicts." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "positions that this combination predicts.\n", + "\n", "__The Lens Equation__\n", "\n", "In Tutorial 2, we used the lens equation to map an image-plane coordinate $\\theta$ to a source-plane coordinate\n", @@ -252,13 +242,8 @@ "\n", "Finding these solutions is harder than evaluating the equation at a known coordinate. The deflection\n", "$\\alpha(\\theta)$ changes with position, and several different coordinates can map to the same source. Simple\n", - "mass profiles have analytic solutions, but more general models require a numerical search." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "mass profiles have analytic solutions, but more general models require a numerical search.\n", + "\n", "__Point Solver__\n", "\n", "The `PointSolver` divides the image-plane into triangles and ray-traces their corners to the source-plane. It\n", @@ -330,13 +315,8 @@ "\n", "Some mass profiles also produce a faint image near the lens centre. Our singular isothermal profile does not\n", "produce a finite central image, so we should not interpret the magnification threshold as removing a fifth\n", - "image in this example." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "image in this example.\n", + "\n", "__Multiple Images and Critical Curves__\n", "\n", "Let's overlay the solved positions on the compact extended source's image. The markers show where a point\n", @@ -374,13 +354,8 @@ "image positions of a point source.\n", "\n", "**\ud83e\udd16 Ask AI:** For a fixed elliptical lens mass profile, why can moving a point source across a caustic change\n", - "the number of its multiple images?" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "the number of its multiple images?\n", + "\n", "__Magnifications__\n", "\n", "Each multiple image has its own magnification. For an unresolved image, we measure this through its total\n", diff --git a/notebooks/chapter_1_introduction/tutorial_5_data.ipynb b/notebooks/chapter_1_introduction/tutorial_5_data.ipynb index 14a3a3c..86453fb 100644 --- a/notebooks/chapter_1_introduction/tutorial_5_data.ipynb +++ b/notebooks/chapter_1_introduction/tutorial_5_data.ipynb @@ -331,13 +331,8 @@ "metadata": {}, "source": [ "**\ud83e\udd16 Ask AI:** How does blurring by a telescope's PSF affect our ability to\n", - "distinguish two nearby features in a gravitationally lensed arc?" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "distinguish two nearby features in a gravitationally lensed arc?\n", + "\n", "__Poisson Noise__\n", "\n", "Even if a galaxy's brightness remains constant, repeated observations do not\n", @@ -591,13 +586,8 @@ "metadata": {}, "source": [ "**\ud83e\udd16 Ask AI:** Why does subtracting the mean background sky from an astronomical\n", - "image leave the sky's Poisson noise behind?" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "image leave the sky's Poisson noise behind?\n", + "\n", "The dataset also stores the PSF used to simulate the observation.\n", "\n", "For real observations, we need an estimate of the PSF, which can come from\n", @@ -841,13 +831,8 @@ "Fluxes and time delays are optional: a dataset can contain only positions\n", "and their uncertainties. For further examples of working with these data,\n", "see the workspace's\n", - "[point-source examples](https://github.com/PyAutoLabs/autolens_workspace/blob/main/scripts/point_source/start_here.py)." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "[point-source examples](https://github.com/PyAutoLabs/autolens_workspace/blob/main/scripts/point_source/start_here.py).\n", + "\n", "__Interferometer Data__\n", "\n", "We can also observe strong lenses at radio and submillimetre wavelengths using\n", @@ -916,13 +901,8 @@ "\n", "The dataset retains the visibilities for analysis. For further examples of\n", "working with these data, see the workspace's\n", - "[interferometer examples](https://github.com/PyAutoLabs/autolens_workspace/tree/main/scripts/interferometer)." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "[interferometer examples](https://github.com/PyAutoLabs/autolens_workspace/tree/main/scripts/interferometer).\n", + "\n", "__Weak Lensing Data__\n", "\n", "Away from the prominent arcs and rings, gravitational lensing can produce\n", @@ -973,13 +953,8 @@ "\n", "Chapter 4 returns to weak lensing in more detail. For further examples of\n", "working with these data, see the workspace's\n", - "[weak-lensing examples](https://github.com/PyAutoLabs/autolens_workspace/blob/main/scripts/weak/start_here.py)." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "[weak-lensing examples](https://github.com/PyAutoLabs/autolens_workspace/blob/main/scripts/weak/start_here.py).\n", + "\n", "__Group and Cluster Lenses__\n", "\n", "Strong lensing also occurs around groups and clusters of galaxies. Their\n", diff --git a/notebooks/chapter_1_introduction/tutorial_6_fitting.ipynb b/notebooks/chapter_1_introduction/tutorial_6_fitting.ipynb index 8bf270f..999fc53 100644 --- a/notebooks/chapter_1_introduction/tutorial_6_fitting.ipynb +++ b/notebooks/chapter_1_introduction/tutorial_6_fitting.ipynb @@ -624,13 +624,8 @@ "the noise expected in that pixel. This is our next step.\n", "\n", "**\ud83e\udd16 Ask AI:** Why can a correct lens model leave larger absolute residuals\n", - "in bright arcs than in the faint background?" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "in bright arcs than in the faint background?\n", + "\n", "The size of a residual only becomes informative when compared with the\n", "uncertainty in that pixel. A difference of 0.1 electrons per second could\n", "be small in a noisy pixel but substantial in a precisely measured one.\n", @@ -1060,13 +1055,8 @@ "cells in the Fitting section.\n", "\n", "**\ud83e\udd16 Ask AI:** Why might the parameters that maximise the likelihood of a\n", - "noisy simulated image differ slightly from those used to generate it?" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "noisy simulated image differ slightly from those used to generate it?\n", + "\n", "__Model Fitting__\n", "\n", "So far, we have compared parameter choices with a known reference model.\n", @@ -1192,13 +1182,8 @@ "\n", "To restore the original starting point, set both centres to `(1.0, 1.0)`,\n", "and set `einstein_radius`, `effective_radius` and `intensity` to 1.0, then\n", - "rerun the cell." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "rerun the cell.\n", + "\n", "You have now performed a simple manual search: choose parameters, predict\n", "an image, compare it with the data, then use that comparison to choose\n", "another set of parameters.\n", @@ -1216,13 +1201,8 @@ "and quantify what we can infer from the data.\n", "\n", "**\ud83e\udd16 Ask AI:** How could changing both the source size and its intensity\n", - "make it difficult to determine either parameter by adjusting them manually?" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "make it difficult to determine either parameter by adjusting them manually?\n", + "\n", "__Likelihood Function__\n", "\n", "Everything we have learned can now be put into a single sequence:\n", diff --git a/notebooks/chapter_1_introduction/tutorial_7_summary.ipynb b/notebooks/chapter_1_introduction/tutorial_7_summary.ipynb index 1e12a4c..bf0c4e3 100644 --- a/notebooks/chapter_1_introduction/tutorial_7_summary.ipynb +++ b/notebooks/chapter_1_introduction/tutorial_7_summary.ipynb @@ -246,13 +246,8 @@ "metadata": {}, "source": [ "**\ud83e\udd16 Ask AI:** Why does evaluating a source light profile on ray-traced source-plane coordinates produce a lensed image,\n", - "whereas evaluating it on a uniform source-plane grid produces an unlensed image?" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "whereas evaluating it on a uniform source-plane grid produces an unlensed image?\n", + "\n", "__Visualization__\n", "\n", "We can combine the lensed image with other quantities calculated from the same tracer.\n", @@ -299,13 +294,8 @@ "component, while `tracer.image_2d_from(grid=grid)` calculates the image of the complete system.\n", "\n", "As your analyses become more complex, this same structure lets you add components and inspect their individual\n", - "contributions." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "contributions.\n", + "\n", "__Source Code__\n", "\n", "The tools used in **HowToLens** are provided by four related projects:\n", @@ -323,13 +313,8 @@ "\n", "[Installation from source](https://pyautolens.readthedocs.io/en/latest/installation/source.html) is described in the\n", "documentation. This is useful if you want to modify or contribute to the libraries; it is optional for continuing\n", - "**HowToLens**." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "**HowToLens**.\n", + "\n", "__Wrap Up__\n", "\n", "You have now followed the path from a mathematical description of a lens system to a predicted observation and a\n", diff --git a/notebooks/chapter_3_pixelizations/tutorial_2_bayesian_regularization.ipynb b/notebooks/chapter_3_pixelizations/tutorial_2_bayesian_regularization.ipynb index f543e5c..c8cc195 100644 --- a/notebooks/chapter_3_pixelizations/tutorial_2_bayesian_regularization.ipynb +++ b/notebooks/chapter_3_pixelizations/tutorial_2_bayesian_regularization.ipynb @@ -418,13 +418,8 @@ "favour giving the source too much freedom.\n", "\n", "**\ud83e\udd16 Ask AI:** Why might a weakly regularized reconstruction have a lower chi-squared but a less trustworthy source\n", - "than a more strongly regularized reconstruction?" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "than a more strongly regularized reconstruction?\n", + "\n", "__Bayesian evidence__\n", "\n", "Choosing the regularization coefficient is a Goldilocks problem. Too little regularization gives the source enough freedom to fit the image\u2019s noise. Too much regularization smooths away source structure that we need to explain the image. We want an amount of regularization that is **\u201cjust right\u201d**.\n", @@ -513,13 +508,8 @@ "source": [ "Compare this reconstruction with the weakly and strongly regularized examples. How much of the irregular source structure has been smoothed away? Are the bright components still reconstructed accurately? What remains in the image residuals?\n", "\n", - "We have selected the best of **three coefficients**, rather than found the overall maximum. A coefficient between our trial values could give a higher log evidence. Next, we will evaluate a wider range and plot the evidence against the coefficient to locate its peak." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "We have selected the best of **three coefficients**, rather than found the overall maximum. A coefficient between our trial values could give a higher log evidence. Next, we will evaluate a wider range and plot the evidence against the coefficient to locate its peak.\n", + "\n", "__Exploring the regularization coefficient__\n", "\n", "Our three examples illustrate too little, moderate and too much regularization. However, there is nothing special about the starting coefficient of `1.0`. A value of `0.5`, `2.0`, or something else entirely might give a higher log evidence.\n", @@ -593,13 +583,8 @@ "\n", "Look for the peak of the curve. Towards small coefficients, the source has more freedom to fit noise. Towards large coefficients, smoothing increasingly prevents it from reproducing the source structure. The peak identifies our Goldilocks coefficient **among the values we have tested**.\n", "\n", - "If the highest point lies at either end of the range, we should extend the range in that direction. We have not yet located a peak between weaker and stronger regularization." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "If the highest point lies at either end of the range, we should extend the range in that direction. We have not yet located a peak between weaker and stronger regularization.\n", + "\n", "__Inspecting the preferred reconstruction__\n", "\n", "Let\u2019s take the coefficient with the highest log evidence and look at its source reconstruction and image residuals.\n", @@ -647,13 +632,8 @@ "\n", "There is also a compromise in applying one coefficient across the whole source. The bright centre may need weaker smoothing to preserve its steep brightness gradient, while the faint outskirts benefit from stronger smoothing. A single coefficient must serve both regions.\n", "\n", - "Keep these limitations in mind. Our later **Fit Problems** tutorial will show why choosing the coefficient is only part of the solution, motivating meshes and regularization that adapt to the source." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "Keep these limitations in mind. Our later **Fit Problems** tutorial will show why choosing the coefficient is only part of the solution, motivating meshes and regularization that adapt to the source.\n", + "\n", "__What is linear, and what requires a search?__\n", "\n", "Our reconstruction contains hundreds of source brightness values. Does this mean we have added hundreds of parameters to the non-linear search?\n", @@ -697,13 +677,8 @@ "3. The inversion solves for the source brightness values.\n", "4. We calculate the log evidence, which the search uses to assess that proposal.\n", "\n", - "This is what makes pixelized sources practical. We can reconstruct hundreds of source brightness values without asking the non-linear search to explore hundreds of additional parameters." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "This is what makes pixelized sources practical. We can reconstruct hundreds of source brightness values without asking the non-linear search to explore hundreds of additional parameters.\n", + "\n", "__Mesh resolution and regularization belong together__\n", "\n", "We have found a preferred regularization coefficient for our current mesh. Would the same coefficient still be preferred if we changed its resolution?\n", @@ -804,13 +779,8 @@ "\n", "A finer mesh may increase the evidence by resolving source structure that the coarser mesh cannot describe. However, more source pixels do not automatically mean a better reconstruction: the additional freedom must be justified by the image data.\n", "\n", - "Before drawing a conclusion, check that each peak lies inside the coefficient range. If a curve is still rising at either end, extend its range. Once a peak is bracketed, try more closely spaced coefficients around it to refine the comparison." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "Before drawing a conclusion, check that each peak lies inside the coefficient range. If a curve is still rising at either end, extend its range. Once a peak is bracketed, try more closely spaced coefficients around it to refine the comparison.\n", + "\n", "__Wrap Up__\n", "\n", "We can now explain the regularization coefficient that we first supplied without much discussion in tutorial 1.\n", @@ -849,13 +819,8 @@ "\n", "Before that, we need to examine another practical issue. Choosing the number of source pixels does not, by itself, determine our source resolution. We also need to consider **how much area those pixels cover**.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "For the underlying equations, the workspace\u2019s [pixelization likelihood guide](https://github.com/PyAutoLabs/autolens_workspace/blob/main/scripts/imaging/features/pixelization/likelihood_function.py) follows the mapping, blurring, regularization and evidence calculations step by step." ] } diff --git a/notebooks/chapter_3_pixelizations/tutorial_3_borders.ipynb b/notebooks/chapter_3_pixelizations/tutorial_3_borders.ipynb index fd9d49d..118befa 100644 --- a/notebooks/chapter_3_pixelizations/tutorial_3_borders.ipynb +++ b/notebooks/chapter_3_pixelizations/tutorial_3_borders.ipynb @@ -20,7 +20,21 @@ "- Switching border relocation on\n", "- What relocation changes\n", "- The mask still needs to cover the lensed source\n", - "- Wrap Up" + "- Wrap Up\n", + "\n", + "__Why the extent of the source mesh matters__\n", + "\n", + "Imagine spreading a `(25, 25)` rectangular mesh across a source-plane region one arcsecond wide. Now spread the same mesh across a region ten arcseconds wide. We still have the same number of brightness values, but the spacing between them is ten times larger. Small source components that the first mesh could resolve may be poorly represented by the second.\n", + "\n", + "What determines this extent?\n", + "\n", + "The uniform rectangular mesh we are using is constructed to cover the ray-traced image coordinates. If some coordinates land far away from the source galaxy, the mesh expands to include them. Much of its resolution can then be spent covering regions where there is little or no source light.\n", + "\n", + "This can happen when image pixels close to the centre of the lens are strongly demagnified and trace far out into the source plane. Adding these central image pixels to a fit can therefore have a surprisingly large effect on the source mesh.\n", + "\n", + "In this tutorial, we will follow those pixels through the calculation and introduce **border relocation**, which prevents these distant coordinates from unnecessarily expanding the reconstruction.\n", + "\n", + "We return to a simpler simulated source, with no foreground lens light, so that we can focus on the geometry." ] }, { @@ -61,25 +75,6 @@ "_setup_colab.setup(\"howtolens\")" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "__Why the extent of the source mesh matters__\n", - "\n", - "Imagine spreading a `(25, 25)` rectangular mesh across a source-plane region one arcsecond wide. Now spread the same mesh across a region ten arcseconds wide. We still have the same number of brightness values, but the spacing between them is ten times larger. Small source components that the first mesh could resolve may be poorly represented by the second.\n", - "\n", - "What determines this extent?\n", - "\n", - "The uniform rectangular mesh we are using is constructed to cover the ray-traced image coordinates. If some coordinates land far away from the source galaxy, the mesh expands to include them. Much of its resolution can then be spent covering regions where there is little or no source light.\n", - "\n", - "This can happen when image pixels close to the centre of the lens are strongly demagnified and trace far out into the source plane. Adding these central image pixels to a fit can therefore have a surprisingly large effect on the source mesh.\n", - "\n", - "In this tutorial, we will follow those pixels through the calculation and introduce **border relocation**, which prevents these distant coordinates from unnecessarily expanding the reconstruction.\n", - "\n", - "We return to a simpler simulated source, with no foreground lens light, so that we can focus on the geometry." - ] - }, { "cell_type": "code", "metadata": {}, @@ -123,13 +118,8 @@ "source": [ "We will compare two masks with the same outer radius. One includes the central image pixels; the other removes them. This lets us isolate how including the centre changes the area covered by the source mesh.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__What we mean by a border__\n", "\n", "Here, the **border** means the outermost boundary of the unmasked image region: the ring of image pixels just inside the mask\u2019s outer edge.\n", @@ -183,13 +173,8 @@ "\n", "We can ray-trace the marked border pixels just as we ray-trace any other image pixels. Their traced positions define a border in the **source plane**, which we will use to identify coordinates that land beyond it.\n", "\n", - "Keep these two operations separate. **Masking** determines which image pixels we fit. **Border relocation** changes how coordinates beyond the traced border enter the source reconstruction; it does not remove those image pixels from the fit." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "Keep these two operations separate. **Masking** determines which image pixels we fit. **Border relocation** changes how coordinates beyond the traced border enter the source reconstruction; it does not remove those image pixels from the fit.\n", + "\n", "__Starting without border relocation__\n", "\n", "Let\u2019s first reconstruct the source using the annular mask, which excludes the central image pixels. We explicitly switch border relocation off so that we can inspect the original ray-traced coordinates.\n", @@ -297,13 +282,8 @@ "\n", "Pay attention to the axis ranges and the spacing of the mesh points. These provide our reference for the next comparison.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Including the central image pixels__\n", "\n", "Now repeat the reconstruction with the circular mask.\n", @@ -378,13 +358,8 @@ "\n", "The masks contain different sets of image pixels, so we should not rank these two fits by their absolute log evidences. For now, compare their geometry and source reconstructions.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Following the central pixels into the source plane__\n", "\n", "Let\u2019s identify exactly where those distant source-plane coordinates came from.\n", @@ -479,13 +454,8 @@ "\n", "There is another potential problem: assigning source brightness at these distant positions may predict additional images outside our mask. Those predictions would not be checked against the excluded image data.\n", "\n", - "Next, we will trace the outer image border into the source plane and use it to control these distant coordinates." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "Next, we will trace the outer image border into the source plane and use it to control these distant coordinates.\n", + "\n", "__Tracing the outer border__\n", "\n", "We have seen that pixels near the image centre can trace far beyond the source galaxy. To decide which coordinates to relocate, we first need a boundary in the source plane.\n", @@ -527,13 +497,8 @@ "\n", "The border relocator uses the traced border to estimate a source-plane boundary. Coordinates beyond that boundary are moved inward for the reconstruction, preventing them from forcing the mesh to extend far beyond the region we want to reconstruct.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Switching border relocation on__\n", "\n", "Let\u2019s repeat the circular-mask fit with border relocation enabled.\n", @@ -628,13 +593,8 @@ "source": [ "These coordinates still correspond to the same image pixels and measured brightnesses. We have changed where they map within the source reconstruction. We have not moved any observed image pixels or excluded them from the fit.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__What relocation changes__\n", "\n", "Does relocating these coordinates mean we have changed the lens model?\n", @@ -731,13 +691,8 @@ "\n", "We have kept the regularization coefficient fixed to show the effect of switching relocation on. As we learned in tutorial 2, changing the mesh geometry can also change the preferred coefficient. A full comparison would therefore repeat the regularization search for each setup.\n", "\n", - "Next, we will examine why border relocation still requires a mask that covers the lensed source." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "Next, we will examine why border relocation still requires a mask that covers the lensed source.\n", + "\n", "__The mask still needs to cover the lensed source__\n", "\n", "Border relocation helps us control the source mesh, but it cannot compensate for an image mask that excludes part of the lensed source.\n", @@ -822,13 +777,8 @@ "\n", "A larger mask can also spread the fixed number of mesh points over a larger source-plane region. If the reconstruction changes, inspect the mesh spacing as well as the newly included data.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Wrap Up__\n", "\n", "The resolution of a rectangular source mesh depends on both its number of points and the area they cover.\n", @@ -874,13 +824,8 @@ "\n", "Next, we will let the mass model vary. This introduces another challenge: a flexible source reconstruction can sometimes produce a close image fit even when the lens mass model is wrong.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "**\ud83e\udd16 Ask AI:** Why can adding central image pixels reduce the useful resolution of a rectangular source mesh, and what does border relocation change?" ] } diff --git a/notebooks/chapter_3_pixelizations/tutorial_4_lens_modeling.ipynb b/notebooks/chapter_3_pixelizations/tutorial_4_lens_modeling.ipynb index a19e694..4154470 100644 --- a/notebooks/chapter_3_pixelizations/tutorial_4_lens_modeling.ipynb +++ b/notebooks/chapter_3_pixelizations/tutorial_4_lens_modeling.ipynb @@ -23,7 +23,27 @@ "- Running the pixelized-source search\n", "- Establishing the foreground light model\n", "- Using that result in the pixelized-source fit\n", - "- Wrap Up" + "- Wrap Up\n", + "\n", + "__Lens modelling with a pixelized source__\n", + "\n", + "In the previous tutorials, we supplied the mass model used to simulate the image. This let us concentrate on reconstructing the source.\n", + "\n", + "For an observed lens, we usually do not know the mass distribution. Measuring it is one of the main reasons we perform lens modelling!\n", + "\n", + "Changing the mass model changes where the image pixels trace in the source plane. For every proposed mass model, we must therefore reconstruct the source again and calculate the corresponding log evidence.\n", + "\n", + "Pixelizations are useful because they can describe complicated source structure without requiring us to choose a separate light profile for every component. However, this flexibility also creates a difficulty: **an incorrect mass model can sometimes fit the image by reconstructing an unusual source**.\n", + "\n", + "For example, instead of tracing several arcs back to a common source component, an incorrect model may reconstruct separate source features to explain them. The image fit can look convincing even though the inferred source and lens mass are wrong.\n", + "\n", + "This problem is illustrated by Figure 1 from [Maresca et al. (2021)](https://arxiv.org/abs/2012.04665).\n", + "\n", + "![Demagnified source reconstructions, Figure 1 of Maresca et al. (2021)](https://raw.githubusercontent.com/PyAutoLabs/HowToLens/main/scripts/chapter_3_pixelizations/images/maresca_fig1.png)\n", + "\n", + "Figure 1 from Maresca et al. (2021). Compare the reconstructed sources with their corresponding image fits. The experiments below use our own simulated dataset. See also the [PyAutoLens demagnified-solutions guide](https://pyautolens.readthedocs.io/en/latest/general/demagnified_solutions.html).\n", + "\n", + "We return to the familiar lens with no foreground light. Its true Einstein radius is `1.6` arcsec, giving us a reference against which to compare deliberately incorrect models." ] }, { @@ -64,31 +84,6 @@ "_setup_colab.setup(\"howtolens\")" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "__Lens modelling with a pixelized source__\n", - "\n", - "In the previous tutorials, we supplied the mass model used to simulate the image. This let us concentrate on reconstructing the source.\n", - "\n", - "For an observed lens, we usually do not know the mass distribution. Measuring it is one of the main reasons we perform lens modelling!\n", - "\n", - "Changing the mass model changes where the image pixels trace in the source plane. For every proposed mass model, we must therefore reconstruct the source again and calculate the corresponding log evidence.\n", - "\n", - "Pixelizations are useful because they can describe complicated source structure without requiring us to choose a separate light profile for every component. However, this flexibility also creates a difficulty: **an incorrect mass model can sometimes fit the image by reconstructing an unusual source**.\n", - "\n", - "For example, instead of tracing several arcs back to a common source component, an incorrect model may reconstruct separate source features to explain them. The image fit can look convincing even though the inferred source and lens mass are wrong.\n", - "\n", - "This problem is illustrated by Figure 1 from [Maresca et al. (2021)](https://arxiv.org/abs/2012.04665).\n", - "\n", - "![Demagnified source reconstructions, Figure 1 of Maresca et al. (2021)](https://raw.githubusercontent.com/PyAutoLabs/HowToLens/main/scripts/chapter_3_pixelizations/images/maresca_fig1.png)\n", - "\n", - "Figure 1 from Maresca et al. (2021). Compare the reconstructed sources with their corresponding image fits. The experiments below use our own simulated dataset. See also the [PyAutoLens demagnified-solutions guide](https://pyautolens.readthedocs.io/en/latest/general/demagnified_solutions.html).\n", - "\n", - "We return to the familiar lens with no foreground light. Its true Einstein radius is `1.6` arcsec, giving us a reference against which to compare deliberately incorrect models." - ] - }, { "cell_type": "code", "metadata": {}, @@ -138,13 +133,8 @@ "source": [ "We use one sample per image pixel to make the mapping demonstrations easy to follow. Before interpreting a scientific fit, we would check that finer sampling does not materially change the result.\n", "\n", - "First, we will deliberately give the inversion the wrong Einstein radius and inspect how it attempts to explain the image." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "First, we will deliberately give the inversion the wrong Einstein radius and inspect how it attempts to explain the image.\n", + "\n", "__Giving the inversion an incorrect mass model__\n", "\n", "Let\u2019s deliberately give the inversion an Einstein radius of `0.8` arcsec\u2014half the value used to simulate the image.\n", @@ -223,13 +213,8 @@ "\n", "This differs from the demagnified central pixels in the borders tutorial. There, we investigated individual coordinates that stretched the mesh. Here, we have an alternative interpretation of the whole lens system. Border relocation is enabled, but it does not prevent this solution.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Comparing with the correct mass model__\n", "\n", "Now restore the Einstein radius to `1.6` arcsec and reconstruct the source again.\n", @@ -312,13 +297,8 @@ "\n", "We have evaluated only two mass models at one regularization coefficient. As tutorial 2 showed, the preferred coefficient can change when the reconstruction changes. These numbers are therefore a first comparison, rather than the outcome of a full search over the mass and regularization parameters.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Seeing how the mass model changes the mappings__\n", "\n", "The source reconstructions show the result. Let\u2019s now inspect the mappings that produce it.\n", @@ -381,13 +361,8 @@ "\n", "Later, we will identify positions that **do** correspond to the same source feature and require them to trace close together. That gives us a way to steer the search away from demagnified solutions.\n", "\n", - "First, we will examine why calculating the evidence does not, by itself, ensure that a search finds the correct mass model." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "First, we will examine why calculating the evidence does not, by itself, ensure that a search finds the correct mass model.\n", + "\n", "__Why evidence does not remove the search difficulty__\n", "\n", "Suppose the correct mass model gives a higher log evidence than our incorrect example. Does that mean a non-linear search will automatically find it?\n", @@ -467,13 +442,8 @@ "source": [ "This one-dimensional scan greatly understates the difficulty of the full search. We have fixed the mass ellipticity, shear and regularization coefficient, whereas a real search varies them together. The resulting parameter space can contain many competing maxima that are absent from this simple slice.\n", "\n", - "This is why we begin with an MGE source and use search chaining. Its restricted geometry makes it harder to compensate for an incorrect mass model by reconstructing arbitrary source structure. We first establish a plausible lensing geometry, then introduce the pixelization and additional constraints to help keep the search away from unphysical solutions." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "This is why we begin with an MGE source and use search chaining. Its restricted geometry makes it harder to compensate for an incorrect mass model by reconstructing arbitrary source structure. We first establish a plausible lensing geometry, then introduce the pixelization and additional constraints to help keep the search away from unphysical solutions.\n", + "\n", "__Starting with a simpler source model__\n", "\n", "We will begin with the multi-Gaussian expansion (MGE) from Chapter 2.\n", @@ -545,13 +515,8 @@ "\n", "Inspect `model_1.info` to see these parameters and their priors. Although we know the simulated Einstein radius is `1.6` arcsec, we have not supplied that answer to the search: it must infer the radius from the image.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Running and inspecting the initial search__\n", "\n", "We are now ready to run the first search.\n", @@ -602,13 +567,8 @@ "\n", "Finally, remember that the plotted maximum-likelihood fit is one model. Use the result summary and parameter distributions, as in Chapter 2, to assess the constraints and whether further exploration is needed.\n", "\n", - "Next, we will use corresponding image positions to help the pixelized-source search avoid demagnified solutions." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "Next, we will use corresponding image positions to help the pixelized-source search avoid demagnified solutions.\n", + "\n", "__Requiring corresponding images to trace together__\n", "\n", "In Chapter 2, Tutorial 5, we introduced positions resampling and the **positions likelihood penalty**. We now use that same approach to help the pixelized-source search avoid the unphysical solutions discussed above.\n", @@ -672,13 +632,8 @@ "\n", "The purpose is to exclude grossly inconsistent mappings while allowing useful changes to the mass model. An excessively tight threshold could instead force the second search to remain too close to an imperfect initial fit.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Replacing the analytic source with a pixelization__\n", "\n", "We are ready to construct the second model.\n", @@ -737,13 +692,8 @@ "\n", "Inspect the priors printed by `model_2.info`. The initial MGE may have left source structure unexplained, potentially shifting its preferred mass model. Prior passing should guide the pixelized search without restricting it so tightly that it cannot correct that initial estimate.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Running the pixelized-source search__\n", "\n", "We now combine the pixelized model with the **positions likelihood penalty**, following the same procedure as Chapter 2, Tutorial 5.\n", @@ -831,13 +781,8 @@ "source": [ "Passing this check confirms that these selected positions trace sufficiently close together. We still need to inspect the image residuals, parameter constraints and search behaviour to assess the overall fit.\n", "\n", - "We have now fitted a free mass model and a pixelized source using search chaining. Next, we will include the foreground lens galaxy\u2019s light and examine how it is fitted alongside the source reconstruction." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "We have now fitted a free mass model and a pixelized source using search chaining. Next, we will include the foreground lens galaxy\u2019s light and examine how it is fitted alongside the source reconstruction.\n", + "\n", "__Establishing the foreground light model__\n", "\n", "So far, our images have contained only the lensed source. In most observed systems, the foreground lens galaxy also contributes light, often overlapping the arcs.\n", @@ -993,13 +938,8 @@ "\n", "Inspect the foreground light model, source and residuals. We want the initial model to separate the foreground galaxy from the arcs sensibly. Significant foreground residuals should be investigated before introducing a more flexible source that might absorb them.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Using that result in the pixelized-source fit__\n", "\n", "Now replace the analytic source with a pixelization.\n", @@ -1127,13 +1067,8 @@ "\n", "If the fixed foreground geometry leaves significant residuals, we can free its parameters in a later chained search. Holding it fixed is a useful intermediate step, rather than a requirement for the final analysis.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Wrap Up__\n", "\n", "We have now fitted a lens mass model alongside a pixelized source, both with and without foreground lens light.\n", @@ -1154,13 +1089,8 @@ "\n", "Even the evidence-preferred constant regularization coefficient must compromise between these demands.\n", "\n", - "In the next tutorial, **Fit Problems**, we will deliberately make the source more compact while keeping the true mass model fixed. This will reveal limitations of the mesh and regularization themselves, and motivate the adaptive methods that follow." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "In the next tutorial, **Fit Problems**, we will deliberately make the source more compact while keeping the true mass model fixed. This will reveal limitations of the mesh and regularization themselves, and motivate the adaptive methods that follow.\n", + "\n", "**\ud83e\udd16 Ask AI:** Why can an incorrect mass model fit the arcs with a pixelized source, and how do search chaining and the positions likelihood penalty address this?" ] } diff --git a/notebooks/chapter_3_pixelizations/tutorial_5_fit_problems.ipynb b/notebooks/chapter_3_pixelizations/tutorial_5_fit_problems.ipynb index 4f7cbab..f26817d 100644 --- a/notebooks/chapter_3_pixelizations/tutorial_5_fit_problems.ipynb +++ b/notebooks/chapter_3_pixelizations/tutorial_5_fit_problems.ipynb @@ -22,51 +22,8 @@ "- Why the mesh becomes a limitation\n", "- Why one regularization coefficient becomes a compromise\n", "- Why maximizing the evidence does not remove the compromise\n", - "- Wrap Up" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "__Google Colab Setup__\n", - "\n", - "Set up Google Colab as in [the introductory notebook](https://github.com/PyAutoLabs/HowToLens/blob/main/notebooks/chapter_1_introduction/tutorial_1_grids_and_galaxies.ipynb).\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "jupyter": { - "source_hidden": true - } - }, - "outputs": [], - "source": [ - "# @title Set up Google Colab { display-mode: \"form\" }\n", - "try:\n", - " import google.colab\n", - "except ImportError:\n", - " from autolens import setup_colab as _setup_colab\n", - "else:\n", - " import importlib\n", - " import subprocess\n", - " import sys\n", - "\n", - " subprocess.check_call(\n", - " [sys.executable, \"-m\", \"pip\", \"install\", \"autonerves\", \"--no-deps\"]\n", - " )\n", - " _setup_colab = importlib.import_module(\"autonerves.setup_colab\")\n", + "- Wrap Up\n", "\n", - "_setup_colab.setup(\"howtolens\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ "__When the best coefficient is not enough__\n", "\n", "We have learned how to reconstruct a source, choose its regularization coefficient using Bayesian evidence, and fit the lens mass model.\n", @@ -98,18 +55,51 @@ "- Separating the limitations of mesh resolution and constant regularization.\n", "- Motivating adaptation to the source\u2019s brightness distribution.\n", "\n", - "---" + "---\n", + "\n", + "__Three progressively compact sources__\n", + "\n", + "We use three S\u00e9rsic profiles with the same centre and elliptical shape.\n", + "\n", + "For the first, we choose a relatively large effective radius and a S\u00e9rsic index of `1.0`. For the next two, we decrease the effective radius and increase the S\u00e9rsic index. Their light becomes more concentrated, with a steeper brightness gradient near the centre." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "__Three progressively compact sources__\n", + "__Google Colab Setup__\n", "\n", - "We use three S\u00e9rsic profiles with the same centre and elliptical shape.\n", + "Set up Google Colab as in [the introductory notebook](https://github.com/PyAutoLabs/HowToLens/blob/main/notebooks/chapter_1_introduction/tutorial_1_grids_and_galaxies.ipynb).\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "cellView": "form", + "jupyter": { + "source_hidden": true + } + }, + "outputs": [], + "source": [ + "# @title Set up Google Colab { display-mode: \"form\" }\n", + "try:\n", + " import google.colab\n", + "except ImportError:\n", + " from autolens import setup_colab as _setup_colab\n", + "else:\n", + " import importlib\n", + " import subprocess\n", + " import sys\n", "\n", - "For the first, we choose a relatively large effective radius and a S\u00e9rsic index of `1.0`. For the next two, we decrease the effective radius and increase the S\u00e9rsic index. Their light becomes more concentrated, with a steeper brightness gradient near the centre." + " subprocess.check_call(\n", + " [sys.executable, \"-m\", \"pip\", \"install\", \"autonerves\", \"--no-deps\"]\n", + " )\n", + " _setup_colab = importlib.import_module(\"autonerves.setup_colab\")\n", + "\n", + "_setup_colab.setup(\"howtolens\")" ] }, { @@ -189,13 +179,8 @@ "\n", "This distinction will become important: the lensing geometry stays the same, but the spatial resolution needed to describe the source\u2019s light changes.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Simulating the images__\n", "\n", "We now simulate the three lensed images using the same image grid and telescope blurring.\n", @@ -313,13 +298,8 @@ "\n", "Next, we will reconstruct them using the same rectangular mesh and constant regularization scheme. For each dataset, we will scan the coefficient and inspect its evidence-preferred reconstruction.\n", "\n", - "We will compare alternative reconstructions **within each dataset**. We will not rank the three different simulated galaxies against one another using their absolute evidence values." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "We will compare alternative reconstructions **within each dataset**. We will not rank the three different simulated galaxies against one another using their absolute evidence values.\n", + "\n", "__Reconstructing the sources__\n", "\n", "We will reconstruct all three sources using a `25 \u00d7 25` rectangular mesh and `Constant` regularization.\n", @@ -371,13 +351,8 @@ "\n", "For now, the only reconstruction setting we vary is the regularization coefficient.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Finding the preferred regularization__\n", "\n", "As in tutorial 2, we evaluate a range of coefficients and compare their log evidences. We do this separately for each dataset: a coefficient suitable for the extended source need not be suitable for either compact source.\n", @@ -471,13 +446,8 @@ "\n", "We can now inspect the reconstructions knowing that we have selected their smoothing strengths systematically.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Starting with the extended source__\n", "\n", "Let\u2019s begin with the extended source.\n", @@ -550,13 +520,8 @@ "source": [ "A value around one is a useful reference for noise-level residuals, but it does not tell us whether those residuals form a spatial pattern. Keep the residual map alongside this number.\n", "\n", - "This extended-source fit provides our baseline. Next, we will make the source more compact without giving the mesh any additional resolution, and examine where the reconstruction begins to struggle." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "This extended-source fit provides our baseline. Next, we will make the source more compact without giving the mesh any additional resolution, and examine where the reconstruction begins to struggle.\n", + "\n", "__Making the source more compact__\n", "\n", "Now inspect the compact source.\n", @@ -627,13 +592,8 @@ "\n", "The question is therefore no longer simply \u201cHave we chosen the wrong coefficient?\u201d We need to ask whether our mesh and regularization scheme can represent this source adequately.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__A very compact source__\n", "\n", "Our final source has an effective radius of just `0.1` arcsec and a S\u00e9rsic index of `4.0`. Its light rises steeply towards the centre, placing a greater demand on the reconstruction.\n", @@ -712,13 +672,8 @@ "\n", "These galaxies also have different brightness distributions and signal-to-noise ratios. We should not interpret the comparison as a measurement of source-size effects alone. Its purpose is to expose the practical difficulty of reconstructing increasingly concentrated light with an unchanged mesh and one smoothing coefficient.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Why the mesh becomes a limitation__\n", "\n", "Let\u2019s examine the first problem: where our source pixels are placed.\n", @@ -771,13 +726,8 @@ "\n", "This motivates adapting the mesh to the **source\u2019s brightness distribution**, which we will explore in the next tutorial.\n", "\n", - "First, there is a second limitation to understand. Even with enough mesh points, should we apply the same regularization coefficient to the bright centre and faint outskirts?" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "First, there is a second limitation to understand. Even with enough mesh points, should we apply the same regularization coefficient to the bright centre and faint outskirts?\n", + "\n", "__Why one regularization coefficient becomes a compromise__\n", "\n", "Suppose we had enough mesh points to resolve the source\u2019s bright centre. Would that solve the problem?\n", @@ -861,13 +811,8 @@ "\n", "These changes need not be equally visible in every dataset. The important point is that changing one coefficient changes the smoothing preference everywhere, even though different source regions may benefit from different strengths.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Why maximizing the evidence does not remove the compromise__\n", "\n", "Why does the Bayesian evidence not simply select a coefficient that fixes both regions?\n", @@ -917,13 +862,8 @@ "\n", "The evidence still decides whether that additional flexibility is useful. We compare the constant and adaptive schemes on the same dataset, allowing their parameters to vary appropriately. Adaptation is a way to improve the available reconstructions, rather than a guarantee that every fit will improve.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Wrap Up__\n", "\n", "We began with three sources whose light became progressively more concentrated, while keeping the lens mass model fixed to its true values.\n", @@ -945,13 +885,8 @@ "\n", "At each stage, we will return to the questions raised here: does the mesh resolve the source\u2019s bright structure, and does regularization preserve that structure while controlling fluctuations elsewhere?\n", "\n", - "We will finish by using these methods in a lens-model fit, with the search-chaining procedure and positions likelihood penalty already introduced." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "We will finish by using these methods in a lens-model fit, with the search-chaining procedure and positions likelihood penalty already introduced.\n", + "\n", "**\ud83e\udd16 Ask AI:** How would you distinguish insufficient source-mesh resolution from over-smoothing when compact-source residuals remain at the evidence-preferred coefficient?" ] } diff --git a/notebooks/chapter_3_pixelizations/tutorial_6_adaptive_reconstruction.ipynb b/notebooks/chapter_3_pixelizations/tutorial_6_adaptive_reconstruction.ipynb index 80782f6..f560fb3 100644 --- a/notebooks/chapter_3_pixelizations/tutorial_6_adaptive_reconstruction.ipynb +++ b/notebooks/chapter_3_pixelizations/tutorial_6_adaptive_reconstruction.ipynb @@ -34,51 +34,8 @@ "- Defining the adaptive Delaunay model\n", "- Running the adaptive Delaunay fit\n", "- Inspecting the source and smoothing pattern\n", - "- Wrap Up" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "__Google Colab Setup__\n", - "\n", - "Set up Google Colab as in [the introductory notebook](https://github.com/PyAutoLabs/HowToLens/blob/main/notebooks/chapter_1_introduction/tutorial_1_grids_and_galaxies.ipynb).\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "jupyter": { - "source_hidden": true - } - }, - "outputs": [], - "source": [ - "# @title Set up Google Colab { display-mode: \"form\" }\n", - "try:\n", - " import google.colab\n", - "except ImportError:\n", - " from autolens import setup_colab as _setup_colab\n", - "else:\n", - " import importlib\n", - " import subprocess\n", - " import sys\n", + "- Wrap Up\n", "\n", - " subprocess.check_call(\n", - " [sys.executable, \"-m\", \"pip\", \"install\", \"autonerves\", \"--no-deps\"]\n", - " )\n", - " _setup_colab = importlib.import_module(\"autonerves.setup_colab\")\n", - "\n", - "_setup_colab.setup(\"howtolens\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ "__Adaptive source reconstruction__\n", "\n", "In **Fit Problems**, we identified two limitations of our reconstruction: too few mesh points across compact source structure, and a single regularization coefficient that must compromise between the bright centre and faint outskirts.\n", @@ -108,18 +65,51 @@ "- Comparing reconstructions on the same dataset.\n", "- Performing a lens-model fit with an adaptive pixelization.\n", "\n", - "---" + "---\n", + "\n", + "__Returning to the compact-source problem__\n", + "\n", + "We return to the very compact source from **Fit Problems**. Keeping the same example lets us examine whether the adaptive methods address the difficulties we have just investigated.\n", + "\n", + "The setup below reproduces that dataset, including its lens model, PSF and Gaussian noise realization. This makes the tutorial runnable independently." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "__Returning to the compact-source problem__\n", + "__Google Colab Setup__\n", "\n", - "We return to the very compact source from **Fit Problems**. Keeping the same example lets us examine whether the adaptive methods address the difficulties we have just investigated.\n", + "Set up Google Colab as in [the introductory notebook](https://github.com/PyAutoLabs/HowToLens/blob/main/notebooks/chapter_1_introduction/tutorial_1_grids_and_galaxies.ipynb).\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "cellView": "form", + "jupyter": { + "source_hidden": true + } + }, + "outputs": [], + "source": [ + "# @title Set up Google Colab { display-mode: \"form\" }\n", + "try:\n", + " import google.colab\n", + "except ImportError:\n", + " from autolens import setup_colab as _setup_colab\n", + "else:\n", + " import importlib\n", + " import subprocess\n", + " import sys\n", "\n", - "The setup below reproduces that dataset, including its lens model, PSF and Gaussian noise realization. This makes the tutorial runnable independently." + " subprocess.check_call(\n", + " [sys.executable, \"-m\", \"pip\", \"install\", \"autonerves\", \"--no-deps\"]\n", + " )\n", + " _setup_colab = importlib.import_module(\"autonerves.setup_colab\")\n", + "\n", + "_setup_colab.setup(\"howtolens\")" ] }, { @@ -228,13 +218,8 @@ "\n", "We know the simulated source profile, but we will not use its brightness distribution to tell the adaptive mesh where to place its points. Brightness adaptation will use a fitted model image, as it would for observed data.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__From an image mesh to a source mesh__\n", "\n", "Previously, we constructed a rectangular mesh directly in the source plane.\n", @@ -311,13 +296,8 @@ "\n", "That is why magnification-based sampling is useful but does not completely address the problem from the previous tutorial.\n", "\n", - "Next, we will connect the traced points into a Delaunay triangulation and use it to reconstruct the source. Border relocation will remain enabled when constructing the fit; the plot above shows the direct ray tracing before that treatment." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "Next, we will connect the traced points into a Delaunay triangulation and use it to reconstruct the source. Border relocation will remain enabled when constructing the fit; the plot above shows the direct ray tracing before that treatment.\n", + "\n", "__Constructing the Delaunay source mesh__\n", "\n", "We now connect the traced mesh points into a **Delaunay triangulation**.\n", @@ -396,13 +376,8 @@ "\n", "The coefficient of `1.0` is only a starting value. We should not judge whether this mesh improves the fit until we have explored its regularization.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Interpolating within a triangle__\n", "\n", "How does a triangular mesh predict the brightness at a ray-traced coordinate?\n", @@ -468,13 +443,8 @@ "\n", "Interpolation still does not determine the vertex brightnesses. Those are solved by the inversion, with regularization controlling how freely they can vary.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Regularization on an irregular mesh__\n", "\n", "We begin with the same `Constant` regularization scheme used earlier.\n", @@ -546,13 +516,8 @@ "\n", "We have changed the mesh geometry, but two limitations remain. The overlay does not respond to the source brightness, and one coefficient still controls regularization everywhere.\n", "\n", - "Next, we will use a fitted image to guide where the image-mesh points are placed, concentrating more sampling on the source emission." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "Next, we will use a fitted image to guide where the image-mesh points are placed, concentrating more sampling on the source emission.\n", + "\n", "__Building an adapt image__\n", "\n", "Our overlay responds to the lensing geometry, but it does not distinguish bright arcs from regions containing only background noise.\n", @@ -591,13 +556,8 @@ "\n", "We will keep this adapt image fixed while comparing mesh settings. It guides the placement of points; it does not fix the brightness values reconstructed on the new mesh.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Placing points with a Hilbert image mesh__\n", "\n", "We now replace the evenly spaced overlay with a **Hilbert image mesh**.\n", @@ -682,13 +642,8 @@ "\n", "Next, let\u2019s examine how its weights are calculated.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Controlling the brightness weighting__\n", "\n", "Two parameters control how the adapt image becomes a weight map: `weight_power` and `weight_floor`.\n", @@ -760,13 +715,8 @@ "\n", "The weight floor helps moderate this effect. Here, every location has a weight of at least `0.05`. This is a minimum weight, not a statement that five per cent of the mesh points must lie in the background.\n", "\n", - "Next, we will vary that floor and reconstruct the source with the Hilbert mesh, checking the residuals and evidence rather than judging the mesh by its appearance alone." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "Next, we will vary that floor and reconstruct the source with the Hilbert mesh, checking the residuals and evidence rather than judging the mesh by its appearance alone.\n", + "\n", "__Keeping resolution in the faint regions__\n", "\n", "Increasing `weight_power` concentrates points on the brightest parts of the adapt image. However, the source may also have faint extended emission that needs to be reconstructed.\n", @@ -832,13 +782,8 @@ "\n", "For our first reconstruction, we return to the mesh already constructed with `weight_power=1.0` and `weight_floor=0.05`. These are starting choices to investigate, rather than settings we have established as optimal.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Reconstructing with the Hilbert mesh__\n", "\n", "We now use the Hilbert coordinates to build a Delaunay source mesh.\n", @@ -918,13 +863,8 @@ "\n", "We have initially reused the overlay\u2019s preferred coefficient. As tutorial 2 showed, changing the mesh can change the preferred smoothing strength. We therefore need to explore regularization again before comparing the two reconstructions.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Comparing overlay and Hilbert reconstructions__\n", "\n", "Let\u2019s scan the regularization coefficient for the Hilbert mesh, using the same range as the overlay.\n", @@ -1027,13 +967,8 @@ "\n", "This tests one Hilbert weighting choice, using a fixed adapt image. It does not establish that every Hilbert mesh outperforms every overlay.\n", "\n", - "We have now addressed where the source points are placed. Next, we will address how to regularize their brightnesses on an irregular mesh, before allowing the smoothing strength to vary between bright and faint regions." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "We have now addressed where the source points are placed. Next, we will address how to regularize their brightnesses on an irregular mesh, before allowing the smoothing strength to vary between bright and faint regions.\n", + "\n", "__Smoothing on an irregular mesh__\n", "\n", "Our Delaunay mesh has vertices with different neighbour counts, edge lengths and directions.\n", @@ -1055,13 +990,8 @@ "\n", "We will first examine `ConstantSplit`, then introduce the brightness-dependent coefficients.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Reconstructing with ConstantSplit__\n", "\n", "Let\u2019s keep the Hilbert mesh and adapt image fixed and change the regularization scheme.\n", @@ -1168,13 +1098,8 @@ "\n", "Split changes how local brightness variations are penalized, but `ConstantSplit` still applies one coefficient everywhere. It therefore retains the centre-versus-outskirts compromise identified in **Fit Problems**.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Allowing different smoothing strengths__\n", "\n", "We now let the regularization strength depend on the source brightness.\n", @@ -1249,13 +1174,8 @@ "\n", "These weights control the regularization penalty; they are not uncertainties on the reconstructed brightnesses.\n", "\n", - "The trial coefficients were chosen to illustrate the mechanism. We have not yet established that they improve the fit or maximize the evidence. Next, we will examine the transition controlled by `signal_scale` and explore the adaptive coefficients systematically." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "The trial coefficients were chosen to illustrate the mechanism. We have not yet established that they improve the fit or maximize the evidence. Next, we will examine the transition controlled by `signal_scale` and explore the adaptive coefficients systematically.\n", + "\n", "__Controlling the transition with `signal_scale`__\n", "\n", "The inner and outer coefficients set the smoothing strengths at the high- and low-signal ends of the source. How do we assign a strength to the regions between them?\n", @@ -1320,13 +1240,8 @@ "\n", "Also distinguish `signal_scale` from the Hilbert mesh\u2019s `weight_power`. Both transform brightness-derived information, but they control different things: `weight_power` changes **where mesh points are placed**, whereas `signal_scale` changes **how their brightnesses are regularized**.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Exploring the adaptive coefficients__\n", "\n", "We now have three regularization parameters: the inner coefficient, outer coefficient and signal scale.\n", @@ -1408,13 +1323,8 @@ "\n", "For a complete modelling analysis, these parameters can be explored by a non-linear search alongside the lens parameters. We will use that approach in the final fit.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Does adaptive regularization address the fit problem?__\n", "\n", "Let\u2019s inspect the highest-evidence adaptive reconstruction we have evaluated and compare it with `ConstantSplit`.\n", @@ -1506,13 +1416,8 @@ "\n", "If substantial residuals remain, inspect where they occur. Adaptive regularization still cannot supply resolution missing from the mesh, and both adaptive steps depend on the initial adapt image.\n", "\n", - "We have now explored the mesh placement and smoothing separately. Next, we will bring them together in an actual lens-model search, using an initial MGE fit, prior passing and the positions likelihood penalty." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "We have now explored the mesh placement and smoothing separately. Next, we will bring them together in an actual lens-model search, using an initial MGE fit, prior passing and the positions likelihood penalty.\n", + "\n", "__Starting the final modelling chain__\n", "\n", "So far, we have supplied the true lens mass model to isolate the effects of mesh placement and regularization.\n", @@ -1597,13 +1502,8 @@ "\n", "Inspect this result before continuing. Does it explain the multiple-image configuration, and does it recover a sensible compact source? We should not use a misleading initial solution to guide the adaptive fit.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Preparing adaptation and the positions likelihood penalty__\n", "\n", "We now derive a new adapt image from the fitted MGE result. This replaces the image used in our fixed-mass experiments.\n", @@ -1724,13 +1624,8 @@ "source": [ "Inspect both the mesh placement and predicted image positions. They should follow the source emission and multiple-image configuration before we use them in the next fit.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Defining the adaptive Delaunay model__\n", "\n", "We now replace the MGE source with a Delaunay pixelization and adaptive Split regularization.\n", @@ -1802,13 +1697,8 @@ "\n", "The regularization priors are starting ranges. After fitting, we will check whether the inferred parameters approach their boundaries and whether broader ranges or further exploration are needed.\n", "\n", - "We are now ready to run the adaptive Delaunay fit and inspect the resulting mass model, source reconstruction and smoothing pattern." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "We are now ready to run the adaptive Delaunay fit and inspect the resulting mass model, source reconstruction and smoothing pattern.\n", + "\n", "__Running the adaptive Delaunay fit__\n", "\n", "The second search combines our Delaunay model, adaptation inputs and positions likelihood penalty.\n", @@ -1862,13 +1752,8 @@ "\n", "Then examine the parameter distributions. A satisfactory-looking best fit does not establish that the search has explored the relevant solutions or measured reliable uncertainties. Check for parameters approaching prior boundaries and use the search diagnostics introduced in Chapter 2.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Inspecting the source and smoothing pattern__\n", "\n", "Let\u2019s look more closely at where the final mesh places its vertices and how strongly those vertices are regularized." @@ -1990,13 +1875,8 @@ "\n", "The additional spatial freedom becomes particularly useful for irregular sources whose structure is difficult to describe using a prescribed arrangement of analytic components.\n", "\n", - "---" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "---\n", + "\n", "__Wrap Up__\n", "\n", "We began this chapter by replacing an analytic source profile with brightness values on a mesh.\n", @@ -2023,13 +1903,8 @@ "\n", "MGE models remain valuable for initial fits, foreground light and sources well described by analytic components. Pixelizations complement them when we need greater freedom to reconstruct the source.\n", "\n", - "We now have the tools to use both together\u2014and to investigate why a reconstruction succeeds or struggles, rather than relying only on how its source plot looks." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "We now have the tools to use both together\u2014and to investigate why a reconstruction succeeds or struggles, rather than relying only on how its source plot looks.\n", + "\n", "**\ud83e\udd16 Ask AI:** How do Hilbert mesh weighting and adaptive Split regularization address different limitations of a compact-source reconstruction?" ] } diff --git a/notebooks/chapter_4_scaling_up_lensing/tutorial_4_group_scale.ipynb b/notebooks/chapter_4_scaling_up_lensing/tutorial_4_group_scale.ipynb index c480243..0b2cae2 100644 --- a/notebooks/chapter_4_scaling_up_lensing/tutorial_4_group_scale.ipynb +++ b/notebooks/chapter_4_scaling_up_lensing/tutorial_4_group_scale.ipynb @@ -647,13 +647,8 @@ "metadata": {}, "source": [ "Adaptive over-sampling was configured around every galaxy centre before the initial fit evaluations. The same\n", - "configuration is retained for the non-linear search." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "configuration is retained for the non-linear search.\n", + "\n", "We first fit the galaxy-only model using `Nautilus` and the familiar `AnalysisImaging` object.\n", "\n", "The likelihood calculation uses the same imaging framework as in earlier chapters, now with the central galaxy\n", diff --git a/scripts/chapter_1_introduction/tutorial_0_visualization.py b/scripts/chapter_1_introduction/tutorial_0_visualization.py index d3ce9dd..c13e293 100644 --- a/scripts/chapter_1_introduction/tutorial_0_visualization.py +++ b/scripts/chapter_1_introduction/tutorial_0_visualization.py @@ -135,9 +135,7 @@ **🤖 Ask AI:** Why can logarithmic colour scaling make faint structure easier to see without increasing its signal-to-noise ratio? -""" -""" __Overlays__ Overlays place reference curves or positions on an image. The `lines=` argument draws curves, while diff --git a/scripts/chapter_1_introduction/tutorial_2_ray_tracing.py b/scripts/chapter_1_introduction/tutorial_2_ray_tracing.py index 6292e31..5228868 100644 --- a/scripts/chapter_1_introduction/tutorial_2_ray_tracing.py +++ b/scripts/chapter_1_introduction/tutorial_2_ray_tracing.py @@ -733,9 +733,7 @@ Tutorial 3 explains the cosmological distances behind this conversion, as part of its optional mathematical discussion. -""" -""" __More Complexity__ Our tracer currently contains one spherical lens and one source. @@ -962,9 +960,7 @@ and caustics. You can follow its physical explanations and figures without working through every equation, and return to the mathematics when you need it. -""" -""" __Env__ Keep the full 101-by-101 grid during automated runs so that the diff --git a/scripts/chapter_1_introduction/tutorial_3_more_ray_tracing.py b/scripts/chapter_1_introduction/tutorial_3_more_ray_tracing.py index 2de94bd..f5e5f00 100644 --- a/scripts/chapter_1_introduction/tutorial_3_more_ray_tracing.py +++ b/scripts/chapter_1_introduction/tutorial_3_more_ray_tracing.py @@ -174,9 +174,7 @@ from an image-plane position to obtain its source-plane position, as we did in tutorial 2. We will continue to call them "deflection angles" throughout this tutorial. -""" -r""" __Convergence__ Imagine adding up the lens galaxy's mass along our line of sight. @@ -248,9 +246,7 @@ This is an average over the enclosed area, rather than the convergence at its edge. We will return to this connection when we use the Einstein radius to measure the lens's mass. -""" -r""" __Deflection Angles__ The convergence describes the projected mass distribution. @@ -299,13 +295,11 @@ array=deflections_x, title="Deflection Angles (x component)", cb_unit=" arcsec" ) -""" +r""" Each component changes sign across the lens centre. Together, they describe the displacement we subtract from each image-plane coordinate when tracing it to the source plane. -""" -r""" __The Lensing Potential__ We can describe the same lens using a single scalar field: @@ -353,13 +347,11 @@ aplt.plot_array(array=potential, title="Lensing Potential", cb_unit=r" arcsec$^2$") -""" +r""" For this spherical lens, the potential has circular contours. Its slope points radially outwards, matching the direction of the deflection vectors used in our lens equation. -""" -r""" We can check $\boldsymbol{\alpha} = \nabla\psi$ directly. First, we calculate the deflections using the mass profile's @@ -791,14 +783,12 @@ print(f"Einstein radius: {einstein_radius:.4f} arcsec") print(f"Einstein radius: {einstein_radius_kpc:.4f} kpc") -""" +r""" The recovered angular radius should be close to the 1.6 arcseconds specified in our spherical mass profile. Small differences arise from locating the critical curve on a finite grid. -""" -r""" Two simple mass distributions have useful expressions for the Einstein radius. @@ -831,9 +821,7 @@ or velocity dispersion to the observer–lens–source geometry. Our `IsothermalSph` profile specifies this angular scale directly through its `einstein_radius` parameter. -""" -r""" For a circular lens, the mean convergence inside the Einstein radius equals one. Its enclosed projected mass is therefore @@ -857,7 +845,7 @@ f"{einstein_mass_solar:.3e} solar masses" ) -""" +r""" This is one of strong lensing's most useful measurements. The image positions constrain the Einstein scale, often giving a tighter constraint on the enclosed projected mass @@ -867,9 +855,7 @@ a useful summary of the lensing scale, but the circular enclosed-mass relation is not generally exact. The detailed mass model and surrounding matter still matter. -""" -r""" __Time Delays__ Different images of the same source follow different paths @@ -934,9 +920,7 @@ at different times. Measuring those delays provides information about cosmological distances as well as the lens's mass distribution. -""" -""" __Wrap Up__ The quantities in this tutorial describe different aspects @@ -972,9 +956,7 @@ In tutorial 4, we turn to point sources, whose individual lensed images provide another way to study a lens. -""" -""" __Env__ Keep the full 250-by-250 grid during automated runs so that diff --git a/scripts/chapter_1_introduction/tutorial_4_point_sources.py b/scripts/chapter_1_introduction/tutorial_4_point_sources.py index 2343aa0..b24b0b2 100644 --- a/scripts/chapter_1_introduction/tutorial_4_point_sources.py +++ b/scripts/chapter_1_introduction/tutorial_4_point_sources.py @@ -112,9 +112,7 @@ A quasar or supernova can also sit inside a visible host galaxy. Such a lens may therefore contain both unresolved point images and extended arcs. -""" -""" __Point Source Tracer__ PyAutoLens represents a point source using the `Point` profile in `al.ps`. This profile specifies its @@ -132,16 +130,14 @@ tracer = al.Tracer(galaxies=[lens_galaxy, source_galaxy]) -""" +r""" The name `point_0` identifies this source within the galaxy. When fitting observations, PyAutoLens uses it to match the source to the dataset containing its multiple images. This becomes particularly useful when a lens contains several background sources. This tracer now contains the lens mass and the point-source position. Next, we will find the multiple image positions that this combination predicts. -""" -r""" __The Lens Equation__ In Tutorial 2, we used the lens equation to map an image-plane coordinate $\theta$ to a source-plane coordinate @@ -160,9 +156,7 @@ Finding these solutions is harder than evaluating the equation at a known coordinate. The deflection $\alpha(\theta)$ changes with position, and several different coordinates can map to the same source. Simple mass profiles have analytic solutions, but more general models require a numerical search. -""" -""" __Point Solver__ The `PointSolver` divides the image-plane into triangles and ray-traces their corners to the source-plane. It @@ -213,9 +207,7 @@ Some mass profiles also produce a faint image near the lens centre. Our singular isothermal profile does not produce a finite central image, so we should not interpret the magnification threshold as removing a fifth image in this example. -""" -""" __Multiple Images and Critical Curves__ Let's overlay the solved positions on the compact extended source's image. The markers show where a point @@ -236,16 +228,14 @@ title="Multiple Images and Critical Curve", ) -""" +r""" Compare the markers with the bright regions of the extended image. Both calculations use the same lens mass and source centre: one evaluates an extended light profile across a grid, while the other finds the discrete image positions of a point source. **🤖 Ask AI:** For a fixed elliptical lens mass profile, why can moving a point source across a caustic change the number of its multiple images? -""" -r""" __Magnifications__ Each multiple image has its own magnification. For an unresolved image, we measure this through its total diff --git a/scripts/chapter_1_introduction/tutorial_5_data.py b/scripts/chapter_1_introduction/tutorial_5_data.py index 4ad21eb..66d20aa 100644 --- a/scripts/chapter_1_introduction/tutorial_5_data.py +++ b/scripts/chapter_1_introduction/tutorial_5_data.py @@ -191,9 +191,7 @@ """ **🤖 Ask AI:** How does blurring by a telescope's PSF affect our ability to distinguish two nearby features in a gravitationally lensed arc? -""" -""" __Poisson Noise__ Even if a galaxy's brightness remains constant, repeated observations do not @@ -360,9 +358,7 @@ """ **🤖 Ask AI:** Why does subtracting the mean background sky from an astronomical image leave the sky's Poisson noise behind? -""" -""" The dataset also stores the PSF used to simulate the observation. For real observations, we need an estimate of the PSF, which can come from @@ -530,9 +526,7 @@ and their uncertainties. For further examples of working with these data, see the workspace's [point-source examples](https://github.com/PyAutoLabs/autolens_workspace/blob/main/scripts/point_source/start_here.py). -""" -""" __Interferometer Data__ We can also observe strong lenses at radio and submillimetre wavelengths using @@ -591,9 +585,7 @@ The dataset retains the visibilities for analysis. For further examples of working with these data, see the workspace's [interferometer examples](https://github.com/PyAutoLabs/autolens_workspace/tree/main/scripts/interferometer). -""" -""" __Weak Lensing Data__ Away from the prominent arcs and rings, gravitational lensing can produce @@ -634,9 +626,7 @@ Chapter 4 returns to weak lensing in more detail. For further examples of working with these data, see the workspace's [weak-lensing examples](https://github.com/PyAutoLabs/autolens_workspace/blob/main/scripts/weak/start_here.py). -""" -""" __Group and Cluster Lenses__ Strong lensing also occurs around groups and clusters of galaxies. Their diff --git a/scripts/chapter_1_introduction/tutorial_6_fitting.py b/scripts/chapter_1_introduction/tutorial_6_fitting.py index 3d7f344..055e371 100644 --- a/scripts/chapter_1_introduction/tutorial_6_fitting.py +++ b/scripts/chapter_1_introduction/tutorial_6_fitting.py @@ -349,7 +349,7 @@ aplt.plot_array(array=fit.residual_map, title="Residual Map") -""" +r""" Because we used the parameters that generated the simulation, we expect the residuals to be dominated by the noise in the data. @@ -362,9 +362,7 @@ **🤖 Ask AI:** Why can a correct lens model leave larger absolute residuals in bright arcs than in the faint background? -""" -r""" The size of a residual only becomes informative when compared with the uncertainty in that pixel. A difference of 0.1 electrons per second could be small in a noisy pixel but substantial in a precisely measured one. @@ -663,9 +661,7 @@ **🤖 Ask AI:** Why might the parameters that maximise the likelihood of a noisy simulated image differ slightly from those used to generate it? -""" -""" __Model Fitting__ So far, we have compared parameter choices with a known reference model. @@ -740,7 +736,7 @@ print("Log Likelihood:") print(fit.log_likelihood) -""" +r""" Try changing one parameter at a time and rerunning the cell above. Record the log likelihood for each choice and use the residual plots to understand what changed. @@ -770,9 +766,7 @@ To restore the original starting point, set both centres to `(1.0, 1.0)`, and set `einstein_radius`, `effective_radius` and `intensity` to 1.0, then rerun the cell. -""" -""" You have now performed a simple manual search: choose parameters, predict an image, compare it with the data, then use that comparison to choose another set of parameters. @@ -791,9 +785,7 @@ **🤖 Ask AI:** How could changing both the source size and its intensity make it difficult to determine either parameter by adjusting them manually? -""" -r""" __Likelihood Function__ Everything we have learned can now be put into a single sequence: diff --git a/scripts/chapter_1_introduction/tutorial_7_summary.py b/scripts/chapter_1_introduction/tutorial_7_summary.py index 9d4142c..166d4ab 100644 --- a/scripts/chapter_1_introduction/tutorial_7_summary.py +++ b/scripts/chapter_1_introduction/tutorial_7_summary.py @@ -150,9 +150,7 @@ """ **🤖 Ask AI:** Why does evaluating a source light profile on ray-traced source-plane coordinates produce a lensed image, whereas evaluating it on a uniform source-plane grid produces an unlensed image? -""" -""" __Visualization__ We can combine the lensed image with other quantities calculated from the same tracer. @@ -189,9 +187,7 @@ As your analyses become more complex, this same structure lets you add components and inspect their individual contributions. -""" -""" __Source Code__ The tools used in **HowToLens** are provided by four related projects: @@ -210,9 +206,7 @@ [Installation from source](https://pyautolens.readthedocs.io/en/latest/installation/source.html) is described in the documentation. This is useful if you want to modify or contribute to the libraries; it is optional for continuing **HowToLens**. -""" -""" __Wrap Up__ You have now followed the path from a mathematical description of a lens system to a predicted observation and a diff --git a/scripts/chapter_3_pixelizations/tutorial_2_bayesian_regularization.py b/scripts/chapter_3_pixelizations/tutorial_2_bayesian_regularization.py index 8f634ad..ce95f49 100644 --- a/scripts/chapter_3_pixelizations/tutorial_2_bayesian_regularization.py +++ b/scripts/chapter_3_pixelizations/tutorial_2_bayesian_regularization.py @@ -299,9 +299,7 @@ def fit_with_coefficient(coefficient): **🤖 Ask AI:** Why might a weakly regularized reconstruction have a lower chi-squared but a less trustworthy source than a more strongly regularized reconstruction? -""" -""" __Bayesian evidence__ Choosing the regularization coefficient is a Goldilocks problem. Too little regularization gives the source enough freedom to fit the image’s noise. Too much regularization smooths away source structure that we need to explain the image. We want an amount of regularization that is **“just right”**. @@ -369,9 +367,7 @@ def fit_with_coefficient(coefficient): Compare this reconstruction with the weakly and strongly regularized examples. How much of the irregular source structure has been smoothed away? Are the bright components still reconstructed accurately? What remains in the image residuals? We have selected the best of **three coefficients**, rather than found the overall maximum. A coefficient between our trial values could give a higher log evidence. Next, we will evaluate a wider range and plot the evidence against the coefficient to locate its peak. -""" -""" __Exploring the regularization coefficient__ Our three examples illustrate too little, moderate and too much regularization. However, there is nothing special about the starting coefficient of `1.0`. A value of `0.5`, `2.0`, or something else entirely might give a higher log evidence. @@ -435,9 +431,7 @@ def fit_with_coefficient(coefficient): Look for the peak of the curve. Towards small coefficients, the source has more freedom to fit noise. Towards large coefficients, smoothing increasingly prevents it from reproducing the source structure. The peak identifies our Goldilocks coefficient **among the values we have tested**. If the highest point lies at either end of the range, we should extend the range in that direction. We have not yet located a peak between weaker and stronger regularization. -""" -""" __Inspecting the preferred reconstruction__ Let’s take the coefficient with the highest log evidence and look at its source reconstruction and image residuals. @@ -475,9 +469,7 @@ def fit_with_coefficient(coefficient): There is also a compromise in applying one coefficient across the whole source. The bright centre may need weaker smoothing to preserve its steep brightness gradient, while the faint outskirts benefit from stronger smoothing. A single coefficient must serve both regions. Keep these limitations in mind. Our later **Fit Problems** tutorial will show why choosing the coefficient is only part of the solution, motivating meshes and regularization that adapt to the source. -""" -""" __What is linear, and what requires a search?__ Our reconstruction contains hundreds of source brightness values. Does this mean we have added hundreds of parameters to the non-linear search? @@ -511,9 +503,7 @@ def fit_with_coefficient(coefficient): 4. We calculate the log evidence, which the search uses to assess that proposal. This is what makes pixelized sources practical. We can reconstruct hundreds of source brightness values without asking the non-linear search to explore hundreds of additional parameters. -""" -""" __Mesh resolution and regularization belong together__ We have found a preferred regularization coefficient for our current mesh. Would the same coefficient still be preferred if we changed its resolution? @@ -593,9 +583,7 @@ def fit_with_coefficient(coefficient, mesh_shape=mesh_shape): A finer mesh may increase the evidence by resolving source structure that the coarser mesh cannot describe. However, more source pixels do not automatically mean a better reconstruction: the additional freedom must be justified by the image data. Before drawing a conclusion, check that each peak lies inside the coefficient range. If a curve is still rising at either end, extend its range. Once a peak is bracketed, try more closely spaced coefficients around it to refine the comparison. -""" -""" __Wrap Up__ We can now explain the regularization coefficient that we first supplied without much discussion in tutorial 1. @@ -624,8 +612,6 @@ def fit_with_coefficient(coefficient, mesh_shape=mesh_shape): Before that, we need to examine another practical issue. Choosing the number of source pixels does not, by itself, determine our source resolution. We also need to consider **how much area those pixels cover**. --- -""" -""" For the underlying equations, the workspace’s [pixelization likelihood guide](https://github.com/PyAutoLabs/autolens_workspace/blob/main/scripts/imaging/features/pixelization/likelihood_function.py) follows the mapping, blurring, regularization and evidence calculations step by step. """ diff --git a/scripts/chapter_3_pixelizations/tutorial_3_borders.py b/scripts/chapter_3_pixelizations/tutorial_3_borders.py index e5ba4c6..803bbca 100644 --- a/scripts/chapter_3_pixelizations/tutorial_3_borders.py +++ b/scripts/chapter_3_pixelizations/tutorial_3_borders.py @@ -16,9 +16,7 @@ - What relocation changes - The mask still needs to cover the lensed source - Wrap Up -""" -""" __Why the extent of the source mesh matters__ Imagine spreading a `(25, 25)` rectangular mesh across a source-plane region one arcsecond wide. Now spread the same mesh across a region ten arcseconds wide. We still have the same number of brightness values, but the spacing between them is ten times larger. Small source components that the first mesh could resolve may be poorly represented by the second. @@ -67,9 +65,7 @@ We will compare two masks with the same outer radius. One includes the central image pixels; the other removes them. This lets us isolate how including the centre changes the area covered by the source mesh. --- -""" -""" __What we mean by a border__ Here, the **border** means the outermost boundary of the unmasked image region: the ring of image pixels just inside the mask’s outer edge. @@ -113,9 +109,7 @@ We can ray-trace the marked border pixels just as we ray-trace any other image pixels. Their traced positions define a border in the **source plane**, which we will use to identify coordinates that land beyond it. Keep these two operations separate. **Masking** determines which image pixels we fit. **Border relocation** changes how coordinates beyond the traced border enter the source reconstruction; it does not remove those image pixels from the fit. -""" -""" __Starting without border relocation__ Let’s first reconstruct the source using the annular mask, which excludes the central image pixels. We explicitly switch border relocation off so that we can inspect the original ray-traced coordinates. @@ -202,9 +196,7 @@ def fit_with_mask_and_border(mask, use_border_relocator): Pay attention to the axis ranges and the spacing of the mesh points. These provide our reference for the next comparison. --- -""" -""" __Including the central image pixels__ Now repeat the reconstruction with the circular mask. @@ -258,9 +250,7 @@ def fit_with_mask_and_border(mask, use_border_relocator): The masks contain different sets of image pixels, so we should not rank these two fits by their absolute log evidences. For now, compare their geometry and source reconstructions. --- -""" -""" __Following the central pixels into the source plane__ Let’s identify exactly where those distant source-plane coordinates came from. @@ -334,9 +324,7 @@ def fit_with_mask_and_border(mask, use_border_relocator): There is another potential problem: assigning source brightness at these distant positions may predict additional images outside our mask. Those predictions would not be checked against the excluded image data. Next, we will trace the outer image border into the source plane and use it to control these distant coordinates. -""" -""" __Tracing the outer border__ We have seen that pixels near the image centre can trace far beyond the source galaxy. To decide which coordinates to relocate, we first need a boundary in the source plane. @@ -368,9 +356,7 @@ def fit_with_mask_and_border(mask, use_border_relocator): The border relocator uses the traced border to estimate a source-plane boundary. Coordinates beyond that boundary are moved inward for the reconstruction, preventing them from forcing the mesh to extend far beyond the region we want to reconstruct. --- -""" -""" __Switching border relocation on__ Let’s repeat the circular-mask fit with border relocation enabled. @@ -444,9 +430,7 @@ def fit_with_mask_and_border(mask, use_border_relocator): These coordinates still correspond to the same image pixels and measured brightnesses. We have changed where they map within the source reconstruction. We have not moved any observed image pixels or excluded them from the fit. --- -""" -""" __What relocation changes__ Does relocating these coordinates mean we have changed the lens model? @@ -522,9 +506,7 @@ def fit_with_mask_and_border(mask, use_border_relocator): We have kept the regularization coefficient fixed to show the effect of switching relocation on. As we learned in tutorial 2, changing the mesh geometry can also change the preferred coefficient. A full comparison would therefore repeat the regularization search for each setup. Next, we will examine why border relocation still requires a mask that covers the lensed source. -""" -""" __The mask still needs to cover the lensed source__ Border relocation helps us control the source mesh, but it cannot compensate for an image mask that excludes part of the lensed source. @@ -588,9 +570,7 @@ def fit_with_mask_and_border(mask, use_border_relocator): A larger mask can also spread the fixed number of mesh points over a larger source-plane region. If the reconstruction changes, inspect the mesh spacing as well as the newly included data. --- -""" -""" __Wrap Up__ The resolution of a rectangular source mesh depends on both its number of points and the area they cover. @@ -626,8 +606,6 @@ def fit_with_mask_and_border(mask, use_border_relocator): Next, we will let the mass model vary. This introduces another challenge: a flexible source reconstruction can sometimes produce a close image fit even when the lens mass model is wrong. --- -""" -""" **🤖 Ask AI:** Why can adding central image pixels reduce the useful resolution of a rectangular source mesh, and what does border relocation change? """ diff --git a/scripts/chapter_3_pixelizations/tutorial_4_lens_modeling.py b/scripts/chapter_3_pixelizations/tutorial_4_lens_modeling.py index e3ab906..f7caf1b 100644 --- a/scripts/chapter_3_pixelizations/tutorial_4_lens_modeling.py +++ b/scripts/chapter_3_pixelizations/tutorial_4_lens_modeling.py @@ -19,9 +19,7 @@ - Establishing the foreground light model - Using that result in the pixelized-source fit - Wrap Up -""" -""" __Lens modelling with a pixelized source__ In the previous tutorials, we supplied the mass model used to simulate the image. This let us concentrate on reconstructing the source. @@ -82,9 +80,7 @@ We use one sample per image pixel to make the mapping demonstrations easy to follow. Before interpreting a scientific fit, we would check that finer sampling does not materially change the result. First, we will deliberately give the inversion the wrong Einstein radius and inspect how it attempts to explain the image. -""" -""" __Giving the inversion an incorrect mass model__ Let’s deliberately give the inversion an Einstein radius of `0.8` arcsec—half the value used to simulate the image. @@ -153,9 +149,7 @@ def fit_at_einstein_radius(einstein_radius, coefficient=1.0): This differs from the demagnified central pixels in the borders tutorial. There, we investigated individual coordinates that stretched the mesh. Here, we have an alternative interpretation of the whole lens system. Border relocation is enabled, but it does not prevent this solution. --- -""" -""" __Comparing with the correct mass model__ Now restore the Einstein radius to `1.6` arcsec and reconstruct the source again. @@ -217,9 +211,7 @@ def fit_at_einstein_radius(einstein_radius, coefficient=1.0): We have evaluated only two mass models at one regularization coefficient. As tutorial 2 showed, the preferred coefficient can change when the reconstruction changes. These numbers are therefore a first comparison, rather than the outcome of a full search over the mass and regularization parameters. --- -""" -""" __Seeing how the mass model changes the mappings__ The source reconstructions show the result. Let’s now inspect the mappings that produce it. @@ -272,9 +264,7 @@ def fit_at_einstein_radius(einstein_radius, coefficient=1.0): Later, we will identify positions that **do** correspond to the same source feature and require them to trace close together. That gives us a way to steer the search away from demagnified solutions. First, we will examine why calculating the evidence does not, by itself, ensure that a search finds the correct mass model. -""" -""" __Why evidence does not remove the search difficulty__ Suppose the correct mass model gives a higher log evidence than our incorrect example. Does that mean a non-linear search will automatically find it? @@ -344,9 +334,7 @@ def fit_at_einstein_radius(einstein_radius, coefficient=1.0): This one-dimensional scan greatly understates the difficulty of the full search. We have fixed the mass ellipticity, shear and regularization coefficient, whereas a real search varies them together. The resulting parameter space can contain many competing maxima that are absent from this simple slice. This is why we begin with an MGE source and use search chaining. Its restricted geometry makes it harder to compensate for an incorrect mass model by reconstructing arbitrary source structure. We first establish a plausible lensing geometry, then introduce the pixelization and additional constraints to help keep the search away from unphysical solutions. -""" -""" __Starting with a simpler source model__ We will begin with the multi-Gaussian expansion (MGE) from Chapter 2. @@ -408,9 +396,7 @@ def fit_at_einstein_radius(einstein_radius, coefficient=1.0): Inspect `model_1.info` to see these parameters and their priors. Although we know the simulated Einstein radius is `1.6` arcsec, we have not supplied that answer to the search: it must infer the radius from the image. --- -""" -""" __Running and inspecting the initial search__ We are now ready to run the first search. @@ -451,9 +437,7 @@ def fit_at_einstein_radius(einstein_radius, coefficient=1.0): Finally, remember that the plotted maximum-likelihood fit is one model. Use the result summary and parameter distributions, as in Chapter 2, to assess the constraints and whether further exploration is needed. Next, we will use corresponding image positions to help the pixelized-source search avoid demagnified solutions. -""" -""" __Requiring corresponding images to trace together__ In Chapter 2, Tutorial 5, we introduced positions resampling and the **positions likelihood penalty**. We now use that same approach to help the pixelized-source search avoid the unphysical solutions discussed above. @@ -507,9 +491,7 @@ def fit_at_einstein_radius(einstein_radius, coefficient=1.0): The purpose is to exclude grossly inconsistent mappings while allowing useful changes to the mass model. An excessively tight threshold could instead force the second search to remain too close to an imperfect initial fit. --- -""" -""" __Replacing the analytic source with a pixelization__ We are ready to construct the second model. @@ -558,9 +540,7 @@ def fit_at_einstein_radius(einstein_radius, coefficient=1.0): Inspect the priors printed by `model_2.info`. The initial MGE may have left source structure unexplained, potentially shifting its preferred mass model. Prior passing should guide the pixelized search without restricting it so tightly that it cannot correct that initial estimate. --- -""" -""" __Running the pixelized-source search__ We now combine the pixelized model with the **positions likelihood penalty**, following the same procedure as Chapter 2, Tutorial 5. @@ -627,9 +607,7 @@ def fit_at_einstein_radius(einstein_radius, coefficient=1.0): Passing this check confirms that these selected positions trace sufficiently close together. We still need to inspect the image residuals, parameter constraints and search behaviour to assess the overall fit. We have now fitted a free mass model and a pixelized source using search chaining. Next, we will include the foreground lens galaxy’s light and examine how it is fitted alongside the source reconstruction. -""" -""" __Establishing the foreground light model__ So far, our images have contained only the lensed source. In most observed systems, the foreground lens galaxy also contributes light, often overlapping the arcs. @@ -764,9 +742,7 @@ def fit_at_einstein_radius(einstein_radius, coefficient=1.0): Inspect the foreground light model, source and residuals. We want the initial model to separate the foreground galaxy from the arcs sensibly. Significant foreground residuals should be investigated before introducing a more flexible source that might absorb them. --- -""" -""" __Using that result in the pixelized-source fit__ Now replace the analytic source with a pixelization. @@ -862,9 +838,7 @@ def fit_at_einstein_radius(einstein_radius, coefficient=1.0): If the fixed foreground geometry leaves significant residuals, we can free its parameters in a later chained search. Holding it fixed is a useful intermediate step, rather than a requirement for the final analysis. --- -""" -""" __Wrap Up__ We have now fitted a lens mass model alongside a pixelized source, both with and without foreground lens light. @@ -886,8 +860,6 @@ def fit_at_einstein_radius(einstein_radius, coefficient=1.0): Even the evidence-preferred constant regularization coefficient must compromise between these demands. In the next tutorial, **Fit Problems**, we will deliberately make the source more compact while keeping the true mass model fixed. This will reveal limitations of the mesh and regularization themselves, and motivate the adaptive methods that follow. -""" -""" **🤖 Ask AI:** Why can an incorrect mass model fit the arcs with a pixelized source, and how do search chaining and the positions likelihood penalty address this? """ diff --git a/scripts/chapter_3_pixelizations/tutorial_5_fit_problems.py b/scripts/chapter_3_pixelizations/tutorial_5_fit_problems.py index 9ff36e7..9f11a63 100644 --- a/scripts/chapter_3_pixelizations/tutorial_5_fit_problems.py +++ b/scripts/chapter_3_pixelizations/tutorial_5_fit_problems.py @@ -18,9 +18,7 @@ - Why one regularization coefficient becomes a compromise - Why maximizing the evidence does not remove the compromise - Wrap Up -""" -""" __When the best coefficient is not enough__ We have learned how to reconstruct a source, choose its regularization coefficient using Bayesian evidence, and fit the lens mass model. @@ -53,9 +51,7 @@ - Motivating adaptation to the source’s brightness distribution. --- -""" -""" __Three progressively compact sources__ We use three Sérsic profiles with the same centre and elliptical shape. @@ -119,9 +115,7 @@ This distinction will become important: the lensing geometry stays the same, but the spatial resolution needed to describe the source’s light changes. --- -""" -""" __Simulating the images__ We now simulate the three lensed images using the same image grid and telescope blurring. @@ -218,9 +212,7 @@ Next, we will reconstruct them using the same rectangular mesh and constant regularization scheme. For each dataset, we will scan the coefficient and inspect its evidence-preferred reconstruction. We will compare alternative reconstructions **within each dataset**. We will not rank the three different simulated galaxies against one another using their absolute evidence values. -""" -""" __Reconstructing the sources__ We will reconstruct all three sources using a `25 × 25` rectangular mesh and `Constant` regularization. @@ -262,9 +254,7 @@ def fit_source(dataset, coefficient): For now, the only reconstruction setting we vary is the regularization coefficient. --- -""" -""" __Finding the preferred regularization__ As in tutorial 2, we evaluate a range of coefficients and compare their log evidences. We do this separately for each dataset: a coefficient suitable for the extended source need not be suitable for either compact source. @@ -348,9 +338,7 @@ def fit_source(dataset, coefficient): We can now inspect the reconstructions knowing that we have selected their smoothing strengths systematically. --- -""" -""" __Starting with the extended source__ Let’s begin with the extended source. @@ -402,9 +390,7 @@ def fit_source(dataset, coefficient): A value around one is a useful reference for noise-level residuals, but it does not tell us whether those residuals form a spatial pattern. Keep the residual map alongside this number. This extended-source fit provides our baseline. Next, we will make the source more compact without giving the mesh any additional resolution, and examine where the reconstruction begins to struggle. -""" -""" __Making the source more compact__ Now inspect the compact source. @@ -454,9 +440,7 @@ def fit_source(dataset, coefficient): The question is therefore no longer simply “Have we chosen the wrong coefficient?” We need to ask whether our mesh and regularization scheme can represent this source adequately. --- -""" -""" __A very compact source__ Our final source has an effective radius of just `0.1` arcsec and a Sérsic index of `4.0`. Its light rises steeply towards the centre, placing a greater demand on the reconstruction. @@ -514,9 +498,7 @@ def fit_source(dataset, coefficient): These galaxies also have different brightness distributions and signal-to-noise ratios. We should not interpret the comparison as a measurement of source-size effects alone. Its purpose is to expose the practical difficulty of reconstructing increasingly concentrated light with an unchanged mesh and one smoothing coefficient. --- -""" -""" __Why the mesh becomes a limitation__ Let’s examine the first problem: where our source pixels are placed. @@ -559,9 +541,7 @@ def fit_source(dataset, coefficient): This motivates adapting the mesh to the **source’s brightness distribution**, which we will explore in the next tutorial. First, there is a second limitation to understand. Even with enough mesh points, should we apply the same regularization coefficient to the bright centre and faint outskirts? -""" -""" __Why one regularization coefficient becomes a compromise__ Suppose we had enough mesh points to resolve the source’s bright centre. Would that solve the problem? @@ -635,9 +615,7 @@ def fit_source(dataset, coefficient): These changes need not be equally visible in every dataset. The important point is that changing one coefficient changes the smoothing preference everywhere, even though different source regions may benefit from different strengths. --- -""" -""" __Why maximizing the evidence does not remove the compromise__ Why does the Bayesian evidence not simply select a coefficient that fixes both regions? @@ -677,9 +655,7 @@ def fit_source(dataset, coefficient): The evidence still decides whether that additional flexibility is useful. We compare the constant and adaptive schemes on the same dataset, allowing their parameters to vary appropriately. Adaptation is a way to improve the available reconstructions, rather than a guarantee that every fit will improve. --- -""" -""" __Wrap Up__ We began with three sources whose light became progressively more concentrated, while keeping the lens mass model fixed to its true values. @@ -702,9 +678,7 @@ def fit_source(dataset, coefficient): At each stage, we will return to the questions raised here: does the mesh resolve the source’s bright structure, and does regularization preserve that structure while controlling fluctuations elsewhere? We will finish by using these methods in a lens-model fit, with the search-chaining procedure and positions likelihood penalty already introduced. -""" -""" **🤖 Ask AI:** How would you distinguish insufficient source-mesh resolution from over-smoothing when compact-source residuals remain at the evidence-preferred coefficient? __Env__ diff --git a/scripts/chapter_3_pixelizations/tutorial_6_adaptive_reconstruction.py b/scripts/chapter_3_pixelizations/tutorial_6_adaptive_reconstruction.py index d9f7645..0cd0f05 100644 --- a/scripts/chapter_3_pixelizations/tutorial_6_adaptive_reconstruction.py +++ b/scripts/chapter_3_pixelizations/tutorial_6_adaptive_reconstruction.py @@ -30,9 +30,7 @@ - Running the adaptive Delaunay fit - Inspecting the source and smoothing pattern - Wrap Up -""" -""" __Adaptive source reconstruction__ In **Fit Problems**, we identified two limitations of our reconstruction: too few mesh points across compact source structure, and a single regularization coefficient that must compromise between the bright centre and faint outskirts. @@ -63,9 +61,7 @@ - Performing a lens-model fit with an adaptive pixelization. --- -""" -""" __Returning to the compact-source problem__ We return to the very compact source from **Fit Problems**. Keeping the same example lets us examine whether the adaptive methods address the difficulties we have just investigated. @@ -169,9 +165,7 @@ We know the simulated source profile, but we will not use its brightness distribution to tell the adaptive mesh where to place its points. Brightness adaptation will use a fitted model image, as it would for observed data. --- -""" -""" __From an image mesh to a source mesh__ Previously, we constructed a rectangular mesh directly in the source plane. @@ -227,9 +221,7 @@ That is why magnification-based sampling is useful but does not completely address the problem from the previous tutorial. Next, we will connect the traced points into a Delaunay triangulation and use it to reconstruct the source. Border relocation will remain enabled when constructing the fit; the plot above shows the direct ray tracing before that treatment. -""" -""" __Constructing the Delaunay source mesh__ We now connect the traced mesh points into a **Delaunay triangulation**. @@ -298,9 +290,7 @@ def fit_delaunay_overlay(coefficient): The coefficient of `1.0` is only a starting value. We should not judge whether this mesh improves the fit until we have explored its regularization. --- -""" -""" __Interpolating within a triangle__ How does a triangular mesh predict the brightness at a ray-traced coordinate? @@ -356,9 +346,7 @@ def fit_delaunay_overlay(coefficient): Interpolation still does not determine the vertex brightnesses. Those are solved by the inversion, with regularization controlling how freely they can vary. --- -""" -""" __Regularization on an irregular mesh__ We begin with the same `Constant` regularization scheme used earlier. @@ -420,9 +408,7 @@ def fit_delaunay_overlay(coefficient): We have changed the mesh geometry, but two limitations remain. The overlay does not respond to the source brightness, and one coefficient still controls regularization everywhere. Next, we will use a fitted image to guide where the image-mesh points are placed, concentrating more sampling on the source emission. -""" -""" __Building an adapt image__ Our overlay responds to the lensing geometry, but it does not distinguish bright arcs from regions containing only background noise. @@ -451,9 +437,7 @@ def fit_delaunay_overlay(coefficient): We will keep this adapt image fixed while comparing mesh settings. It guides the placement of points; it does not fix the brightness values reconstructed on the new mesh. --- -""" -""" __Placing points with a Hilbert image mesh__ We now replace the evenly spaced overlay with a **Hilbert image mesh**. @@ -517,9 +501,7 @@ def fit_delaunay_overlay(coefficient): Next, let’s examine how its weights are calculated. --- -""" -""" __Controlling the brightness weighting__ Two parameters control how the adapt image becomes a weight map: `weight_power` and `weight_floor`. @@ -581,9 +563,7 @@ def fit_delaunay_overlay(coefficient): The weight floor helps moderate this effect. Here, every location has a weight of at least `0.05`. This is a minimum weight, not a statement that five per cent of the mesh points must lie in the background. Next, we will vary that floor and reconstruct the source with the Hilbert mesh, checking the residuals and evidence rather than judging the mesh by its appearance alone. -""" -""" __Keeping resolution in the faint regions__ Increasing `weight_power` concentrates points on the brightest parts of the adapt image. However, the source may also have faint extended emission that needs to be reconstructed. @@ -639,9 +619,7 @@ def fit_delaunay_overlay(coefficient): For our first reconstruction, we return to the mesh already constructed with `weight_power=1.0` and `weight_floor=0.05`. These are starting choices to investigate, rather than settings we have established as optimal. --- -""" -""" __Reconstructing with the Hilbert mesh__ We now use the Hilbert coordinates to build a Delaunay source mesh. @@ -711,9 +689,7 @@ def fit_delaunay_hilbert(coefficient): We have initially reused the overlay’s preferred coefficient. As tutorial 2 showed, changing the mesh can change the preferred smoothing strength. We therefore need to explore regularization again before comparing the two reconstructions. --- -""" -""" __Comparing overlay and Hilbert reconstructions__ Let’s scan the regularization coefficient for the Hilbert mesh, using the same range as the overlay. @@ -795,9 +771,7 @@ def fit_delaunay_hilbert(coefficient): This tests one Hilbert weighting choice, using a fixed adapt image. It does not establish that every Hilbert mesh outperforms every overlay. We have now addressed where the source points are placed. Next, we will address how to regularize their brightnesses on an irregular mesh, before allowing the smoothing strength to vary between bright and faint regions. -""" -""" __Smoothing on an irregular mesh__ Our Delaunay mesh has vertices with different neighbour counts, edge lengths and directions. @@ -820,9 +794,7 @@ def fit_delaunay_hilbert(coefficient): We will first examine `ConstantSplit`, then introduce the brightness-dependent coefficients. --- -""" -""" __Reconstructing with ConstantSplit__ Let’s keep the Hilbert mesh and adapt image fixed and change the regularization scheme. @@ -908,9 +880,7 @@ def fit_hilbert_with_regularization(regularization): Split changes how local brightness variations are penalized, but `ConstantSplit` still applies one coefficient everywhere. It therefore retains the centre-versus-outskirts compromise identified in **Fit Problems**. --- -""" -""" __Allowing different smoothing strengths__ We now let the regularization strength depend on the source brightness. @@ -969,15 +939,13 @@ def fit_hilbert_with_regularization(regularization): aplt.subplot_fit_imaging(fit=adaptive_fit) -""" +r""" Compare the regularization-weight map with the reconstructed source. Smaller weights should occur where the adapt image indicates stronger source emission, allowing greater local brightness variation. Larger weights encourage more smoothing in low-signal regions. These weights control the regularization penalty; they are not uncertainties on the reconstructed brightnesses. The trial coefficients were chosen to illustrate the mechanism. We have not yet established that they improve the fit or maximize the evidence. Next, we will examine the transition controlled by `signal_scale` and explore the adaptive coefficients systematically. -""" -r""" __Controlling the transition with `signal_scale`__ The inner and outer coefficients set the smoothing strengths at the high- and low-signal ends of the source. How do we assign a strength to the regions between them? @@ -1032,9 +1000,7 @@ def fit_hilbert_with_regularization(regularization): Also distinguish `signal_scale` from the Hilbert mesh’s `weight_power`. Both transform brightness-derived information, but they control different things: `weight_power` changes **where mesh points are placed**, whereas `signal_scale` changes **how their brightnesses are regularized**. --- -""" -""" __Exploring the adaptive coefficients__ We now have three regularization parameters: the inner coefficient, outer coefficient and signal scale. @@ -1106,9 +1072,7 @@ def fit_hilbert_with_regularization(regularization): For a complete modelling analysis, these parameters can be explored by a non-linear search alongside the lens parameters. We will use that approach in the final fit. --- -""" -""" __Does adaptive regularization address the fit problem?__ Let’s inspect the highest-evidence adaptive reconstruction we have evaluated and compare it with `ConstantSplit`. @@ -1179,9 +1143,7 @@ def fit_hilbert_with_regularization(regularization): If substantial residuals remain, inspect where they occur. Adaptive regularization still cannot supply resolution missing from the mesh, and both adaptive steps depend on the initial adapt image. We have now explored the mesh placement and smoothing separately. Next, we will bring them together in an actual lens-model search, using an initial MGE fit, prior passing and the positions likelihood penalty. -""" -""" __Starting the final modelling chain__ So far, we have supplied the true lens mass model to isolate the effects of mesh placement and regularization. @@ -1256,9 +1218,7 @@ def fit_hilbert_with_regularization(regularization): Inspect this result before continuing. Does it explain the multiple-image configuration, and does it recover a sensible compact source? We should not use a misleading initial solution to guide the adaptive fit. --- -""" -""" __Preparing adaptation and the positions likelihood penalty__ We now derive a new adapt image from the fitted MGE result. This replaces the image used in our fixed-mass experiments. @@ -1347,9 +1307,7 @@ def fit_hilbert_with_regularization(regularization): Inspect both the mesh placement and predicted image positions. They should follow the source emission and multiple-image configuration before we use them in the next fit. --- -""" -""" __Defining the adaptive Delaunay model__ We now replace the MGE source with a Delaunay pixelization and adaptive Split regularization. @@ -1411,9 +1369,7 @@ def fit_hilbert_with_regularization(regularization): The regularization priors are starting ranges. After fitting, we will check whether the inferred parameters approach their boundaries and whether broader ranges or further exploration are needed. We are now ready to run the adaptive Delaunay fit and inspect the resulting mass model, source reconstruction and smoothing pattern. -""" -""" __Running the adaptive Delaunay fit__ The second search combines our Delaunay model, adaptation inputs and positions likelihood penalty. @@ -1457,9 +1413,7 @@ def fit_hilbert_with_regularization(regularization): Then examine the parameter distributions. A satisfactory-looking best fit does not establish that the search has explored the relevant solutions or measured reliable uncertainties. Check for parameters approaching prior boundaries and use the search diagnostics introduced in Chapter 2. --- -""" -""" __Inspecting the source and smoothing pattern__ Let’s look more closely at where the final mesh places its vertices and how strongly those vertices are regularized. @@ -1549,9 +1503,7 @@ def fit_hilbert_with_regularization(regularization): The additional spatial freedom becomes particularly useful for irregular sources whose structure is difficult to describe using a prescribed arrangement of analytic components. --- -""" -""" __Wrap Up__ We began this chapter by replacing an analytic source profile with brightness values on a mesh. @@ -1579,9 +1531,7 @@ def fit_hilbert_with_regularization(regularization): MGE models remain valuable for initial fits, foreground light and sources well described by analytic components. Pixelizations complement them when we need greater freedom to reconstruct the source. We now have the tools to use both together—and to investigate why a reconstruction succeeds or struggles, rather than relying only on how its source plot looks. -""" -""" **🤖 Ask AI:** How do Hilbert mesh weighting and adaptive Split regularization address different limitations of a compact-source reconstruction? __Env__ diff --git a/scripts/chapter_4_scaling_up_lensing/tutorial_4_group_scale.py b/scripts/chapter_4_scaling_up_lensing/tutorial_4_group_scale.py index b1e12c0..192043f 100644 --- a/scripts/chapter_4_scaling_up_lensing/tutorial_4_group_scale.py +++ b/scripts/chapter_4_scaling_up_lensing/tutorial_4_group_scale.py @@ -462,10 +462,7 @@ """ Adaptive over-sampling was configured around every galaxy centre before the initial fit evaluations. The same configuration is retained for the non-linear search. -""" - -""" We first fit the galaxy-only model using `Nautilus` and the familiar `AnalysisImaging` object. The likelihood calculation uses the same imaging framework as in earlier chapters, now with the central galaxy