Render fieldcompare diff visualizations - #883
Conversation
MakisH
left a comment
MakisH
left a comment
There was a problem hiding this comment.
Thank you for the PR! Some first comments, I will continue once I can run it.
Run diff PNG rendering through a dedicated Docker stage with libosmesa6 so local and CI follow the same headless path, and include generated PNGs in the _logs artifact with a job-summary link.
| max_abs_value = float(np.max(np.abs(values))) | ||
| color_limit = max_abs_value if max_abs_value > 0 else 1.0 | ||
| plotter.add_mesh( | ||
| glyphs, | ||
| scalars=scalar_name, | ||
| cmap="coolwarm", | ||
| clim=(-color_limit, color_limit), | ||
| scalar_bar_args={"title": field_name}, | ||
| ) |
There was a problem hiding this comment.
In this run, fieldcompare complained about the force in the last time window:
field-compare-1 | Comparing '/runs/perpendicular-flap_fluid-openfoam-solid-calculix_2026-08-05-084014/precice-exports/Solid-Mesh-Solid.dt500.vtu'
field-compare-1 | and '/runs/perpendicular-flap_fluid-openfoam-solid-calculix_2026-08-05-084014/reference-results-unpacked/fluid-openfoam_solid-calculix/Solid-Mesh-Solid.dt500.vtu'
field-compare-1 | Reading '/runs/perpendicular-flap_fluid-openfoam-solid-calculix_2026-08-05-084014/precice-exports/Solid-Mesh-Solid.dt500.vtu'
field-compare-1 | Reading '/runs/perpendicular-flap_fluid-openfoam-solid-calculix_2026-08-05-084014/reference-results-unpacked/fluid-openfoam_solid-calculix/Solid-Mesh-Solid.dt500.vtu'
field-compare-1 | -- Comparing the field 'Force': FAILED
field-compare-1 | -- Report: Deviation above tolerance detected -> [ 8.05858e-03 -5.47671e-05 0.00000e+00] vs. [ 8.05464e-03 -5.46938e-05 0.00000e+00] ([ 0.05 -0.13 inf] %)
field-compare-1 | -- Predicate: DefaultEquality (abs_tol: 0., rel_tol: 3.e-07)
field-compare-1 | Wrote diff into '/runs/perpendicular-flap_fluid-openfoam-solid-calculix_2026-08-05-084014/precice-exports/diff_Solid-Mesh-Solid.dt500.vtu.vtu'
field-compare-1 | File comparison FAILED with 2 PASSED / 1 FAILED / 0 SKIPPED
but looking at the image, this difference is not clearly visible:
The [-1.0,1.0] range is a bit arbitrary. Why is that being picked? I would generally rely on the color scale and range to understand how large a regression is.
Also, if the difference is visualized (and not the absolute value of it), why have the range symmetric? There must be a minimum value and a maximum value.
There was a problem hiding this comment.
Notes from our discussion:
- We should show the range of the diff files
- Even if the values tend to zero, if that doesn't cause any issues, let's have the real values: seeing a range
-1.23e-33 to +4.56e-33is also clear, and it is how ParaView does it - We noticed that the visualizations for the
Solidparticipant are missing from the picture.
|
|
||
|
|
||
| SUPPORTED_SUFFIXES = {".vtk", ".vtp", ".vtu"} | ||
| WINDOW_SIZE = (1024, 768) |
There was a problem hiding this comment.
In some cases, this resolution is a bit small. See a run for the heat exchanger:
Can pyvista export PDF files (vector graphics)? If we cannot include PNG files in the GHA summary, then PDF files would also do, and maybe they would make the archive smaller.
| runs/*/system-tests-run.log | ||
| runs/*/system-tests-compare.log | ||
| runs/*/*/system-tests_*.log | ||
| runs/*/diff-results/visualizations/**/*.png |
There was a problem hiding this comment.
It now looks like the _logs archive is several MB large, which defeats the original purpose of having something small to download for inspection on a bad connection.
Let's move these to a new archive (_diffs).
| plotter.add_text( | ||
| f"{source_file.name}\npoint field: {field_name}", | ||
| font_size=10, | ||
| color="black", | ||
| ) |
There was a problem hiding this comment.
An additional useful text here would be what the exact field operation is: Difference: computed - reference or Difference: reference - computed?
github-project-automation
Bot
moved this to Planned next
in GSoC 2026: System tests improvements
MakisH
moved this from Planned next
to Needs review
in GSoC 2026: System tests improvements
2 participants
Summary
Render archived fieldcompare diff VTK files as PNG images when result comparison fails.
Generate one image per numeric point field using distinct sphere glyphs and a generic automatically selected view.
Store images under
diff-results/visualizations/in the system-test artifacts.Keep visualization best-effort so rendering errors do not hide the original fieldcompare failure.
Fixes #441.
Test plan
Generated PNGs from synthetic scalar and vector VTK fields.
Verified the
archive-to-visualizationfailure path locally.Rendered five real
CI diff VTK files, producing 13 PNGs.Checked zero and non-zero differences visually.
Ran
pre-commitchecks successfully.