From 38a577e6e8893df3a2638cec103cce3a4b2b5424 Mon Sep 17 00:00:00 2001 From: Fabian Isensee Date: Fri, 6 Nov 2026 16:39:16 +0100 Subject: [PATCH] Note that to_cframe() keeps the stored chunks of containers with a postfilter Blosc/c-blosc2#858 makes blosc2_schunk_to_buffer() and friends write the chunks as they are stored for every storage. Before, a container with a postfilter that was not an in-memory contiguous frame was serialized with the postfilter's output, so SChunk.to_cframe() and NDArray.to_cframe() change for those containers once the bundled C-Blosc2 includes it. Release note and Notes sections in both docstrings, with how to serialize the postfiltered values instead. Co-Authored-By: Claude Opus 5.5 --- RELEASE_NOTES.md | 17 +++++++++++++++++ src/blosc2/ndarray.py | 6 ++++++ src/blosc2/schunk.py | 7 +++++++ 3 files changed, 30 insertions(+) diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 2a463b384..888938137 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -122,6 +122,23 @@ for array/table examples, persistence policy and current limitations. Release certification is tracked separately; this development entry is not an announcement that 4.15.0 has been published. +### Serializing containers that have a postfilter + +- **Behaviour change:** `SChunk.to_cframe()` and `NDArray.to_cframe()` now return + the chunks as they are stored for every storage, also when a postfilter is set + (`SChunk.postfilter`). Before, they returned the postfilter's output unless the + container was an in-memory contiguous frame (`contiguous=True` without + `urlpath`). In-memory containers are sparse by default, so the frame usually + held the postfiltered values, and setting the postfilter again on the container + loaded from it applied it twice. Code that loads such a frame without setting + the postfilter again now reads the stored values. To serialize the postfiltered + values, serialize a copy that holds them: `NDArray.copy()` applies the + postfilter, and for an `SChunk`, a new one made from its decompressed data does. + Containers without a postfilter give the same frames as before. With a + prefilter or a postfilter, `to_cframe()` no longer decompresses and recompresses + every chunk, which makes it much faster. This comes with the bundled C-Blosc2 + (Blosc/c-blosc2#858). + ## Changes from 4.14.1 to 4.14.2 XXX version-specific blurb XXX diff --git a/src/blosc2/ndarray.py b/src/blosc2/ndarray.py index bf739cb7d..f0238d0e4 100644 --- a/src/blosc2/ndarray.py +++ b/src/blosc2/ndarray.py @@ -5476,6 +5476,12 @@ def to_cframe(self) -> bytes: :func:`~blosc2.ndarray_from_cframe` This function can be used to reconstruct a NDArray from the serialized bytes. + Notes + ----- + The buffer holds the chunks as they are stored: a postfilter set on + :attr:`schunk` is not applied, and it is not part of the buffer either. + For a buffer with the postfiltered values, serialize a :meth:`copy`. + Examples -------- >>> import blosc2 diff --git a/src/blosc2/schunk.py b/src/blosc2/schunk.py index 33452acae..b48282cff 100644 --- a/src/blosc2/schunk.py +++ b/src/blosc2/schunk.py @@ -1383,6 +1383,13 @@ def to_cframe(self) -> bytes: -------- :func:`~blosc2.schunk_from_cframe` + Notes + ----- + The buffer holds the chunks as they are stored: a postfilter (see + :meth:`postfilter`) is not applied, and it is not part of the buffer + either. Set it again on the deserialized :ref:`SChunk` to read the same + values as from this one. + Examples -------- >>> import blosc2