Skip to content

ResourceWarnings when opening file-level-compressed CDFs under Python 3.14 #341

Description

@jameswilburlewis

I've been trying to track down some ResourceWarnings I've been seeing in recent PySPEDAS test logs. For example:

06-Aug-26 02:58:05: Downloading remote index: https://ergsc.isee.nagoya-u.ac.jp/data/ergsc/satellite/erg/orb/def/2017/
06-Aug-26 02:58:07: File is current: data/ergsc/satellite/erg/orb/def/2017/erg_orb_l2_20170327_v05.cdf
06-Aug-26 02:58:07: /opt/hostedtoolcache/Python/3.14.6/x64/lib/python3.14/tempfile.py:484: ResourceWarning: Implicitly cleaning up <_TemporaryFileWrapper file=<_io.BufferedRandom name='/tmp/tmp4fa5j8sh.cdf'>>
  _warnings.warn(self.warn_message, ResourceWarning)

At least some of these warnings appear to be coming from cdflib's handling of compressed CDFs. I have a minimal self-contained test case (below) that reproduces the ResourceWarnings under Python 3.14. Older versions of Python don't seem to exhibit this behavior.

The reproducer specifically exercises _uncompress_file() in cdfread.py. That method uses Path(tempfile.NamedTemporaryFile(suffix=".cdf").name), which retains only the filename while immediately discarding the _TemporaryFileWrapper without explicitly closing it. Python 3.14 reports the wrapper’s implicit finalization as a ResourceWarning, producing one warning for each compressed CDF opened. _unstream_file() uses the same construction and is likely affected as well, although this reproducer does not exercise that path.

One possible fix would be to use mkstemp() and explicitly close its returned descriptor:

fd, temp_name = tempfile.mkstemp(suffix=".cdf")
os.close(fd)
self.temp_file = Path(temp_name)

Alternatively, cdflib could retain and explicitly close a NamedTemporaryFile(delete=False) wrapper before reopening its path. The existing CDF.del() cleanup could continue removing the temporary path afterward.

Here's a minimal self-contained test case that reproduces the issue:

# reproduce_cdflib_resourcewarning.py

import gc
import tempfile
import warnings
from pathlib import Path

import numpy as np
from cdflib import cdfread, cdfwrite


cdf_path = Path(tempfile.gettempdir()) / "cdflib_resourcewarning.cdf"

try:
    # Generate a small file-level-compressed CDF.
    with cdfwrite.CDF(
        cdf_path,
        cdf_spec={"Compressed": 6},
        delete=True,
    ) as writer:
        writer.write_var(
            {
                "Variable": "data",
                "Data_Type": cdfwrite.CDF.CDF_INT4,
                "Num_Elements": 1,
                "Rec_Vary": True,
                "Dim_Sizes": [],
            },
            var_data=np.arange(1000, dtype=np.int32),
        )

    print("CDF marker:", cdf_path.read_bytes()[:8].hex())
    # Expected: cdf30001cccc0001
    # "cccc0001" marks a file-level-compressed CDF.

    warnings.simplefilter("always", ResourceWarning)

    for iteration in range(3):
        reader = cdfread.CDF(cdf_path)
        reader.varget("data")
        del reader
        gc.collect()
        print(f"Completed iteration {iteration + 1}")

finally:
    cdf_path.unlink(missing_ok=True)

Output (Python: 3.14.3, cdflib: 1.3.8)

(python_conda_314) jwl@jwl-new-mac /tmp % python --version
Python 3.14.3
(python_conda_314) jwl@jwl-new-mac /tmp % python -m reproduce_cdflib_resourcewarning   
CDF marker: cdf30001cccc0001
/Users/jwl/opt/anaconda3/envs/python_conda_314/lib/python3.14/tempfile.py:484: ResourceWarning: Implicitly cleaning up <_TemporaryFileWrapper file=<_io.BufferedRandom name='/var/folders/_4/hbrtgs_j1qlgl2f3fwjw2_kc0000gn/T/tmp191a9d8y.cdf'>>
  _warnings.warn(self.warn_message, ResourceWarning)
Completed iteration 1
/Users/jwl/opt/anaconda3/envs/python_conda_314/lib/python3.14/tempfile.py:484: ResourceWarning: Implicitly cleaning up <_TemporaryFileWrapper file=<_io.BufferedRandom name='/var/folders/_4/hbrtgs_j1qlgl2f3fwjw2_kc0000gn/T/tmptepdlukm.cdf'>>
  _warnings.warn(self.warn_message, ResourceWarning)
Completed iteration 2
/Users/jwl/opt/anaconda3/envs/python_conda_314/lib/python3.14/tempfile.py:484: ResourceWarning: Implicitly cleaning up <_TemporaryFileWrapper file=<_io.BufferedRandom name='/var/folders/_4/hbrtgs_j1qlgl2f3fwjw2_kc0000gn/T/tmp63_a0g9i.cdf'>>
  _warnings.warn(self.warn_message, ResourceWarning)
Completed iteration 3

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions