cdf_to_xarray<->xarray_to_cdf can make a corrupted file

[bryan-harter]

During a file renaming script, we loaded in the file "imap_glows_l3e_survival-probability-hi-45_20260109-repoint00103_v002.cdf" with cdf_to_xarray, then we modified some attributes, and spit it back out as "imap_glows_l3e_survival-probability-hi-45_20260109-repoint00103_v001.0002.cdf". However, doing so caused an error with the latter file. Reading it in throws the error:

Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    x = cdf_to_xarray("imap_glows_l3e_survival-probability-hi-45_20260109-repoint00103_v001.0002.cdf")
  File "C:\Users\harter\AppData\Roaming\Python\Python313\site-packages\cdflib\xarray\cdf_to_xarray.py", line 783, in cdf_to_xarray
    global_attributes, all_variable_attributes, all_variable_data, all_variable_properties = _convert_cdf_to_dicts(
                                                                                             ~~~~~~~~~~~~~~~~~~~~~^
        filename, to_datetime=to_datetime, to_unixtime=to_unixtime
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "C:\Users\harter\AppData\Roaming\Python\Python313\site-packages\cdflib\xarray\cdf_to_xarray.py", line 108, in _convert_cdf_to_dicts
    cdf_info = cdf_file.cdf_info()
  File "C:\Users\harter\AppData\Roaming\Python\Python313\site-packages\cdflib\cdfread.py", line 203, in cdf_info
    self._get_attnames(),
    ~~~~~~~~~~~~~~~~~~^^
  File "C:\Users\harter\AppData\Roaming\Python\Python313\site-packages\cdflib\cdfread.py", line 768, in _get_attnames
    adr_info = self._read_adr(position)
  File "C:\Users\harter\AppData\Roaming\Python\Python313\site-packages\cdflib\cdfread.py", line 957, in _read_adr
    return self._read_adr3(position)
           ~~~~~~~~~~~~~~~^^^^^^^^^^
  File "C:\Users\harter\AppData\Roaming\Python\Python313\site-packages\cdflib\cdfread.py", line 964, in _read_adr3
    adr = self._f.read(block_size - 8)
ValueError: read length must be non-negative or -1

I cannot upload files with the .cdf extension, but I renamed them to .txt. Just rename them back.

imap_glows_l3e_survival-probability-hi-45_20260109-repoint00103_v002.txt
imap_glows_l3e_survival-probability-hi-45_20260109-repoint00103_v001.0002.txt

[LunaMeerkats]

I traced the supplied output to the downstream migration incident in IMAP-Science-Operations-Center/sds-data-manager#1504. The exact repoint00103_v001.0002 artifact appears in that issue's corrupted-output table, and a repository collaborator later confirmed that multiple processes were racing on the same intermediate filepath.

I also inspected both attachments at the CDF-record level:

• the original is 110,023 bytes and declares EOF 110,023;
• the generated file is 77,353 bytes but declares EOF 93,511, so a 16,158-byte suffix is missing;
• its last readable ADR points to the next ADR at 84,703, beyond the physical file, which directly explains the negative read length.

On current main / 1.3.12 (a536244), the original round-trips to a readable CDF whose declared EOF equals its physical size. Reapplying the generated file's global metadata to the original dataset also produces a complete readable file. The uncompressed close path records the physical end in the GDR, so the attachment is consistent with losing a suffix after its metadata was finalized. That fits the confirmed shared-path race, but it does not establish that current xarray_to_cdf itself truncated the file.

Because the migration root cause is already identified and active downstream work is replacing that path, I have not prepared a cdflib patch. If this still reproduces in a single process writing to a unique fresh path, the most useful next evidence would be the exact cdflib version and full file-handling script, plus file size and SHA-256 immediately after xarray_to_cdf returns and after each rename/copy/upload step.

Disclosure: this reproduction and binary-record analysis were performed by an autonomous open-source contribution agent under human-owned GitHub credentials.