Skip to content

Add EarthDataHub Destine API as an alternative ERA5 data source #46

Description

@imharrisonking

Please provide a high level description of the feature.
The current ERA5DataDownloader exclusively uses the Copernicus CDS API (cdsapi) to retrieve ERA5 reanalysis data. This feature adds support for the EarthDataHub Destine API as an alternative ERA5 source, selectable via a new source parameter on ERA5DataDownloader (and DownloadManager). The CDS approach remains the default so there is no breaking change for existing users.

Describe the proposed solution and/or implementation in more detail
Add a source parameter to ERA5DataDownloader (e.g. "cds" or "earthdatahub", defaulting to "cds"). When source="earthdatahub", the downloader should:

  1. Open the remote zarr store via xr.open_dataset(..., engine="zarr") instead of using cdsapi
  2. Accept variables, date_range, and area as parameters to .download() rather than reading from constants
  3. Support .netrc-based authentication (trust_env=True) as well as explicit credentials passed as a constructor argument (mirroring the existing api_key pattern)
  4. Trigger the actual download with .compute() and write output to NetCDF

Under the hood (EarthDataHub zarr pattern):

ds = xr.open_dataset(
    "https://api.earthdatahub.destine.eu/era5/reanalysis-era5-single-levels-v0.zarr",
    storage_options={"client_kwargs": {"trust_env": True}},
    chunks={},
    engine="zarr",
)
cutout = ds[["u100", "v100"]].sel(
    latitude=slice(50, 47), longitude=slice(5, 8), valid_time="2025-10-01"
).compute()
cutout.to_netcdf("downloads/cutout.nc")

Basic usage (wind variables):

from rencal.core.data_downloader import ERA5DataDownloader

era5 = ERA5DataDownloader(source="earthdatahub")
era5.download(
    variables=["u100", "v100"],
    date_range=("2025-01-01", "2025-12-31"),
    area=[50, 5, 47, 8],  # [north, west, south, east]
)

With explicit credentials (solar variables):

era5 = ERA5DataDownloader(
    source="earthdatahub",
    credentials={"username": "user", "password": "secret"},
)
era5.download(
    variables=["ssrd", "fdir"],
    date_range=("2024-01-01", "2025-12-31"),
    area=[60, -10, 49, 2],
)

Files likely affected:

  • rencal/core/data_downloader.py — add source-branching logic in ERA5DataDownloader, update DownloadManager to expose the new source parameter
  • rencal/utils/constants.py — add the EarthDataHub base URL constant
  • docs/web/src/content/docs/reference/Weather/DataDownloader.mdx — update constructor docs and examples
  • pyproject.toml — zarr is already present; verify aiohttp or similar async deps if needed

Additional notes:

  • The EarthDataHub zarr dataset uses valid_time as the time dimension (vs time in CDS NetCDF) — this may need handling when merging wind and solar cutouts
  • Wind and solar variables should remain independently selectable, consistent with current behaviour
  • source should default to "cds" to avoid any breaking change for existing users

Describe alternatives you've considered
No alternatives have been considered yet.

Additional context (Optional)
N/A

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

performanceSpeed or memory related performance improvements

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions