Proposed new feature or change:
While diagnosing #1444 and developing #1667, we noticed an increasing scope. It started with being concerned about a specific instance of time leaking onto a grid object, causing downstream issues while using subset.
After finding a fix for this scenario, we decided we wanted to make a more robust grid object cleaning function, that can clean grid objects in a way that avoids other potentially damaging situations like this. Doing so we found other interactions with the IO, like #1717 that showed we need to further investigate making rules around cleaning, as there is a surprisingly unique range of incoming data variables, with characteristics that are uncommon, but comparable to what we imagine "leaked" data to look like.
A few main goals:
- Reduce the amount of potential incoming grid
data variables
- Based on what we don't need or don't expect to have after a grid file is read in
- Reduce the amount of potential incoming
coordinates
- Based on what we don't need or don't expect to have after a grid file is read in, and if they are 0 dimensional (dims=() or "scalar")
- Make a method that can run without needing model/format specific solutions
- Ensure that no models have their unique attributes removed if they are needed
- Ensure that the function is placed within the file read process so that it doesn't disrupt other existing parts
Added notes:
- We are interested in keeping at a minimum our existing
GRID_DIMS:
['n_edge', 'n_face', 'n_max_edge_edges', 'n_max_edge_faces',
'n_max_edge_nodes', 'n_max_face_edges', 'n_max_face_faces',
'n_max_face_nodes', 'n_max_node_edges', 'n_max_node_faces',
'n_node', 'two']
- We want cases like we've seen, and cases we've not seen, to be covered by this fix (in these examples,
time should be assumed to be any variable we don't want/need):
time present as a dimension coordinate (1-D, time(time), ndim == 1)
time present as a scalar (0-dimensional) coordinate (dims == (), ndim == 0)
time present as a dimension without a coordinate (a bare time dimension carrying data variables, no time coordinate)
- Meaningful dimensions (dimensions within
DIM_NAMES) that should be there, but aren't, due to it's unique IO process
These are some of the instances we've seen and want to keep them all in mind when finding a solution. This may be blocked until we understand #1717 better.
Proposed new feature or change:
While diagnosing #1444 and developing #1667, we noticed an increasing scope. It started with being concerned about a specific instance of
timeleaking onto a grid object, causing downstream issues while usingsubset.After finding a fix for this scenario, we decided we wanted to make a more robust grid object cleaning function, that can clean grid objects in a way that avoids other potentially damaging situations like this. Doing so we found other interactions with the IO, like #1717 that showed we need to further investigate making rules around cleaning, as there is a surprisingly unique range of incoming data variables, with characteristics that are uncommon, but comparable to what we imagine "leaked" data to look like.
A few main goals:
data variablescoordinatesAdded notes:
GRID_DIMS:timeshould be assumed to be any variable we don't want/need):timepresent as a dimension coordinate (1-D, time(time), ndim == 1)timepresent as a scalar (0-dimensional) coordinate (dims == (), ndim == 0)timepresent as a dimension without a coordinate (a bare time dimension carrying data variables, no time coordinate)DIM_NAMES) that should be there, but aren't, due to it's unique IO processThese are some of the instances we've seen and want to keep them all in mind when finding a solution. This may be blocked until we understand #1717 better.