[WIP] Adding HIT l1b Science CDF Attributes by daralynnrhode · Pull Request #3484 · IMAP-Science-Operations-Center/imap_processing

daralynnrhode · 2026-09-21T23:25:19Z

Added the L1b Science Attributes for HIT

Overview

Adds complete, ISTP-compliant CDF variable attributes for HIT L1B science data products (standard-rates, summed-rates, sectored-rates). Previously only housekeeping attributes existed in imap_hit_l1b_variable_attrs.yaml; science variables had no CDF metadata at all.

All attribute values are sourced directly from the HIT CDF Product Definitions spreadsheet .

The spreadsheet has no row for he_energy_delta_plus/he_energy_delta_minus (every other summed-rates particle has one). Flagged with a comment in the YAML; Will need to be confirmed/corrected.

closes #3070

File changes

imap_processing/cdf/config/imap_hit_l1b_variable_attrs.yaml — full science attribute section added/rebuilt from the spreadsheet, replacing the placeholder/HK-only content. Includes dynamic_threshold_state, all standard-rates rate/uncertainty variables, and all summed/sectored-rates particle rate/uncertainty/energy-coordinate variables.
imap_processing/hit/l1b/hit_l1b.py — wires the attribute dictionary into .attrs for every L1B science dataset (process_science_data()), for all data vars and for epoch.
imap_processing/hit/hit_utils.py — add_energy_variables() and add_summed_particle_data_to_dataset() now accept an optional attr_mgr so the per-particle energy coordinates created fresh during summed-rates processing (which the shared attrs loop can't reach, since it only iterates data_vars, not coords) get real CDF attributes instead of none. Kept optional so L1A's existing call site (which sets attrs separately, later) isn't broken.

Testing

NA

lacoak21 · 2026-09-22T18:23:13Z


+
+
+# <=== Science Data Variable Attributes (source: HIT CDF Product Definitions spreadsheet) ===>


Is this spreadsheet in galaxy? Can we link that page here?

daralynnrhode · 2026-09-22T18:25:50Z

@lacoak21 Its this link https://lasp.colorado.edu/galaxy/spaces/IMAP/pages/207619340/HIT+CDF+Product+Designs

lacoak21

Excellent jon!! I had a few minor comments about attribute structure. Wow there were a lot more than I thought 😆

@bryan-harter since these are not l2 attrs and dont need to be fully ISTP compliant can Daralynn skip sending them to Andriy for review or should she do that anyway?

lacoak21 · 2026-09-22T18:26:18Z

+  DEPEND_2: sngrates_index
+  DISPLAY_TYPE: time_series
+  FIELDNAM: sngrates_count_rate
+  FILLVAL: -1e+31


You can also define this as a variable at the top of the file e.g.:

double_fillval: &double_fillval -1.0E31

and then here you can use it like :

FILLVAL: *double_fillval

lacoak21 · 2026-09-22T18:34:18Z

@@ -546,3 +544,2526 @@ ebox_p2d0vd:
  UNITS: V
  VALIDMAX: 10000000000

+
+
+# <=== Science Data Variable Attributes (source: HIT CDF Product Definitions spreadsheet) ===>
+
+dynamic_threshold_state:
+  CATDESC: Raw dynamic threshold state per integration
+  DEPEND_0: epoch
+  DISPLAY_TYPE: time_series
+  FIELDNAM: Dynamic Threshold State
+  FILLVAL: -128
+  FORMAT: I1
+  LABLAXIS: State
+  SCALEMAX: 3
+  SCALEMIN: 0
+  SCALETYP: linear
+  UNITS: ' '
+  VALIDMAX: 3
+  VALIDMIN: 0
+  VAR_TYPE: data
+  dtype: int8
+
+
+# Standard Rates Data Variable Attributes
+
+sngrates:
+  CATDESC: Standard sngrates Count Rates
+  DELTA_MINUS_VAR: sngrates_stat_uncert_minus
+  DELTA_PLUS_VAR: sngrates_stat_uncert_plus
+  DEPEND_0: epoch
+  DEPEND_1: gain
+  DEPEND_2: sngrates_index
+  DISPLAY_TYPE: time_series
+  FIELDNAM: sngrates_count_rate
+  FILLVAL: -1e+31
+  FORMAT: F15.3
+  LABLAXIS: "Count Rate"
+  LABL_PTR_1: gain_label
+  LABL_PTR_2: sngrates_index_label
+  SCALEMAX: 10000000000
+  SCALEMIN: 0.0001
+  SCALETYP: log
+  UNITS: "counts / livetime fraction"
+  VALIDMAX: 10000000000
+  VALIDMIN: 0
+  VAR_TYPE: data
+
+coinrates:
+  CATDESC: Standard coinrates Count Rates
+  DELTA_MINUS_VAR: coinrates_stat_uncert_minus
+  DELTA_PLUS_VAR: coinrates_stat_uncert_plus
+  DEPEND_0: epoch
+  DEPEND_1: coinrates_index
+  DISPLAY_TYPE: time_series
+  FIELDNAM: coinrates_count_rate
+  FILLVAL: -1e+31
+  FORMAT: F15.3
+  LABLAXIS: "Count Rate"
+  LABL_PTR_1: coinrates_index_label
+  SCALEMAX: 10000000000
+  SCALEMIN: 0.0001
+  SCALETYP: log
+  UNITS: "counts / livetime fraction"
+  VALIDMAX: 10000000000
+  VALIDMIN: 0
+  VAR_TYPE: data
+
+pbufrates:
+  CATDESC: Standard pbufrates Count Rates
+  DELTA_MINUS_VAR: pbufrates_stat_uncert_minus
+  DELTA_PLUS_VAR: pbufrates_stat_uncert_plus
+  DEPEND_0: epoch
+  DEPEND_1: pbufrates_index
+  DISPLAY_TYPE: time_series
+  FIELDNAM: pbufrates_count_rate
+  FILLVAL: -1e+31
+  FORMAT: F15.3
+  LABLAXIS: "Count Rate"
+  LABL_PTR_1: pbufrates_index_label
+  SCALEMAX: 10000000000
+  SCALEMIN: 0.0001
+  SCALETYP: log
+  UNITS: "counts / livetime fraction"
+  VALIDMAX: 10000000000
+  VALIDMIN: 0
+  VAR_TYPE: data
+
+l2fgrates:
+  CATDESC: Standard l2fgrates Count Rates
+  DELTA_MINUS_VAR: l2fgrates_stat_uncert_minus
+  DELTA_PLUS_VAR: l2fgrates_stat_uncert_plus
+  DEPEND_0: epoch
+  DEPEND_1: l2fgrates_index
+  DISPLAY_TYPE: time_series
+  FIELDNAM: l2fgrates_count_rate
+  FILLVAL: -1e+31
+  FORMAT: F15.3
+  LABLAXIS: "Count Rate"
+  LABL_PTR_1: l2fgrates_index_label
+  SCALEMAX: 10000000000
+  SCALEMIN: 0.0001
+  SCALETYP: log
+  UNITS: "counts / livetime fraction"
+  VALIDMAX: 10000000000
+  VALIDMIN: 0
+  VAR_TYPE: data
+
+l2bgrates:
+  CATDESC: Standard l2bgrates Count Rates
+  DELTA_MINUS_VAR: l2bgrates_stat_uncert_minus
+  DELTA_PLUS_VAR: l2bgrates_stat_uncert_plus
+  DEPEND_0: epoch
+  DEPEND_1: l2bgrates_index
+  DISPLAY_TYPE: time_series
+  FIELDNAM: l2bgrates_count_rate
+  FILLVAL: -1e+31
+  FORMAT: F15.3
+  LABLAXIS: "Count Rate"
+  LABL_PTR_1: l2bgrates_index_label
+  SCALEMAX: 10000000000
+  SCALEMIN: 0.0001
+  SCALETYP: log
+  UNITS: "counts / livetime fraction"
+  VALIDMAX: 10000000000
+  VALIDMIN: 0
+  VAR_TYPE: data
+
+l3fgrates:
+  CATDESC: Standard l3fgrates Count Rates
+  DELTA_MINUS_VAR: l3fgrates_stat_uncert_minus
+  DELTA_PLUS_VAR: l3fgrates_stat_uncert_plus
+  DEPEND_0: epoch
+  DEPEND_1: l3fgrates_index
+  DISPLAY_TYPE: time_series
+  FIELDNAM: l3fgrates_count_rate
+  FILLVAL: -1e+31
+  FORMAT: F15.3
+  LABLAXIS: "Count Rate"
+  LABL_PTR_1: l3fgrates_index_label
+  SCALEMAX: 10000000000
+  SCALEMIN: 0.0001
+  SCALETYP: log
+  UNITS: "counts / livetime fraction"
+  VALIDMAX: 10000000000
+  VALIDMIN: 0
+  VAR_TYPE: data
+
+l3bgrates:
+  CATDESC: Standard l3bgrates Count Rates
+  DELTA_MINUS_VAR: l3bgrates_stat_uncert_minus
+  DELTA_PLUS_VAR: l3bgrates_stat_uncert_plus
+  DEPEND_0: epoch
+  DEPEND_1: l3bgrates_index
+  DISPLAY_TYPE: time_series
+  FIELDNAM: l3bgrates_count_rate
+  FILLVAL: -1e+31
+  FORMAT: F15.3
+  LABLAXIS: "Count Rate"
+  LABL_PTR_1: l3bgrates_index_label
+  SCALEMAX: 10000000000
+  SCALEMIN: 0.0001
+  SCALETYP: log
+  UNITS: "counts / livetime fraction"
+  VALIDMAX: 10000000000
+  VALIDMIN: 0
+  VAR_TYPE: data
+
+penfgrates:
+  CATDESC: Standard penfgrates Count Rates
+  DELTA_MINUS_VAR: penfgrates_stat_uncert_minus
+  DELTA_PLUS_VAR: penfgrates_stat_uncert_plus
+  DEPEND_0: epoch
+  DEPEND_1: penfgrates_index
+  DISPLAY_TYPE: time_series
+  FIELDNAM: penfgrates_count_rate
+  FILLVAL: -1e+31
+  FORMAT: F15.3
+  LABLAXIS: "Count Rate"
+  LABL_PTR_1: penfgrates_index_label
+  SCALEMAX: 10000000000
+  SCALEMIN: 0.0001
+  SCALETYP: log
+  UNITS: "counts / livetime fraction"
+  VALIDMAX: 10000000000
+  VALIDMIN: 0
+  VAR_TYPE: data
+
+penbgrates:
+  CATDESC: Standard penbgrates Count Rates
+  DELTA_MINUS_VAR: penbgrates_stat_uncert_minus
+  DELTA_PLUS_VAR: penbgrates_stat_uncert_plus
+  DEPEND_0: epoch
+  DEPEND_1: penbgrates_index
+  DISPLAY_TYPE: time_series
+  FIELDNAM: penbgrates_count_rate
+  FILLVAL: -1e+31
+  FORMAT: F15.3
+  LABLAXIS: "Count Rate"
+  LABL_PTR_1: penbgrates_index_label
+  SCALEMAX: 10000000000
+  SCALEMIN: 0.0001
+  SCALETYP: log
+  UNITS: "counts / livetime fraction"
+  VALIDMAX: 10000000000
+  VALIDMIN: 0
+  VAR_TYPE: data
+
+ialirtrates:
+  CATDESC: Standard ialirtrates Count Rates
+  DELTA_MINUS_VAR: ialirtrates_stat_uncert_minus
+  DELTA_PLUS_VAR: ialirtrates_stat_uncert_plus
+  DEPEND_0: epoch
+  DEPEND_1: ialirtrates_index
+  DISPLAY_TYPE: time_series
+  FIELDNAM: ialirtrates_count_rate
+  FILLVAL: -1e+31
+  FORMAT: F15.3
+  LABLAXIS: "Count Rate"
+  LABL_PTR_1: ialirtrates_index_label
+  SCALEMAX: 10000000000
+  SCALEMIN: 0.0001
+  SCALETYP: log
+  UNITS: "counts / livetime fraction"
+  VALIDMAX: 10000000000
+  VALIDMIN: 0
+  VAR_TYPE: data
+
+l4fgrates:
+  CATDESC: Standard l4fgrates Count Rates
+  DELTA_MINUS_VAR: l4fgrates_stat_uncert_minus
+  DELTA_PLUS_VAR: l4fgrates_stat_uncert_plus
+  DEPEND_0: epoch
+  DEPEND_1: l4fgrates_index
+  DISPLAY_TYPE: time_series
+  FIELDNAM: l4fgrates_count_rate
+  FILLVAL: -1e+31
+  FORMAT: F15.3
+  LABLAXIS: "Count Rate"
+  LABL_PTR_1: l4fgrates_index_label
+  SCALEMAX: 10000000000
+  SCALEMIN: 0.0001


I noticed that the last ~6 of these attrs are repeated. Can we make a "rate_default" attr that goes with the rest of the defaults above?

It would look something like this: ( I might be missing others that are also the same)

default_rate_attrs: &default_rate_attrs
Add FilesPaste, drop, or click to add filesCancelAdd review comment
SCALEMIN: 0.0001
SCALETYP: log
UNITS: "counts / livetime fraction"
VALIDMAX: 10000000000
VALIDMIN: 0
VAR_TYPE: data

and then the actual rates vars can inherit the defaults with this notation

l4bgrates:
<<: *default_rate_attrs

lacoak21 · 2026-09-22T18:34:55Z

+  FIELDNAM: l4bgrates_count_rate
+  FILLVAL: -1e+31
+  FORMAT: F15.3
+  LABLAXIS: "Count Rate"


Andriy once told me these should preferably be unique. Can you add the rate name to this string?

lacoak21 · 2026-09-22T18:35:29Z

+  FILLVAL: -1e+31
+  FORMAT: F5.1
+  LABLAXIS: "Energy"
+  SCALETYP:


missing log.

lacoak21 · 2026-09-22T18:35:58Z

+
+#Summed & Sectored Rates - Particle Variable Attributes
+
+h_energy_mean:


Might be worth making a default energy attr for any overlap here too.

lacoak21 · 2026-09-22T18:40:01Z

+
+# NOTE: not provided in the excel file, will need to be confirmed
+
+he_energy_delta_plus:


Nice! I think you're right and we definitely want these.

bryan-harter · 2026-09-23T21:02:57Z

Excellent jon!! I had a few minor comments about attribute structure. Wow there were a lot more than I thought 😆

@bryan-harter since these are not l2 attrs and dont need to be fully ISTP compliant can Daralynn skip sending them to Andriy for review or should she do that anyway?

Yup, we don't need to send them to Andriy if they are below L2

daralynnrhode added 2 commits September 21, 2026 17:08

adding hit l1b science cdf attr

482774a

adding he energy delta +- for istp compliance

f67072a

daralynnrhode added this to the September 2026 milestone Sep 21, 2026

daralynnrhode self-assigned this Sep 21, 2026

daralynnrhode added the Student Tasks suitable for student work label Sep 21, 2026

daralynnrhode added this to IMAP Sep 21, 2026

alphabatizing the attrs

f75f1f6

lacoak21 reviewed Sep 22, 2026

View reviewed changes

Navigation Menu

[WIP] Adding HIT l1b Science CDF Attributes - #3484

[WIP] Adding HIT l1b Science CDF Attributes#3484
daralynnrhode wants to merge 3 commits into
IMAP-Science-Operations-Center:devfrom
daralynnrhode:hit_attributes

daralynnrhode commented Sep 21, 2026 •

edited

Loading

Uh oh!

lacoak21 Sep 22, 2026

Uh oh!

daralynnrhode commented Sep 22, 2026

Uh oh!

lacoak21 left a comment

Uh oh!

lacoak21 Sep 22, 2026

Uh oh!

lacoak21 Sep 22, 2026

Uh oh!

lacoak21 Sep 22, 2026

Uh oh!

lacoak21 Sep 22, 2026

Uh oh!

lacoak21 Sep 22, 2026

Uh oh!

lacoak21 Sep 22, 2026

Uh oh!

bryan-harter commented Sep 23, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants




		# <=== Science Data Variable Attributes (source: HIT CDF Product Definitions spreadsheet) ===>


		#Summed & Sectored Rates - Particle Variable Attributes

		h_energy_mean:


		# NOTE: not provided in the excel file, will need to be confirmed

		he_energy_delta_plus:

Conversation

daralynnrhode commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Added the L1b Science Attributes for HIT

Overview

File changes

Testing

Uh oh!

lacoak21 Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

daralynnrhode commented Sep 22, 2026

Uh oh!

lacoak21 left a comment

Choose a reason for hiding this comment

Uh oh!

lacoak21 Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

lacoak21 Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

lacoak21 Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

lacoak21 Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

lacoak21 Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

lacoak21 Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

bryan-harter commented Sep 23, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

daralynnrhode commented Sep 21, 2026 •

edited

Loading