Skip to content

Select reversibility heuristics per thermodynamic data source (eQuilibrator gets its own cutoff) - #285

Open
Cooper-Taylor wants to merge 8 commits into
ModelSEED:devfrom
Cooper-Taylor:eQuilibrator-fix
Open

Select reversibility heuristics per thermodynamic data source (eQuilibrator gets its own cutoff)#285
Cooper-Taylor wants to merge 8 commits into
ModelSEED:devfrom
Cooper-Taylor:eQuilibrator-fix

Conversation

@Cooper-Taylor

@Cooper-Taylor Cooper-Taylor commented Aug 18, 2026

Copy link
Copy Markdown

Summary

The reaction-direction cascade had exactly one rule set — DEFAULT_HEURISTICS, derived from the Group-Contribution era — applied to every thermodynamic source. This splits it into a registry keyed by source:

set cascade basis
GC (default) ATPS → ABCT → MdeltaG bounds → mMdeltaG band → lowE → = Jankowski 2008
EQ ATPS → ABCT → undecomposable → uncorrected transport → ln Γ ± 1σ → = Noor 2012 index, Beber 2022 uncertainty
EQ2 same, index as a bare point estimate Flamholz 2012

GC is the default for any source without a set of its own, so both dGPredictor variants are unchanged. The GC cascade itself is untouched.

Why

Three defects in the eQuilibrator path. (Points 1 and 2 were re-characterised after review — see the "Corrected after review" note at the bottom. The code did not change; the explanation of why it was needed was wrong.)

1. The EQ run stopped reading eQuilibrator energies in 2023. The design was correct. From c263e233 (2019) through 17c9739b (2023-09-11), Update_Reaction_eQuilibrator_Energies.py overwrote the canonical energy and stamped the note that gates the reversibility step:

reactions_dict[rxn]['deltag']    = float(eq_reactions[rxn]['dg'])
reactions_dict[rxn]['deltagerr'] = float(eq_reactions[rxn]['dge'])
if('EQU' not in notes_list): notes_list.append('EQU')

So reading deltag gated on EQU was reading eQuilibrator energies, exactly as intended.

3e50646b (2023-09-13) removed the write half and left the read half in place — the updater moved to thermodynamics['eQuilibrator'] only, dropping both the deltag overwrite and the EQU stamp. Later GC rebuilds (Convention A, ad34d6ab) and the PR #265 promotion then rewrote deltag from other sources. Today only 1,797 of 25,028 reactions with an eQuilibrator record have deltag == thermodynamics['eQuilibrator'][0]; restricted to the 17,094 still carrying EQU, only 1,586 do. EQU is now a fossil of the pre-2023 pipeline. Pointing the EQ run at the eQuilibrator sublist restores the original intent rather than inventing a new one.

2. For most σ-flagged reactions eQuilibrator returns no energy at all — we stored a transform term as if it were one. GibbsEnergyPredictor.standard_dg short-circuits when a component falls outside both the reactant- and group-contribution spans:

mu, sigma_fin, sigma_inf, residual = self.get_reaction_prediction(reaction)
if residual:
    return Q_(0, "kJ/mol").plus_minus(self.preprocess.RMSE_inf)

Literally zero — the computed mean is discarded, not projected. standard_dg_prime, which our retrieval script calls, then adds the Legendre/pH transform on top, so the nonzero ΔG′° we store is only the transform. rxn00017's −20.46 kcal/mol is the pH-7 transform of an energy eQuilibrator declined to estimate.

Of the 4,607 σ-flagged reactions: 2,895 are this zeroed case (σ exactly RMSE_inf; 61 still exactly 0.000000, having no net proton change for the transform to act on — e.g. rxn02677, rxn04059), and 1,712 carry RMSE_inf·‖σ_inf‖ and do have a real mean but with an undetermined covariance direction. Neither supports a directional call; the GC bounds rule cannot fire at that width, so all of them fell through to a permissive =.

3. Transport energies are compartment-collapsed. Retrieve_eQuilibrator_Reactions_Energies.py keys its MetaNetX formula on compound id and discards compartment, so any species on both sides nets out — 1,102 transport reactions carry a ΔG′° for a different reaction (rxn12518 at +524 kcal/mol). A further 76 non-transport reactions collapse via stereo-neutral InChIKey matching (rxn00816: D-glucose and galactose merge). Beber 2022 separately notes the transformed framework needs a −N_H·RT·ln(10^ΔpH) − Q·FΔΦ term we never apply.

Separately: the reversibility index eQuilibrator has been computing for us — column 4 of MetaNetX_Reaction_Energies.tbl — was parsed and discarded on every run, and make_ln_reversibility_index_heuristic was never invoked. It now decides 8,944 reactions, and the bare no-evidence default fallback drops from 7,120 to zero.

Results

Per-method operators vs current dev:

source records changed
Group contribution 56,002 0
eQuilibrator 25,028 7,103
dGPredictor 27,715 0
dGPredictor-ModelSEED 31,924 0

Only eQuilibrator moves. The three columns scored with the untouched GC rule set are byte-identical, which isolates the change to the new cutoff rather than the refactor.

Canonical reversibility: > 12,670→9,584 (−3,086) · < 2,323→1,603 (−720) · = 15,059→14,002 (−1,057) · ? 25,960→30,823 (+4,863)

The database gets less confident, and that is the point. The +4,863 unknowns track the σ-flagged records almost exactly — reactions that had been getting a permissive = on the strength of a number that, for 2,895 of them, is eQuilibrator's literal zero plus a pH transform.

Verification

  • New Scripts/Tests/test_eq_heuristics.py — the ln Γ implementation reproduces eQuilibrator's own published ln_reversibility_index on 17,771 reactions; every residual is a MetaNetX-collapsed reaction, asserted rather than assumed.
  • Scripts/Tests/test_reaction_direction.py now separates invariant sources (GC, dGPredictor — must match exactly) from intentionally re-scored ones (eQuilibrator), with --strict to require equality everywhere. Passes.
  • Rerun_Thermodynamics.sh is idempotent: a second run reproduces every file byte-for-byte.
  • Spot checks: rxn00001> (ln Γ −9.18) · rxn00017? · rxn12518? · rxn08173 ATP synthase → =.

Full write-up: Papers/NAR_Update_2026/data/equilibrator_reversibility_heuristics_2026-08-18.md.

Scope notes for review

  • Canonical deltag is untouched and mostly remains the Group-Contribution value. Reconciling it is Promote_Reaction_Thermodynamics_to_Canonical.py, deliberately not part of Rerun_Thermodynamics.sh and not run here.
  • The ~3,086 lost > calls are the main review surface: some are the index genuinely disagreeing with the GC bounds rule, others are reactions whose only evidence was the zeroed value. Separable from the generated report on request.
  • Follow-ups worth considering: is_using_group_contribution() would replace the σ-threshold with eQuilibrator's own verdict; standard_dg_prime_multi() returns the covariance the current per-reaction loop cannot; and the MetaNetX collapse in the retrieval script is fixable at source.

Corrected after review

Commit a86f6adf revises the framing of defects 1 and 2 following review. The original text claimed the EQ run "never" read eQuilibrator energies (it did, correctly, until 2023) and that the σ marker was "read as an error bar" (understated — the energy itself is a discarded mean replaced by zero). No code changed.

🤖 Generated with Claude Code

Cooper-Taylor and others added 8 commits August 18, 2026 07:45
The direction cascade had exactly one rule set, DEFAULT_HEURISTICS, derived
from the Group-Contribution era and applied to every source. Split it into a
registry keyed by source, with GC as the default for anything without a set of
its own.

  GC   unchanged Jankowski 2008 cascade -- the default
  EQ   eQuilibrator: Noor 2012 reversibility index, gated by Beber 2022
       uncertainty (new)
  EQ2  eQuilibrator 2.0: the same index as a bare point estimate

Three defects in the eQuilibrator path motivated this:

1. `Estimate_Reaction_Reversibility.py EQ` never read eQuilibrator energies.
   It pulled the canonical `deltag`, merely gated on eQuilibrator eligibility.
   Since the additive-thermodynamics refactor nothing overwrites `deltag`, so
   only 1,797 of 25,028 reactions with an eQuilibrator record actually had
   `deltag == thermodynamics['eQuilibrator'][0]`; the other 23,140 were scored
   on the Group-Contribution number and labelled eQuilibrator. EQ runs now read
   the eQuilibrator sublist's own dG and sigma.

2. eQuilibrator's ~1e5 kJ/mol "cannot decompose this reaction" marker was being
   consumed as an error bar. 4,933 records carry it; the GC bounds rule cannot
   fire that wide, so they fell through to a permissive "=". They now return
   "?". Observed real sigma tops out at 65.35 kcal/mol against a marker of
   23,900.57, so the cut at 1e4 kJ/mol sits in an empty gap.

3. `Retrieve_eQuilibrator_Reactions_Energies.py` keys its MetaNetX formula on
   compound id and so discards compartment, collapsing any species present on
   both sides; 1,102 transport reactions carry a dG for a different reaction.
   Beber 2022 separately notes the transformed framework needs a
   -N_H*RT*ln(10^dpH) - Q*F*dPhi term across a membrane that we never apply.
   Transport is now decided structurally (ATPS/ABCT) or returns "?".

The reversibility index eQuilibrator has been computing for us since the table
was first generated -- column 4 of MetaNetX_Reaction_Energies.tbl -- was parsed
and discarded on every run, and `make_ln_reversibility_index_heuristic` was
never invoked. It now decides 8,944 reactions, and the bare no-evidence
`default` fallback drops from 7,120 to zero.

Add_Reaction_Thermodynamics_Operators, _thermo_helpers and Promote_* now pass
the source label through, so each method's stored operator is computed with its
own rule set instead of GC's.

Scripts/Tests/test_eq_heuristics.py verifies the ln(Gamma) implementation
reproduces eQuilibrator's own published ln_reversibility_index on 17,771
reactions; every residual is a MetaNetX-collapsed reaction.
test_reaction_direction.py now distinguishes invariant sources (GC,
dGPredictor -- must match exactly) from intentionally re-scored ones
(eQuilibrator), with --strict to require equality everywhere.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Output of Scripts/Thermodynamics/Rerun_Thermodynamics.sh with the per-source
rule sets. Verified idempotent: a second run reproduces every file byte-for-byte
("Entries refreshed/added: 0").

Per-method operators, vs the previous dev state:

  Group contribution      56,002 records        0 changed
  eQuilibrator            25,028 records    7,103 changed
  dGPredictor             27,715 records        0 changed
  dGPredictor-ModelSEED   31,924 records        0 changed

Only eQuilibrator moves. The three columns scored with the untouched GC rule
set are byte-identical, which is what isolates the change to the new cutoff
rather than the refactor.

Canonical reversibility:  >  12,670 -> 9,584   (-3,086)
                          <   2,323 -> 1,603     (-720)
                          =  15,059 -> 14,002  (-1,057)
                          ?  25,960 -> 30,823  (+4,863)

The database gets less confident, and that is the point: the +4,863 unknowns
track the 4,933 undecomposable records almost exactly. Those had been receiving
a permissive "=" or an inherited direction on the strength of a dG carrying
+/-23,900 kcal/mol -- an unread sentinel, not evidence.

Deciding rule for the EQ run:

  Incomplete (no eQuilibrator energy)   24,828
  EQ:lnGamma                             8,944
  EQ:reversible                          8,905
  Incomplete (GCC, keeps GC answer)      6,146
  EQ:undecomposable                      4,933
  ABCT                                   1,151
  EQ:ambiguous                             871
  EQ:transport-uncorrected                 209
  ATPS                                      15
  Empty                                     10

Canonical deltag is untouched and mostly remains the Group-Contribution value;
reconciling it is Promote_Reaction_Thermodynamics_to_Canonical.py, which is
deliberately not part of Rerun_Thermodynamics.sh and was not run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Write-up of the work behind PR ModelSEED#285, following the convention of the other
Papers/NAR_Update_2026/data/ investigation notes.

Records what dev was actually doing and how that was established (the
eQuilibrator reversibility index was computed, stored in column 4 of
MetaNetX_Reaction_Energies.tbl, and discarded unread on every run), the three
defects in the eQuilibrator path, and the results of the rerun.

Section 3 is the part worth keeping for a methods section: eQuilibrator ships
no directionality classifier, Beber 2022 defines no reversibility index and
defers directionality to downstream tools, and the table there separates the
parts of EQ_HEURISTICS that are faithful reimplementation (the ln Gamma
formula, verified against 17,771 published values) from the parts that are
ours (the +/-1 sigma margin, the undecomposable and transport gates).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both corrections came from supervisor review of PR ModelSEED#285. The code is unchanged;
the characterisation of why it was needed was wrong.

1. "The EQ run never read eQuilibrator energies" was wrong. The design was
   correct: from c263e23 (2019) through 17c9739 (2023-09-11),
   Update_Reaction_eQuilibrator_Energies.py overwrote canonical deltag/deltagerr
   with the eQuilibrator values and stamped the EQU note, so
   `Estimate_Reaction_Reversibility.py EQ` reading deltag gated on EQU was
   reading eQuilibrator energies exactly as intended.

   3e50646 (2023-09-13) removed the write half -- moving the updater to
   thermodynamics['eQuilibrator'] only, dropping both the deltag overwrite and
   the EQU stamp -- and left the read half in place. Later GC rebuilds and the
   PR ModelSEED#265 promotion then rewrote deltag from other sources. Only 1,586 of the
   17,094 reactions still carrying EQU have deltag == the eQuilibrator value.
   EQU is a fossil of the pre-2023 pipeline, and is distinct from EQC (which
   the retrieval script writes to mean "all reagents had structures").

2. "The marker was read as an error bar" understated it. For the residual case
   GibbsEnergyPredictor.standard_dg returns Q_(0, "kJ/mol") outright -- the
   computed mean is discarded, not projected. standard_dg_prime then adds the
   Legendre/pH transform, so the nonzero dG'o stored for those reactions is the
   transform alone. Of 4,607 sigma-flagged reactions, 2,895 are this zeroed
   case (61 still exactly 0.000000, having no net proton change) and 1,712 have
   a real mean with an undetermined covariance direction.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Standalone companion to the heuristics write-up, aimed at a reader who wants to
follow the defect end to end rather than the design of the fix.

Covers: the pipeline step by step against origin/dev's code with file:line
references; what eQuilibrator returns when it cannot decompose a reaction
(standard_dg returns Q_(0, "kJ/mol") and standard_dg_prime adds the Legendre
transform on top, so the stored number is the transform alone); the 61
reactions still sitting at exactly 0.000000 because they have no net proton
change; the three separate breakages introduced by 3e50646 on 2023-09-13 and
why each later version preserved them; and why the 2026-07-05 eQuilibrator
rerun turned a handful of such records into 4,934.

Impact table contrasts dev (0 of 4,934 marked "?", 2,940 given a hard
direction) against this branch (4,933 marked "?"), and attributes dev's answers
to the specific GC rules that produced them -- 627 directional calls from
low_energy_heuristic, which reads the sign of what is only a pH transform.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Third companion document, focused on the defect where the EQ reversibility step
scores canonical deltag rather than the eQuilibrator estimate. Leads with
verification rather than assertion -- four independently reproducible checks:

  1. No script in Rerun_Thermodynamics.sh writes canonical deltag. The only
     writer is Promote_Reaction_Thermodynamics_to_Canonical.py, which is not in
     the rerun. deltag is static data, not pipeline output.
  2. Estimate_Reaction_Reversibility.py:74 nonetheless reads it as the energy
     source; reversibility_heuristics.py:94 looks up the eQuilibrator pair,
     discards it, and returns deltag tagged with eQuilibrator's label.
  3. deltag matches no currently-stored source for 49.4% of reactions, and for
     EQ-eligible reactions matches the eQuilibrator value only 7.2% of the time.
     Canonical deltagerr is never the eQuilibrator sentinel (0 of 4,934).
  4. Replaying both energies through the cascade gives different directions for
     1,575 reactions where eQuilibrator is confident (sigma <= 5 kcal/mol), e.g.
     rxn00003 flips '=' to '<' on a 1.13 kcal/mol input difference.

Traces the cause to 3e50646 (2023-09-13) deleting the writer's half of a
two-sided contract, explains why it failed silently (EQU is persisted data,
deltag stayed populated, the cascade cannot see provenance), and records that no
later commit repaired it.

Also states plainly what is NOT fixed: the GC path reads the wrong energy 79.3%
of the time for the identical structural reason, left alone deliberately so the
byte-identical GC column remains a control for PR ModelSEED#285. And that deltag is
currently neither pipeline output nor a curated field, which needs a decision.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Promotes the GC path from a "not fixed" footnote to a full section, with the
impact actually measured rather than inferred from the energy drift.

Simulating the full pipeline both ways (GC step then EQ step, in
Rerun_Thermodynamics.sh order) and comparing final canonical reversibility:
1,156 reactions differ. But the failure mode is not eQuilibrator's -- only 5 are
hard directional flips and 105 turn a confident call into "?", because the stale
values are mostly older GC numbers sitting a median of 2.07 kcal/mol from the
current ones, close enough that the cascade usually lands the same way.

The dominant effect is lost coverage: 843 reactions move from "?" to a confident
call. 892 reactions hold a stored Group-contribution energy that is unreachable
because canonical deltag is the 10000000 sentinel, so _energy_for returns None
and _incomplete_decision emits "?" despite the energy being present.

Records why the GC fix cannot be the same one-line swap the EQ fix was: of those
892, three carry |dG| > 1000 kcal/mol and five carry error > 100, and they are
currently invisible precisely because Promote_Reaction_Thermodynamics_to_Canonical.py
applies MAX_ABS_DG/MAX_ERR before writing deltag. Reading deltag is wrong but
protective -- it is accidentally acting as a quality filter. The EQ rule set
carries its own gate (eq_undecomposable_heuristic) and so needed no equivalent;
the GC cascade has none, since stored_bounds_heuristic abstains on a wide error
bar rather than rejecting it.

A follow-up PR should swap the GC source, port the plausibility guards into the
GC rule set, and report the ~884 recovered reactions as an intended coverage
gain. Kept out of PR ModelSEED#285 so the byte-identical GC column remains a control.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Restructured from 343 lines to 197, led by the mental model rather than the
evidence. The core is now: every reaction stores energies in two places (a
worked rxn01442 record where canonical deltag is visibly dGPredictor's number,
not eQuilibrator's), the two-step handoff that used to keep them in sync, and
what 3e50646 broke -- with a sticky-note/whiteboard analogy for a reader who
does not know the schema.

Verification is kept but compressed to the four numbers that carry it: no
pipeline script writes deltag, deltag matches no current source for 49.4% of
reactions, it is eQuilibrator's value only 7.2% of the time in the EQ path, and
1,575 directions differ. The long code excerpts and per-rule cascade tables are
dropped -- they live in the zero-energy companion note.

The GC section is retained in full since it carries the finding that changes the
follow-up plan: deltag is accidentally acting as a quality filter, so that fix
needs a plausibility gate ported from Promote_*, not just a source swap.

Also keeps the correction on record: the EQ step read eQuilibrator energies
correctly for four years; "never" was wrong.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant