Skip to content

Template missing rez ids - #151

Merged
EllieKallmier merged 5 commits into
mainfrom
template-missing-rez-ids
Oct 30, 2026
Merged

EllieKallmier merged 5 commits into
mainfrom
template-missing-rez-ids

Conversation

@EllieKallmier

Copy link
Copy Markdown
Member

Template N0/V0 non-REZ areas and map Q8a/b/c sub-areas to Q8

In the new-format templates, every unit's geo_id should exist in
network_geography. Two groups of 7.5 units broke that rule, and the CLI end-to-end
test carried placeholder exceptions for both:

  • N0/V0 (resolves Non-REZ IDs N0/V0 not templated in network_geography #133): AEMO's "non-REZ" areas in NSW and VIC. They host lower resource quality new-entrant VRE outside the formal REZs. They have their own traces, LCFs, connection costs and resource limits, but they aren't in renewable_energy_zones.
  • Q8a/b/c (Sub-REZ IDs: Q8a/b/c #150): three sub-areas of the Darling Downs REZ (Q8) that appear in
    the REZ ID column of the existing/planned summary. AEMO splits Q8 this way to
    reflect differences in resource quality within the REZ; the sub-areas aren't
    separate/actual REZs.

This change adds N0/V0 as REZ rows and maps Q8a/b/c back to Q8. The CLI test now
asserts that every templated geo_id is a real geography, with no exceptions.

src/ispypsa/templater/
├── geography.py          ← _NON_REZ_GEOGRAPHIES + _add_non_rez_geographies_to_rez_table
├── create_template.py    ← adds N0/V0 at the top of the new-format branch
├── transmission.py       ← N0/V0 excluded from the "absent from initial_transmission_limits" warning
└── existing_planned.py   ← Q8a/b/c → Q8 added to _IASR_TABLE_REPLACEMENTS
tests/
├── test_templater/       ← tests for the new geography helper and the warning exclusion
└── test_cli/             ← placeholder geo_id exceptions removed; expected counts include N0/V0

(Sorry for big text to follow - recording some context + decisions here to keep together)

Handling: N0/V0 as REZs, not on the sub-region bus

In the PLEXOS model: N0/V0 generators attach directly to the CNSW/SEV nodes. Doing
that here (by just setting geo_id to the corresponding subregion) would then require
special-case handling wherever data is keyed by REZ ID (traces, LCFs, connection costs,
resource limits), because N0/V0 wouldn't be geo_ids. As REZ rows, they go through
the existing path unchanged and retain their own information.

Their transmission limit is left blank. That follows the same path as any REZ
missing from initial_transmission_limits: one collapsed NaN row, which
the translator fills with transmission_default_limit. N0/V0 have no REZ
augmentation options, so they get no expansion options either.

The net effect matches the PLEXOS model, where there's no REZ limit. Group constraints
and other custom constraints apply by unit name, which doesn't change (unless the geo_id
in the name were a subregion - another pro of taking this approach). The cost is an extra bus
and link for each area, and some maybe not-perfectly-accurate naming.

Points to note:

  • I made the "REZs absent from initial_transmission_limits" warning exclude N0/V0
    (transmission._log_rez_paths_absent_from_limits), because hey'd otherwise trigger it
    on every run (and they're a known quantity there). But I'm happy to be overridden on
    that and add drop the exclusion.
  • The added rows are hand-maintained. _add_non_rez_geographies_to_rez_table raises if a
    future workbook adds N0/V0 to renewable_energy_zones itself.
  • Also added logging with an 'IASR TABLE EDIT' prefix - kind of a test run to get
    thoughts on that kind of approach for other edits/manual fixes on input tables?

Q8a/b/c mapped to Q8, not added as geographies

Most other tables keyed by REZ (renewable_energy_zones, LCFs, connection costs,
transmission limits, and the REZ ID column of new_entrants_summary) use 'Q8'. New
entrants already get geo_id Q8, so only existing/planned units change. The mapping
goes through the existing _IASR_TABLE_REPLACEMENTS list. That works on a copy, so
the raw sub-area IDs are still in iasr_tables.

AEMO does treat the sub-areas as distinct elsewhere: per-sub-area resource limits
(whose members include some existing/planned units), capacity factors, and
coefficients in the PLEXOS export constraints. The constraint coefficients are keyed
by unit name, so they're unaffected. Resource limits aren't templated yet - when they
are, they can read sub-area membership from the raw IASR tables (the IASR ID prefix
for new entrants, REZ ID for existing units). That work, including which existing
units count towards each limit, is in #150.

Note: This PR doesn't add a templater guard for a future workbook that splits another
REZ. An unmapped ID would pass through silently for now. I might add a guard/check
that covers this and a few other geo_id/geography-related workbook quirks in a subsequent
PR (see #131 for some of those
quirks).

@EllieKallmier EllieKallmier added type: feature New feature or request module: templater Covers contents of `templater` module labels Oct 14, 2026
@codecov

codecov Bot commented Oct 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines Coverage Δ
src/ispypsa/templater/create_template.py 93.40% <100.00%> (+0.07%) ⬆️
src/ispypsa/templater/existing_planned.py 100.00% <ø> (ø)
src/ispypsa/templater/geography.py 95.91% <100.00%> (+1.97%) ⬆️
src/ispypsa/templater/mappings.py 100.00% <100.00%> (ø)
src/ispypsa/templater/transmission.py 98.80% <100.00%> (+<0.01%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nick-gorman nick-gorman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good Ellie,

  • I think my main question relates more to how the Q8 sub zones are being handled for new entrants, so a bit off topic. For new entrants, how will the right resource traces be retreived if the geo_id is Q8 for new entrants in a, b, and c?
  • I think the exclusion of N0 and V0 from the missing limits warning is good.
  • I also like the IASR TABLE EDIT pattern, and think it would work to roll out to similar cases. Although a broader roll out might require a refactor. For example, adding logging to _apply_iasr_table_replacements could lead to repeated logging of the same fixes. The refactor would be to lift those fixes up to the top level in create_template and then pass the fixed tables down to the helpers. Probably not a this PR thing.
  • I like the idea of the guard against future REZ splitting, but yep, follow up PR sounds good.
  • Also looks like there is the potential for a bug in _apply_iasr_table_replacements, where if you tried to apply multiple corrections to the same table only the last one would be applied. Not an issue now but could easily happen in the future.

Nothing blocking

),
# Sub-REZ IDs Q8(a/b/c) in ECAA summary table 'REZ ID' column are not real REZs,
# but IDs used for resource-quality based build limits inside the Q8 REZ.
dict(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably a silly question, but is the reason that this works because the resource traces for the assocaited units are all mapped via their IASR ID, and while the a, b, and c areas are resource quality areas, they don't actuall get used for mapping to resource traces?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not at all, and yep - it's IASR ID based mapping for the resource traces (pretty much - I think there are some typos/slightly differently-named traces but the new config setup in the trace-parser also has an 'iasr_aliases' field to cover those cases too).

It is pretty similar for the new entrants too - not exactly the same ordering of name/id components but the traces are named like 'REZ_Q8a_Darling_Downs_SAT...', for the corresponding templated new entrant 'name'='Q8a_SAT_Darling Downs'. I thought about alternatives, but they'd have flow-on effects that I didn't feel made them worth it (yet, at least). Still leaving the related issue open to come back to cause I don't think it's 100% perfectly resolved, but this works for now I think...

@EllieKallmier
EllieKallmier merged commit 9332a82 into main Oct 30, 2026
15 checks passed
@EllieKallmier
EllieKallmier deleted the template-missing-rez-ids branch October 30, 2026 00:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module: templater Covers contents of `templater` module type: feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Non-REZ IDs N0/V0 not templated in network_geography

2 participants