Repository navigation
Feature: erwin import - #43
Merged
Merged
Conversation
Small additive IR fields the erwin importer needs so a real erwin model imports faithfully and a re-export round-trips: - LogicalEntity.physical_name and Attribute.physical_name (erwin Physical_Name, distinct from the logical name) - Relationship.verb_phrase / inverse_verb_phrase (erwin Parent_To_Child / Child_To_Parent verb phrases, e.g. "places" / "is placed by") All optional and defaulted to None, so a model that doesn't use them is unchanged and dumps clean (exclude_none drops them). Round-trip test confirms they survive write+load; core suite green.
The old import_erwin was written against an invented flat shape (<Entity name=
subject_area=>, parent=/child= by name) that does not match real erwin exports at all,
so it silently produced an empty model. Rewritten against the actual erwin Data Modeler
format (from erwin/schemas/*.xsd):
- Namespaced {http://www.erwin.com/dm}erwin -> {.../dm/data}Model, GUID-referenced
objects, properties as element TEXT inside a sibling <XxxProps> (not XML attributes).
- Streaming parse (ElementTree.iterparse + elem.clear() per top-level object) so a
7-10MB export never materialises as a DOM. Verified on a 2000-entity >1MB fixture.
- Two-pass: index every object by id, then resolve cross-refs (Parent/Child_Entity_Ref,
Key_Group_Member/Attribute_Ref, Parent_Domain_Ref, subject-area attached-object arrays).
- Maps: Entity -> Conceptual+Logical (logical name lowercased, Physical_Name kept),
per-entity Key_Group -> KeyGroup (pk/alternate/unique), Relationship -> Relationship
(child=from/many, parent=to/one, verb phrases, cardinality, identifying), subtype
relationship -> Category (with discriminator), Subject_Area -> SubjectArea membership,
Domain -> Domain. FK columns inherit the parent PK's domain (erwin type migration).
- Defensive: unknown groups (View, ER_Diagram, transforms) and dangling refs skip with a
warning, never fatal. Returns ErwinImport(model, warnings).
9 tests on a real-shaped fixture: entities/attrs/keys, relationship + verb phrases,
subtype->Category, subject-area membership, domains, skip-with-warnings, the imported
model validates clean, and the streaming large-export path. CLI wiring follows in the
next commit.
…ply bridge)
The canvas /api/import and the CLI apply path consume to_commands's {op,payload} list,
but to_commands only accepts the thin table-oriented ImportedModel. Erwin produces a full
IR Model (subject areas, categories, domains, conceptual entities, keys) that would be
flattened away going through ImportedModel.
model_to_commands(model) walks a full Model and emits the same op vocabulary
(create_subject_area, create_domain, create_entity with subject-area home tag,
set_subject_area_members for the inclusion list, add_attribute, create_key_group,
create_relationship, create_category with discriminator), minting fresh ULIDs in a
dependency-safe order. This lets the erwin importer ride the exact preview/apply path the
simple importers use, without losing its richness.
3 round-trip tests (structure reconstructs, the result validates clean, a category with a
discriminator round-trips). apply_command reconstructs the model faithfully.
…o an existing one Rewrites the erwin import command for the new streaming reader + rich Model: - passes the file PATH to import_erwin so the 7-10MB streaming path is used - default: write a fresh model at --out (an erwin export is a whole model, like reverse), with the same no-clobber divert to a -reversed-v<N> sibling unless --force - --apply: merge the erwin objects INTO an existing model at --out via model_to_commands -> apply_command (the validated mutation engine), for adding erwin to a model you have - surfaces the reader's warnings as yellow note: lines 3 CLI tests: write-a-model (validates, notes shown), --apply into an existing model (merges, validates), and the divert-on-existing behaviour. 471 tests pass across reverse/cli/emit-erd/core.
- Server /api/import gains an `erwin` branch: import_erwin(content) → model_to_commands,
returning the same {ok, changes, tables, warnings} shape as the other formats, so the
canvas previews/applies it through the existing path. Uses the rich Model bridge, not
the thin table parsers, so subject areas / categories / domains survive.
- Canvas Import menu: adds an "erwin XML" format option and `.xml` to the file accept.
Rebuilt into the server static.
2 server tests (erwin returns the right ops + warnings; empty content is 422). Canvas
typechecks + builds.
…ight-click Three entry points for the erwin importer, no dedicated panel frame (import is a one-shot action, not monitored state — a button-only frame would be the blank-void anti-pattern): - `modelith.importErwin` command -> Cmd+Shift+P "Modelith: Import Erwin XML" (file picker, .xml filter). Resolves a target dir via the reverse no-clobber flow (update in place vs new folder), runs `mdl import erwin <file> -o <dir>` under progress (5-min timeout for 10MB files), then opens the canvas on the result and refreshes the panels. Surfaces the reader's notes. - Reverse Review title-bar button (beside Reverse / Reverse Live / Compare) — erwin import is a "bring an external model in" action, its natural home. - Explorer right-click on any .xml -> "Import Erwin XML", passing the clicked URI so it skips the picker. All 6 regression gates pass (42 commands registered); typecheck clean.
Documents `mdl import erwin` (streaming, rich mapping — keys, relationships + verb phrases, subtype categories, subject areas, domains, physical names; graceful skip of views/layout/transforms) and the three VS Code entry points.
…and on edges
The importer populated verb_phrase/inverse_verb_phrase and physical_name, but they were
invisible in the UI — imported data the user couldn't see or edit. Now fully surfaced:
- Projection emits verb_phrase/inverse_verb_phrase per relationship and physical_name per
entity and attribute.
- Inspector: a "physical" name field under the entity title (editable), a physical column
name beside an attribute when it differs, and the relationship rows show the two verb
phrases ("is party to" / "is with") inline and editable.
- ER edge label shows the relationship's verb phrase when selected (the erwin reading
"<parent> <verb> <child>"), falling back to the name.
- Editing rides the existing command engine: update_relationship gains verb_phrase/
inverse_verb_phrase, update_attribute + rename_entity gain physical_name (empty clears).
Verified in the browser against an erwin-imported model: the Inspector shows PHYSICAL
COUNTERPARTY and the "is party to / is with" verb phrases; edits persist. 4 tests
(projection emits, commands edit all four fields). Canvas rebuilt; extension 0.3.14.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Title: Real erwin XML import — streaming parser, rich mapping, three entry points, surfaced in the canvas
Body:
What this changes
Turns the parked erwin import into a real feature. The existing importer was written against an invented flat XML shape and silently produced empty models on real exports — this rewrites it against the actual erwin Data Modeler schema (from the extracted
erwin/schemas/*.xsd) and surfaces the imported data end to end.mdl_reverse/erwin.pyis rewritten: namespaced, GUID-referenced, properties read as element text inside<XxxProps>. It usesET.iterparse+elem.clear()so a 7-10MB export never materializes as a DOM (verified on a 2000-entity, >1MB fixture), then resolves cross-references in a second pass.Model, and a newmodel_to_commands()bridges it to the same{op, payload}list the canvas/CLI apply path uses — so subject areas, categories, and domains survive instead of being squeezed through the thin table-only importer.mdl import erwin <file> -o <dir>writes a fresh model (with the reverse no-clobber divert);--applymerges into an existing model via the mutation engine.Modelith: Import Erwin XML), a Reverse Review title-bar button, and an.xmlExplorer right-click.erwinbranch in/api/import.update_relationship,update_attribute,rename_entitygained the new fields).Small additive IR fields make this faithful:
Relationship.verb_phrase/inverse_verb_phrase,LogicalEntity.physical_name,Attribute.physical_name— all optional, so existing models are unchanged.How it was tested
uv run pytest— 900 passed, 1 skipped. New tests: the reader on a real-shaped fixture (entities/keys, relationship + verb phrases, subtype → category, subject-area membership, domains, skip-with-warnings, the model validates, streaming large export);model_to_commandsround-trips; the CLI (write /--apply/ divert); the/api/importerwin branch; and the projection-emits + commands-edit the new fields.uv run ruff check packages/— clean.npm run typecheck && npm run regression(Node 20) — all 6 gates, 42 commands registered.npm run build(Node 20) — typechecks + builds into the server static.Versions
CLI 0.6.6, extension 0.3.14.