diff --git a/.gitignore b/.gitignore index 666715b..6d36814 100644 --- a/.gitignore +++ b/.gitignore @@ -26,6 +26,14 @@ demo/**/target/ demo/**/logs/ demo/**/dbt_packages/ +# demo recorder toolchain (tapes/pages are committed; the recorder's deps + rendered +# media are regenerable — `demos/standalone/build.mjs` + `record.mjs` rebuild them). +# The committed copies live where they're used: README GIFs in docs/assets/demos/, +# landing videos in demos/landing/media/. +demos/standalone/node_modules/ +demos/standalone/package-lock.json +demos/standalone/out/ + # duckdb build artifacts *.duckdb *.duckdb.wal @@ -39,3 +47,6 @@ demo/**/dbt_packages/ # Vendored erwin XSD schemas — reference only, must NOT ship in a public repo. erwin/ + +# Modelith CLI demo renders (regenerable via demos/render.sh) +demos/out/ diff --git a/README.md b/README.md index 6f88dc1..6e3da00 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,10 @@ owns state. [](#license) [](https://docs.astral.sh/ruff/) + + +Design in git → generate dbt → compile to a contract. [See all six flows ↓](#video-demos) +  ## Try it in 2 minutes @@ -42,6 +46,46 @@ term — the demo starts a local ontology server for you. See [Ontology and knowledge graph, step by step](#ontology-and-knowledge-graph-step-by-step) for the ontology workflow. +## Video demos + +Six flows, end to end. Every command and every line of output is real — the `mdl` CLI on the +left, the live ER diagram on the right. + +### Design-first — a blank repo to running dbt +Scaffold a model, add entities (ULIDs minted for you), validate, and compile to +contract-enforced dbt plus an ODCS data contract. + + + +### Adopt a warehouse — reverse a legacy dbt project +Point it at a warehouse with no data model: staging excluded, SCD2 and Data Vault detected, +surrogate keys stripped, every inference in a reviewable ledger. + + + +### The drift gate — model and warehouse in lockstep +A column lands in the warehouse; drift classifies it (additive, not breaking) and reconcile +folds the safe change into the model. Nothing is ever lost. + + + +### Ontology anchoring — align to an industry vocabulary +Search FIBO, align an entity to a standard term, and track coverage — reproducible, offline. + + + +### Erwin import — bring an erwin model into git +Import an erwin XML export straight into clean logical YAML you can version, diff, and +generate from. + + + +### One model, many targets — a contract factory +From one definition: contract-enforced dbt, an ODCS contract, Pydantic, SQL DDL, Mermaid, +DBML, RDF/SHACL, and a Neo4j graph schema — each generated deterministically. + + + ## Why Warehouse teams keep their meaning in three disconnected places: an ER tool that diff --git a/demos/DESIGN-BRIEF.md b/demos/DESIGN-BRIEF.md new file mode 100644 index 0000000..7a01734 --- /dev/null +++ b/demos/DESIGN-BRIEF.md @@ -0,0 +1,210 @@ +# Design brief: Modelith animated demo (code-based, for `/design`) + +Paste this whole brief into `/design`. It builds **one canvas with 3 artboards**, each an +**animated terminal + ER diagram** demo of one Modelith story. Everything below is real — +real `mdl` commands, real output, real entities — captured from the working CLI. Do not invent +commands or output; use exactly what's here. + +## Ground it in the REAL repo (read these first) + +This animation must reproduce the **real Modelith Studio canvas**, not an approximation. Before +building, read these in the repo: + +- **`demos/design-assets/canvas-reference.jpg`** — a real screenshot of the live Studio canvas + (the IBOR model). This is the visual target: the exact entity-card look, the amber top border, + the gold-key PK rows, the dim mono types, the dashed crow's-foot relationship lines, the + warm-charcoal dotted background. Match it. +- **`demos/design-assets/canvas-tokens.css`** — the REAL design tokens and entity-card CSS, + extracted verbatim from `canvas/src/styles.css`. Use these exact values and class structure + for the ER cards in the animation. +- For deeper fidelity, the real source is `canvas/src/styles.css` (the `:root` tokens, + `.entity-node`, `.entity-header`, `.badge`, `.attr-row`) and `canvas/src/EntityNode.tsx` + (the card markup: header + badges, a keys section, then the attribute rows). + +The terminal panel is a plain dark terminal (same background/mono font); it doesn't need to copy +a specific app chrome. + +--- + +## What to build + +A horizontal filmstrip canvas, **3 artboards** (16:9, 1920×1080 each), named: +`Main.dc.html` = **1-Design-first**, `Adopt.dc.html` = **2-Adopt-a-warehouse**, +`Drift.dc.html` = **3-Drift-gate**. + +Each artboard is a **split scene**: a terminal panel on the LEFT (~55% width) and a live +**ER diagram** on the RIGHT (~45%). The terminal types commands and reveals output; the ER +diagram animates in sync (entities appear, columns highlight, a change folds in). Animations +loop or play once on view — your call; prefer a ~12–18s timed sequence per artboard that loops. + +## Visual identity (match the Modelith product) + +- Background `#14110f` (warm charcoal). Panels `#1c1815`, cards `#221d19`, borders `#33291f`. +- Text `#ece6dd`, dim text `#a89a86`. **Accent amber `#e0a63a`** (the brand color). +- Success green `#6cbf73`, warn/amber `#e6a94a`, error `#f0857a`, PK gold `#f2c14e`. +- Monospace for the terminal: "JetBrains Mono", ui-monospace, Menlo. Sans for labels: Inter. +- Terminal prompt is `$`. Typed commands animate character-by-character (~40ms/char feel). + Output reveals as a block after a short beat. Keep a blinking amber cursor. +- ER entity cards: rounded, dark card, a 3px amber top border, a bold entity name header, then + attribute rows (name left, type right). PK rows show a gold `PK` chip; FK rows an amber `FK` + chip. Relationship lines connect FK column to the referenced entity (crow's-foot if easy). + +--- + +## Artboard 1 — "Design-first" (greenfield) + +**Tagline (small, top):** "Design in git. Compile to contract-enforced dbt." + +**Terminal sequence (type + reveal, in order):** + +``` +$ mdl model render +# model: shop +entity customer customer_id [pk], email, full_name, created_at +entity order order_id [pk], customer_id, order_date, status +entity order_line order_line_id [pk], order_id, product_id, quantity +entity product product_id [pk], sku, name, unit_price + +$ mdl new entity payment --definition "A settlement against an order" +created entity 'payment' (01M3WX64NZ...) + +$ mdl validate +validation passed + +$ mdl generate +wrote 9 files to target/dbt + +$ mdl export contract +wrote contract to datacontract.yaml +``` + +**ER diagram animation (synced):** +- As `model render` reveals, the 4 entity cards **fade/slide in**: `customer`, `order`, + `order_line`, `product`. Draw FK relationship lines: `order.customer_id → customer`, + `order_line.order_id → order`, `order_line.product_id → product`. +- On `mdl new entity payment`, a **5th card `payment` animates in** (amber glow pulse), with a + `payment_id [pk]` row (it's a fresh entity, minimal attrs). +- On `mdl validate` → a green "✓ valid" badge pulses over the diagram. +- On `mdl generate` / `export contract` → two small output chips slide in at the bottom: + a dbt logo-ish chip "dbt • contract enforced" and a "ODCS data contract" chip — the + "one model, many targets" idea. Keep subtle. + +**Entity/attribute data (use verbatim):** +- customer: customer_id (bigint, PK), email (string), full_name (string), created_at (timestamp) +- order: order_id (bigint, PK), customer_id (bigint, FK→customer), order_date (date), status (string) +- order_line: order_line_id (bigint, PK), order_id (bigint, FK→order), product_id (bigint, FK→product), quantity (integer) +- product: product_id (bigint, PK), sku (string), name (string), unit_price (decimal) +- payment: payment_id (bigint, PK) [appears on `new entity`] + +--- + +## Artboard 2 — "Adopt a warehouse" (brownfield / reverse) + +**Tagline:** "Point it at a legacy warehouse. Get a clean model back." + +**Terminal sequence:** + +``` +$ ls models/staging models/marts models/vault +stg_customers.sql stg_orders.sql int_order_enriched.sql +dim_customer.sql dim_product.sql fct_orders.sql +hub_customer.sql link_order.sql sat_customer_details.sql + +$ mdl reverse --project target/manifest.json --out owned +reversed 6 entities (4 staging/intermediate excluded) + kept as business entities (4): dim_customer, dim_product, fct_orders, hub_customer + excluded as staging/intermediate (4): stg_customers, stg_orders, ... + SCD2 pattern detected (2): dim_customer, dim_product + Data Vault detected (3): hub_customer, link_order, sat_customer_details + surrogate keys stripped (7) + +$ mdl validate -m owned +validation passed +``` + +**Reverse output must keep the color coding:** "kept as business entities" in GREEN, "managed +but no business key" in AMBER, "excluded" in BLUE-ish, "SCD2 / Data Vault" headers in blue, +"surrogate keys stripped" header in blue. This color-coded classification is the hero of the shot. + +**ER diagram animation:** +- Start with a **messy pile** of raw SQL file chips (stg_*, dim_*, fct_*, hub_*, link_*, sat_*) + scattered/greyed on the right. +- On `mdl reverse`, animate the transform: staging chips **fade out** (excluded); the business + entities **snap into clean ER cards** — `dim_customer [SCD2]`, `dim_product [SCD2]`, + `fct_orders`, `hub_customer [HUB]` — with `[SCD2]`/`[HUB]`/`[LINK]` badge tags. Draw FK lines: + `fct_orders.customer_id → dim_customer`, `fct_orders.product_id → dim_product`. +- Small floating annotations pulse as each pattern is "detected": a tag "SCD2" on dim_customer, + "Data Vault" on the hub/link/sat group, "surrogate key stripped" briefly on a removed `_sk` + column. These mirror the real detection. + +**Entity/attribute data (verbatim from the real reverse):** +- dim_customer [SCD2]: customer_id (integer, PK), name (string), tier (string) +- dim_product [SCD2]: product_id (integer, PK), sku (string), category (string), list_price (decimal) +- fct_orders: order_id (integer, PK), customer_id (FK→dim_customer), product_id (FK→dim_product), order_date (date), amount (decimal) +- hub_customer [HUB]: customer_id (integer, PK), load_dts (date) +- link_order [LINK], sat_customer_details — shown as tagged cards, fewer attrs + +--- + +## Artboard 3 — "The drift gate" (the climax) + +**Tagline:** "The model and the warehouse, always in lockstep." + +**Terminal sequence (this is the emotional peak — pace it):** + +``` +$ mdl drift -m model --manifest target/manifest.json +no drift detected + +# a teammate adds a broker_code column to the warehouse +$ mdl drift -m model --manifest target/manifest.json --check +drift vs target 'duckdb_dev': + [additive] 1 + - column transaction.broker_code exists in dbt but not the model + +$ mdl drift -m model --manifest target/manifest.json --reconcile +reconciled 1 delta(s); skipped 0 breaking + + add column transaction.broker_code + +$ mdl drift -m model --manifest target/manifest.json +no drift detected +``` + +**Color:** `no drift detected` neutral/green; `[additive] 1` in AMBER; `reconciled 1 delta(s); +skipped 0 breaking` in GREEN; `+ add column transaction.broker_code` with a green `+`. + +**ER diagram animation (the key moment):** +- Center the **`transaction`** entity card: transaction_id [PK], trade_date, amount, + portfolio_code [FK→portfolio], counterparty_id [FK→counterparty]. (Optionally small + portfolio + counterparty cards to its sides with FK lines.) +- Scene 1 — a steady state; a subtle "model ⇄ warehouse in sync" indicator (two panels linked, + green). +- Scene 2 — a **new `broker_code` row** appears on the WAREHOUSE side only (right), glowing + AMBER, with a floating "additive" tag. The sync indicator turns amber. This is drift. +- Scene 3 — on `--reconcile`, the `broker_code` row **animates from the warehouse side INTO the + model's `transaction` card** (slides over, snaps in, turns from amber to normal). The sync + indicator returns to GREEN. A small "nothing lost" / "✓ reconciled" note pulses. +- End on both sides matching, green, calm. + +**Entity data (verbatim):** +- transaction: transaction_id (string, PK), trade_date (date), amount (decimal), + portfolio_code (string, FK→portfolio), counterparty_id (bigint, FK→counterparty) +- the added column: broker_code (string) — additive, folded in by reconcile. + +--- + +## Overall feel + +- Calm, confident, "engineering-grade" — not flashy. Amber accent used sparingly for emphasis + (the cursor, a badge, a glow on the key moment). Everything else restrained and dark. +- Each artboard should read in one glance (what story) and reward watching (the animation). +- No fake UI chrome that impersonates another product. The terminal + ER diagram are generic + Modelith surfaces. +- If sound/voice is off the table (it is, in /design), let the on-screen taglines + the typed + commands carry the narrative. + +## Source of truth + +These flows are verified real. The matching VHS reference clips and the exact `.tape` scripts +are under `demos/tapes/films/`, and the live commands run against `demos/fixtures/shop`, +`demo/legacy-warehouse`, and `demo/ibor`. Nothing here is mocked. diff --git a/demos/README.md b/demos/README.md new file mode 100644 index 0000000..b86212f --- /dev/null +++ b/demos/README.md @@ -0,0 +1,72 @@ +# Modelith CLI demos + +Three short **films** of the `mdl` CLI, each a realistic project followed end to end as one +continuous session — the project is set up once and evolves on screen, no mid-story resets. +Scripted with [VHS](https://github.com/charmbracelet/vhs) (not screen-captured) so every take +is identical, typo-free, re-renderable when the CLI changes, and visually consistent. + +**Everything is real** — real `mdl` commands, real `dbt parse`, real output. Changes are shown +on-camera; nothing is faked or staged. + +## The three films + +### 1 — Design-first (greenfield, ~45s) +From a blank repo to running dbt + a data contract. +`mdl model render` (the designed model) → `mdl new entity` (mints ULIDs) → `mdl validate` → +`mdl generate` (protected regions + an ENFORCED contract) → `mdl export contract` (one model → +an ODCS data contract too). **Story: design in git, compile to more than dbt.** + +### 2 — Adopt a warehouse (brownfield, ~55s) +Take over a legacy dbt project that has no data model. +`ls` the warehouse (staging / marts / vault) → `mdl reverse` (real SCD2 / Data Vault / +surrogate-key detection) → `mdl decisions list` (every inference, confidence-tagged, reviewable) +→ `mdl validate` → `mdl model render` (clean owned model). **Story: inherit a mess, own it in +one command, nothing is a black box.** + +### 3 — The drift gate (the climax, ~60s) +The model and the warehouse stay in lockstep. +`mdl drift` (clean) → a teammate adds a `broker_code` column to a dbt model + its contract (a +real edit) → `dbt parse` → `mdl drift --check` (real: classified **additive**) → `mdl drift +--reconcile` (real: folds the safe change into the model) → `mdl drift` (clean) + `mdl validate`. +**Story: drift is caught and classified; safe change folds in; work is never lost.** + +The Studio **canvas** is a browser UI, not terminal output — recorded separately as a browser +screen capture, not a VHS tape. (Spotlight clips for structural merge, multi-target compile, and +the AI/MCP surface can be added the same way these films are built.) + +## Rendering + +Requirements (macOS): + +```bash +brew install vhs ffmpeg +brew install --cask font-jetbrains-mono +uv sync # so .venv/bin/mdl exists — the tapes put it on PATH for a fast, clean `mdl` +``` + +Then: + +```bash +demos/render.sh # render all films → demos/out/*.mp4 +demos/render.sh 3 # just film 3 +``` + +## How the tapes work + +- **`tapes/_settings.tape`** — shared look: JetBrains Mono 22px, 1280×900 (tall enough that a + continuous session doesn't scroll earlier scenes off the top), the Modelith amber theme, 70ms + typing. `Source`d by every film; tune once here. +- Each film has ONE **hidden setup block** (`Hide … Show`) that puts a fast `mdl` on PATH from + the project venv and copies the film's fixture to a throwaway `/tmp/mdl-film*` dir. After that + the whole arc runs in one session; the on-screen `clear`s are scene breaks, not resets. +- `setopt interactivecomments` lets the narration `# …` lines show as comments in zsh. +- Timing is **fixed `Sleep`s** sized to the CLI's ~2s startup plus a reading beat (not `Wait`, + which raced the slow start and echoed keystrokes). +- Fixtures: the bundled `demo/` models (ibor, legacy-warehouse), the greenfield `fixtures/shop` + model, and `fixtures/add-broker-code.sh` (the real warehouse edit for film 3). + +## Re-rendering when the CLI changes + +The films drive the real `mdl` + `dbt`, so a change can alter the output. After upgrading: +`uv sync`, then `demos/render.sh`, and watch the new films. If a command's flags changed, edit +the matching `.tape` (commands are plain text) and re-render just that one. diff --git a/demos/design-assets/canvas-reference.jpg b/demos/design-assets/canvas-reference.jpg new file mode 100644 index 0000000..bafda7c Binary files /dev/null and b/demos/design-assets/canvas-reference.jpg differ diff --git a/demos/design-assets/canvas-tokens.css b/demos/design-assets/canvas-tokens.css new file mode 100644 index 0000000..1e4e963 --- /dev/null +++ b/demos/design-assets/canvas-tokens.css @@ -0,0 +1,63 @@ +/* REAL design tokens + entity-card CSS, extracted verbatim from the Modelith Studio canvas + (canvas/src/styles.css). Use these EXACT values in the animation so the ER diagram matches + the real product. The screenshot canvas-reference.jpg shows how these render live. */ + +:root { + --bg: #14110f; /* warm charcoal background */ + --bg-raised: #1c1815; /* panels, toolbar */ + --bg-card: #221d19; /* entity cards, inputs */ + --border: #33291f; + --border-strong: #4a3c2c; + --text: #ece6dd; + --text-dim: #a89a86; + --accent: #e0a63a; /* brand amber */ + --accent-soft: rgba(224, 166, 58, 0.15); + --accent-ink: #241a08; + --pk: #f2c14e; /* primary-key gold */ + --err: #f0857a; + --warn: #e6a94a; + --ok: #6cbf73; /* success green */ + --font: "Inter", -apple-system, "Segoe UI", Roboto, sans-serif; + --mono: "JetBrains Mono", ui-monospace, "Cascadia Code", Menlo, monospace; +} + +/* --- the real entity card (ER node) --- */ +.entity-node { + width: 264px; background: var(--bg-card); + border: 1px solid var(--border); + border-top: 3px solid var(--accent); /* amber top edge — the signature */ + border-radius: 10px; overflow: hidden; + box-shadow: 0 4px 18px rgba(0, 0, 0, 0.35); + font-size: 12px; +} +.entity-header { + display: flex; align-items: center; justify-content: space-between; + padding: 10px 12px; background: rgba(255, 255, 255, 0.03); + border-bottom: 1px solid var(--border); +} +.entity-name { font-weight: 700; font-size: 13px; letter-spacing: 0.01em; color: var(--text); } +.entity-badges { display: flex; gap: 4px; } +/* pattern badge: SCD2 / HUB / LINK / SAT tags (amber pill). ontology badge is blue. */ +.badge { + font-size: 10px; padding: 1px 6px; border-radius: 99px; + background: var(--accent-soft); color: var(--accent); + text-transform: uppercase; letter-spacing: 0.05em; font-weight: 600; +} +.badge.ontology { background: rgba(147, 197, 253, 0.12); color: #93c5fd; } + +/* the PK/FK key rows sit in their own faintly-gold section at the top of the card body */ +.attr-section.keys { border-bottom: 1px solid var(--border); background: rgba(252, 211, 77, 0.04); } +.attr-section { padding: 4px 0; } +.attr-row { + display: flex; align-items: center; gap: 6px; + padding: 3px 12px; height: 26px; line-height: 20px; +} +.attr-icon { width: 14px; text-align: center; font-size: 10px; flex: none; } /* role glyph, e.g. a key for PK */ +.attr-name { flex: 1; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; color: var(--text); } +.attr-name.pk { color: var(--pk); font-weight: 600; } /* primary key rows render in gold */ +.attr-type { color: var(--text-dim); font-family: var(--mono); font-size: 10px; flex: none; } /* type, right-aligned, mono, dim */ + +/* relationship lines between cards: thin, dim, dashed; crow's-foot at the "many" end */ +.rel-path { stroke: var(--border-strong); stroke-width: 1.6; } /* in the live canvas these are dashed amber-grey */ + +/* the canvas background is var(--bg) with a faint dot grid (#33291f dots, ~22px gap) */ diff --git a/demos/fixtures/add-broker-code.sh b/demos/fixtures/add-broker-code.sh new file mode 100644 index 0000000..83a4363 --- /dev/null +++ b/demos/fixtures/add-broker-code.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +# A teammate adds a broker_code column to the transaction model and its contract. +set -e +sql=transform/warehouse/models/transaction.sql +schema=transform/warehouse/models/schema.yml + +# add the column to the model's SELECT (a real dbt column) +perl -0pi -e 's/( counterparty_id\n)(from source\n-- mdl:generated-end)/$1 , cast(null as varchar) as broker_code\n$2/' "$sql" + +# declare it in the enforced contract (so dbt knows its type) +python3 - "$schema" <<'PY' +import sys +f = sys.argv[1] +s = open(f).read() +i = s.find("- name: transaction\n") +j = s.find("columns:\n", i) + len("columns:\n") +block = (" - name: broker_code\n" + " data_type: VARCHAR\n" + " meta:\n" + " mdl_ulid: 01M3XBROKERCODE0000000000X\n") +open(f, "w").write(s[:j] + block + s[j:]) +PY +echo "warehouse: + transaction.broker_code (SQL + contract)" diff --git a/demos/fixtures/shop/.gitignore b/demos/fixtures/shop/.gitignore new file mode 100644 index 0000000..ef142c7 --- /dev/null +++ b/demos/fixtures/shop/.gitignore @@ -0,0 +1,9 @@ +# dbt build artifacts (generated projects live here by default) +target/ +dbt_packages/ +logs/ +# Fetched ontology layers — pinned by .mdl/lock.yaml, not committed (spec §3). +# Run `mdl ontology fetch` to repopulate. Like node_modules to package-lock.json. +.mdl/ontology-cache/ +# Reverse/generation working state (the committed lock/decisions stay tracked). +.mdl/state/ diff --git a/demos/fixtures/shop/.mdl/lock.yaml b/demos/fixtures/shop/.mdl/lock.yaml new file mode 100644 index 0000000..84c4e63 --- /dev/null +++ b/demos/fixtures/shop/.mdl/lock.yaml @@ -0,0 +1,6 @@ +dbt: '1.9' +osi: + tag: osi-0.1.1-rc1 + version: 0.1.1 +vocabularies: {} +profiles: {} diff --git a/demos/fixtures/shop/conceptual/entities/customer.yaml b/demos/fixtures/shop/conceptual/entities/customer.yaml new file mode 100644 index 0000000..416d784 --- /dev/null +++ b/demos/fixtures/shop/conceptual/entities/customer.yaml @@ -0,0 +1,3 @@ +id: 01M3WS6GDGNDJSQ8J69WPTEC02 +kind: conceptual_entity +name: Customer diff --git a/demos/fixtures/shop/conceptual/entities/order.yaml b/demos/fixtures/shop/conceptual/entities/order.yaml new file mode 100644 index 0000000..7b30bf8 --- /dev/null +++ b/demos/fixtures/shop/conceptual/entities/order.yaml @@ -0,0 +1,3 @@ +id: 01M3WS6MYM4HWTS7Z022BFS98T +kind: conceptual_entity +name: Order diff --git a/demos/fixtures/shop/conceptual/entities/order_line.yaml b/demos/fixtures/shop/conceptual/entities/order_line.yaml new file mode 100644 index 0000000..651d06c --- /dev/null +++ b/demos/fixtures/shop/conceptual/entities/order_line.yaml @@ -0,0 +1,3 @@ +id: 01M3WS6Q5Y8C3J4GH1B3Q3TCDR +kind: conceptual_entity +name: OrderLine diff --git a/demos/fixtures/shop/conceptual/entities/product.yaml b/demos/fixtures/shop/conceptual/entities/product.yaml new file mode 100644 index 0000000..efff24c --- /dev/null +++ b/demos/fixtures/shop/conceptual/entities/product.yaml @@ -0,0 +1,3 @@ +id: 01M3WS6JBQ4NRR17WNYA2N7527 +kind: conceptual_entity +name: Product diff --git a/demos/fixtures/shop/logical/domains/identifier_bigint.yaml b/demos/fixtures/shop/logical/domains/identifier_bigint.yaml new file mode 100644 index 0000000..b89b16f --- /dev/null +++ b/demos/fixtures/shop/logical/domains/identifier_bigint.yaml @@ -0,0 +1,5 @@ +id: 01M3WS6D96RM44FGPXP6CQ52VA +kind: domain +name: identifier_bigint +base_type: bigint +definition: A surrogate or business identifier stored as a 64-bit integer. diff --git a/demos/fixtures/shop/logical/entities/customer.yaml b/demos/fixtures/shop/logical/entities/customer.yaml new file mode 100644 index 0000000..1c642eb --- /dev/null +++ b/demos/fixtures/shop/logical/entities/customer.yaml @@ -0,0 +1,25 @@ +id: 01M3WS6GDGEEH1Y8WHQ6MV3V3F +kind: logical_entity +name: customer +realises: 01M3WS6GDGNDJSQ8J69WPTEC02 +attributes: +- id: 01NHT44FHT243W0EC7NYE7XT5G + name: customer_id + domain: identifier_bigint + role: business_key + nullable: false +- id: 019KNST7XJ2DKXTC87G0EA3HKH + name: email + domain: string + role: attribute + nullable: false +- id: 019GDP66FGFQKT68CFVR3ZM8FQ + name: full_name + domain: string + role: attribute + nullable: false +- id: 01QP2BSPBXG463YDY605ZJ69J9 + name: created_at + domain: timestamp + role: attribute + nullable: false diff --git a/demos/fixtures/shop/logical/entities/order.yaml b/demos/fixtures/shop/logical/entities/order.yaml new file mode 100644 index 0000000..d20dd5e --- /dev/null +++ b/demos/fixtures/shop/logical/entities/order.yaml @@ -0,0 +1,25 @@ +id: 01M3WS6MYM4WC27B98H8N0FMEE +kind: logical_entity +name: order +realises: 01M3WS6MYM4HWTS7Z022BFS98T +attributes: +- id: 01FRDRCCE2C2AE7XW43T7G043M + name: order_id + domain: identifier_bigint + role: business_key + nullable: false +- id: 014BKSP4BRDZZSKSTC1T5T3SVK + name: customer_id + domain: identifier_bigint + role: attribute + nullable: false +- id: 01A7ZKYFT9P1Y63R1W01RC442D + name: order_date + domain: date + role: attribute + nullable: false +- id: 010BHNZSXXY1G3J01ZSR27NP9R + name: status + domain: string + role: attribute + nullable: false diff --git a/demos/fixtures/shop/logical/entities/order_line.yaml b/demos/fixtures/shop/logical/entities/order_line.yaml new file mode 100644 index 0000000..bdfdc6b --- /dev/null +++ b/demos/fixtures/shop/logical/entities/order_line.yaml @@ -0,0 +1,25 @@ +id: 01M3WS6Q5YM32P4ZWX3DQQF4TG +kind: logical_entity +name: order_line +realises: 01M3WS6Q5Y8C3J4GH1B3Q3TCDR +attributes: +- id: 01E27JA246C69XXSNHG3G5NXSM + name: order_line_id + domain: identifier_bigint + role: business_key + nullable: false +- id: 01WCBNMDRE9XY1MFNG29G1XKHH + name: order_id + domain: identifier_bigint + role: attribute + nullable: false +- id: 01EBXNY3SSNY6DVMFR1W3KQQVS + name: product_id + domain: identifier_bigint + role: attribute + nullable: false +- id: 01M260C4FKNHRBXRFKY69QG3R3 + name: quantity + domain: integer + role: attribute + nullable: false diff --git a/demos/fixtures/shop/logical/entities/product.yaml b/demos/fixtures/shop/logical/entities/product.yaml new file mode 100644 index 0000000..5057fa5 --- /dev/null +++ b/demos/fixtures/shop/logical/entities/product.yaml @@ -0,0 +1,25 @@ +id: 01M3WS6JBQXS2JC5461EEYEZJD +kind: logical_entity +name: product +realises: 01M3WS6JBQ4NRR17WNYA2N7527 +attributes: +- id: 01A3Y3VYBXYFQKQP4CBWFG7NY9 + name: product_id + domain: identifier_bigint + role: business_key + nullable: false +- id: 0146BZKSG0FJ1T646003JCC9NK + name: sku + domain: string + role: attribute + nullable: false +- id: 01KG5MDJ2A5Y3RDSHQZNGFRBXT + name: name + domain: string + role: attribute + nullable: false +- id: 0121ZQHKJC7J9SQKXPA9SQY3ME + name: unit_price + domain: decimal + role: attribute + nullable: false diff --git a/demos/fixtures/shop/mdl-project.yaml b/demos/fixtures/shop/mdl-project.yaml new file mode 100644 index 0000000..c4e3277 --- /dev/null +++ b/demos/fixtures/shop/mdl-project.yaml @@ -0,0 +1,37 @@ +# Modelith project config (spec §2.2) +name: shop +dbt_target: duckdb_dev +platform_targets: + - duckdb_dev +# Ontology sources plug in by declaration. A source is either a local file +# vocabulary (default type: local) or a remote resolver browsed live for +# autocomplete/search (types: ols, ols-compatible, ontoportal, collibra). FIBO is +# only one example — ACORD / FHIR / ISO 20022 / your own plug in the same way. +# ontology_stack: +# - name: fibo # local file vocabulary +# layer: industry +# format: turtle +# path: ontologies/industry/fibo/2024.03 +# modules: [fnd, fbc] +# prefixes: +# fibo-fnd-pty-pty: "https://spec.edmcouncil.org/fibo/ontology/FND/Parties/Parties/" +# - name: ols # public OLS4 (no auth) — live search only +# layer: industry +# type: ols +# url: https://www.ebi.ac.uk/ols4/api +# - name: bioportal # OntoPortal/BioPortal (needs an API key) +# layer: domain +# type: ontoportal +# url: https://data.bioontology.org +# apikey_env: BIOPORTAL_APIKEY +# - name: collibra # Collibra Ontology Domains (bearer token) +# layer: core +# type: collibra +# url: https://acme.collibra.com +# token_env: COLLIBRA_TOKEN +# domain_types: [Ontology] +ontology_stack: [] +naming: + logical_case: snake + physical_case: upper_snake + abbreviations: {} diff --git a/demos/landing/index.html b/demos/landing/index.html new file mode 100644 index 0000000..5b6312c --- /dev/null +++ b/demos/landing/index.html @@ -0,0 +1,261 @@ + + +
+ + +Design entities and relationships in git, generate contract-enforced dbt, and + catch drift before it ships. One model every tool reads and writes — the CLI, the canvas, + VS Code, and CI.
+ +Every command and every line of output below is real. Nothing is mocked.
+ +Scaffold a model, add entities (ULIDs minted for you), validate, and compile to + contract-enforced dbt — plus an ODCS data contract from the same model.
+ mdl init · new entity · validate · generate +Point it at a warehouse with no data model. It excludes staging, detects SCD2 and + Data Vault, strips surrogate keys, and logs every inference in a reviewable ledger.
+ mdl reverse --project target/manifest.json +A column lands in the warehouse. Drift classifies it — additive, not breaking — + and reconcile folds the safe change into the model. Nothing is ever lost.
+ mdl drift --check · --reconcile +Search FIBO (or any OLS / OntoPortal / Collibra source), align an entity to a + standard term, and track coverage — anchored, reproducible, offline.
+ mdl ontology search · align · status +Import an erwin XML export straight into clean logical YAML you can version, diff, + and generate from. The migration path off a legacy modeling tool.
+ mdl import erwin fund.xml -o fund +From one definition: contract-enforced dbt, an ODCS data contract, Pydantic models, + SQL DDL, Mermaid, DBML, RDF/SHACL, and a Neo4j graph schema — each deterministic.
+ mdl generate · export · emit +Warehouse teams scatter their meaning across an ER tool that never sees + production, dbt SQL that drifts from the design, and a catalog nobody edits. Modelith + collapses them into one model every surface reads and writes.
+Generate dbt-core models with enforced + contracts, key constraints, and relationship tests. Regeneration runs a three-way merge, + so hand edits and generated blocks both survive.
Compare the committed model to the compiled
+ warehouse. Breaking / additive / cosmetic, classified in seconds. --check
+ fails the PR; --reconcile folds the safe deltas back in.
A live database, a dbt manifest, or raw + SQL DDL, lifted into a clean model — SCD2 and Data Vault detected, surrogates stripped, + every inference reviewable.
A four-layer vocabulary stack with SKOS + alignments and provenance. Pin bundles like dependencies; resolve against OLS, OntoPortal, + or Collibra — reproducibly, offline.
A ULID-keyed merge driver: two people add + different attributes to the same entity and merge clean. A rename is one cosmetic change, + not a delete-plus-add.
The model is plain YAML in git. The + CLI, the web canvas, the VS Code extension, and CI all read and write it — none of them + owns state.
Each generated deterministically from the single definition.
+mdl generatemdl export contractmdl emit pydanticmdl export graphmdl emit semanticmdl export rdfmdl export sqlmdl export mermaidmdl export dbmlNo cloud warehouse. The IBoR demo ships a model and + a dbt project over a bundled DuckDB, so it builds on a laptop.
+ +