Own the SQLMesh start date here and move dbt to 1.12 - #10
Merged
Merged
Conversation
SQLMesh refuses to load a dbt project without a backfill start date, and it only looks in the `models:` block of dbt_project.yml. The datamart carried `+start: Jan 1 2000` there purely for this stack - dbt v2 rejects it as an unrecognised key, and it reads as a stray to anyone working on the shared model. It moves here, as model_defaults on sqlmesh_config, which is where a consumer-specific setting belongs. This is safe to merge before the datamart drops `+start`: SQLMesh takes the config.py value whether or not dbt_project.yml still has one. It has to merge before DATAMART_REF moves to a tag without it. dbt-core 1.10 is also deprecated as of the v2 release. SQLMesh declares dbt-core<2, so 1.12.5 with dbt-duckdb 1.11.0 is fine; verified by running sqlmesh plan against the datamart branch that drops `+start` (23 models, audits passed). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Two things, both prompted by the dbt v2 release.
The start date. SQLMesh refuses to load a dbt project without a backfill start date, and it only looks for one in the
models:block ofdbt_project.yml. The shared datamart carried+start: Jan 1 2000there purely for this stack — it reads as a stray to anyone working on the model, and the improvement plan had it down for deletion. dbt v2 settles it:+startis an unrecognised key and a hard parse error.So the start date moves here, as
model_defaultsonsqlmesh_config, which is where a consumer-specific setting belongs.dbt 1.10 is deprecated as of the v2 release. SQLMesh declares
dbt-core<2, so 1.12 needs nothing else from us.Merge order
This is safe to merge before the datamart drops
+start: SQLMesh takes theconfig.pyvalue whether or notdbt_project.ymlstill has one. It has to merge beforeDATAMART_REFmoves to a tag without it.Verification
Ran
sqlmesh planagainst the datamart branch that drops+start, on dbt-core 1.12.5 + dbt-duckdb 1.11.0:READ_PARQUET(...)— the datamart branch declaresexternal_locationunder bothconfigandconfig.metaso v2 and SQLMesh each find it+startstill present, which is the merge-order case aboveduckdb stays at 1.4.4 so the warehouse remains readable by Superset.
🤖 Generated with Claude Code