Repository navigation
feat: a scan reads each database as of one instant, and fixes from a version and platform matrix - #149
Merged
Conversation
A view the target has and the source lacks is dropped by the schema check's fix and its grants revoked by the roles check's. When the drop ran first, each revoke failed with "does not exist", was reported as left out, and the run exited 1 — though the view, and its grants with it, were gone as intended. A fix that only revokes privileges on one object, still failing that way once retrying it after the others no longer helps, is now counted as applied, with a note that there was nothing left to revoke. A revoke whose object a later fix creates is still retried after it. Fixes #147 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… has A restore or apply runs in one transaction, which holds a lock on every relation it creates, alters or drops until it commits, in a table the whole server shares. A large one ran out part way, after minutes of work, and rolled everything back with "out of shared memory". The statements are now counted before the transaction starts — each relation once, and what a new table brings with it: its TOAST table and index, an index per key, an identity sequence — against the target's lock table, sized from max_locks_per_transaction and its backend slots, less what other sessions hold. When they look likely not to fit, the run says so first, with both numbers, raising max_locks_per_transaction and --no-transaction as the ways round it. It warns rather than refuses: the table can grow into spare shared memory, so a count over its size may still fit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
USAGE on a schema of the project's own is what lets a role reach the tables in it at all, and it was not compared: a target without it read as matching, and a sync left the schema out of the Data API's reach. The roles check now reads schema grants as it reads table and routine grants, offers the GRANT or REVOKE, and leaves the grants on a schema only the target has to the schema check, as it does for routines. PostgreSQL 15 stopped granting CREATE on public to everyone, so between servers either side of that change the difference is each version's default; copying it into a newer server would undo its hardening, and it is not compared there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
roles.sql held the grants on tables, columns and routines, and the schema layer is dumped without privileges, so a restore lost USAGE on the project's own schemas — leaving their tables out of reach — and every ALTER DEFAULT PRIVILEGES, so new tables were no longer granted what they were on the source. Both are captured now: a schema's grants as GRANT ... ON SCHEMA, and each role's defaults as the ALTER DEFAULT PRIVILEGES that make them, against PostgreSQL's built-in defaults for a global entry. The roles they name are created first on a target that lacks them, as for any grant. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A source newer than the target writes what the target has no syntax or feature for: UNIQUE NULLS NOT DISTINCT before PostgreSQL 15, a VIRTUAL generated column before 18, security_invoker before 15. The fix failed with a syntax error, and the whole apply rolled back with it. The apply now knows the source's version. On a target that is older, a syntax error, an unsupported feature or an unknown parameter means the target cannot hold that fix: it is left out and reported, as one needing a missing extension is, and the rest is applied. --prove rehearses it the same way. On a target as new as the source those are still real failures and roll everything back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…of the schema check dbdiff now compares default privileges, and between plain PostgreSQL and Supabase it offered to change those supabase_admin holds — the platform's, which postgres cannot alter — so the apply failed. Default privileges for a platform role are left out, as their grants are; postgres's own are the project's and still compared. pgmq's schema holds the queues' tables, which the queues check compares through pgmq; the schema check no longer reads them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A version mismatch was fixed with ALTER EXTENSION ... UPDATE TO the source's version whichever way it went. PostgreSQL has no way down a version, and a server may not ship the one asked for: between Supabase 15 and 17, pg_cron 1.6 against 1.6.4 failed, and the whole apply with it. The target's own update paths now decide: a version it can reach gets the ALTER, any other is reported with no fix and why. An extension in a schema of its own was created into it, which plain PostgreSQL may not have — Supabase's `extensions` — and the missing schema failed the apply. The schema is made first; an extension the server does not ship is still left out with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A snapshot of a newer PostgreSQL holds what an older one has no syntax or feature for — a VIRTUAL generated column before 18, UNIQUE NULLS NOT DISTINCT before 15 — and in one transaction that statement rolled the whole restore back. The snapshot's version is read from pg_dump's header in schema.sql; restoring into an older server, such a statement is left out and named, as one needing a missing extension is, and the rest is restored. Into a server as new, it still fails the restore. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The checks run many queries, concurrently, over seconds or minutes, and a change made meanwhile — a migration, a policy someone adds — was seen by some of them and not others: the report could describe a database that never existed. While a scan runs, one connection to each database holds a REPEATABLE READ, READ ONLY transaction that exports its snapshot, and each pooled connection joins it once. A query that fails ends that connection's transaction, which joins again next time; a definition rendered with a temporary object is undone to a savepoint. Both are let go when the scan ends, before anything is applied. A server that cannot export a snapshot is read as before. As with pg_dump, what the scan reads stays locked until it ends; --no-snapshot reads each query on its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hema
pg_cron is in pg_catalog on Supabase, and creating the schema an
extension goes in first wrote CREATE SCHEMA IF NOT EXISTS "pg_catalog",
which PostgreSQL refuses ("unacceptable schema name") — failing the
whole apply from Supabase to plain PostgreSQL. A system schema is
always there; only another one is made first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| * is let go; a read that fails, and a definition rendered with a temporary | ||
| * object, leave the reads after them working. | ||
| */ | ||
| import { describe, it, expect, beforeAll, afterAll, afterEach } from 'vitest' |
An extension in a schema of its own makes that schema first, so its fix no longer began with CREATE EXTENSION, and one the target's server does not ship — pg_graphql on plain PostgreSQL — was no longer recognised as such: it failed the whole apply instead of being left out. The fix is now recognised by a CREATE EXTENSION anywhere in it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From PostgreSQL 18 a NOT NULL is a constraint with a name: a table or column renamed keeps the old one, and one made afresh gets a default. The proof compares the migrated copy with the source's structure made anew on the target's server, so a rename from an older source read as its NOT NULLs missing and others unexpected, and the migration was refused. They are now matched by the column they hold for. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One commit per change, each with its own tests. Most came out of a smoke matrix: every pair of Supabase 15 and 17 and PostgreSQL 14–18, each way. Each pair ran a first sync, drift with live rows on the target (renames, a retype under a generated column and views, an enum label,
--prove), and a snapshot and restore.--no-transactionandmax_locks_per_transactionas the ways round it.USAGEon a schema of the project's own wasn't compared, so a target without it read as matching while its tables were out of the Data API's reach. PostgreSQL 15's change to the defaultCREATEonpublicisn't compared across that version boundary.roles.sqlheld table, column and routine grants. Restore lost schemaUSAGEand everyALTER DEFAULT PRIVILEGES.UNIQUE NULLS NOT DISTINCTon 14 or aVIRTUALcolumn on 17 failed with a syntax error and rolled back the whole apply. On an older target that fix is now left out and reported, and the rest applies.--proverehearses it the same way.supabase_admin's default privileges are the platform's, andpostgrescan't change them. pgmq's tables are compared by the queues check.UPDATE TOthe source's version, even downwards (pg_cron 1.6 against 1.6.4), which failed the apply. The target's update paths now decide. An extension in its own schema creates that schema first; plain PostgreSQL may lackextensions.pg_dump's header.--no-snapshotreads query by query.pg_catalogon Supabase, andCREATE SCHEMA "pg_catalog"is refused.CREATE EXTENSION, and one the server doesn't ship (pg_graphql on plain PostgreSQL) failed the whole apply instead of being left out.--provematches a NOT NULL constraint by its column, not its nameTests:
🤖 Generated with Claude Code