Skip to content

Gate the main test suite on test-build; add build-results; require ON_ERROR_STOP in test/install - #109

Open
jnasbyupgrade wants to merge 1 commit into
Postgres-Extensions:masterfrom
jnasbyupgrade:fix-test-build-ordering
Open

Gate the main test suite on test-build; add build-results; require ON_ERROR_STOP in test/install#109
jnasbyupgrade wants to merge 1 commit into
Postgres-Extensions:masterfrom
jnasbyupgrade:fix-test-build-ordering

Conversation

@jnasbyupgrade

@jnasbyupgrade jnasbyupgrade commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

test-build had no real dependency edge on installcheck, so the main suite (test/install + test/sql) actually ran before test-build's own sanity check in the default config, and raced with it under make -j -- see #108 for the empirical proof. Add an explicit installcheck: test-build edge (guarded to avoid recursing into test-build's own nested installcheck invocation) so a broken build blocks the main suite entirely: there's no point validating install/query behavior against a build that doesn't even come up cleanly.

Add make build-results to refresh test/build/expected/*.out from the last test-build run, mirroring make results for the main suite. It refuses to bless any file whose actual output contains an ERROR: line -- accepting an errored build as the new baseline would defeat the point of test-build -- and reports exactly which file to bless by hand if the error is intentional.

test/install/*.sql files never get a real diff: their actual output is written to the exact same file as their expected output (self-overwriting), so a content difference can never fail the build. Add check-test-install-error-stop, enforcing that every test/install/*.sql file sets ON_ERROR_STOP (directly or via \i test/pgxntool/psql.sql) -- the only thing that still turns a hard SQL error into a build failure -- closing #97. test/install/*.out is now gitignored, since it's rewritten by every run and never meaningfully compared.

Documented all of the above for both humans and AI agents reading base.mk/CLAUDE.md directly, since none of this is obvious from the code alone.

Fixes #108.
Fixes #97.

Related pgxntool-test PR: Postgres-Extensions/pgxntool-test#79

…_ERROR_STOP in test/install

test-build had no real dependency edge on installcheck, so the main suite
(test/install + test/sql) actually ran before test-build's own sanity
check in the default config, and raced with it under `make -j`. Add an
explicit `installcheck: test-build` edge (guarded to avoid recursing into
test-build's own nested installcheck invocation) so a broken build blocks
the main suite entirely -- its results would be meaningless otherwise.

Add `make build-results` to refresh test/build/expected/*.out from the
last test-build run, mirroring `make results` for the main suite. Refuses
to bless any file whose actual output contains an ERROR: line.

test/install/*.sql files never get a real diff -- their actual output
overwrites their expected output in place, so a content difference can
never fail the build. Add check-test-install-error-stop, enforcing that
every test/install/*.sql file sets ON_ERROR_STOP (directly or via
test/pgxntool/psql.sql), since that's the only thing that still turns a
hard SQL error into a build failure. test/install/*.out is now gitignored
since it's rewritten by every run and never meaningfully compared.

Fixes Postgres-Extensions#108.
Fixes Postgres-Extensions#97.

Related changes in pgxntool-test:
- Template's test/install file now sets ON_ERROR_STOP; its .out is no
  longer committed
- Test coverage for the new ordering edge, build-results, and
  check-test-install-error-stop

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 20, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: e3db3704-167d-438a-8cb9-312abbef158c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant