Gate the main test suite on test-build; add build-results; require ON_ERROR_STOP in test/install - #109
Open
jnasbyupgrade wants to merge 1 commit into
Conversation
…_ERROR_STOP in test/install test-build had no real dependency edge on installcheck, so the main suite (test/install + test/sql) actually ran before test-build's own sanity check in the default config, and raced with it under `make -j`. Add an explicit `installcheck: test-build` edge (guarded to avoid recursing into test-build's own nested installcheck invocation) so a broken build blocks the main suite entirely -- its results would be meaningless otherwise. Add `make build-results` to refresh test/build/expected/*.out from the last test-build run, mirroring `make results` for the main suite. Refuses to bless any file whose actual output contains an ERROR: line. test/install/*.sql files never get a real diff -- their actual output overwrites their expected output in place, so a content difference can never fail the build. Add check-test-install-error-stop, enforcing that every test/install/*.sql file sets ON_ERROR_STOP (directly or via test/pgxntool/psql.sql), since that's the only thing that still turns a hard SQL error into a build failure. test/install/*.out is now gitignored since it's rewritten by every run and never meaningfully compared. Fixes Postgres-Extensions#108. Fixes Postgres-Extensions#97. Related changes in pgxntool-test: - Template's test/install file now sets ON_ERROR_STOP; its .out is no longer committed - Test coverage for the new ordering edge, build-results, and check-test-install-error-stop Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
test-build had no real dependency edge on
installcheck, so the main suite (test/install + test/sql) actually ran before test-build's own sanity check in the default config, and raced with it undermake -j-- see #108 for the empirical proof. Add an explicitinstallcheck: test-buildedge (guarded to avoid recursing into test-build's own nestedinstallcheckinvocation) so a broken build blocks the main suite entirely: there's no point validating install/query behavior against a build that doesn't even come up cleanly.Add
make build-resultsto refreshtest/build/expected/*.outfrom the last test-build run, mirroringmake resultsfor the main suite. It refuses to bless any file whose actual output contains anERROR:line -- accepting an errored build as the new baseline would defeat the point of test-build -- and reports exactly which file to bless by hand if the error is intentional.test/install/*.sqlfiles never get a real diff: their actual output is written to the exact same file as their expected output (self-overwriting), so a content difference can never fail the build. Addcheck-test-install-error-stop, enforcing that everytest/install/*.sqlfile setsON_ERROR_STOP(directly or via\i test/pgxntool/psql.sql) -- the only thing that still turns a hard SQL error into a build failure -- closing #97.test/install/*.outis now gitignored, since it's rewritten by every run and never meaningfully compared.Documented all of the above for both humans and AI agents reading
base.mk/CLAUDE.mddirectly, since none of this is obvious from the code alone.Fixes #108.
Fixes #97.
Related pgxntool-test PR: Postgres-Extensions/pgxntool-test#79