Handle every error kind the test schema's invalid attribute defines - #131
Open
bryantaustin13 wants to merge 1 commit into
Open
bryantaustin13 wants to merge 1 commit into
bryantaustin13 wants to merge 1 commit into
Conversation
testSchema.xsd defines five values for InvalidType — false, syntax, semantic, execution and true — four of which say the test expects the engine to fail, differing only in how. The runner recognised only `true` and `semantic`, so a test declaring `invalid="syntax"` fell through to value comparison and was scored a failure even though the engine had produced exactly the syntax error the test asked for. The semantics were duplicated across five call sites, so they now live in one place as two predicates: - expectsError() — all four error kinds. The test passes only if the engine errored, and needs no <output> to compare against. - preventsTranslation() — syntax and semantic only. Those cannot survive CQL-to-ELM translation, so build-cql must leave them out of a generated library; one such define would fail the translation of every define around it. Runtime and execution errors translate fine and are kept. Two further sites needed the same widening and would otherwise have broken: - The results schema enumerated only ["true", "false", "semantic"], so writing invalid="syntax" into a report would have failed results validation. - build-cql excluded only `semantic` from generated libraries, so a syntax-invalid define would have been emitted and broken its whole library. Both currently affected tests are exactly that case. Verified against the reference engine: 1639/170 -> 1641/168, with the two invalid="syntax" tests (CeilingIntegerGreaterThanMaxInteger and CeilingIntegerLessThanMinInteger) passing and no other test changing status. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
testSchema.xsd defines five values for InvalidType — false, syntax, semantic, execution and true — four of which say the test expects the engine to fail, differing only in how. The runner recognised only
trueandsemantic, so a test declaringinvalid="syntax"fell through to value comparison and was scored a failure even though the engine had produced exactly the syntax error the test asked for.The semantics were duplicated across five call sites, so they now live in one place as two predicates:
Two further sites needed the same widening and would otherwise have broken:
semanticfrom generated libraries, so a syntax-invalid define would have been emitted and broken its whole library. Both currently affected tests are exactly that case.Verified against the reference engine: 1639/170 -> 1641/168, with the two invalid="syntax" tests (CeilingIntegerGreaterThanMaxInteger and CeilingIntegerLessThanMinInteger) passing and no other test changing status.